mirror of
https://github.com/oven-sh/bun.git
synced 2026-10-25 21:40:30 +08:00
Dylan Conwayandautofix-ci[bot]
cf712112fa
## What The scripts CI runs are TypeScript now, typed under the same strict settings as `scripts/build/`, and the ones CI does not run are gone. | before | after | | --- | --- | | `.buildkite/ci.mjs` | `.buildkite/ci.ts` | | `scripts/utils.mjs` | gone: see "utils is dissolved" below | | `scripts/agent.mjs` | `scripts/agent.ts` | | `scripts/runner.node.mjs` | `scripts/runner.node.ts` | | `scripts/features.mjs` | `scripts/features.ts` | | `scripts/machine.mjs`, `azure.mjs`, `docker.mjs`, `tart.mjs` | `scripts/ci-image.ts` (only the two commands CI runs: `bake-image` and `wait-image`) | | `scripts/p-limit.mjs`, `scripts/yocto-queue.mjs` (vendored) | a small limiter inside the runner | All but `features.ts`, which the freshly built bun runs, run under plain `node` (type stripping), so only erasable syntax is used, and `scripts/tsconfig.json` covers them. The Lint workflow now runs `tsc --noEmit` over `scripts/tsconfig.json` and over `scripts/build/tsconfig.json`. Nothing typechecked the build system in CI before; it had four errors (the two byte-class generators index a 16-entry table with a computed number), which now go through the build's `assert()`. The generated `.h` and `.rs` files are byte-identical. ## utils is dissolved `utils.mjs` was a grab bag. By the time it was dissolved (converted, and with the helpers only `machine.mjs` used already gone), it exported 74 values: 9 imported by nothing, 38 by exactly one script, and 27 by two or more. Going by what each script actually reaches: - What only one script used moved into that script: cloud metadata, the AWS and Azure secret readers and the SigV4 signer into `agent.ts`; the canary revision, the YAML writer, the bootstrap version and the emoji table into `ci.ts`; unzip, the live output filter and a few more into the runner; parsing compiler output into annotations into the build system, as `scripts/build/annotations.ts`. - What was left went two ways. What is known about the machine went to `agent.ts`, which reports it as the agent's tags: the platform, os, arch, abi, distro, kernel, hostname and temp directory, plus the few small helpers it needs for itself and now exports (`output(command)`: what a command prints, or undefined; `run(command)`: a command on this process's stdio that throws unless it exits with 0; `which`, `requireCommand`, `request`). What is known about the CI environment is `scripts/buildkite.ts`: what Buildkite and GitHub say about the build, secrets, meta-data, artifacts, annotations, log groups, `getJson`, `getPullRequestFiles`. An optional environment variable is read as `process.env.NAME`; `getEnv(name)` is for the ones that have to be set. - Removed outright: `spawn()`/`spawnSafe()`/`spawnSync()` and their result record (every caller wanted `output` or `run`), the `privileged` spawn option (only the agent's install used it, always right after writing a root-owned file, so the sudo/doas probing behind it never mattered), a `debugLog()` only the Windows image step switched on, four one-line wrappers over `node:fs`, `curl()` options nothing passed (it is `getJson(url)` on the agent's `request()` now, so there is one retry loop), `cwd` parameters nothing passed, parameters and branches of the annotation parser and formatter nothing reached, helpers nothing called, and two pieces that could not do anything: `getWindowsExitReason()`, which looked for `ntstatus.h` under a relative path and so only ever returned `undefined`, and a `lastGroup` in `startGroup()` that was only assigned inside a branch that needed it set already. - The runner had its own copies of `escapeHtml`, `escapeCodeBlock` and `unescapeGitHubAction`, and its own `stripAnsi` that only removed `ESC[<digits>m`; there is one of each now. Its own `spawnSafe`, a different function from the shared one under the same name, is `spawnWithTimeout`. The moves change no code; the generated pipeline is byte-identical across them. The converted code follows `scripts/build/`'s style: dot access on typed objects, `interface` for object types, no comments that restate a `!`. `scripts/tsconfig.json` extends the build system's. Fixes that ride along, each in its own commit: the runner's results never carried the exit code, signal and duration that `--results-json` and the JUnit report read (the report's times were all 0), and the link of a failed test never got its error's line (it does now, when the error is in the test file and not in a helper); the manual build's form asked whether to run tests "even if no test files have changed", which nothing ever decided on, so the question is gone; build meta-data is read and written one value at a time instead of ten at once (only a manual build's options step notices). Also: `spawn()` resolved on the child's `exit`, which can come before its output has been read (with 60 children at a time running `echo true`, 418 to 540 of 600 calls came back empty; resolving on `close`, none do), and `curl()` kept the error of a failed attempt after a later attempt succeeded. ## Conversions No logic changes were intended in the conversions; JSDoc types became real types (no `any`, no `@ts-ignore`), with overloads where a helper's return type depends on its options (`getEnv`, `getSecret`, `which`), so callers get a `string` without a cast. - **`ci.ts`**: `Target`, `Platform`, the agent a step asks for, retry rules and `Step` (group, command and block steps) are typed. The generated pipeline is byte-identical to the one `ci.mjs` generated, for a normal commit and for a `[build images]` commit (checked by generating both before and after). - **`ci-image.ts`**: the pipeline passes the Windows image name it computed (`bake-image --arch --name`), so the name is derived in one place instead of two that had to agree. - **`features.ts`** drops its `// @bun` pragma, which hands a file to the engine untranspiled: fine for a `.mjs`, a trap for a `.ts`. - **`runner.node.ts`**, **`agent.ts`**: typed; the agent's per-platform paths come from one function. Typing turned up code that could only throw, which is fixed: `setEnv` appended to an undeclared `outputPath` (the variable two lines up was meant), `getCloudMetadata` used `inspect` without importing it, the runner called `server.off()` without a listener, passed numbers as environment values (which also broke `SHOW_SPAWN_COMMANDS`) and took `Object.entries` of an env that may be absent. Wrong JSDoc (a return type, a block sitting on the wrong function, fields declared but never emitted) is corrected or dropped. Two small behaviour changes besides those: a platform key from a manual build's options step that the build does not have (asan on `main`) is now an error naming the key, where it used to produce steps called `undefined-undefined-build-bun`; and the build steps drop `--experimental-strip-types`, since every CI image has Node 26. ## Removed - Everything `machine.mjs` did besides the Windows Packer bake and `wait-image`: booting a machine in a cloud or VM and opening a shell on it (`azure.mjs`, `docker.mjs`, `tart.mjs`, the six `machine:*` package scripts), and the files only that reached (`.buildkite/Dockerfile`, `.buildkite/Dockerfile-bootstrap.sh`, the Linux Packer template). CI uses none of it. - `.buildkite/bootstrap.yml`: nothing reads it, the pipeline step is configured in the Buildkite UI, and the copy had drifted from it. ## agent.ts is the one file that runs on a CI machine `agent.ts` imports nothing but Node: it uses `process.env`, `child_process`, `fetch` with a retry for the cloud's metadata and secret services, and a small PATH lookup. `install` copies that one file into the agent's home as `agent.mts` (an ES module by its extension, wherever it sits) on every platform (it only copied on macOS before) and registers the service against the copy. The Linux image bake runs `node ./scripts/agent.ts install` from its own checkout instead of expecting the script to have been put on the machine beforehand, and the Windows Packer templates upload the one file instead of an esbuild bundle. ## Two one-line files Two names are set outside a checkout, so each keeps a one-line file that imports the TypeScript one: - `.buildkite/ci.mjs`: the pipeline step's command is configured in the Buildkite UI, for every branch at once. - `scripts/runner.node.mjs`: the command hook installed on the macOS tart hosts (`scripts/darwin-ci/hooks/command.ts`) recognises a test step by that name, so the macOS test steps keep calling it. The hook in the repo now matches either name; once the hosts have it, the steps can call `runner.node.ts` and this file can go. ## Testing - `tsc --noEmit` over both projects: 0 errors. - Pipeline generation: byte-identical before and after the `ci.ts` conversion; after that, the only differences are the command lines that name the renamed scripts, the Windows image step's arguments, and the dropped `--experimental-strip-types`. - `bun test test/internal/runner-junit.test.ts test/internal/source-lints/ci-annotations.test.ts test/internal/source-lints/build-rust.test.ts test/internal/ci-slowest-tests.test.ts`: pass. - The runner starts under `node`, directly and through `runner.node.mjs`, and selects nothing for a filter that matches nothing. Not exercised yet: the image bake path (`agent.ts install` on a fresh Linux box, the Windows Packer upload, the service starting from `agent.mts`). That needs a `[build images]` run, which in turn needs the matching change on the CI provisioner side to be live first. This PR's own CI exercises everything else: pipeline generation, builds, and the test runner on every platform. --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
15 lines
543 B
JSON
15 lines
543 B
JSON
{
|
|
// The CI scripts: pipeline generation, the agent service, the test runner
|
|
// and what they share. They run under plain `node` (type stripping), so only
|
|
// erasable syntax.
|
|
"extends": "./build/tsconfig.json",
|
|
"compilerOptions": {
|
|
"erasableSyntaxOnly": true,
|
|
// runner.node.ts imports test/docker/prestart-map.mjs.
|
|
"allowJs": true,
|
|
// `extends` does not rebase this path.
|
|
"types": ["../packages/bun-types"]
|
|
},
|
|
"include": ["agent.ts", "buildkite.ts", "runner.node.ts", "ci-image.ts", "../.buildkite/ci.ts"]
|
|
}
|