Files
Dylan Conway fc297d4658 ci: content-addressed machine images (#43608)
## What

CI machine images are content-addressed. An image's name is
`<key>-<hash>`: the key is the operating system or distro and the
architecture (`linux-x64-debian`, `windows-aarch64`), and the hash
covers everything its bake runs, the release and the base image
included. A CI build asks whether each image exists and bakes the ones
that do not before anything else runs; every later build (the PR's next
push, `main` after the merge) finds them by name. The `[build images]` /
`[publish images]` commit tags, the `# Version:` numbers and the
`-build-<n>` names are gone: to change what is on a CI machine, edit one
file.

## One file

`scripts/build/ci-images/spec.ts` is everything about CI's machines:

1. **The data**: the files copied onto machines, every version (`pins`),
where a bake puts things (`locations`), the six Linux and two Windows
images each with its exact base image, the package lists, and how Packer
bakes Windows.
2. **The tools**: one function per thing a bake sets up on a machine,
and the order they run in. A tool does not contain shell. It returns
steps that say what should be true (`download`, `unpack`,
`installExecutable`, `directory`, `systemUser`, `service`,
`registryValue`, `scheduledTaskAtStartup`, …), and one description
serves every system the tool exists on: `bun` is one function for Linux,
macOS and Windows.
3. **The machinery**: the vocabulary the tools are written in, its
renderers (sh as root for Linux, sh with `sudo` where a system path is
written for macOS, PowerShell for Windows), the Packer template, and the
generator. No shell is kept in strings: the generated scripts have no
helper functions, a step renders the lines themselves (a download is a
`curl` line on every system), and in PowerShell every statement that ran
a native program is followed by a check of its exit code.

`bun run ci:images [key…]` writes `build/ci-images/<key>/`
(`image.json`, `bootstrap.sh` or `bootstrap.ps1`, the files the script
puts on the machine, and for Windows the Packer template) and prints
each name, which is the sha256 of that directory. The generated script
is what is read, linted and debugged: straight-line, with every value in
it, looping only over what is known on the machine. The output depends
only on committed files. Besides `spec.ts`, only `scripts/agent.ts`
(copied whole onto every baked machine) and the vendored
`scripts/build/xmac.mjs` (the build image) can rename an image;
`spec.ts` lists both.

`.buildkite/ci.ts` generates every image, asks the cloud for each name's
state (`scripts/ci-image.ts`: AWS by name among the account's own
images, Azure by gallery version), and for a missing or failed one
uploads its directory and emits a bake step; for one another build is
already baking, only a wait step. A fork's build cannot bake (a bake
runs the branch's code on a machine that becomes everyone's image), so
it does not ask the cloud anything and reads none of the cloud's
credentials: it uses the images that exist, and fails with an
explanation if its diff touches a bake input. A Linux bake runs
`bootstrap.sh <commit> <image name>` as root on a machine started from
the exact base image the spec names; a Windows bake runs Packer, pinned
and checksummed, as the step's single command so that a cancel reaches
it.

The build system's LLVM, Node.js, xwin, Windows SDK, macOS SDK, Android
API level and FreeBSD versions import from `pins`, and its sysroot and
download-cache lookups from `locations`. On a Buildkite agent the build
compares `bun`, `cmake`, `node`, `clang` and `ld.lld` with the pins; a
bake installs and does not judge versions.
`test/internal/source-lints/ci-image-pins.test.ts` keeps
`rust-toolchain.toml` and the GitHub workflows (Bun, LLVM, the Rust
nightly) in step with the spec, since other programs read those.

Every bake writes `bun-image.json` on the machine and publishes it as
the bake job's artifact. It is for keying caches of build outputs, so it
holds what the name cannot: what arrived on the day of the bake. `Tool`
requires an `identity`, so a tool that does not say how what it installs
is known does not compile: `pinned` (the spec's value is written as it
is), `observed` (a step rendered into the script, whose output is
recorded: the glibc and musl cross sysroots as a digest of their trees,
and on Windows the MSVC toolset and Windows SDK versions the Visual
Studio installer chose), `packageDatabase` (the record lists apt's,
apk's or Scoop's database with exact versions), `configuration`, or
`notRecorded` with the reason (the prefetch). The file is deterministic:
no times, hostnames or instance ids.

## What changes on the machines

- The Rust toolchain CI builds with is installed by the bake, from the
pinned channel with its components and targets.
- Bun is 1.4.2 on the images and in the GitHub workflows (they were
1.3.13 and 1.3.14).
- Pinned where they floated: the base images (exact AMI names, exact
Marketplace versions), Bun and buildkite-agent on Windows, rustup-init,
Packer, nssm. Floating, by name, as before: distro, Scoop and Homebrew
packages, Docker, Tailscale, Chrome, Visual Studio, the patch release of
LLVM from apt.
- Bun's build of ninja (oven-sh/ninja) is baked into every machine, in a
directory of its own that is not on PATH; the `ninja` the build runs is
still the machine's own.
- The prefetch (dependency sources, test Docker images, `bun install`
cache) is one tool for Linux and Windows: it clones the commit being
built, fetches, and removes the clone. Its contents stay outside the
hash, so a dependency bump does not rename an image. Raising
`prefetchTriggerVersion` in the spec is how the caches are refreshed:
the number is written into that tool's section of the generated scripts,
which renames every baked image, and the next build bakes them all.
- The Windows image no longer carries one set of SSH host keys for every
machine, and password logins are really off.
- Smart App Control is turned off by the bake alone; the test runner's
copy, for images baked before the bake did it, is removed (on machines
from these images it found the policy already off).
- Not carried over, because CI never ran it: the FreeBSD, dnf/yum/zypper
and `--gcc-13` paths, the `qemu-user` packages, Strawberry Perl (the old
script skipped it; `perl` on Windows is Git's).

## macOS

The macOS test machines are in the spec too, with the same pinned
Node.js, Bun, curl-h3, LLVM (through Homebrew) and Rust toolchain. CI
does not bake or name them: `scripts/darwin-ci` generates
`build/ci-images/darwin-<arch>/bootstrap.sh` on the host and runs it in
the Tart guest image it builds, or on a bare host.
`scripts/bootstrap.sh`, `scripts/bootstrap.ps1` and `scripts/packer/`
are removed. The generated macOS script passes shellcheck and every URL
in it exists, but it has not run on a Mac: `darwin-ci bake --ref <this
branch>` on one Tart host is the test, and it leaves the host's image
alone unless the toolchain check passes.

## Verified

- An earlier revision of this branch, with hand-written scripts per
tool, baked all eight images and ran the full build and test pipeline on
them. The generated Linux scripts have since baked all six Linux images
on real machines.
- Four reviews of the generated output against those hand-written
scripts (Linux; Windows and macOS; the generator's correctness, with
probes; conventions) and one security review against main. What they
found is fixed: `DEBIAN_FRONTEND` exported once so installers that call
apt inherit it, files written from PowerShell without a byte-order mark
(node-gyp read its `installVersion` as 0), `unzip` replacing on a Mac
that is set up again, PowerShell literals in expressions always quoted.
The security review found nothing looser than main except that the
pipeline read cloud credentials in fork builds, which it no longer does.
- For the generated scripts: shellcheck on all eight sh scripts,
PowerShell's parser on both Windows scripts and the SSH key script,
`packer validate` on both templates with exactly the pinned Packer; the
Debian 13 and Alpine 3.23 x64 scripts run to completion in containers up
to the prefetch; a simulated Buildkite run with every image missing
uploads eight directories and emits eight bakes.
- `tsc` is clean for both script projects; the source lints and the
generator test pass.
2026-09-20 03:38:01 -07:00

177 lines
5.5 KiB
Nix

{
description = "Bun - A fast all-in-one JavaScript runtime";
# Uncomment this when you set up Cachix to enable automatic binary cache
# nixConfig = {
# extra-substituters = [
# "https://bun-dev.cachix.org"
# ];
# extra-trusted-public-keys = [
# "bun-dev.cachix.org-1:REPLACE_WITH_YOUR_PUBLIC_KEY"
# ];
# };
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
flake-utils.url = "github:numtide/flake-utils";
};
outputs = { self, nixpkgs, flake-utils }:
flake-utils.lib.eachDefaultSystem (system:
let
pkgs = import nixpkgs {
inherit system;
config = {
allowUnfree = true;
};
};
# LLVM 23 - the major of pins.llvm in scripts/build/ci-images/spec.ts (actual version from nixpkgs-unstable)
llvm = pkgs.llvmPackages_23.llvm;
clang = pkgs.llvmPackages_23.clang;
lld = pkgs.llvmPackages_23.lld;
# Node.js 26 - the major of pins.nodejs in scripts/build/ci-images/spec.ts (actual version from nixpkgs-unstable)
nodejs = pkgs.nodejs_26;
# Build tools and dependencies
packages = [
# Core build tools
pkgs.cmake # Expected: 3.30+ on nixos-unstable as of 2025-10
pkgs.ninja
pkgs.pkg-config
pkgs.ccache
# Compilers and toolchain - version pinned to LLVM 23
clang
llvm
lld
pkgs.gcc
pkgs.rustc
pkgs.cargo
pkgs.go
# Bun itself (for running build scripts via `bun bd`)
pkgs.bun
# Node.js - version pinned to 26
nodejs
# Python for build scripts
pkgs.python3
# Other build dependencies, as on the CI images (scripts/build/ci-images/spec.ts)
pkgs.libtool
pkgs.ruby
pkgs.perl
# Libraries
pkgs.openssl
pkgs.zlib
pkgs.libxml2
pkgs.libiconv
# Development tools
pkgs.git
pkgs.curl
pkgs.wget
pkgs.unzip
pkgs.xz
# Additional dependencies for Linux
] ++ pkgs.lib.optionals pkgs.stdenv.isLinux [
pkgs.gdb # for debugging core dumps
# Chromium dependencies for Puppeteer testing
# X11 and graphics libraries
pkgs.xorg.libX11
pkgs.xorg.libxcb
pkgs.xorg.libXcomposite
pkgs.xorg.libXcursor
pkgs.xorg.libXdamage
pkgs.xorg.libXext
pkgs.xorg.libXfixes
pkgs.xorg.libXi
pkgs.xorg.libXrandr
pkgs.xorg.libXrender
pkgs.xorg.libXScrnSaver
pkgs.xorg.libXtst
pkgs.libxkbcommon
pkgs.mesa
pkgs.nspr
pkgs.nss
pkgs.cups
pkgs.dbus
pkgs.expat
pkgs.fontconfig
pkgs.freetype
pkgs.glib
pkgs.gtk3
pkgs.pango
pkgs.cairo
pkgs.alsa-lib
pkgs.at-spi2-atk
pkgs.at-spi2-core
pkgs.libgbm # for hardware acceleration
pkgs.liberation_ttf # fonts-liberation
pkgs.atk
pkgs.libdrm
pkgs.xorg.libxshmfence
pkgs.gdk-pixbuf
] ++ pkgs.lib.optionals pkgs.stdenv.isDarwin [
# macOS specific dependencies
pkgs.darwin.apple_sdk.frameworks.CoreFoundation
pkgs.darwin.apple_sdk.frameworks.CoreServices
pkgs.darwin.apple_sdk.frameworks.Security
];
in
{
devShells.default = (pkgs.mkShell.override {
stdenv = pkgs.clangStdenv;
}) {
inherit packages;
hardeningDisable = [ "fortify" ];
shellHook = ''
# Set up build environment
export CC="${pkgs.lib.getExe clang}"
export CXX="${pkgs.lib.getExe' clang "clang++"}"
export AR="${llvm}/bin/llvm-ar"
export RANLIB="${llvm}/bin/llvm-ranlib"
export CMAKE_C_COMPILER="$CC"
export CMAKE_CXX_COMPILER="$CXX"
export CMAKE_AR="$AR"
export CMAKE_RANLIB="$RANLIB"
export CMAKE_SYSTEM_PROCESSOR="$(uname -m)"
export TMPDIR="''${TMPDIR:-/tmp}"
'' + pkgs.lib.optionalString pkgs.stdenv.isLinux ''
export LD="${pkgs.lib.getExe' lld "ld.lld"}"
export NIX_CFLAGS_LINK="''${NIX_CFLAGS_LINK:+$NIX_CFLAGS_LINK }-fuse-ld=lld"
export LD_LIBRARY_PATH="${pkgs.lib.makeLibraryPath packages}''${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
'' + ''
# Print welcome message
echo "====================================="
echo "Bun Development Environment"
echo "====================================="
echo "Node.js: $(node --version 2>/dev/null || echo 'not found')"
echo "Bun: $(bun --version 2>/dev/null || echo 'not found')"
echo "Clang: $(clang --version 2>/dev/null | head -n1 || echo 'not found')"
echo "CMake: $(cmake --version 2>/dev/null | head -n1 || echo 'not found')"
echo "LLVM: ${llvm.version}"
echo ""
echo "Quick start:"
echo " bun bd # Build debug binary"
echo " bun bd test <test-file> # Run tests"
echo "====================================="
'';
# Additional environment variables
CMAKE_BUILD_TYPE = "Debug";
ENABLE_CCACHE = "1";
};
}
);
}