Alexandru DimaandCopilot
fe721a5460
* ci: only warm missing node_modules caches on main Every push to main started six cache warming jobs on 1ES and macOS runners, which found their caches present and exited in about 30 seconds in 93 of the last 100 runs, while waiting up to 29 minutes for runners shared with pull request jobs. A job on a GitHub-hosted runner now computes the cache keys of each job, including the CRLF checkouts of the Windows runners, and lists the caches on main. Jobs only run when one of their caches is missing, and all jobs run when the detection fails. At most one run is in progress so that concurrent pushes don't install the same missing dependencies several times. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * ci: also match cache versions when detecting missing caches actions/cache only restores a cache whose key and version both match, and the version depends on the cache path, compression method and OS. Detecting caches by key alone could skip warming a cache that the jobs can no longer restore, for example after a runner image changes its home directory. The detection now also compares the cache versions, mirroring actions/cache with the cache paths of the runners. A cache with another version counts as missing, so the job runs and saves a compatible one. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
5.0 KiB
node_modules cache scripts
Shared helpers used by the GitHub Actions workflows to store and restore the
node_modules cache. They exist so the archive format and compression flags
live in a single place instead of being duplicated across every workflow.
Each script takes an action argument:
| Script | Platform | archive |
extract |
|---|---|---|---|
cache.sh <action> |
Linux / macOS | Create node-modules.tzst (cache miss) |
Restore node-modules.tzst (cache hit) |
cache.ps1 <action> |
Windows | Create cache.7z (cache miss) |
Restore cache.7z (cache hit) |
Linux/macOS use multi-threaded zstd (node-modules.tzst); Windows uses 7-Zip
(cache.7z). The archive must NOT be named cache.tzst: actions/cache names
its own tarball cache.tzst and passes --exclude cache.tzst, which would drop
our archive and save an empty cache. The two families are not interchangeable,
so the cache key already encodes the OS (node_modules-linux-*,
node_modules-windows-*, …).
Example usage from a workflow step:
- name: Extract node_modules cache
if: steps.cache-node-modules.outputs.cache-hit == 'true'
run: ./.github/workflows/node_modules_cache/cache.sh extract
- name: Create node_modules archive
if: steps.cache-node-modules.outputs.cache-hit != 'true'
run: ./.github/workflows/node_modules_cache/cache.sh archive
If you change the archive format or flags, update both the archive and
extract branch of the script, and bump build/.cachesalt to invalidate
existing caches.
Warming the caches on main
Pull requests can only restore caches saved on their base branch, so pr-node-modules.yml saves the node_modules, Electron, built-in extensions and WSL installer caches on every push to main. Most pushes don't change any of them, so the Detect Missing Caches job first computes each platform job's cache keys on a GitHub-hosted runner and lists the caches on main (see detect-missing-caches). A platform job only starts its runner when one of its caches is missing, for example after a dependency change or a cache eviction. All platform jobs run when the detection fails.
The detection mirrors the cache keys of the platform jobs, including the CRLF line endings of the Windows checkouts, and the cache versions that actions/cache matches in addition to the key: they depend on the cache paths, including the home directory of the runners for Electron. A cache with another version counts as missing, so the job runs and saves a compatible one; if the detection then keeps warning about a version, update its cache paths. When you change a cache key, path, key prefix, or the jobs of the workflow, update cacheWarmingJobs and getCaches in detect.ts; build/lib/test/detectMissingCaches.test.ts fails until they match.
At most one run of the workflow is in progress, and newer pushes replace its pending run, so concurrent pushes don't install the same missing dependencies several times.
Electron download cache
The platform jobs in pr-node-modules.yml also warm a
separate Electron download cache on main. The
setup-electron action shares cache
handling and download retries with the Linux, macOS, and Windows PR test
workflows:
- By default, restore cached archives and validate/prepare Electron, downloading with up to three attempts as needed. PR jobs use this mode without saving.
lookup-only: 'true'only checks for a cache entry, without restoring archives, preparing Electron, or saving. This mode does not require npm dependencies.save-cache: 'true'also saves after successful setup on a cache miss.
The cache stores @electron/get's original download archives, not the prepared
.build/electron application. Its key includes the runner OS, target
architecture, and a hash of .npmrc and build/checksums/electron.txt. Ordinary
package-lock.json changes do not invalidate it. If a downloader update changes
the cache format, bump the electron-download-v1 prefix in setup-electron.
The paths match @electron/get's platform defaults, including XDG_CACHE_HOME
on Linux and LOCALAPPDATA on Windows. Cache paths and keys are handled within
the action.
Even on a cache hit, PR jobs run npm run electron to validate and prepare the
application from the cached archives using the current checkout. Checksum
validation remains enabled: the current downloader still fetches
SHASUMS256.txt from GitHub, so this cache avoids the large ZIP downloads but
does not make Electron setup entirely network-independent.
On main, an Electron cache hit uses lookup-only checks and skips downloading
Electron. On a miss, the platform job restores node_modules (or installs them
if that cache also misses), then invokes setup-electron with save-cache: 'true'.
This checks the cache again before preparing and saving Electron, while keeping
already-warm jobs from extracting dependency archives unnecessarily.