mirror of
https://github.com/electron/electron.git
synced 2026-10-28 11:44:46 +08:00
Samuel Attard
d36eebfc2e
* build: trim the non-link overhead of macOS release builds Four independent, low-risk changes aimed at the macOS release job, which today spends well over an hour outside the ThinLTO links themselves: * Zip Symbols: `tar --options xz:threads=0` does not actually multi-thread on macOS (Apple's liblzma emits a single block), so the ~7 GB of dSYMs were being compressed by one core at preset -6, ~30 minutes per job. Pipe the tar stream through the xz binary with -T0 instead (falling back to the tar options when xz is not on PATH), and use preset -3, which is ~5x faster than -6 for ~12% larger output. Same .tar.xz format. * siso: set SISO_EXPERIMENTS=no-fallback on macOS. When a remote compile exceeds siso's 2 minute client-side deadline (RBE queueing, an upload stall) siso re-runs it locally; on the 5-core / 14 GB runners a local ThinLTO compile takes 5-10 minutes and lands on the critical path (75 such fallbacks in a recent release build, 570 in a testing build). no-fallback retries remotely with a 4x longer deadline instead. * GN: thin_lto_enable_cache=false for macOS release builds. CI starts from a fresh out dir, so the ThinLTO cache is written (~100 GB across a release build's links) and never read, competing for IO with links that are already swapping. Local builds keep the default. * build-stats: hash object files on worker threads and only walk obj/ and <toolchain>/obj/. The previous recursive readdir of the whole out dir (~800k entries) plus single-threaded sha256 of ~6 GB took 2-3 minutes; this takes seconds. Objects outside those trees (SDK crt*.o, rust sysroot copies) are no longer counted, which does not affect the change rate metric since only files present in both snapshots are compared. No-Verification-Needed: CI config and build scripts only, no product code * build: use context managers for the tar|xz pipeline (pylint R1732) * build: read only the tail of siso.INFO in build-stats.mjs siso.INFO grows with the step count and a release build's already exceeds V8's maximum string length once a second toolchain is in the graph, which made build-stats.mjs fail with 'Invalid string length'. The stats line is at the end of the log, so read the last 16 MB instead of the whole file. * build: drop SISO_EXPERIMENTS=no-fallback on macOS Under RBE queue pressure a remote compile can stay EXECUTING past siso's deadline; with no-fallback siso retries (deduped onto the same running op) and then fails the build outright, which is worse than a slow local fallback for PR CI. Give remote compiles a longer deadline in build-tools' siso config instead and keep the fallback as the safety net. * build: link the macOS snapshot tools without ThinLTO in release builds (#52884) * build: link the macOS snapshot tools without ThinLTO in release builds In official builds v8_snapshot_toolchain is the target toolchain, so mksnapshot, node_mksnapshot, v8_context_snapshot_generator and the natives code-cache generator are each a full ThinLTO+PGO link of V8 (and Blink, and Node) with dsymutil. On the 5-core / 14 GB macOS CI runners those four links take 12 + 6 + 20 + 2 minutes (21 + 46 min on x64, where they also run under Rosetta), run serially through a link pool of depth 1, and page heavily (5-9 GB RSS each), all before the Electron Framework link can start. Their outputs are snapshot/code-cache data that does not depend on how optimized the tool binary is. Add //electron/build/toolchain/mac with variants of the mac toolchain that turn off ThinLTO, PGO and debug info (symbol_level=0, so the extra objects are small and dsymutil has nothing to link) and point macOS release builds' v8_snapshot_toolchain at them: clang_arm64_snapshot for arm64 and clang_x64_snapshot for x64 (still x64 so mksnapshot.zip keeps shipping an x64 mksnapshot). V8/Blink/Node compile a second time in that toolchain, but remotely and cacheably. Because the tools now live under $root_out_dir/<toolchain>/, mksnapshot.zip is flattened on macOS so its layout is unchanged (mksnapshot, v8_context_snapshot_generator, their dylibs and icudtl.dat at the root), and build/zip.py learns to apply PATHS_TO_SKIP relative to a toolchain dir and to drop duplicates when flattening. The resulting file set is identical to the default-toolchain one. Testing builds are unaffected (no LTO there, so nothing to gain from a second toolchain). No-Verification-Needed: build config and packaging script only * build: update build-tools for remote generators and longer remote compile deadlines (#52896) Picks up electron/build-tools#889 and #890: * siso runs the mojom, Blink bindings, grit strings and TypeScript/WebUI generators on RBE workers instead of locally (all hosts). On the 5-core macOS runners those steps are about a CPU-hour of python/node per release build, contending with siso's include scanning during the fetch phase; remotely they are also cached, so unchanged generators become cache hits on every host. * remote clang steps on macOS hosts get a 10 minute client-side deadline instead of 2 minutes, so ordinary RBE queueing no longer triggers the 5-10 minute local ThinLTO fallback compiles that land on the critical path (75 in one release build, 570 in a testing build). Local fallback stays as the safety net.