Files
Bryan ChenandCopilot a58fdc7c54 Engineering - fetch Chromium build scripts through the GitHub API with fallbacks (#344354)
* Engineering - fail and retry the Chromium clang script download

setup-env.sh piped 'curl -s' of Chromium's tools/clang/scripts/update.py from
raw.githubusercontent.com straight into python. When GitHub rate-limits the
request (429), python runs the error page and the install fails with
'SyntaxError: illegal target for annotation': 15 Linux PR CI runs in 30 days,
10 of them on Oct 26. Download the script to a file with curl -f, retry
transient errors, authenticate with GITHUB_TOKEN when it's set (all six
callers set it) to get the higher rate limit, and only then run it.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Engineering - fetch Chromium build scripts from Gitiles before GitHub

raw.githubusercontent.com rate-limits shared agent IPs per network, so
authenticated requests and short retries do not help during an outage. It
failed the releasing insider 489293 (Linux ARMHF Prepare deb package, all
five dpkg-shlibdeps.pl attempts got 429) and main CI 489263 (Linux X64 and
Quality Install dependencies, update.py).

Add build/lib/chromiumSource.ts, which downloads a file from the Chromium
tree at the version pinned in cgmanifest.json, from Chromium's canonical
Gitiles host first, then the GitHub mirror (authenticated, then anonymous),
retrying the set after 10, 30 and 60 s, and only writes content that looks
like a script. Use it for dpkg-shlibdeps.pl in calculate-deps.ts and for
clang's update.py in setup-env.sh, which also drops setup-env.sh's second
hard-coded copy of the Chromium version.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Bound each Chromium script download so a stalled source fails over

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Log which host served each Chromium script

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Try the GitHub API first: Gitiles answers 403 to ADO agents

Validation 489314 showed chromium.googlesource.com returning 403 to every
request from the ADO Linux agents, so it could not cover a raw GitHub 429.
raw.githubusercontent.com limits per network even with a token, while the
GitHub REST contents API limits authenticated requests per token. Try it
first when a token is set, then raw (authenticated, anonymous), then Gitiles.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-10-26 14:32:32 -07:00
..