Review on #124676: keep reusing the MasterCI build that compiles master with
the PR flags, but make it explicit that its binary is the performance
reference, and keep the cmake options that tell an official build from a PR
build in one place so the reference cannot drift from a PR build.
- Rename `Build (amd|arm_release_pr_cache_warmup)` to
`Build (amd|arm_release_performance_reference)`, with the artifacts
`CH_AMD|ARM_RELEASE_PERFORMANCE_REFERENCE`. The job still compiles every
object with the shared sccache read-write, and the arm one still uploads
the build profile baseline, now under the new check name.
- `flavor_cmake_options` in build_clickhouse.py is the only place that adds
`-DCLICKHOUSE_OFFICIAL_BUILD=1` (official) or `-DDISABLE_ALL_DEBUG_SYMBOLS=1`
(PR, public, ThinLTO). A reference build takes the PR options, the cmake
line of its release build type and that build's PGO/BOLT decision.
Behavior is unchanged: for every build type, master/PR, public/private and
with/without PGO+BOLT profiles, main() produces the same cmake and build
commands as before the refactor (280 combinations), and the reference cmake
equals the PR release cmake for amd and arm.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The merge-queue flaky check reran only the PR's own new/changed tests. That
catches a `master` change breaking the PR's tests, but not the reverse: the PR
breaking a test that landed on `master` after the PR's last CI run. This is how
https://github.com/ClickHouse/ClickHouse/pull/124131 broke
`05339_string_filters_during_scan_parquet_short_dictionary_values` from
https://github.com/ClickHouse/ClickHouse/pull/124507 and was reverted by
https://github.com/ClickHouse/ClickHouse/pull/126396.
The `Config Workflow` job of the merge queue, which has the commit history, now
finds the tests changed on the first-parent history of the queue base after the
PR head commit time (or after the merge base, if the PR merged `master` since),
and stores them for the flaky check. When there are more than 30, the ones whose
names share more words with the names of the files changed by the PR come first.
For the merge-queue commit of #124131 (`d64fd503b42e`), the range has 819 tests,
and 05339 is selected second.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Resolve conflicts in the generated `.github/workflows/master.yml` and
`.github/workflows/pull_request.yml` by regenerating them with
`python3 -m ci.praktika yaml`. `contrib/llvm-project` fast-forwards to the
PR's pointer, which already contains master's.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`ReleaseBranchCI` already runs `Build (amd_illumos)`, so backports should be
checked by the same job before they merge into a release branch.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With the label a pull request runs the LLVM coverage family and nothing else:
`Build (llvm_coverage_build)`, the `amd_llvm_coverage` stateless, integration and
unit test shards, the `excluded_from_llvm` jobs with their `Build (amd_binary)`,
and the `LLVM Coverage` merge job. The flaky check of the changed tests runs on
the `amd_binary` build, because the coverage shards run without random settings
and never fail the pipeline. The style check stays. Every other job is skipped.
Made for pull requests that only add tests and need the coverage report, such as
the pull requests of the coverage agent. They use `ci-force-all` so far: about
4,400 machine minutes per push, against about 1,400 with this label.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Add toolchain and ci job to build for illumos, modeled after freebsd
cross-compilation.
Note: we disable a number of features that don't currently build on
illumos. These can be reenabled in the future if desired.
Resolves#23777.
Runs the LLVM coverage build, the unit, stateless and integration coverage shards and the LLVM Coverage report job every 3 hours (and on dispatch), reusing the shared job configs. Rows in `coverage_ci.coverage_data` keep `branch = 'master'` and `pull_request_number = 0`.
Master no longer builds `amd_llvm_coverage` or runs the LLVM coverage functional, integration and unit test jobs, the `excluded_from_llvm` integration job and the `LLVM Coverage` report job. The `DBReplicated`, `ParallelReplicas` and `AsyncInsert` stateless configurations and the full integration run now use the `arm_binary` jobs defined by https://github.com/ClickHouse/ClickHouse/pull/123790, so master runs the same configurations as pull requests.
Related: https://github.com/ClickHouse/ClickHouse/issues/124015
In `master_head` mode the PR performance comparison used master's official
release build as the reference. That build keeps debug info and sets
`-DCLICKHOUSE_OFFICIAL_BUILD=1`, while a PR release build has
`-DDISABLE_ALL_DEBUG_SYMBOLS=1` and no official flag. Both change the
generated code (XRay instruments ~11k more functions when debug info is
present, and the official flag adds code), so the same sources give two
binaries with different code layout. The CI-only #123849 has 0x65500 bytes
less code than the official build of its own base e95690ab45 and was
21.6% slower on `window_functions_distinct #2`. On some master bases a hot
loop lands on a slow placement only in the PR flavour, and every PR on that
base then reds (seen on #100391).
MasterCI's `Build (amd|arm_release_pr_cache_warmup)` jobs already compile
master with exactly the PR cmake flags. They now also link
`clickhouse-self-extracting` and upload it, and take the PGO/BOLT decision
of the release build they mirror, as a PR build does.
For PR runs of the public repository, `find_prev_build` prefers that binary.
When the base has none it walks back only over commits where MasterCI
skipped the warmup job (a skipped job means its inputs did not change, so
the older build has the same sources); anything else falls back to the
official build. Replayed over 14 days of PR references, 332 of 356 get a
build with identical compiled sources and none gets different sources.
Master runs (official vs official) and local runs are unchanged. The report
parsing of `classify_missing_prev_master_run` moves into
`master_job_status`, shared by both walks.
The arm warmup's build-profile upload stays what it was (compiled objects):
the server link trace is deleted before extraction and the symbol table of
the linked binary is not uploaded, so the shared build-profile cluster gets
no new load.
Cost: ~24 min (amd) and ~39 min (arm) of ThinLTO link per warmup run and
~540 MB of S3 per run with default retention. MasterCI wall clock is
unchanged and no PR job is added.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With the report enabled, praktika stores the workflow result and the job
result on S3 under `result_<normalized name>.json`. The workflow and its only
job were both named `CreateRelease`, so the job's gzipped result overwrote the
workflow result, and the HTML hook failed to read it back with
`UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b`. The job uploaded
its log files before that failure.
Dry run: https://github.com/ClickHouse/ClickHouse/actions/runs/37448847921
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`release_job.py` attached `/tmp/reprepro.log`, `/tmp/createrepo_c.log` and
`~/fuse_mount.log` (the geesefs log) to its `Result.files`, but two gaps kept
them off S3:
- `CreateRelease` had no `enable_report`, so praktika never ran the HTML hook
that uploads `Result.files` and links them to the result.
- A run killed by the 7200 s timeout never reached `complete_job`, so its
result carried no files at all.
Enable the report, and checkpoint a `RUNNING` result with the log files
before each step. On a timeout the runner marks that result `ERROR` and
uploads its files. The logs are created up front, so a run killed inside the
export step still names them.
Run 36585485987 timed out with no log uploaded:
https://github.com/ClickHouse/ClickHouse/actions/runs/36585485987
Related: https://github.com/ClickHouse/clickhouse-private/issues/78182
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
