mirror of
https://github.com/ClickHouse/ClickHouse.git
synced 2026-10-28 16:18:23 +08:00
Groene AI
0f6a7d6a64
MergeTreeIndexAggregatorSet::buildFilter builds one hashing state per index
granule while its columns span the whole insert block, so before the range
argument the batched key pack ran once per granule over the whole block. The
argument cannot change any result, because the state is only ever asked about
rows [pos, pos + limit), so no functional test can distinguish the two
behaviours and the only oracle that can is a measurement.
Order-balanced A/B of origin/master dc050cf699 against this branch, both
RelWithDebInfo out of the same build dir, one INSERT of 5M rows with default
block and granule settings, single insert thread, arms interleaved per
repetition: master 1.711 s against 0.302 s here, 5.66x, n=16 each. Two negative
controls hold the mechanism rather than the speedup: a single index column
hashes through HashMethodOneNumber, which never batched, and measures 1.05x
over 14 interleaved repetitions, which is the test's control query; an index
granule as large as the block makes the range cover the whole block, so both
binaries batch, 0.93x. Master in both arms measures 0.99x, so the harness has
no position bias.
The effect is the number of index granules per block, one run each at 5M rows:
6.0x at the default 8192, 15.8x at 2048, 18.9x at 1024, 25.6x at 512. The test
keeps the default, because tests/performance/scripts/perf.py applies
max_execution_time = 15 to every measured query on the reference server as well
as on the pull request's, so a benchmark whose old side exceeds that limit
fails the job on a correct pull request. Replaying the committed statements
with that setting, the reference side passes at 8.1x under the limit and this
branch at 48x. That also means the test reports rather than gates: one slower
query stays under the job's own threshold of ten.
The mutant is master rather than this branch with the one line reverted,
because an INSERT with a set index reaches no other code this branch changes.
Both controls measuring at parity is what shows the two binaries do not
otherwise differ on that path.
The test is a separate file because it needs SYSTEM STOP MERGES, which would
neutralise the OPTIMIZE TABLE FINAL fill query of the aggregation test.
25 lines
1.5 KiB
XML
25 lines
1.5 KiB
XML
<test>
|
|
<!-- A `set` index over more than one fixed-width column packs its grouping key through the batched
|
|
path, which covers the whole insert block. The index aggregator is called once per index
|
|
granule, so an insert used to pay that pack `block_rows / index_granularity` times over. Both
|
|
sides stay well inside the harness's 15-second per-query limit, which applies to the reference
|
|
server too: measured 1.71 s against 0.30 s. -->
|
|
|
|
<settings>
|
|
<max_insert_threads>1</max_insert_threads>
|
|
</settings>
|
|
|
|
<create_query>CREATE TABLE set_index_insert_two_keys (a UInt64, b UInt64, ord UInt64, INDEX idx (a, b) TYPE set(1000) GRANULARITY 1) ENGINE = MergeTree ORDER BY ord</create_query>
|
|
<create_query>CREATE TABLE set_index_insert_one_key (a UInt64, b UInt64, ord UInt64, INDEX idx a TYPE set(1000) GRANULARITY 1) ENGINE = MergeTree ORDER BY ord</create_query>
|
|
<create_query>SYSTEM STOP MERGES</create_query>
|
|
|
|
<query>INSERT INTO set_index_insert_two_keys SELECT number % 20, number % 23, number FROM numbers(5000000)</query>
|
|
|
|
<!-- Control: a single index column hashes through a method that never batched, so this must not move. -->
|
|
<query>INSERT INTO set_index_insert_one_key SELECT number % 20, number % 23, number FROM numbers(5000000)</query>
|
|
|
|
<drop_query>SYSTEM START MERGES</drop_query>
|
|
<drop_query>DROP TABLE IF EXISTS set_index_insert_two_keys</drop_query>
|
|
<drop_query>DROP TABLE IF EXISTS set_index_insert_one_key</drop_query>
|
|
</test>
|