mirror of
https://github.com/ClickHouse/ClickHouse.git
synced 2026-10-28 16:18:23 +08:00
Groene AIandClaude Opus 5.5
2728ff6ecb
Every timed `INSERT INTO FUNCTION file('test_file', ...)` appended its
whole output (29-318 MB) to the same file and nothing reset it between
runs, so each server appended about 6 GB per CI run into the page cache
the two servers share. On master (2026-10-22..24, query_metric_runs_v1),
slow runs (5-7x the query's minimum, 4-36% of runs per query) wrote the
same bytes with the same CPU as fast runs; all the extra time was inside
write() waiting for I/O (DiskWriteElapsedMicroseconds, OSIOWaitMicroseconds),
and the slow state persisted across runs (P(slow | previous slow)
0.68-0.81 vs 0.05-0.07) and across the two servers (0.63-0.78 vs
0.04-0.07). 267 of 2238 query runs were over the report floors and the
learned thresholds grew to 3.9-10.8.
Truncating on every insert (972ffff551) was reverted as too slow
(f3651363d5). `/dev/null` is allowed by StorageFile for performance
tests and already used by select_format.xml and formats_hits_write.xml:
every format's writer still produces the same bytes, but no run leaves
state for the next one.
Local check, CI's perf.py on two servers of the official master build
pinned to the same 8 cores: old text, the file ends at 24-26 GB per
server and write() takes 38-74% of a run; new text, OSWriteBytes is 0
for every execution and write() at most 2% (per-query median); queries
with stat_threshold above 0.25 go from 2-4 of 24 to 0.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
44 lines
1.5 KiB
XML
44 lines
1.5 KiB
XML
<test>
|
|
|
|
<substitutions>
|
|
<substitution>
|
|
<name>format</name>
|
|
<values>
|
|
<value>TabSeparated</value>
|
|
<value>TabSeparatedWithNames</value>
|
|
<value>TabSeparatedWithNamesAndTypes</value>
|
|
<value>CSV</value>
|
|
<value>CSVWithNames</value>
|
|
<value>Values</value>
|
|
<value>JSONEachRow</value>
|
|
<value>JSONCompactEachRow</value>
|
|
<value>JSONCompactEachRowWithNamesAndTypes</value>
|
|
<value>TSKV</value>
|
|
<value>RowBinary</value>
|
|
<value>Native</value>
|
|
<value>MsgPack</value>
|
|
</values>
|
|
</substitution>
|
|
<substitution>
|
|
<name>partitions_count</name>
|
|
<values>
|
|
<value>5</value>
|
|
<value>50</value>
|
|
<value>500</value>
|
|
</values>
|
|
</substitution>
|
|
</substitutions>
|
|
|
|
<!-- Writes go to /dev/null: appending to a file would carry the earlier runs' dirty page cache into every run. -->
|
|
<query>
|
|
INSERT INTO FUNCTION file('/dev/null', '{format}', 'key UInt64, value UInt64')
|
|
SELECT number, number FROM numbers(10000000)
|
|
</query>
|
|
|
|
<query>
|
|
INSERT INTO FUNCTION file('/dev/null', '{format}', 'key UInt64, value1 UInt64, value2 UInt64, value3 UInt64, value4 UInt64, value5 UInt64')
|
|
SELECT number, number, number, number, number, number FROM numbers(1000000)
|
|
</query>
|
|
|
|
</test>
|