mirror of
https://github.com/ClickHouse/ClickHouse.git
synced 2026-10-28 16:18:23 +08:00
Groene AI
5499eb4aed
Three review points on the previous commit, plus one defect in the new performance test that made it unable to run at all. The stop-table comment claimed that stopping the scan on 0xE2 keeps the `end - it >= 3` lookahead inside a single run, so it could neither read past `end` nor straddle a run boundary. Both halves were wrong. The lookahead reads `it[1]` and `it[2]`, which are 0x80 and 0xA8 or 0xA9, and the table deliberately does not contain those, so by construction they fall in the following run. And the bounds safety comes from the `end - it >= 3` test itself, which is byte-identical to master, so this change creates no property of it. The comment now states the invariant that does hold: 0xE2 has to be a stop byte because otherwise a bulk copy would swallow the lead byte and emit U+2028 or U+2029 raw. Deleting that one entry makes 00490_special_line_separators_and_characters_ outside_of_bmp fail, which is how the claim is checkable. The scan was written twice, once above the loop and once at its tail. All four bulk-copy writers in this header scan once, as the first statement inside a `while (true)`, and the argument for this change is that the function should get the shape its siblings already have. It now does: `run_end` lives inside the loop, and two copies of the scan condition become one. Pure refactor, so it is proven rather than asserted: the emitted bytes are unchanged across 1579 shapes (all 256 byte values alone and embedded under both stop tables, the multi-byte carrier cases, nine type wrappers, FixedString with and without trailing-NUL trimming, a U+2028 planted at forty offsets across a buffer refill, twelve output formats, JSONExtractRaw), while the build with 0xE2 removed still diverges on 226 of them. The performance test was sized against only one of the three constraints tests/performance/scripts/README.md documents. The query timeout was fine, but a Memory table bigger than 1 GB is a named cause of instability, and the fill query built 1.39 GiB; and a query below about 0.2 s is the most frequent cause of instability, which the TSV scope control was at 149 ms, making the least stable query in the file the one whose flatness carries the scope argument. So the row count drops to 1,700,000, measured at 811.93 MiB, and the control now formats all five columns instead of one, which raises it to 837 ms and makes it exercise exactly the bytes the JSON arms use. All six query medians are now between 333 ms and 837 ms on one thread. The value lengths and the escape-heavy scenario are unchanged: the gain scales with run length, and the regressing query stays in the file deliberately. Finally, the fill query could never have run. perf.py passes every create and fill query through `str.format` to expand substitutions, so the literal braces of the JSON document it builds parsed as a substitution field named `"u"` and raised KeyError before a single query executed. The braces are doubled, which renders back to the same SQL.
37 lines
2.6 KiB
XML
37 lines
2.6 KiB
XML
<test>
|
|
<!-- Cost of emitting String values in JSON row output formats, which is paid once per String
|
|
cell per row. `max_threads = 1` keeps that per-row cost visible instead of hiding it
|
|
behind threading, and `formatRow` keeps the output in memory so the sink is never the
|
|
bottleneck. -->
|
|
|
|
<create_query>
|
|
CREATE TABLE json_string_value_formatting (clean String, escaped String,
|
|
separator String, escape_heavy String, doc String) ENGINE = Memory
|
|
</create_query>
|
|
<fill_query>
|
|
INSERT INTO json_string_value_formatting
|
|
SELECT
|
|
leftPad(toString(number), 70, 'x'),
|
|
concat(leftPad(toString(number), 58, 'x'), '"a"', unhex('5c'), 'c/d', unhex('0a'), 'e'),
|
|
concat(leftPad(toString(number), 64, 'x'), unhex('e280a8'), 'ab'),
|
|
concat(repeat(concat('"', unhex('5c'), '/', unhex('0a')), 17), leftPad(toString(number), 2, 'x')),
|
|
concat('{{"u":"https://host.example/', toString(number), '/a/b/c","s":"plain value"}}')
|
|
FROM numbers(1700000)
|
|
</fill_query>
|
|
|
|
<!-- A long value with nothing to escape: the whole value is one run -->
|
|
<query>SELECT count() FROM json_string_value_formatting WHERE NOT ignore(formatRow('JSONEachRow', clean)) SETTINGS max_threads = 1</query>
|
|
<!-- Same length, a handful of escapes: several short runs -->
|
|
<query>SELECT count() FROM json_string_value_formatting WHERE NOT ignore(formatRow('JSONEachRow', escaped)) SETTINGS max_threads = 1</query>
|
|
<!-- Contains U+2028, whose three-byte sequence needs a lookahead from a run boundary -->
|
|
<query>SELECT count() FROM json_string_value_formatting WHERE NOT ignore(formatRow('JSONEachRow', separator)) SETTINGS max_threads = 1</query>
|
|
<!-- Almost every byte needs escaping, so there are no runs to copy in bulk -->
|
|
<query>SELECT count() FROM json_string_value_formatting WHERE NOT ignore(formatRow('JSONEachRow', escape_heavy)) SETTINGS max_threads = 1</query>
|
|
<!-- JSONExtractRaw writes its result through the same function with forward slashes unescaped -->
|
|
<query>SELECT count() FROM json_string_value_formatting WHERE NOT ignore(JSONExtractRaw(doc)) SETTINGS max_threads = 1</query>
|
|
<!-- Scope control: every one of those values through a format that does not use this function, so it must not move -->
|
|
<query>SELECT count() FROM json_string_value_formatting WHERE NOT ignore(formatRow('TSV', clean, escaped, separator, escape_heavy, doc)) SETTINGS max_threads = 1</query>
|
|
|
|
<drop_query>DROP TABLE IF EXISTS json_string_value_formatting</drop_query>
|
|
</test>
|