mirror of
https://github.com/ClickHouse/ClickHouse.git
synced 2026-10-28 16:18:23 +08:00
Groene AIandClaude Opus 5
a768cc1310
`JSONUtils::writeFieldFromColumn` took the field title as `const std::optional<String> &`. No caller owns an `optional`: of the six call sites, the four that pass a title pass a `String` lvalue owned by a format member, so each of those calls materialised a temporary `std::optional<String>`, copy-constructing and destroying a `std::string` once per field per row. The other two pass no title, so the compact formats never paid this cost. Above libc++'s small-string buffer that is an `operator new` / `operator delete` pair, and ClickHouse replaces global `new`/`delete` with memory-tracking versions, so it costs more than a bare `malloc`; the allocation is observable, so it cannot be elided. The body then passed `name->data()`, discarding the length it already had, which `writeCString` recovered with `strlen`. Both costs are functions of the column header alone, so they were per-row repetitions of work whose input never changes. Carry the title as `std::optional<std::string_view>` and `std::string_view`, and write it by size. The parameter stays an `optional` on purpose: an `empty` test would drop the `"":` form an empty column name would produce. Such a name is unconstructible today, so this preserves shape, not a reachable case. The four `write*Start(const char * title)` signatures are deliberately left alone. `JSONObjectEachRow` passes a raw column value through one of them, where `strlen` truncation at an embedded NUL is the documented fail-closed behaviour behind that format's `MayProduceRawBytesChecker`; their `writeTitle` call now converts implicitly, one `strlen`, exactly as before. The direct-buffer fast path proposed in the issue is not included. It only pays off where `WriteBuffer::write` is out of line, and the shipped release builds set `ENABLE_THINLTO=1` (`ci/jobs/build_clickhouse.py`), so the evidence that would justify the extra code cannot be produced in the configuration users get; it would also bypass the `finalized`/`canceled` checks in `WriteBuffer::write`. The allocation and the `strlen` removed here are immune to link-time optimisation, so the shipped win does not depend on the inliner. Output is byte-identical, verified across 67 shapes covering every JSON output format that reaches `writeTitle`, keys containing NUL, non-UTF-8 and 4-byte UTF-8 bytes, quotes, tabs, newlines and forward slashes, eight `output_format_json_*` settings at 0 and 1, empty and non-empty after-colon delimiters, 200k-300k-row outputs, and the HTTP exception-framing path. Each shape was shown to be live by a third build emitting a semicolon instead of the colon after the key, which changed 62 of the 67 and left exactly the five keyless controls untouched. A 492-test `json` sweep gives identical failing-name sets before and after. Measured with `-DCMAKE_BUILD_TYPE=None -DENABLE_THINLTO=0`, `clang-22`, x86_64, 15 interleaved rounds, medians with bootstrap confidence intervals: one short key 8.9% faster, one key longer than the small-string buffer 37.3% faster, three keys per row 27.6% faster. A `TSV` query, which emits no keys, moves 1.2% against a 1.1% concurrent noise floor, so it sits at the measurement boundary; no `TSV` translation unit is recompiled, leaving binary layout as the only route. Release builds enable ThinLTO, so `tests/performance/json_key_formatting.xml` is added to get the release figure on x86 and ARM from CI; this path had no perf coverage before. Reported by snelheidai in https://github.com/ClickHouse/ClickHouse/issues/117043 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
25 lines
1.6 KiB
XML
25 lines
1.6 KiB
XML
<test>
|
|
<!-- Cost of emitting the object keys in JSON row output formats, which is paid once per field
|
|
per row. `max_threads = 1` keeps that per-row cost visible instead of hiding it behind
|
|
threading, and `formatRow` keeps the output in memory so the sink is never the bottleneck. -->
|
|
|
|
<create_query>
|
|
CREATE TABLE json_key_formatting (id UInt64, event_count UInt64,
|
|
a_column_name_longer_than_the_small_string_limit UInt64) ENGINE = Memory
|
|
</create_query>
|
|
<fill_query>
|
|
INSERT INTO json_key_formatting SELECT number, number + 1, number + 2 FROM numbers(10000000)
|
|
</fill_query>
|
|
|
|
<!-- One narrow field, key short enough to fit a small-string buffer -->
|
|
<query>SELECT count() FROM json_key_formatting WHERE NOT ignore(formatRow('JSONEachRow', id)) SETTINGS max_threads = 1</query>
|
|
<!-- Same, with a key too long for a small-string buffer -->
|
|
<query>SELECT count() FROM json_key_formatting WHERE NOT ignore(formatRow('JSONEachRow', a_column_name_longer_than_the_small_string_limit)) SETTINGS max_threads = 1</query>
|
|
<!-- Three keys per row -->
|
|
<query>SELECT count() FROM json_key_formatting WHERE NOT ignore(formatRow('JSONEachRow', id, event_count, a_column_name_longer_than_the_small_string_limit)) SETTINGS max_threads = 1</query>
|
|
<!-- Scope control: a format that emits no keys, so it must not move -->
|
|
<query>SELECT count() FROM json_key_formatting WHERE NOT ignore(formatRow('TSV', id)) SETTINGS max_threads = 1</query>
|
|
|
|
<drop_query>DROP TABLE IF EXISTS json_key_formatting</drop_query>
|
|
</test>
|