mirror of
https://github.com/ClickHouse/ClickHouse.git
synced 2026-10-28 16:18:23 +08:00
Raúl MarínandClaude Opus 5.5
5e0958a869
`WriteBufferValidUTF8::nextImpl` and `writeValidUTF8` call `skipASCIIBlocks` once per sequence they step over. In non-ASCII text the next such byte is usually within 64 bytes, so the 64-byte check failed and the 16-byte check ran after it: two vector checks per step where the old SSE2 and NEON loops did one. Now one 16-byte block is checked first, and the function returns at once if it has a byte of `0x80` or more. The result is the same. Long ASCII runs pay one extra 16-byte check. `clickhouse local` on Neoverse-V2, `max_threads = 1`, best of 9: - `FORMAT XML`, accented Latin, 300000 rows of 720 bytes: 1.14 -> 0.89 s - `FORMAT XML`, Cyrillic, 300000 rows of 1 KB: 0.96 -> 0.79 s - `JSONEachRow` with `output_format_json_validate_utf8 = 1`, Cyrillic: 1.05 -> 0.88 s - `toValidUTF8` of 63 `é` and an `a`, 2M rows: 0.57 -> 0.52 s - ASCII `FORMAT XML` and `toValidUTF8`: unchanged The x86 code of the two callers changes too: CI showed about +6% on `to_valid_utf8` there as well. Add a non-ASCII `XML` case to the `writing_valid_utf8` performance test. Closes: https://github.com/ClickHouse/ClickHouse/issues/123346 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
28 lines
1.2 KiB
XML
28 lines
1.2 KiB
XML
<test>
|
|
<settings>
|
|
<output_format_parallel_formatting>0</output_format_parallel_formatting>
|
|
</settings>
|
|
|
|
<substitutions>
|
|
<substitution>
|
|
<name>format</name>
|
|
<values>
|
|
<value>JSON</value>
|
|
<value>JSONCompact</value>
|
|
<value>XML</value>
|
|
</values>
|
|
</substitution>
|
|
</substitutions>
|
|
|
|
<create_query>CREATE TABLE IF NOT EXISTS table_{format} ENGINE = File({format}, '/dev/null') AS SELECT SearchPhrase, ClientIP6, URL, Referer, URLDomain FROM test.hits limit 0</create_query>
|
|
|
|
<!-- Non-ASCII text, where the next non-ASCII byte is always close. -->
|
|
<create_query>CREATE TABLE IF NOT EXISTS table_non_ascii (s String) ENGINE = File(XML, '/dev/null')</create_query>
|
|
|
|
<query>INSERT INTO table_{format} SELECT SearchPhrase, ClientIP6, URL, Referer, URLDomain FROM test.hits LIMIT 100000</query>
|
|
<query>INSERT INTO table_non_ascii SELECT repeat('Съешь же ещё этих мягких французских булок, да выпей чаю. ', 10) FROM numbers(100000)</query>
|
|
|
|
<drop_query>DROP TABLE IF EXISTS table_{format}</drop_query>
|
|
<drop_query>DROP TABLE IF EXISTS table_non_ascii</drop_query>
|
|
</test>
|