`CODEC(GDeflate[(level)])`, behind `enable_gdeflate_codec`, levels 1..12,
default 6, method byte 0xa1. GDeflate is a variant of Deflate whose bit stream
is split into 32 interleaved sub-streams, the format that nvcomp and
DirectStorage expand on a GPU, many chunks in parallel.
A block is compressed in independent chunks of 64 KiB, each a GDeflate page,
from a multiple of 4 bytes, after the number of chunks and a descriptor of
each. Of a column of integers, `Int*`, `UInt*`, `Decimal32`, `Decimal64`,
`Date*` or `DateTime*`, a chunk is also bit-packed before it is compressed:
each integer as its difference from the least of the chunk, in as many bits as
the largest difference takes, up to 56. The smaller of the two forms is kept,
and the block says which. The format is that of the GPU codec of
https://github.com/ClickHouse/ClickHouse/pull/119502, so that a GPU can expand
the chunks of a column written by this codec.
The codec uses the GDeflate pages of NVIDIA's fork of libdeflate
(`contrib/gdeflate`), whose compressed pages nvcomp expands on the GPU, and
whose expansion reads what nvcomp compresses, as checked on the columns of
TPC-H `lineitem` at every level. The fork exports the same symbols as
libdeflate, so `contrib/gdeflate-cmake` builds only its GDeflate parts and
renames them.
`contrib/gdeflate` points at the upstream repository; it must be replaced with
a fork under github.com/ClickHouse before merging.
The `contrib/nlp-data` submodule has no license and no record of where its
data came from. Its only file still in use, `charset.zst`, held per-encoding
byte-bigram frequencies embedded into `FrequencyHolder.cpp`, used only by the
experimental `detectCharset` and `detectLanguageUnknown` functions.
Language detection remains available via `detectLanguage` and
`detectLanguageMixed` (cld2).
Removed:
- the `contrib/nlp-data` submodule;
- `FrequencyHolder` and `FunctionsCharsetClassification.cpp`, plus the
`OBJECT_DEPENDS` entry in `src/CMakeLists.txt`;
- the two names from the query fuzzer's function-substitution group and the
vale token list;
- the related queries of `02133_classification` and the
`04401_detect_charset_language_large_input_ubsan` test.
The sections in `nlp-functions.mdx` are inside the generated region and are
removed by the docs regeneration from `system.functions`.
Closes: https://github.com/ClickHouse/ClickHouse/issues/89035
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Upstream builds the float string inside a SIMD register and trims the
double path: `Float32` formatting is about 1.25x faster and `Float64`
1.02x to 1.14x on x86-64-v3 (more on x86-64-v2). The output is
byte-identical to the previous version over all `Float32` values and
320M sampled `Float64` values.
The submodule now points at the ClickHouse fork, which carries the patch
(`ClickHouse/main-20261015`). Upstream swapped the arguments of
`zmij::detail::write`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Retarget the submodule from the ClickHouse fork to upstream
VectorCamp/vectorscan and pin tag vectorscan/5.4.13 (acd7363aadea).
The fork carried a single patch (moving `#include <vector>` out of the
`HAVE_BOOST_CONTAINER_SMALL_VECTOR` fallback in `src/util/small_vector.h`);
it is obsolete because upstream merged it (VectorCamp/vectorscan#359), so
no fork branch is needed.
Adapt the explicit source list in `contrib/vectorscan-cmake/CMakeLists.txt`
to the upstream file changes: `src/fdr/teddy.c` -> `src/fdr/teddy.cpp`,
drop `src/fdr/teddy_avx2.c`, add `src/fdr/teddy_fat.cpp`. `teddy_fat.cpp`
is guarded by `HAVE_AVX2`, which is not defined in ClickHouse's
non-fat build, so it compiles to nothing here.
Update `contrib/vectorscan-cmake/common/hs_version.h` to 5.4.13 /
2026-08-23 (`HS_VERSION_STRING`, `HS_PATCH`); `HS_VERSION_32BIT` is left
unchanged, as in the previous bump.
The parser is not regenerated with ragel: upstream changed `Parser.rl`
(new nested-class / `&&` rejection rules) but regenerating is a
user-visible behaviour change deferred to a separate PR.
Includes the upstream stack buffer overflow fix in
`rvermicelliDoubleExecReal` (`src/nfa/vermicelli_simd.cpp`, fe94004d).
Co-Authored-By: Claude Code <noreply@anthropic.com>
`getNumaNodesTotalMemory` was the only user of libnuma. Replace it with the
`get_mempolicy` syscall plus `/sys/devices/system/node/node<N>/meminfo`, which
also drops libnuma's `numa_init` constructor from every process startup.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add https://github.com/apache/parquet-testing as `contrib/parquet-testing`
and a stateless test that reads every `.parquet` and `.parquet.encrypted`
file in it. For each file the test prints the inferred schema, the row count
and an order-independent hash of all values, or the error name if the file
cannot be read. Files that ship with an `*_expect.csv` are compared with
the expected contents.
The stateless test job does not check out submodules, so
`ci/jobs/functional_tests.py` now checks out `contrib/parquet-testing`
before the tests run.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The DataStax `cpp-driver` (with `libuv`) was used only to connect, authenticate
with a password, run a paged `SELECT` query, and decode a few scalar types.
Implement exactly that over the native protocol v4 in `src/Dictionaries/CassandraClient`,
with bounds checks on every read from the server.
Bring back the integration test of Cassandra dictionaries, which was disabled
because of instability, and limit the JVM heap of the Cassandra container.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This reverts #125030 (merge 18b5a991f0),
re-applying #124678 unchanged. The `ThreadSanitizer` race it exposed is
fixed by the next commit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Master moved secret-argument masking out of `src/Parsers/FunctionSecretArgumentsFinder.cpp`
into `SecretArgumentsSpec` registrations. The `ipcrypt*` functions now register
`findIPCryptSecretArguments` as a `custom` spec in `src/Functions/ipcrypt.cpp`, like `HMAC`,
and `InterpreterExplainQuery.cpp` takes master's `isFunctionWithSecretArguments`, which
picks up the spec from `FunctionFactory`. In `Settings.cpp` both `enable_ipcrypt` and
master's `enable_xgboost` are kept.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conflict resolution: master made codec family names case-insensitive and stopped
upper-casing codec strings, so the PR's `upperCaseCodecFamilyNames` is dropped in
favor of master's lookup and `checkCodecChainIsNotColumnLevelOnly`; the PR's
`Default` comparisons now use `equalsCaseInsensitive`.
aws-sdk-cpp 1.11.907 requires the newer AWS CRT APIs, so the whole
chain is co-bumped in one commit:
- contrib/aws -> ClickHouse/1.11.907 @ e235f91d162c (fork, ClickHouse patches rebased onto 1.11.907)
- contrib/aws-crt-cpp -> ClickHouse/0.43.8 @ fb9638dc096a (fork, 2 patches)
- contrib/aws-c-io -> ClickHouse/v0.27.6 @ ad2b310237ca (fork, 1 MSan patch)
- contrib/aws-c-cal -> upstream v0.9.15 @ 8aa2a48a09f9 (fork carries no patches; .gitmodules URL points back at awslabs)
- aws-c-auth, aws-c-common, aws-c-event-stream, aws-c-mqtt, aws-c-s3, aws-c-sdkutils -> the pins required by aws-crt-cpp 0.43.8
contrib/aws-cmake/CMakeLists.txt: add the new aws-cpp-sdk-core source
directories (smithy/client, smithy/client/schema, smithy/identity,
utils/local, utils/local/stream) and drop the removed utils/crypto/openssl
glob. The smithy/tracing/impl/opentelemetry sources are intentionally not
built: they need an external OpenTelemetry C++ SDK that ClickHouse does
not vendor, and upstream does not compile them either.
Co-Authored-By: Claude Code <noreply@anthropic.com>
Updates the ClickHouse fork pin to ClickHouse/2.5.5 (70de9e880072),
which is upstream 2.5.5 plus the ClickHouse patch series: `strerror_l`
for musl/FreeBSD, `mlib_have_posix_clocks` detection on Darwin/FreeBSD/musl,
the reply memory leak on retryable cursor server-selection failure, the
redundant store in `_mongoc_handshake_freeze`, and a new patch that stops
printing option values in the deprecated-option conflict error.
Upstream 2.5.5 fixes an out-of-bounds write in the poll-set handling of
`mongoc_async_run` (CDRIVER-6404), tracked as CVE-2026-96746.
Declares the fork URL without the `.git` suffix, matching the other
ClickHouse forks in .gitmodules.
Co-Authored-By: Claude Code <noreply@anthropic.com>
Resolve the conflict in `src/Core/Settings.cpp` with the rename of `allow_experimental_time_series_table`.
Move the history record of `allow_experimental_ipcrypt_functions` into its `DECLARE` (current version 26.10),
as `master` now keeps settings history inline.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Konstantin Vedernikov