Laurens Kuiper 0b272c23b1 BlockAllocator improvements (#27472) ...

This extends the `BlockAllocator` with time-based decay and bounded
background reclamation, and adds `AllocationCache` for large
power-of-two allocations that favors CPU-local reuse. Both pooled blocks
and cached fallback allocations are included in memory-pressure
handling.

On `c7a.metal-48xl`, total runtime and RSS compared vs. the base branch:

| Workload | Threads | Runtime (s) | Runtime reduction | Peak RSS (GiB)
| RSS reduction |
|---|---:|---:|---:|---:|---:|
| TPC-H SF100 | 32 | 7.44 → 6.48 | 12.9% | 27.8 → 25.6 | 7.8% |
| TPC-H SF100 | 192 | 4.35 → 3.96 | 8.9% | 38.8 → 31.3 | 19.5% |
| TPC-DS SF100 | 32 | 27.46 → 23.49 | 14.5% | 43.1 → 39.0 | 9.6% |
| TPC-DS SF100 | 192 | 20.11 → 17.61 | 12.5% | 52.1 → 45.6 | 12.4% |
| ClickBench native | 32 | 5.48 → 5.01 | 8.6% | 20.9 → 16.8 | 19.5% |
| ClickBench native | 192 | 2.83 → 2.54 | 10.3% | 32.8 → 24.3 | 25.9% |
| ClickBench Parquet | 32 | 8.36 → 7.96 | 4.8% | 27.1 → 10.1 | 62.8% |
| ClickBench Parquet | 192 | 5.63 → 4.81 | 14.4% | 26.5 → 17.2 | 35.0% |

The goal is to enable `BlockAllocator` by default in a future PR, as it
now has improved performance (like before), but now also with reduced
RSS (which was much higher and not free-able before).

2026-10-19 18:46:32 +00:00

..

ci

Treat isolated HASH_ZERO test timeouts as warnings

2026-10-16 07:35:55 +00:00

lldb

Compile the LLDB fixture with DuckDB’s minimum C++ standard

2026-10-13 15:52:38 +00:00

parser

Compact PEG parse result source positions

2026-10-19 13:57:01 +02:00

regression

Allow regression base runner to use its own TPC-DS answers

2026-10-13 06:00:00 +00:00

settings_scripts

Optimize prepare job tooling

2026-10-10 13:41:48 +02:00

__init__.py

Use changed-tests from prepare in smoke tests, and refactor tests

2026-04-02 15:35:18 +02:00

add_git_hooks.py

Fix problems with pre-commit hook

2026-08-05 09:54:43 +02:00

amalgamation.py

Fix mixed path separators for extension/loader in amalgamation

2026-08-25 22:41:14 +02:00

append_metadata.cmake

Fix typos in scripts/

2026-04-22 08:53:31 +02:00

apply_extension_patches.py

Support submodules in extensions patching process

2026-06-10 12:20:40 +01:00

architecture_flags.cmake

Use haswell for windows amd64 (GCC 10 does not support -march=x86-64-v3)

2026-08-26 10:22:40 +02:00

asset-upload-gha.py

…

banned_symbols_check.py

Parallelize banned symbols check

2026-04-02 12:44:42 +02:00

cancel_workflows.py

…

capi_v1_regen.sh

Optimize prepare job tooling

2026-10-10 13:41:48 +02:00

capi_v2_regen.sh

Regen script now formats as well

2026-08-28 15:50:50 +02:00

ccache_workflow_summary.py

Parse CI logs better in ccache summary script

2026-09-08 07:21:53 +02:00

check_coverage.py

Revert "fix: drop useless python import"

2025-03-25 22:41:46 +01:00

check_extension_abi.py

Use new generator

2026-08-03 12:25:10 +02:00

check_extension_patches.cmake

Accept duckdb_extension_statically_link in the extension patch check

2026-09-29 17:27:19 +02:00

check-issue-for-code-formatting.py

…

clang-tidy-diff.py

…

cleanup_config_skip_tests.py

Remove peg parser for now

2026-02-03 16:31:48 +01:00

compute-extension-hash.sh

Allow concurrent extension signing/upload and verify in PR's CI runs

2026-06-06 15:29:08 +02:00

coverage_check.sh

Add build/*/test/run alias for test runner

2026-06-04 13:10:43 +02:00

create_local_extension_repo.py

Fix typos in scripts/

2026-04-22 08:53:31 +02:00

create_patch.py

add script to create patch from changes in a repo

2025-08-15 15:39:18 +02:00

exported_symbols_check.py

Fix symbol checks on main (different with Clang)

2026-08-31 15:13:06 +02:00

extension-upload-repository.sh

Print upload count and fail if none found

2026-03-23 13:58:42 +01:00

extension-upload-single.sh

tighten rclone ext push, avoid pre-PUT HEAD call

2026-08-28 13:00:36 +02:00

fix_loop_iterators.py

bring back fix_loop_iterators.py

2026-04-28 14:41:04 +02:00

format_test_benchmark.py

Use errors='surrogateescape' for format check scripts

2026-04-14 13:15:25 +02:00

format.py

Run the typos check in chunks to bound its memory usage

2026-10-14 11:47:30 +00:00

generate_csv_header.py

…

generate_enum_util.py

BlockAllocator improvements (#27472)

2026-10-19 18:46:32 +00:00

generate_enums.py

Revert "fix: drop useless python import"

2025-03-25 22:41:46 +01:00

generate_extensions_function.py

Add legacy_parser extension

2026-10-08 08:33:27 +02:00

generate_functions.py

Internal #8572: Window Function Metadata

2026-04-03 14:20:29 -07:00

generate_html_template.py

Generate html instead of inlining it

2026-06-28 14:43:15 +02:00

generate_metrics.py

Restructure profiler related files to src/main/profiler/

2026-06-11 12:59:13 +02:00

generate_plan_storage_version.py

Revert "fix: drop useless python import"

2025-03-25 22:41:46 +01:00

generate_presigned_url.sh

Expand HTTPUtil with new client interface

2025-05-12 17:25:57 +02:00

generate_serialization.py

Add support for dropping nested schemas

2026-06-28 13:18:13 +02:00

generate_settings.py

integrate the existing functionality of each setting with the new API

2024-09-13 15:09:12 +02:00

generate_static_extension_loader.py

Add STATICALLY_LINK_EXTENSIONS and STATICALLY_LINK_CAPABILITIES

2026-09-29 17:16:29 +02:00

generate_storage_info.py

make it backwards compatible

2026-05-18 09:41:01 +02:00

generate_tpcds_results.py

Revert "fix: drop useless python import"

2025-03-25 22:41:46 +01:00

generate_tpcds_schema.py

removing the amalgamation build, so long

2026-04-22 13:27:31 +02:00

generate_util.py

Merge origin/main into match-recognize

2026-09-14 14:40:51 +02:00

github_cache_usage.py

Move valgrind CI job with increased timeout to NightlyTests

2026-06-03 14:07:17 +02:00

list_vcpkg_registry_packages.py

add registries, add script to list the packages of the registry

2025-04-15 12:41:49 +02:00

merge_vcpkg_deps.py

Route aws-sdk-cpp to vcpkg-duckdb-ports in merged vcpkg manifest

2026-09-23 16:20:55 +02:00

modify_distribution_matrix.py

…

null.txt

…

osx_import_codesign_certificate.sh

…

package_build.py

Build installing and loading external extensions as its own library

2026-09-29 17:16:29 +02:00

package_release_artifact.sh

Register static extensions through a descriptor filled by their root

2026-09-21 09:02:11 +02:00

parser_test.py

…

plan_cost_runner.py

Fix script import problem

2026-08-26 16:32:54 +02:00

prepare_build_artifact.sh

Upload unittest binary in release artifact

2026-10-01 21:17:03 +02:00

python_helpers.py

improve format scripts for working on windows environment

2026-01-06 01:37:06 +08:00

regression_check.py

Revert "fix: drop useless python import"

2025-03-25 22:41:46 +01:00

regression_test_extension_size.py

Compare CLI binary size also in regression test

2026-09-16 08:41:15 +02:00

regression_test_python.py

Fix monotonic clock usage

2026-08-02 22:06:05 -07:00

regression_test_storage_size.py

Fix script import problem

2026-08-26 16:32:54 +02:00

rerun_workflows.py

Refactor StringEnumCast to use new iterators / writers

2026-04-15 09:52:35 +02:00

run_benchmark.py

Add run_benchmark.py script

2024-12-10 17:03:09 +01:00

run_extension_medata_tests.sh

Remove DONT_LINK: configs ask for linking with duckdb_extension_statically_link

2026-09-29 17:27:19 +02:00

run-clang-tidy.py

Enable bugprone-exception-escape and run clang-tidy in NightlyTests

2026-09-15 07:10:46 +02:00

run-clangd-tidy.py

Add TableFunctionMultiFileWrapper which allows a single-file-read table function to become a multi-file-read function, and migrate read_json to it (#25471)

2026-09-09 18:51:29 +00:00

samply_profile_launcher.py.in

Show HTTP requests and execution markers in samply profiler

2026-10-14 09:53:21 +02:00

samply_profile.py

Fix format of python code

2026-10-14 09:53:22 +02:00

sync_out_of_tree_extensions.py

sync_out_of_tree_extensions: an empty vcpkg.json needs no vcpkg toolchain

2026-10-15 16:20:59 +02:00

test_benchmark_sql_runner.py

Write db files under build/ and remove afterward

2026-05-28 07:41:50 +02:00

test_compile.py

some more 11->17

2026-03-13 09:01:16 +01:00

test_config_compare.py

formatting....

2025-11-27 15:27:32 +01:00

test_package_build_version.py

Simplify release versioning for 2.x releases

2026-09-01 08:07:10 +02:00

test_peg_parser.py

Format fix and add moves

2025-10-21 09:54:29 +02:00

test_samply_profile.py

Fix format of python code

2026-10-14 09:53:22 +02:00

test_serialization_bwc.py

Add build/*/test/run alias for test runner

2026-06-04 13:10:43 +02:00

test_storage_compatibility.py

test harness: remote temp-dir root, LOCAL_* dirs, --env-passthrough

2026-07-31 15:07:39 +02:00

test_zero_initialize.py

test harness: remote temp-dir root, LOCAL_* dirs, --env-passthrough

2026-07-31 15:07:39 +02:00

try_timeout.py

Revert "fix: drop useless python import"

2025-03-25 22:41:46 +01:00

typos.toml

Exclude image version file from typos check

2026-09-04 07:09:07 +02:00

upload-assets-to-staging.sh

Dont deploy on arm64 compat, retry choco if missing

2026-09-24 07:42:31 +02:00

verify_enum_integrity.py

Revert "fix: drop useless python import"

2025-03-25 22:41:46 +01:00

verify-extension-signing.sh

Replace slow 'dd bs=1' calls and verify extensions concurrently

2026-06-22 14:15:44 +02:00

windows_ci.py

…