### Rationale for this change
For older versions of CMake the variable `arrow_SOURCE_DIR` is overriden by FetchContent's one because `project(arrow)` sets it as a cache variable while FetchContent sets a normal variable with precedence. Arrow can't find `thirdparty/versions.txt` because it is pointing to the tarball root instead of the cpp subfolder. See policy https://cmake.org/cmake/help/latest/policy/CMP0180.html
This has been tested and found at Iceberg-cpp here:
- https://github.com/apache/iceberg-cpp/pull/1029
### What changes are included in this PR?
Use `ARROW_SOURCE_DIR` instead of `arrow_SOURCE_DIR` so FetchContent's `arrow_SOURCE_DIR` does not have precedence.
### Are these changes tested?
Yes locally on the iceberg PR on Ubuntu 24.04.
### Are there any user-facing changes?
No
### Was AI used for this PR?
In accordance to the [AI generation guidelines](https://arrow.apache.org/docs/dev/developers/overview.html#ai-generated-code), please disclose below whether and how AI was used in this PR.
**PR code and description written by:**
- [x] Human
- [ ] AI
I did use AI to initially find the root cause.
**Reviewed before submission by:**
- [x] Human
- [ ] AI
- [ ] Not reviewed
* GitHub Issue: #53995
Authored-by: Raúl Cumplido <raulcd@apache.org>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
### Rationale for this change
There is a security problem in bundled Brotli (1.0.9), pinned vcpkg Brotli (1.2.0) and rtools' Brotli (1.0.9).
We should update them.
### What changes are included in this PR?
* Bump the bundled Brotli to 1.2.1 from 1.0.9.
* Bump vcpkg to 2026.10.24 (`61d7eaeb09`) from 2026.05.27 (`9b965a1168`).
* Use the bundled Brotli instead of the rtools one for R Windows binaries.
### Are these changes tested?
Yes.
### Are there any user-facing changes?
Yes.
### Was AI used for this PR?
**PR code and description written by:**
- [x] Human
- [x] AI
**Reviewed before submission by:**
- [x] Human
- [ ] AI
- [ ] Not reviewed
* GitHub Issue: #55165
Lead-authored-by: Raúl Cumplido <raulcd@apache.org>
Co-authored-by: Curt Hagenlocher <curt@hagenlocher.org>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
### Rationale for this change
CRAN failures on 26.0.0 release due to regex not catching incompatible MacOSX11.sdk version
### What changes are included in this PR?
Make regex more permissive
### Are these changes tested?
No
### Are there any user-facing changes?
No
**PR code and description written by:**
- [x] Human
- [x] AI
**Reviewed before submission by:**
- [x] Human
- [x] AI
- [ ] Not reviewed
* GitHub Issue: #55267
Authored-by: Nic Crane <thisisnic@gmail.com>
Signed-off-by: Nic Crane <thisisnic@gmail.com>
### Rationale for this change
Trying to reduce the size of libarrow.so and remove google-cloud-cpp SDK on some builds. Allow for users to plug and play based on requirements and divide our functionality into cleaner modules.
### What changes are included in this PR?
Unconditionally build GCS and the google-cloud-cpp SDK into a different module libarrow_gcs.so outside of libarrow.so.
Update bindings to link against the new libarrow_gcs.so library.
Update the Linux Package jobs to have the new module into a different package.
### Are these changes tested?
Yes via CI
### Are there any user-facing changes?
Yes, users will need to either link against libarrow_s3.so or register using LoadFileSystemFactories
### Was AI used for this PR?
In accordance to the [AI generation guidelines](https://arrow.apache.org/docs/dev/developers/overview.html#ai-generated-code), please disclose below whether and how AI was used in this PR.
**PR code and description written by:**
- [x] Human
- [x] AI
**Reviewed before submission by:**
- [x] Human
- [ ] AI
- [ ] Not reviewed
* GitHub Issue: #54902
Authored-by: Raúl Cumplido <raulcd@apache.org>
Signed-off-by: Raúl Cumplido <raulcd@apache.org>
### Rationale for this change
gRPC 1.76.0 doesn't compile with OpenSSL 4. gRPC 1.85.0 includes OpenSSL 4 support: https://github.com/grpc/grpc/pull/41932
### What changes are included in this PR?
gRPC 1.85.0 requires a newer Abseil. gRPC 1.76.0 doesn't compile with the newer Abseil. So we need to upgrade both of them together.
We use Abseil 20260817.0 not 20260526.0 that is used by gRPC 1.85.0 because Abseil 20260526.0 can't be built with MSVC with `/arch:AVX2`: https://github.com/abseil/abseil-cpp/issues/2116
gRPC 1.85.0 moved its .proto files to the third_party/grpc-proto submodule. Source archives don't include submodules and we disable gRPC_DOWNLOAD_ARCHIVES. So we download grpc-proto by ourselves.
Abseil 20260817.0 removed some libraries and added some new libraries. So we update the list of bundled Abseil libraries.
Abseil 20260817.0 uses `O_NONBLOCK` that isn't available on MinGW. So we define `O_NONBLOCK` as 0 for MinGW like the upstream fix: https://github.com/abseil/abseil-cpp/commit/ddbdebc6c8
Abseil 20260817.0 includes `<cstdint>` by itself. So we remove the `-include stdint.h` workaround for GCC 13 or later.
gRPC headers don't use deprecated `std::iterator`. So we remove the `-Wno-error=deprecated-declarations` workaround.
We didn't bundle upb static libraries used by gRPC. So static linking with `libarrow_bundled_dependencies.a` failed with undefined upb symbols. gRPC 1.85.0 also requires the new `upb_descriptor_lib`. So we bundle all upb static libraries.
Other bundled dependencies also need to be upgraded for the new Abseil:
Protobuf 31.1 can't be built with Abseil 20260817.0 with MSVC. So we upgrade bundled Protobuf to 36.2. Protobuf 36.2 headers include `utf8_validity.h` in utf8_range. So we add the utf8_range include directory for ORC built by ExternalProject.
google-cloud-cpp headers use deprecated Abseil APIs such as `absl::visit()`. They cause compilation errors with `-Werror` with Abseil 20260817.0. They are fixed in google-cloud-cpp 3.9.0 or later but google-cloud-cpp 3.9.0 or later requires libcurl 8.7.1 or later. We can't require it because some supported platforms such as Ubuntu 24.04 ship older libcurl. So we use `-Wno-error=deprecated-declarations` for bundled google-cloud-cpp instead of upgrading it.
### Are these changes tested?
Yes.
### Are there any user-facing changes?
Yes.
### Was AI used for this PR?
In accordance to the [AI generation guidelines](https://arrow.apache.org/docs/dev/developers/overview.html#ai-generated-code), please disclose below whether and how AI was used in this PR.
**PR code and description written by:**
- [x] Human
- [x] AI
**Reviewed before submission by:**
- [x] Human
- [x] AI
- [ ] Not reviewed
* GitHub Issue: #54638
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Raúl Cumplido <raulcd@apache.org>
### Rationale for this change
`simdjson >= 5` uses GCC builtins such as `__builtin_ctz` when `__SSE2__` is defined. So we can't build Apache Arrow C++ with `simdjson >= 5` and MSVC because we define `__SSE2__` globally.
### What changes are included in this PR?
We don't need to define `__SSE2__` because xsimd detects SSE2 by `_M_X64` with MSVC.
We still need to define `__SSE4_1__`/`__SSE4_2__` for xsimd because MSVC doesn't define them even with `/arch:SSE4.2`. But we define them only where SSE4.2 is enabled. So they are no longer defined with `ARROW_SIMD_LEVEL=NONE`.
### Are these changes tested?
Yes.
### Are there any user-facing changes?
Yes.
### Was AI used for this PR?
In accordance to the [AI generation guidelines](https://arrow.apache.org/docs/dev/developers/overview.html#ai-generated-code), please disclose below whether and how AI was used in this PR.
**PR code and description written by:**
- [x] Human
- [x] AI
**Reviewed before submission by:**
- [x] Human
- [ ] AI
- [ ] Not reviewed
* GitHub Issue: #51690
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
### Rationale for this change
The bundled Apache Thrift dependency (0.22.0) is affected by
CVE-2026-55969, an integer overflow vulnerability in
`TTransport::checkReadBytesAvailable()` that could bypass message
size checks when reading maliciously crafted Thrift-encoded data.
Arrow's Parquet module relies on Thrift's compact protocol to
deserialize Parquet file metadata, so this is relevant to Arrow.
### What changes are included in this PR?
- Bump the bundled Apache Thrift version from 0.22.0 to 0.24.0 in
`cpp/thirdparty/versions.txt` (including the SHA256 checksum).
- Remove the Clang-only `thrift-3187.patch` and its application logic
in `ThirdpartyToolchain.cmake`. This patch pre-applied Thrift's
upstream fix for THRIFT-3268 (a compiler warning), which has since
been merged into Thrift itself and is already included in 0.24.0.
Keeping the patch would make `git apply`/`patch` fail during the
bundled build with Clang.
### Are these changes tested?
- Verified that `thrift-0.24.0.tar.gz` downloads correctly and its
SHA256 checksum matches the value published at
downloads.apache.org.
- Confirmed the CVE fix is present in Thrift 0.24.0 by diffing
`TProtocol.h`, `TCompactProtocol.h`, `TBinaryProtocol.h`, and
`TTransport.h` against 0.22.0.
- Confirmed `thrift-3187.patch` no longer applies cleanly against
0.24.0 sources (already fixed upstream), which is why it was
removed rather than kept as a no-op.
- Built Arrow C++ locally on macOS (arm64, AppleClang) with
`-DARROW_PARQUET=ON -DARROW_BUILD_TESTS=ON -DThrift_SOURCE=BUNDLED`,
on top of the latest `main` (after rebasing), and ran the following
Parquet tests. All of them passed with no failures (some tests were
skipped, e.g. those requiring Snappy, which was not enabled in my
build):
`parquet-internals-test`, `parquet-file-deserialize-test`,
`parquet-schema-test`, `parquet-reader-test`,
`parquet-writer-test`, `parquet-arrow-reader-writer-test`,
`parquet-arrow-metadata-test`, `parquet-arrow-index-test`, and
`parquet-arrow-internals-test`.
- Not tested locally: the removed patch was only applied for
`CMAKE_CXX_COMPILER_ID STREQUAL "Clang"`, which my AppleClang build
does not hit, so building the bundled Thrift with Clang on Linux
relies on CI. I also did not test encryption, Windows/MSVC, or the
Python/R packaging builds locally.
### Are there any user-facing changes?
No.
* GitHub Issue: #51354
---
Disclosure: this change was prepared with the assistance of an AI
coding tool (Claude Code). I reviewed the diff, verified the CVE fix
and checksum myself, and ran the test suite locally before opening
this PR.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Authored-by: Hanayoshi-8744 <201203709+Hanayoshi-8744@users.noreply.github.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
### Rationale for this change
This PR continues the simdjson migration by replacing the RapidJSON-based parsing implementation used by the JSON parser.
The existing parser uses RapidJSON's SAX/handler interface to parse JSON values and populate Arrow builders. This change replaces that implementation with simdjson's ondemand API while retaining the existing builder and type-inference logic.
### Changes
* Replace the RapidJSON parser and handler interface with simdjson's ondemand API.
* Parse JSON documents using `simdjson::ondemand::parser::iterate_many`.
* Use `ResolveSimdjsonResult()` consistently when handling simdjson results.
* Preserve support for nested objects and arrays.
* Preserve explicit-schema and inferred-field behavior.
* Preserve unexpected-field handling for `Error`, `Ignore`, and `InferType`.
* Continue storing numeric values as raw JSON tokens.
* Trim trailing whitespace from numeric raw tokens to preserve existing behavior.
* Preserve JSON parse error propagation through `Status::Invalid`.
* Remove the parser's RapidJSON-specific dependencies.
### Are there any user-facing changes?
No
### Was AI used for this PR?
In accordance to the [AI generation guidelines](https://arrow.apache.org/docs/dev/developers/overview.html#ai-generated-code), please disclose below whether and how AI was used in this PR.
**PR code and description written by:**
- [X] Human
- [X] AI
**Reviewed before submission by:**
- [X] Human
- [X] AI
- [ ] Not reviewed
Fixes: #51037
* GitHub Issue: #51037
Authored-by: Aaditya Srinivasan <aadityasri03@gmail.com>
Signed-off-by: Antoine Pitrou <antoine@python.org>
### Rationale for this change
Static Arrow linking can fail for bundled GCS and system Abseil.
### What changes are included in this PR?
Export Abseil dependencies via CMake target and pkg-config metadata.
### Are these changes tested?
To be additionally verified by CI.
### Are there any user-facing changes?
This improves downstream static linking.
### Was AI used for this PR?
**PR code and description written by:**
- [X] Human
- [X] AI
**Reviewed before submission by:**
- [X] Human
- [X] AI
- [ ] Not reviewed
* GitHub Issue: #51651
Authored-by: Tadeja Kadunc <tadeja.kadunc@gmail.com>
Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com>
### Rationale for this change
The vendored uriparser in `cpp/src/arrow/vendored/uriparser/` is 0.9.3, which is old and misses upstream security fixes. Instead of refreshing the vendored copy, this makes uriparser an external dependency like our other third-party dependencies.
### What changes are included in this PR?
- Add `uriparser` to `ARROW_THIRDPARTY_DEPENDENCIES`
- Add `build_uriparser()` that builds uriparser 1.0.2 with `FetchContent` for `uriparser_SOURCE=BUNDLED` (static; docs, tests, tools and `wchar_t` support are disabled). The built library is bundled into `arrow_bundled_dependencies`.
- Require uriparser 0.9.6 or later for `uriparser_SOURCE=SYSTEM`. All uriparser APIs used by Arrow are available in 0.9.6, so supported platforms such as Ubuntu 22.04 can use their distribution package.
- Add `cmake_modules/FinduriparserAlt.cmake`. It tries uriparser's CMake package configuration first and falls back to pkg-config (`liburiparser.pc`), because Debian and Ubuntu's `liburiparser-dev` provides only the pkg-config file.
- Add `uriparser::uriparser` to `ARROW_STATIC_INSTALL_INTERFACE_LIBS` for `uriparser_SOURCE=SYSTEM`, so that static library users link uriparser.
- meson: Use the system uriparser 0.9.6 or later if available. Otherwise, build uriparser 1.0.2 via `subprojects/uriparser.wrap` with the CMake subproject support.
- Remove `cpp/src/arrow/vendored/uriparser/` and its section in `LICENSE.txt`.
- Packaging: Install uriparser in our Docker images, conda environment, `Brewfile`, vcpkg manifests, MSYS2 setup and Linux packages build environments.
- Packaging: Pass `-Duriparser_SOURCE` in `ci/scripts/cpp_build.sh`.
- Packaging: Add uriparser to the dependency list in `docs/source/developers/cpp/building.rst`.
***NOTICE*** : The floor is 0.9.6 rather than 1.0.2, so supported platforms such as Ubuntu 22.04 can use their distro package and CI does not have to build uriparser from source.
### Are these changes tested?
All existing tests pass.
### Are there any user-facing changes?
Now Arrow requires uriparser or builds it . No user facing changes as such because no API changes .
**This PR contains a "Critical Fix".** It has security fixes that has gone in uriparser.
**AI Usage**:
- Used for testing <-> building loop.
* GitHub Issue: #51007
Authored-by: Narendra <naskastark11@gmail.com>
Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com>
### Rationale for this change
Fix#51329
Additional downstream static-linking test, after #50195 moved S3 into the separate ArrowS3 library.
### What changes are included in this PR?
Test downstream static linking of `ArrowS3::arrow_s3_static` via minimal_build example and `cpp_test.sh`,
Fix dependencies for static Azure and GCS consumers: append `libcurl` for bundled GCS, append `libcurl` and `LibXml2` for bundled Azure(NOT WIN32),
append links for system Azure SDK
and pkg-config --static arrow to include `libcurl` and `libxml-2.0` for bundled GCS/Azure
### Are these changes tested?
Verified by CI
### Are there any user-facing changes?
Downstream consumers now have required static dependencies for Azure/GCS
* GitHub Issue: #51329
Authored-by: Tadeja Kadunc <tadeja.kadunc@gmail.com>
Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com>
### Rationale for this change
The current version of google-cloud-cpp used by Arrow does not build with OpenSSL 4.x
### What changes are included in this PR?
This change applies a patch to google-cloud-cpp as a workaround for OpenSSL 4.x compatibility.
### Are these changes tested?
Yes.
### Are there any user-facing changes?
No.
### Was AI used for this PR?
In accordance to the [AI generation guidelines](https://arrow.apache.org/docs/dev/developers/overview.html#ai-generated-code), please disclose below whether and how AI was used in this PR.
**PR code and description written by:**
- [x] Human
- [ ] AI
**Reviewed before submission by:**
- [x] Human
- [ ] AI
- [ ] Not reviewed
* GitHub Issue: #51634
Lead-authored-by: Hiroyuki Sato <hiroysato@gmail.com>
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
### Rationale for this change
The FlightSQL ODBC driver was previously using dynamic linkage for the Arrow deps it relies on. This worked but prevented a challenge for code signing because (I think) we'd have to sign each DLL and signing costs money. We can use static linkage with a dynamically linked MSVC runtime so we only have one DLL to sign.
Ref: https://github.com/apache/arrow/issues/49404
### What changes are included in this PR?
Updates to CMake and CI for the build to allow static linkage with dynamic MSVC runtime (`x64-windows-static-md`).
### Are these changes tested?
Yes.
### Are there any user-facing changes?
No.
* GitHub Issue: #49538
Authored-by: Bryce Mecum <petridish@gmail.com>
Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com>
### Rationale for this change
Try to use MSVC's new preprocessor to handle C++20 grammar such as `__VA_OPT__`.
### What changes are included in this PR?
Change the common flag under MSVC.
### Are these changes tested?
Tested by existed tests.
### Are there any user-facing changes?
No.
* GitHub Issue: #50978
Authored-by: Zehua Zou <zehuazou2000@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
### Rationale for this change
Trying to reduce the size of `libarrow.so` and remove AWS SDK on some builds. Allow for users to plug and play based on requirements and divide our functionality into cleaner modules.
### What changes are included in this PR?
Unconditionally build S3 and the AWS SDK into a different module `libarrow_s3.so` outside of `libarrow.so`.
Update bindings to link against the new `libarrow_s3.so` library.
Update the Linux Package jobs to have the new module into a different package.
### Are these changes tested?
Yes via CI
### Are there any user-facing changes?
Yes, users will need to either link against `libarrow_s3.so` or register using `LoadFileSystemFactories`
* GitHub Issue: #50194
Authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com>
### Rationale for this change
Reproducible builds are currently failing since simdjson changes were merged.
### What changes are included in this PR?
Disable Precompiled Headers on simdjson to avoid failing reproducible builds.
### Are these changes tested?
Yes on CI
### Are there any user-facing changes?
No
### Was AI used for this PR?
In accordance to the [AI generation guidelines](https://arrow.apache.org/docs/dev/developers/overview.html#ai-generated-code), please disclose below whether and how AI was used in this PR.
**PR code and description written by:**
- [x] Human
- [ ] AI
**Reviewed before submission by:**
- [x] Human
- [ ] AI
- [ ] Not reviewed
Note: AI was used to diagnose the reproducible build failure otuput from diffoscope, failure on this gist:
https://gist.github.com/raulcd/b2961cc053ae2d3033742e790d474b5b
* GitHub Issue: #51343
Authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
### Rationale for this change
The latest version of boost is 1.92.0. It fixes some bugs in the version used now. See #50978
### What changes are included in this PR?
Update boost.
### Are these changes tested?
Yes.
### Are there any user-facing changes?
Yes.
* GitHub Issue: #51116
Authored-by: Zehua Zou <zehuazou2000@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
### Rationale for this change
Fix#51140, CMake configuration should not accept simdjson earlier than v4.3.0.
### What changes are included in this PR?
Set ARROW_SIMDJSON_REQUIRED_VERSION to "4.3.0"
### Are these changes tested?
Yes. Locally, CMake configuration rejects system simdjson 4.2.3 now
`CMake Error at cmake_modules/ThirdpartyToolchain.cmake:316 (message):`
` Couldn't find simdjson >= 4.3.0`
### Are there any user-facing changes?
No.
* GitHub Issue: #51140
Authored-by: Tadeja Kadunc <tadeja.kadunc@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
### Rationale for this change
JSON integration tests fail on deeply nested lists we want to fix that.
### What changes are included in this PR?
We give Emscripten test executable a bigger stack size (1 MiB) and hope it's enough.
### Are these changes tested?
By CI.
### Are there any user-facing changes?
No, integration testing only change.
* GitHub Issue: #51101
Authored-by: Rok Mihevc <rok@mihevc.org>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
### Rationale for this change
Parquet uses JsonWriter when ARROW_JSON=OFF, but its implementation was only built with Arrow JSON, causing link failures.
### What changes are included in this PR?
Move JsonWriter to the simdjson utilities and update its callers and CMake/Meson builds.
### Are these changes tested?
Yes. CMake shared/static and Meson Parquet builds pass with JSON disabled. Unit tests and pre-commit checks also pass.
### Are there any user-facing changes?
No. This only fixes the affected build configuration.
AI disclosure - this was AI generated to test alternative approach to #50900.
* GitHub Issue: #50859
Authored-by: Rok Mihevc <rok@mihevc.org>
Signed-off-by: Antoine Pitrou <antoine@python.org>
### Rationale for this change
Fix#50774. After #50650, the Pyodide build failed while loading PyArrow with ` ImportError: dynamic module does not define module export function (PyInit_lib)`.
The ImportError is misleading because the `PyInit_lib` is present. The actual failure is an earlier abort during static initialization of `libarrow_python.so`, in Protobuf descriptor registration.
#50650 changed the order of libraries merged into `libarrow_bundled_dependencies.a`. The archive contains duplicate weak Protobuf/Abseil symbols emitted by the Protobuf runtime and generated code in Substrait and in ORC. These symbols had inconsistent visibility:
a) Protobuf was compiled with *hidden* visibility,
b) Substrait and ORC generated Protobuf code used default visibility,
so the order change caused `wasm-ld` to select symbols with different visibility.
(e.g. `google::protobuf::internal::ZeroFieldsBase::Clear()` changed from hidden in Protobuf's `empty.pb.cc.o` to default visible in Substrait's `algebra.pb.cc.o`)
The failure showed up in Emscripten, no failures on other platforms, but matching visibility globally is desirable to prevent any unintended exports and symbol interposition.
### What changes are included in this PR?
Compile bundled Substrait and ORC with the same hidden visibility for symbols as Protobuf so not depending on archive order.
Also add debug for Emscripten CI builds `--profiling-funcs`.
### Are these changes tested?
Yes, `test-conda-python-emscripten` passes
(Local repro, debug and fix verification with `ARCH=amd64 archery docker run conda-python-emscripten`, for symbols `llvm-nm` and `wasm-ld`)
### Are there any user-facing changes?
No.
* GitHub Issue: #50774
Authored-by: Tadeja Kadunc <tadeja.kadunc@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
### Rationale for this change
GH-50697
I have tested the RPM ODBC installer on AlmaLinux 9. The ODBC driver works as expected on AlmaLinux 9. The only issue I found was that the `connection-options.md` guide was not included in the package installation. This PR adds `connection-options.md` to the RPM ODBC package so users can reference the connection options documentation after installation.
Summary of testing results:
1. Clean install - driver works as expected and registered successfully, arrow installer installs `isql` and `unixODBC` during installation.
2. Uninstall - driver is uninstalled and unregistered automatically, arrow installer flags and uninstalls `isql` and `unixODBC` as unused dependency during installation.
If user wants to avoid this behavior, they can add `--noautoremove `flag to make `dnf` keep `isql` and `unixODBC` during ODBC uninstallation.
3. Upgrade - driver can be upgraded to new version.
4. Data retrieval works as expected, no dependency issues found.
### What changes are included in this PR?
- Add `connection-options.md` to the RPM ODBC package installation.
- Example location after install: `/usr/share/doc/arrow2600-flight-sql-odbc-libs/connection-options.md`
- Fix Windows ODBC installer to include docs `LICENSE.txt` and `NOTICE.txt`
- Add ODBC installer description to ODBC `README.md`
### Are these changes tested?
Yes
### Are there any user-facing changes?
Yes
* GitHub Issue: #50697
Lead-authored-by: Alina (Xi) Li <Alina.Li@improving.com>
Co-authored-by: Alina (Xi) Li <96995091+alinaliBQ@users.noreply.github.com>
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
Co-authored-by: justing-bq <justin.gossett@improving.com>
Signed-off-by: David Li <li.davidm96@gmail.com>
### Rationale for this change
Unlike RapidJSON, simdjson is not header-only and needs to be linked to explicitly when linking against libarrow.a.
This should fix the JNI builds on the C++ Extra workflow.
### Are these changes tested?
Yes, by existing CI builds.
### Are there any user-facing changes?
No.
* GitHub Issue: #50739
Authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com>
### Rationale for this change
I noticed while editing a .pyx file that my changes weren't
showing up after a rebuild — I had to build twice. Traced it
to `BYPRODUCTS` being commented out in `UseCython.cmake`.
### What changes are included in this PR?
I uncommented `BYPRODUCTS ${_generated_files}` in
`cpp/cmake_modules/UseCython.cmake`. Without it, CMake doesn't
realize the .cpp was updated in the same build pass, so it
skips recompiling the .so until the next build.
The line was commented out for older CMake compatibility — but
the project requires CMake >= 3.25 now, and BYPRODUCTS has
worked since 3.2, so that's no longer a concern.
### Are these changes tested?
This is a build system fix so there's no unit test for it.
The CI builds pyarrow from source and runs the full test suite,
which will validate the build still works correctly.
### Are there any user-facing changes?
No — this only improves the dev experience when iterating on
.pyx files. One build instead of two.
* GitHub Issue: #50702
Authored-by: Pratyush Adhikari <pratyushadk990@gmail.com>
Signed-off-by: Antoine Pitrou <antoine@python.org>
### Rationale for this change
This should fix a number of C++ and Python CI builds now that simdjson is used for parsing geospatial and modular encryption metadata.
### Are these changes tested?
By existing CI builds.
### Are there any user-facing changes?
Just a bugfix in the build chain.
* GitHub Issue: #50716
Authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Antoine Pitrou <antoine@python.org>
### Rationale for this change
This change enables building Arrow with `SIMDJSON_EXCEPTIONS=0` by updating the JSON object parser to use simdjson's non-throwing API.
### What changes are included?
- Enable `SIMDJSON_EXCEPTIONS=0` for the bundled simdjson dependency.
- Replace uses of `simdjson_result::value()` in `ObjectParser` with the non-throwing `get()` API.
- Preserve existing error handling by returning Arrow `Status` values on simdjson errors.
### Are these changes tested?
Yes.
Fixes: #50654
* GitHub Issue: #50654
Authored-by: Aaditya Srinivasan <aadityasri03@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
### What changes are included?
This PR introduces a reusable `JsonWriter` wrapper around simdjson's JSON builder API and migrates the integration JSON writer to use it instead of RapidJSON.
Specifically, this PR:
- Adds a reusable `JsonWriter` abstraction in `arrow/json`.
- Migrates the integration JSON writer implementation to `JsonWriter`.
- Updates the integration tests to use `JsonWriter`.
- Adds unit tests for `JsonWriter`.
This is part of the incremental migration from RapidJSON to simdjson.
### Performance
I compared the serialization performance of `IntegrationJsonWriter::WriteRecordBatch()` + `Finish()` using a temporary benchmark with a 1M-row `RecordBatch`. The benchmark was run 10 times on both `main` and this branch.
| Branch | Average Time |
|--------|-------------:|
| `main` | 2157.4 ms |
| This PR | 2153.6 ms |
No measurable performance regression was observed.
### Are these changes tested?
Yes.
I added unit tests for `JsonWriter` and verified that both JSON and integration tests pass locally:
- `arrow-json-test`
- `arrow-json-integration-test`
### Are there any user-facing changes?
No.
* GitHub Issue: #50567
Authored-by: Aaditya Srinivasan <aadityasri03@gmail.com>
Signed-off-by: Antoine Pitrou <antoine@python.org>
### Rationale for this change
The workaround introduced for the macOS 11.3 SDK currently uses a global `add_compile_definitions(SIMDJSON_CONCEPT_DISABLED=1)`.
Since this workaround is only needed when building the bundled simdjson dependency, scope it to the `simdjson` target inside `build_simdjson()` instead of applying it globally.
### What changes are included in this PR?
- Move the macOS SDK 11.3 workaround into `build_simdjson()`.
- Replace the global `add_compile_definitions()` with `target_compile_definitions(simdjson PUBLIC SIMDJSON_CONCEPT_DISABLED=1)`.
* GitHub Issue: #50631
Authored-by: Aaditya Srinivasan <aadityasri03@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
### Rationale for this change
We must define options explicitly.
### What changes are included in this PR?
* Add `ARROW_FLIGHT_SQL_ODBC_INSTALLER` to `DefineOptions.cmake`
* Don't enable `ARROW_FLIGHT_SQL_ODBC_INSTALLER` by default in `features-maximal` preset because it's available only on Windows and macOS
### Are these changes tested?
Yes.
### Are there any user-facing changes?
Yes.
* GitHub Issue: #50519
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
### Rationale
The R build may pick up a system-installed simdjson when `simdjson_SOURCE` is not explicitly set. This causes build failures in environments where the system version is incompatible or unsuitable for the build.
### What changes are included?
This change defaults `simdjson_SOURCE` to `BUNDLED` when it is not already set, ensuring C++ wrapper builds invoked through `build_arrow_static.sh` use the bundled simdjson by default, while still allowing users to override it explicitly.
This addresses the macOS CRAN release and ubuntu-r-only-r (AUTO deps) failures reported in GH-50581.
Related: #50581
* GitHub Issue: #50581
Authored-by: Aaditya Srinivasan <aadityasri03@gmail.com>
Signed-off-by: Rok Mihevc <rok@mihevc.org>
### Rationale for this change
Fix#50597. Nightly Crossbow job `test-conda-python-emscripten` sometimes finishes with success like [2026-07-18 Docker Test conda-python-emscripten](https://github.com/ursacomputing/crossbow/actions/runs/29628907267/job/88038728709#step:6:7261)
But often hangs like [2026-07-19 Docker Test conda-python-emscripten](https://github.com/ursacomputing/crossbow/actions/runs/29672419322/job/88153721397#step:6:6851)
and [2026-07-20 Docker Test conda-python-emscripten](https://github.com/ursacomputing/crossbow/actions/runs/29715959996/job/88269203693#step:6:6846)
```
driver.load_arrow()
...
selenium.common.exceptions.TimeoutException: Message: script timeout
```
Additionally, main nightly started failing during `snappy_ep` configure with CMake 4.4: `CMake Error: The warning category "linkflags" is not known.`
### What changes are included in this PR?
import `TimeoutException`, introduce 300s timeout + 3 attempts around load_arrow() and full Chrome restart on timeout,
ignore harmless `BrokenPipeError` / `ConnectionResetError` when Chrome has to be killed mid-download for retry.
Also remove the redundant standalone `-Wno-error=linkflags` CMake argument from Snappy's Emscripten configure arguments. The flag remains in `CMAKE_SHARED_LINKER_FLAGS`
### Are these changes tested?
Yes, locally by `docker compose run --rm -e SETUPTOOLS_SCM_PRETEND_VERSION=26.0.0.dev0 conda-python-emscripten`.
Example test run hit the PyArrow load timeout once, restarted Chrome, and then completed successfully.
### Are there any user-facing changes?
No.
* GitHub Issue: #50597
Authored-by: Tadeja Kadunc <tadeja.kadunc@gmail.com>
Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com>
### Rationale for this change
On macOS, two independently-linked mimalloc v3 instances built with default TLS settings can end up using the same hard-coded TLS slots and crash due to conflicting expectations. See upstream issue at https://github.com/microsoft/mimalloc/issues/1327
This can manifest when PyArrow is loaded side-by-side with another Python extension module that bundles its own instance of mimalloc.
### What changes are included in this PR?
1. Bump mimalloc to 3.4.1, for the availability of the required CMake option.
2. Configure macOS to use C thread-local variables for thread-local storage, avoiding conflicting accesses to hard-coded TLS slots on macOS.
3. Also, unrelatedly, make sure the default malloc is not overriden by our mimalloc build on macOS.
### Are these changes tested?
By existing CI jobs.
### Are there any user-facing changes?
No, just a bugfix.
* GitHub Issue: #50428
Lead-authored-by: Antoine Pitrou <antoine@python.org>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
Signed-off-by: Antoine Pitrou <antoine@python.org>
### Rationale for this change
This PR is the first in a series of PRs for GH-35460 to migrate Arrow's JSON implementation from RapidJSON to simdjson.
As the foundation for the migration, this PR introduces simdjson into Arrow's build system, updates the bundled simdjson version to v4.6.4, and migrates the self-contained `arrow::json::internal::ObjectParser`.
### What changes are included in this PR?
This PR:
- Introduces `simdjson` as a third-party dependency.
- Adds CMake support for discovering and building `simdjson`.
- Updates the bundled simdjson dependency to **v4.6.4**.
- Adds system package and vcpkg integration required for simdjson across supported build environments.
- Links `simdjson` into the Arrow JSON library.
- Migrates `arrow::json::internal::ObjectParser` from RapidJSON to the simdjson DOM API.
- Adds dedicated unit tests covering:
- String retrieval
- Boolean retrieval
- String map extraction
- Invalid JSON
- Missing keys
- Incorrect types
- Non-object root documents
- Empty objects
This establishes the build infrastructure needed for the remaining JSON migration, which will be completed incrementally in follow-up PRs.
### Performance
On the ObjectParser benchmark, the simdjson implementation achieved approximately **6.3×** higher throughput than the previous RapidJSON implementation (about **84%** lower execution time).
### Are these changes tested?
Yes.
- Added dedicated unit tests for `ObjectParser`.
- Verified the ObjectParser JSON test suite locally.
- Updated the build system and validated the dependency integration across supported CI configurations (currently 41 CI checks passing; remaining failures are limited to two Windows-specific workflows under investigation).
* GitHub Issue: #50566
Authored-by: Aaditya Srinivasan <aadityasri03@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
### Rationale for this change
Fix#50424 `Python Emscripten Crossbow` job with multiple failing points
A) failing to install Chrome - `Requested Chrome major 148, but apt repo currently publishes 150.0.7871.100`,
B) and underlying Abseil failure - `error: expected '(' before 'ABSL_INTERNAL_CONSTEXPR_SINCE_CXX17'` [logB](https://github.com/ursacomputing/crossbow/actions/runs/28963702667/job/85941369345#step:6:2995)
C) pytest 8.1.1 failing on three tests with the message - `E TypeError: importorskip() got an unexpected keyword argument 'exc_type'` [logC](https://github.com/ursacomputing/crossbow/actions/runs/28972366846/job/85971005157#step:6:6354)
The remaining intermittent issue be best dealt with in a follow up PR:
D) intermittent `loadPyodide` / Selenium timeout hang - `selenium.common.exceptions.TimeoutException: Message: script timeout` [logD](https://github.com/ursacomputing/crossbow/actions/runs/28997397611/job/86049819728#step:6:6856) and [hang log with debug](https://github.com/ursacomputing/crossbow/actions/runs/29015193964/job/86108340808#step:6:7248)
### What changes are included in this PR?
A) Changed `chrome_version: "148"` to `chrome_version: "latest"`
B) Disabled zlib for protobuf build of the host in order to avoid conda include path overriding vendor Abseil headers.
C) Added work-around for current Pyodide's bundled pytest version 8.1.1 which lacks `importorskip(..., exc_type=...)`
### Are these changes tested?
Yes, test-conda-python-emscripten job completed successfully on rerun. Some earlier runs hit the intermittent Chrome/Selenium timeout mentioned above.
### Are there any user-facing changes?
No.
* GitHub Issue: #50424
Authored-by: Tadeja Kadunc <tadeja.kadunc@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
### Rationale for this change
FetchContent suggested as better approach in GH-45303.
### What changes are included in this PR?
Use FetchContent for xsimd and solve a bug where xsimd is not available at configure time.
### Are these changes tested?
Yes, locally.
### Are there any user-facing changes?
No.
* GitHub Issue: #50222
Authored-by: AntoinePrv <AntoinePrv@users.noreply.github.com>
Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com>
### Rationale for this change
We want to reduce CI time as much as possible for developer experience and avoiding consuming shared resources.
### What changes are included in this PR?
* Use `apache/infrastructure-actions/stash` instead of `actions/cache` to use larger cache storage
* Enable ccache
* Increase ccache size because 500MB is small for our case
* Fix Docker image cache
* Disable RapidJSON's ccache detection that causes needless `ccache` call such as `ccache ccache cc ...`
* Use Ninja instead of Make
### Are these changes tested?
Yes.
CI time is reduced to 20min-60min from 60min-120min.
### Are there any user-facing changes?
No.
* GitHub Issue: #50170
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
### Rationale for this change
We don't need to build unit tests for the bundled RE2 library, and it avoids `ctest` picking those tests when run from the build directory.
### Are these changes tested?
Manually.
### Are there any user-facing changes?
No.
* GitHub Issue: #49834
Authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
### Rationale for this change
Use xsimd cpu feature instead of maitaining them here.
Stacked on GH-49922.
Should also handle issue from GH-30368.
### What changes are included in this PR?
- Use xsimd for CPu feature detection, cache sizes are still using the same implementation
- We are loosing CPU model name (which was unused)
- Simplify CpuInfo (remove Pimpl pattern)
### Are these changes tested?
in CI
### Are there any user-facing changes?
No
* GitHub Issue: #45331
Authored-by: AntoinePrv <AntoinePrv@users.noreply.github.com>
Signed-off-by: Antoine Pitrou <antoine@python.org>
### Rationale for this change
`PARQUET_REQUIRE_ENCRYPTION` needs `ARROW_JSON`. So `ARROW_JSON` should be enabled automatically when `PARQUET_REQUIRE_ENCRYPTION` is used.
### What changes are included in this PR?
Add missing `ARROW_JSON` dependency to `PARQUET_REQUIRE_ENCRYPTION`.
### Are these changes tested?
Yes.
### Are there any user-facing changes?
Yes.
* GitHub Issue: #50129
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com>
### Rationale for this change
Some Parquet utilities call the Arrow filesystem API.
Also make sure that compute benchmarks are not built when the full compute kernels registry is not enabled.
### Are these changes tested?
Manually.
### Are there any user-facing changes?
No.
* GitHub Issue: #50098
Authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
### Rationale for this change
- Fix for the current SVE ODR violation in the latest version
https://github.com/xtensor-stack/xsimd/pull/1311
- Pre-requisit to GH-45331
### What changes are included in this PR?
Bump version and cleanup backports.
### Are these changes tested?
Yes, in CI.
### Are there any user-facing changes?
No
* GitHub Issue: #49921
Authored-by: AntoinePrv <AntoinePrv@users.noreply.github.com>
Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com>
### Rationale for this change
Reproducible builds are a requirement to be able to add automated signing. We would like to be able to do automated signing for our Linux packages too as discussed here:
https://github.com/apache/arrow/issues/47058#issuecomment-3243601724
### What changes are included in this PR?
Some minor changes to our build to fix some Reproducible tests errors found. The main addition is adding a new step to our Linux Packaging GitHub actions workflow to run reprotest on the rake tasks to build the packages and compare the artifacts built.
### Are these changes tested?
Yes on CI as part of the APT Packaging Linux jobs.
### Are there any user-facing changes?
* GitHub Issue: #49988
Lead-authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Co-authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
### Rationale for this change
We already set this variable when adding projects through `ExternalProject_Add()`, but the config file that RapidJSON exports sets `cmake_minimum_required()` to a version older than 3.5. Set the policy minimum when searching for the just-built RapidJSON.
### What changes are included in this PR?
Set `CMAKE_POLICY_VERSION_MINIMUM` in `FindRapidJSONAlt.cmake`
### Are these changes tested?
Tested locally
### Are there any user-facing changes?
* GitHub Issue: #48801
Authored-by: Kyle Edwards <kyedwards@nvidia.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
### Rationale for this change
`find_package()`'s `VALIDATOR`'s result variable must be initialized to `TRUE` by caller.
We reuse the `VALIDATOR` (`validate_apple_libtool()`) when we specify `CMAKE_LIBTOOL` explicitly. But we didn't initialize the result variable to `TRUE`.
### What changes are included in this PR?
Initialize `is_apple_libtool` explicitly.
### Are these changes tested?
Yes.
### Are there any user-facing changes?
Yes.
* GitHub Issue: #49974
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
### Rationale for this change
We are currently using 1.17.2 which is 5 years old. There's been plenty of new versions released we should regularly upgrade our bundled dependencies.
### What changes are included in this PR?
Update bundled c-ares from 1.17.2 to 1.34.6
### Are these changes tested?
Yes via CI
### Are there any user-facing changes?
No
* GitHub Issue: #49938
Authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com>
### Rationale for this change
The `134` version fails to be installed from stable releases on chrome due to a new stable version released.
### What changes are included in this PR?
- Update version of latest stable chrome driver to `148` and how we look for version on instrall script.
- Update selenium to newer version to work with newer chrome.
- Fixes to cross-compilation build after upgrading Abseil and Protobuf and moving to FetchContent.
- Updated timeout so Chrome job doesn't time out.
- Skip substrait tests which currently fail on Pyodide.
### Are these changes tested?
Yes, via archery job.
### Are there any user-facing changes?
No
* GitHub Issue: #49522
Authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
### Rationale for this change
With the bump versions for gRPC/Protobuf/Abseil and others a bug was introduced where we build Abseil outside of resolve dependency. This caused to interpret Abseil as System dependency for Arrow Flight when it was bundled and failing to find it when trying to build PyArrow.
### What changes are included in this PR?
Reorder resolve_dependency abseil to be done before resolving protobuf in order to not require manually building abseil.
### Are these changes tested?
Yes via CI, via archery and validated the local build that previously failed now succeeds
### Are there any user-facing changes?
No
* GitHub Issue: #49764
Authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
Raúl Cumplido