### Rationale for this change
Resolves#49046
`python/scripts/test_imports.py` and `python/scripts/test_leak.py` are leftovers from an older
test layout. They are not referenced by CI, packaging or the pytest suite, and are not
documented anywhere. Their `test_`-prefixed names have also caused accidental pytest collection
in the past (ARROW-1033). This follows the suggestion on the issue to remove them.
Deletes `python/scripts/test_imports.py` and `python/scripts/test_leak.py`.
The other two scripts in that directory are untouched and still in use:
`run_emscripten_tests.py` (invoked from `ci/scripts/python_test_emscripten.sh`) and
`update_stub_docstrings.py`.
The issue description raised the option of keeping `test_leak.py` as a documented snippet for
checking leaks. This PR removes it, following the later comment. Happy to restore and document
it under the developer docs instead if reviewers prefer.
### Are these changes tested?
No new tests, this only removes files that nothing references. I grepped `ci/`, `dev/`,
`python/setup.py`, `python/pyproject.toml` and `python/MANIFEST.in` for references and found
none. The only remaining mention in the repository is a historical `CHANGELOG.md` entry for
ARROW-1033, left as is.
### Are there any user-facing changes?
No.
### AI usage disclosure
I used Claude Code to grep the repository for remaining references to the two scripts and to
help draft this description. The change itself is a two file deletion, and I reviewed both the
search results and the diff.
* GitHub Issue: #49046
Authored-by: Guja <127162872+GujaLomsadze@users.noreply.github.com>
Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com>
### Rationale for this change
Fix#50597. Nightly Crossbow job `test-conda-python-emscripten` sometimes finishes with success like [2026-07-18 Docker Test conda-python-emscripten](https://github.com/ursacomputing/crossbow/actions/runs/29628907267/job/88038728709#step:6:7261)
But often hangs like [2026-07-19 Docker Test conda-python-emscripten](https://github.com/ursacomputing/crossbow/actions/runs/29672419322/job/88153721397#step:6:6851)
and [2026-07-20 Docker Test conda-python-emscripten](https://github.com/ursacomputing/crossbow/actions/runs/29715959996/job/88269203693#step:6:6846)
```
driver.load_arrow()
...
selenium.common.exceptions.TimeoutException: Message: script timeout
```
Additionally, main nightly started failing during `snappy_ep` configure with CMake 4.4: `CMake Error: The warning category "linkflags" is not known.`
### What changes are included in this PR?
import `TimeoutException`, introduce 300s timeout + 3 attempts around load_arrow() and full Chrome restart on timeout,
ignore harmless `BrokenPipeError` / `ConnectionResetError` when Chrome has to be killed mid-download for retry.
Also remove the redundant standalone `-Wno-error=linkflags` CMake argument from Snappy's Emscripten configure arguments. The flag remains in `CMAKE_SHARED_LINKER_FLAGS`
### Are these changes tested?
Yes, locally by `docker compose run --rm -e SETUPTOOLS_SCM_PRETEND_VERSION=26.0.0.dev0 conda-python-emscripten`.
Example test run hit the PyArrow load timeout once, restarted Chrome, and then completed successfully.
### Are there any user-facing changes?
No.
* GitHub Issue: #50597
Authored-by: Tadeja Kadunc <tadeja.kadunc@gmail.com>
Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com>
### Rationale for this change
Fix#50424 `Python Emscripten Crossbow` job with multiple failing points
A) failing to install Chrome - `Requested Chrome major 148, but apt repo currently publishes 150.0.7871.100`,
B) and underlying Abseil failure - `error: expected '(' before 'ABSL_INTERNAL_CONSTEXPR_SINCE_CXX17'` [logB](https://github.com/ursacomputing/crossbow/actions/runs/28963702667/job/85941369345#step:6:2995)
C) pytest 8.1.1 failing on three tests with the message - `E TypeError: importorskip() got an unexpected keyword argument 'exc_type'` [logC](https://github.com/ursacomputing/crossbow/actions/runs/28972366846/job/85971005157#step:6:6354)
The remaining intermittent issue be best dealt with in a follow up PR:
D) intermittent `loadPyodide` / Selenium timeout hang - `selenium.common.exceptions.TimeoutException: Message: script timeout` [logD](https://github.com/ursacomputing/crossbow/actions/runs/28997397611/job/86049819728#step:6:6856) and [hang log with debug](https://github.com/ursacomputing/crossbow/actions/runs/29015193964/job/86108340808#step:6:7248)
### What changes are included in this PR?
A) Changed `chrome_version: "148"` to `chrome_version: "latest"`
B) Disabled zlib for protobuf build of the host in order to avoid conda include path overriding vendor Abseil headers.
C) Added work-around for current Pyodide's bundled pytest version 8.1.1 which lacks `importorskip(..., exc_type=...)`
### Are these changes tested?
Yes, test-conda-python-emscripten job completed successfully on rerun. Some earlier runs hit the intermittent Chrome/Selenium timeout mentioned above.
### Are there any user-facing changes?
No.
* GitHub Issue: #50424
Authored-by: Tadeja Kadunc <tadeja.kadunc@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
### Rationale for this change
The `134` version fails to be installed from stable releases on chrome due to a new stable version released.
### What changes are included in this PR?
- Update version of latest stable chrome driver to `148` and how we look for version on instrall script.
- Update selenium to newer version to work with newer chrome.
- Fixes to cross-compilation build after upgrading Abseil and Protobuf and moving to FetchContent.
- Updated timeout so Chrome job doesn't time out.
- Skip substrait tests which currently fail on Pyodide.
### Are these changes tested?
Yes, via archery job.
### Are there any user-facing changes?
No
* GitHub Issue: #49522
Authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
### Rationale for this change
Updating docstrings is failing on Windows trying to remove arrow.dll from tempdir because its still loaded in memory.
### What changes are included in this PR?
Added `ignore_cleanup_errors=True`
### Are these changes tested?
Yes via archery
### Are there any user-facing changes?
No
* GitHub Issue: #49531
Authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com>
### Rationale for this change
The emscripten job is currently failing due to pyarrow-stubs building.
### What changes are included in this PR?
- Do not build doc stubs for emscripten build as it tries to install pyarrow which is unavailable.
- Minor fix for chrome as it was timing out without being able to download wasm wheel.
- Fix loading tzdata as some tests were failing without it.
### Are these changes tested?
Yes via archery.
### Are there any user-facing changes?
No
* GitHub Issue: #49426
Authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Rok Mihevc <rok@mihevc.org>
### Rationale for this change
This is the first in series of PRs adding type annotations to pyarrow and resolving #32609.
### What changes are included in this PR?
This PR establishes infrastructure for type checking:
- Adds CI workflow for running mypy, pyright, and ty type checkers on linux, macos and windows
- Configures type checkers to validate stub files (excluding source files for now)
- Adds PEP 561 `py.typed` marker to enable type checking
- Updates wheel build scripts to include stub files in distributions
- Creates initial minimal stub directory structure
- Updates developer documentation with type checking workflow
### Are these changes tested?
No. This is mostly a CI change.
### Are there any user-facing changes?
This does not add any actual annotations (only `py.typed` marker) so user should not be affected.
* GitHub Issue: #32609
* GitHub Issue: #49102
Lead-authored-by: Rok Mihevc <rok@mihevc.org>
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
Co-authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Rok Mihevc <rok@mihevc.org>
### Rationale for this change
When looking for the wheel the script was falling back to returning a 404 even when the wheel was found:
```
+ python scripts/run_emscripten_tests.py dist/pyarrow-24.0.0.dev31-cp312-cp312-pyodide_2024_0_wasm32.whl --dist-dir=/pyodide --runtime=chrome
127.0.0.1 - - [27/Jan/2026 01:14:50] code 404, message File not found
```
Timing out the job and failing.
### What changes are included in this PR?
Correct logic and only return 404 if the file requested wasn't found.
### Are these changes tested?
Yes via archery
### Are there any user-facing changes?
No
* GitHub Issue: #47692
Authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com>
### Rationale for this change
Installing the GCS testbench using the same Python that's being used to test PyArrow is fragile: some testbench versions may not be compatible, or there could be conflicts among the dependencies of the respective libraries.
### What changes are included in this PR?
Use `pipx` to install the GCS testbench in a separate, controlled environment, using an appropriate Python version.
### Are these changes tested?
Yes, by CI.
### Are there any user-facing changes?
No.
* GitHub Issue: #38183
Authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
### Rationale for this change
The tests are failing on windows when using numpy 2.0 RC, probably related to default integer bitwidth.
### What changes are included in this PR?
### Are these changes tested?
### Are there any user-facing changes?
* GitHub issue: #41319
* GitHub Issue: #41924
Authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
Signed-off-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
An array object was failing to be decref'd on the DatetimeTZ conversion path. The code is slightly complicated by the different calling reference ownership semantics of the Array/ChunkedArray conversion path (which expects to own the created array when it calls `GetSeriesResult` while the `GetResultBlock` code retains its array in a `OwnedRefNoGIL`). This was the simplest thing that fixed the memory leak for me. There is leak checking code that can be used to verify this in python/scripts/test_leak.py (just run the script).
Closes#7522 from wesm/ARROW-8801
Authored-by: Wes McKinney <wesm@apache.org>
Signed-off-by: Wes McKinney <wesm@apache.org>
Warning: hold your nose for this one =) I think this can be made cleaner, but I just wanted to triage all the problems and make sure we don't introduce a hard dependency on pandas again
Also resolves ARROW-4794: make pandas an optional dependency
Author: Wes McKinney <wesm+git@apache.org>
Closes#3893 from wesm/ARROW-4637 and squashes the following commits:
3c353b6b <Wes McKinney> do not override orc global mark in test_orc.py
59fdf8ab <Wes McKinney> Tweak pyarrow._orc import to see if it fixes MSVC
747c8937 <Wes McKinney> Address Python 2.7 unicode interaction issue with Cython
4d94b2e1 <Wes McKinney> Add benchmark from ARROW-4629
f7bf7741 <Wes McKinney> Cythonize pandas API shim for better performance
4a2549aa <Wes McKinney> Remove TF testing from travis_script_manylinux1.sh
5061c6e1 <Wes McKinney> Do not require pandas to run unit tests
385cfe55 <Wes McKinney> Finish pandas API shim; do not eagerly import pandas, add to CI
804587a0 <Wes McKinney> add import test script
bb0240e2 <Wes McKinney> Begin to refactor to make references to pandas more centralized and lazy
Destroy RowGroupMetadataBuilder after each row group is completed
Author: Wes McKinney <wesm+git@apache.org>
Closes#3261 from tanyaschlusser/ARROW-3324 and squashes the following commits:
5f3876706 <Wes McKinney> Refine case a bit
4f2bdcdce <Wes McKinney> Destroy RowGroupMetadataBuilder object after completing a row group to reduce memory usage
Author: Wes McKinney <wesm+git@apache.org>
Closes#2137 from wesm/ARROW-2395 and squashes the following commits:
d9e6e9f3 <Wes McKinney> Fix Cython flake8 warnings in examples
059a6d6b <Wes McKinney> Make functions in benchmarks/common.py py2-compatible
13791863 <Wes McKinney> Fix flake8 warnings outside of pyarrow/ directory. Check in CI
cc @BryanCutler
Author: Wes McKinney <wes.mckinney@twosigma.com>
Closes#704 from wesm/ARROW-1053 and squashes the following commits:
2f90337c [Wes McKinney] Remove unnecessary Py_INCREF in PyBuffer causing memory leak
Notes:
* `PyList_Append` increments ref count, so new objects must be DECREF'd after being inserted
* `PyArray_SimpleNewFromDescr` does not set `NPY_ARRAY_OWNDATA`, neither does `NewFromDescr`
Author: Wes McKinney <wes.mckinney@twosigma.com>
Closes#685 from wesm/ARROW-1017 and squashes the following commits:
8459123 [Wes McKinney] Fix memory leak caused by list append ref count, lack of setting NPY_ARRAY_OWNDATA
Guja