### Rationale for this change
Installing the GCS testbench using the same Python that's being used to test PyArrow is fragile: some testbench versions may not be compatible, or there could be conflicts among the dependencies of the respective libraries.
### What changes are included in this PR?
Use `pipx` to install the GCS testbench in a separate, controlled environment, using an appropriate Python version.
### Are these changes tested?
Yes, by CI.
### Are there any user-facing changes?
No.
* GitHub Issue: #38183
Authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
### Rationale for this change
The tests are failing on windows when using numpy 2.0 RC, probably related to default integer bitwidth.
### What changes are included in this PR?
### Are these changes tested?
### Are there any user-facing changes?
* GitHub issue: #41319
* GitHub Issue: #41924
Authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
Signed-off-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
An array object was failing to be decref'd on the DatetimeTZ conversion path. The code is slightly complicated by the different calling reference ownership semantics of the Array/ChunkedArray conversion path (which expects to own the created array when it calls `GetSeriesResult` while the `GetResultBlock` code retains its array in a `OwnedRefNoGIL`). This was the simplest thing that fixed the memory leak for me. There is leak checking code that can be used to verify this in python/scripts/test_leak.py (just run the script).
Closes#7522 from wesm/ARROW-8801
Authored-by: Wes McKinney <wesm@apache.org>
Signed-off-by: Wes McKinney <wesm@apache.org>
Warning: hold your nose for this one =) I think this can be made cleaner, but I just wanted to triage all the problems and make sure we don't introduce a hard dependency on pandas again
Also resolves ARROW-4794: make pandas an optional dependency
Author: Wes McKinney <wesm+git@apache.org>
Closes#3893 from wesm/ARROW-4637 and squashes the following commits:
3c353b6b <Wes McKinney> do not override orc global mark in test_orc.py
59fdf8ab <Wes McKinney> Tweak pyarrow._orc import to see if it fixes MSVC
747c8937 <Wes McKinney> Address Python 2.7 unicode interaction issue with Cython
4d94b2e1 <Wes McKinney> Add benchmark from ARROW-4629
f7bf7741 <Wes McKinney> Cythonize pandas API shim for better performance
4a2549aa <Wes McKinney> Remove TF testing from travis_script_manylinux1.sh
5061c6e1 <Wes McKinney> Do not require pandas to run unit tests
385cfe55 <Wes McKinney> Finish pandas API shim; do not eagerly import pandas, add to CI
804587a0 <Wes McKinney> add import test script
bb0240e2 <Wes McKinney> Begin to refactor to make references to pandas more centralized and lazy
Destroy RowGroupMetadataBuilder after each row group is completed
Author: Wes McKinney <wesm+git@apache.org>
Closes#3261 from tanyaschlusser/ARROW-3324 and squashes the following commits:
5f3876706 <Wes McKinney> Refine case a bit
4f2bdcdce <Wes McKinney> Destroy RowGroupMetadataBuilder object after completing a row group to reduce memory usage
Author: Wes McKinney <wesm+git@apache.org>
Closes#2137 from wesm/ARROW-2395 and squashes the following commits:
d9e6e9f3 <Wes McKinney> Fix Cython flake8 warnings in examples
059a6d6b <Wes McKinney> Make functions in benchmarks/common.py py2-compatible
13791863 <Wes McKinney> Fix flake8 warnings outside of pyarrow/ directory. Check in CI
cc @BryanCutler
Author: Wes McKinney <wes.mckinney@twosigma.com>
Closes#704 from wesm/ARROW-1053 and squashes the following commits:
2f90337c [Wes McKinney] Remove unnecessary Py_INCREF in PyBuffer causing memory leak
Notes:
* `PyList_Append` increments ref count, so new objects must be DECREF'd after being inserted
* `PyArray_SimpleNewFromDescr` does not set `NPY_ARRAY_OWNDATA`, neither does `NewFromDescr`
Author: Wes McKinney <wes.mckinney@twosigma.com>
Closes#685 from wesm/ARROW-1017 and squashes the following commits:
8459123 [Wes McKinney] Fix memory leak caused by list append ref count, lack of setting NPY_ARRAY_OWNDATA
