### Rationale for this change
See: https://github.com/apache/arrow/issues/49227#issue-3926177634
### What changes are included in this PR?
Deprecation warnings for `pyarrow.gandiva` added for version 24.0.0
### Are these changes tested?
Yes, tested locally that warnings are raised on import. CI tests verify the warnings are filtered in out test suite.
### Are there any user-facing changes?
Yes, `pyarrow.gandiva` is deprecated and will be removed in a feature PyArrow version.
* GitHub Issue: #49227
Authored-by: AlenkaF <frim.alenka@gmail.com>
Signed-off-by: AlenkaF <frim.alenka@gmail.com>
### Rationale for this change
Running `python -m pytest --doctest-cython` directly on `.pxi` files doesn't work because Cython `.pxi` files are included in `.pyx` files at compile time. This isn't documented, which could _potentially_ confuse contributors.
Please see discussion in #49279 (comment) by @ AlenkaF for additional context.
### What changes are included in this PR?
Added a `note` to the Doctest section of the Developing PyArrow documentation explaining that:
- `.pxi` files cannot be tested directly with `--doctest-cython`
- Doctests should be run on the `.pyx` file they are included in (e.g., `lib.pyx`)
- Errors will appear under the `.pyx` file, not the original `.pxi` filename
### Are these changes tested?
Documentation-only change. No code changes.
### Are there any user-facing changes?
No.
* GitHub Issue: #49503
Lead-authored-by: Bhavana Sundar <bhavana7899@gmail.com>
Co-authored-by: Alenka Frim <AlenkaF@users.noreply.github.com>
Signed-off-by: AlenkaF <frim.alenka@gmail.com>
### Rationale for this change
Move our PyArrow build backend from setuptools and a custom setup.py to scikit-build-core which is just build backend for CMake related projects.
### What changes are included in this PR?
Move from setuptools to scikit-build-core and remove PyArrow setup.py. Update some of the build requirements and minor fixes.
A custom build backend has been also been created in order to wrap scikit-build-core in order to fix problems on License files for monorepos.
pyproject.toml metadata validation expects license files to exist before exercising the build backend that's why we create symlinks. Our thin build backend will just make those symlinks hard-links in order for license and notice files to contain the contents and be added as part of the sdist.
Remove flags that are not used anymore (were only part of setup.py) and documented and validated how the same flags have to be used now.
### Are these changes tested?
Yes all Python CI tests, wheels and sdist are successful.
### Are there any user-facing changes?
Yes, users building PyArrow will now require the new build dependencies to exercise the build and depending on the flags used they might require to use the new documented way of using those flags.
* GitHub Issue: #36411
Lead-authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
Co-authored-by: Rok Mihevc <rok@mihevc.org>
Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com>
### Rationale for this change
This is the first in series of PRs adding type annotations to pyarrow and resolving #32609.
### What changes are included in this PR?
This PR establishes infrastructure for type checking:
- Adds CI workflow for running mypy, pyright, and ty type checkers on linux, macos and windows
- Configures type checkers to validate stub files (excluding source files for now)
- Adds PEP 561 `py.typed` marker to enable type checking
- Updates wheel build scripts to include stub files in distributions
- Creates initial minimal stub directory structure
- Updates developer documentation with type checking workflow
### Are these changes tested?
No. This is mostly a CI change.
### Are there any user-facing changes?
This does not add any actual annotations (only `py.typed` marker) so user should not be affected.
* GitHub Issue: #32609
* GitHub Issue: #49102
Lead-authored-by: Rok Mihevc <rok@mihevc.org>
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
Co-authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Rok Mihevc <rok@mihevc.org>
### Rationale for this change
As discussed on the issue we don't seem to have run asv benchmarks on Python for the last years. It is probably broken.
### What changes are included in this PR?
Remove asv benchmarking related files and docs.
### Are these changes tested?
No, Validate CI and run preview-docs to validate docs.
### Are there any user-facing changes?
No
* GitHub Issue: #46008
Authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com>
### Rationale for this change
In many places in the Python User Guide the code exampels are written with IPython directive (elsewhere code-block is used). IPython directives are converted to IPython format (`In` and `Out` during the doc build). This can lead to slower builds.
### What changes are included in this PR?
IPython directives are converted to runnable code-block (with `>>>` and `...`) and pytest doctest support for `.rst` files is added to the `conda-python-docs` CI job. This means the code in the Python User Guide is tested separately to the building of the documentation.
### Are these changes tested?
Yes, with the CI.
### Are there any user-facing changes?
Changes to the Python User Guide examples will have to be tested with `pytest --doctest-glob='*.rst' docs/source/python/file.rst`
* GitHub Issue: #28859
Lead-authored-by: AlenkaF <frim.alenka@gmail.com>
Co-authored-by: Alenka Frim <AlenkaF@users.noreply.github.com>
Co-authored-by: tadeja <tadeja@users.noreply.github.com>
Signed-off-by: AlenkaF <frim.alenka@gmail.com>
### Rationale for this change
The structure of the Python build and development documentation has become unclear due to multiple incremental changes over time. This PR aims to improve clarity and organization.
### What changes are included in this PR?
- Introduces tabs to improve navigation.
- Splits the existing documentation into two focused pages: Building and Developing PyArrow.
### Are these changes tested?
Yes. With docs preview build.
### Are there any user-facing changes?
No. Only docs changes.
* GitHub Issue: #20125
Authored-by: AlenkaF <frim.alenka@gmail.com>
Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com>
Alenka Frim