mirror of
https://github.com/apache/arrow.git
synced 2026-10-28 16:47:19 +08:00
Aaditya SrinivasanandAntoine Pitrou
3cb299408d
### Rationale for this change
This PR continues the simdjson migration by replacing the RapidJSON-based JSON boundary detection used by the JSON chunker.
The existing implementation uses RapidJSON's streaming parser to identify complete JSON values. This change replaces that logic with the simdjson streaming parser.
### Are these changes tested?
Yes, by existing tests.
### Are there any user-facing changes?
No, except perhaps slight differences in error message and timing of error reporting when reading from an invalid JSON stream (i.e. there may be cases where a JSON parse error is reported earlier or later while iterating over RecordBatches).
* GitHub Issue: #50944
Lead-authored-by: Aaditya Srinivasan <aadityasri03@gmail.com>
Co-authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Antoine Pitrou <antoine@python.org>
Python library for Apache Arrow
This library provides a Python API for functionality provided by the Arrow C++ libraries, along with tools for Arrow integration and interoperability with pandas, NumPy, and other software in the Python ecosystem.
Installing
Across platforms, you can install a recent version of pyarrow with the conda package manager:
conda install pyarrow -c conda-forge
On Linux, macOS, and Windows, you can also install binary wheels from PyPI with pip:
pip install pyarrow
If you encounter any issues importing the pip wheels on Windows, you may need to install the latest Visual C++ Redistributable for Visual Studio.
Development
See Python Development in the documentation subproject.
Building the documentation
See documentation build instructions in the documentation subproject.