### Rationale for this change
CSV and JSON options lack a nice repr/str dunder method
### What changes are included in this PR?
Add both these methods
### Are these changes tested?
Will be once it's ready for review
### Are there any user-facing changes?
No
* GitHub Issue: #47389
Lead-authored-by: Nic Crane <thisisnic@gmail.com>
Co-authored-by: AlenkaF <frim.alenka@gmail.com>
Co-authored-by: Alenka Frim <AlenkaF@users.noreply.github.com>
Signed-off-by: AlenkaF <frim.alenka@gmail.com>
### Rationale for this change
The C++ arrow has a JSON streaming reader which is not exposed on the Python interface.
### What changes are included in this PR?
This PR is based on #33761. It adds the `open_json` method to open a streaming reader for a JSON file.
### Are these changes tested?
Yes
### Are there any user-facing changes?
Yes. A new `open_json` method has been added to the Python interface, located at `pyarrow.json.open_json`, and its parameters are the same as the `pyarrow.json.read_json`
* GitHub Issue: #14932
Lead-authored-by: pxc <panxuchen.pxc@alibaba-inc.com>
Co-authored-by: Akshay Subramanian <asubramanian@grailbio.com>
Signed-off-by: Antoine Pitrou <antoine@python.org>
This PR supports for reading JSON Datasets With Python. As mentioned in [#34216](https://github.com/apache/arrow/issues/34216), only the reading ability are supported.
Please compare the difference between my implemenation of _json.pyx, _json.pyd and _csv.pyx _csv.pyd.
Cause _csv.pyd utilize pointer for cpp class and my implementation doesn't.
**What changes are included in this PR?**
C++: add inclusion for file_json.h
Python: reference C++ codes and support reading JSON Datasets
**Are these changes tested?**
Yes
6 test samples added in tests/test_dataset.py
* Closes: #34216
Lead-authored-by: Junming Chen <junming.chen.r@outlook.com>
Co-authored-by: Weston Pace <weston.pace@gmail.com>
Signed-off-by: Weston Pace <weston.pace@gmail.com>
Use Scenario:
I was using ray.data.read_json to read json data source, in while i got error says that pyarrow.json.ReadOptions & pyarrow.json.ParseOptions does not support pickling so the task could not be passed between workers.
Goal:
Support pickling options class for json.
Closes#12632 from hankvyang/ARROW-15939
Authored-by: hankvyang <hank_yang@sina.cn>
Signed-off-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
Address all docstrings to make sure they pass `archery numpydoc --allow-rule PR01`
Closes#11245 from amol-/ARROW-13637
Authored-by: Alessandro Molina <amol@turbogears.org>
Signed-off-by: Antoine Pitrou <antoine@python.org>
By default, Cython extension classes (defined with "cdef class") don't have a weakref slot, so add one to all of them.
This adds just one memory word to each object, which IMHO is acceptable.
Closes#7758 from pitrou/ARROW-9469-py-weakrefable-objects
Authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Krisztián Szűcs <szucs.krisztian@gmail.com>
Author: Antoine Pitrou <antoine@python.org>
Closes#4521 from pitrou/ARROW-5556-py-json-docs and squashes the following commits:
40064de97 <Antoine Pitrou> Address review comments
65f357a6b <Antoine Pitrou> ARROW-5556: Document JSON reader
