### Rationale for this change
We have many deprecated APIs in Python we should slowly remove.
### What changes are included in this PR?
Several deprecation messages are acted on.
### Are these changes tested?
By the existing test suite.
### Are there any user-facing changes?
Yes, some deprecated user-facing APIs are.removed.
### Was AI used for this PR?
In accordance to the [AI generation guidelines](https://arrow.apache.org/docs/dev/developers/overview.html#ai-generated-code), please disclose below whether and how AI was used in this PR.
**PR code and description written by:**
- [ ] Human
- [x] AI
**Reviewed before submission by:**
- [x] Human
- [x] AI
- [ ] Not reviewed
* GitHub Issue: #49153
Authored-by: Rok Mihevc <rok@mihevc.org>
Signed-off-by: Antoine Pitrou <antoine@python.org>
See #38584 for original PR. Will be quoted for this PR description.
### Rationale for this change
> support multi sortkey nulls first.
```
order by i nulls first, j, k nulls first;
```
> The current null sorting only supports all sortkeys, not a certain sortkey, so NullPlacement is extended to the SortKey field. Since the underlying framework is very well written, when modifying this function, you only need to pass the null_placement of each SortKey in. That’s it.
### What changes are included in this PR?
> 1.SortKey structure, NullPlacemnt transfer logic, sorting logic and Ording related, test related
> 2.Substriait related.
> 3.c_glib related.
> 4.SelectK related.
> 5.RankOptions related.
### Are these changes tested?
> yes, I changed the code inside vector_sort_test.cc and performed additional tests.
### Are there any user-facing changes?
> yes, pg database include null sorting of multiple sort keys.
**This PR includes breaking changes to public APIs.** (If there are any breaking changes to public APIs, please explain which changes are breaking. If not, you can remove this.)
I amended the original PR to be less breaking in public APIs.
Still Ordering, SortOptions, RankOptions, and RankQuantileOptions now accept a `std::optional<NullPlacement>` instead of NullPlacement, which did lead to some changes in downstream APIs and bindings. ~I also need some help with fixing the `c_glib` bindings.~
* GitHub Issue: #38558
Lead-authored-by: Alexander Taepper <alexander.taepper@gmail.com>
Co-authored-by: light-city <455954986@qq.com>
Co-authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Antoine Pitrou <antoine@python.org>
### Rationale for this change
To close or discuss #48167
`inverse_permutation` and `scatter` functions got implemented via #44393, PR #44394.
### What changes are included in this PR?
Python tests for `scatter`, `inverse_permutation` kernels and bindings for `InversePermutationOptions` and `ScatterOptions`.
### Are these changes tested?
Yes, tests added in test_compute.py.
### Are there any user-facing changes?
Bindings for `InversePermutationOptions` and `ScatterOptions` are added.
#### This PR includes breaking changes to public APIs.
Options `InversePermutationOptions` changed from accepting parameter
`std::shared_ptr<DataType> output_type = NULLPTR`
to
`std::optional<std::shared_ptr<DataType>> output_type = std::nullopt`
* GitHub Issue: #48167
Lead-authored-by: Tadeja Kadunc <tadeja.kadunc@gmail.com>
Co-authored-by: Rossi Sun <zanmato1984@gmail.com>
Co-authored-by: tadeja <tadeja@users.noreply.github.com>
Signed-off-by: Rossi Sun <zanmato1984@gmail.com>
### What does this PR do?
Adds a new compute kernel `utf8_zero_fill` to Arrow Compute module. This kernel zero-pads UTF-8 strings while preserving leading signs.
### Why is it needed?
Python's `str.zfill()` behavior is useful and expected in many data cleaning scenarios. Arrow lacked a direct equivalent.
### Implementation details
- Introduces `Utf8ZeroFillTransform` with dedicated `ZeroFillOptions`
- Registers `utf8_zero_fill` as a compute kernel
- Adds Python bindings and a clean alias: `utf8_zfill = utf8_zero_fill`
- Includes sign-aware padding logic
### Are these changes tested?
- C++ unit tests: `TestStringKernels, Utf8ZeroFill`
- Python tests: `test_utf8_zfill()`
Ran both tests
* GitHub Issue: #46683
Authored-by: iabhi4 <iamonecool@gmail.com>
Signed-off-by: Antoine Pitrou <pitrou@free.fr>
### Rationale for this change
The current docstring for `pyarrow.compute.scalar()` is incomplete and potentially misleading. It states that it only accepts "a subset of types" when in reality it accepts any value that can be converted to a `pyarrow.Scalar`. Additionally, there's no clear explanation of the difference between `pyarrow.scalar()` and `pyarrow.compute.scalar()`, which can be confusing for users since both functions have the same name but serve different purposes.
### What changes are included in this PR?
This PR updates the docstring for `pyarrow.compute.scalar()` to clarify that it accepts any value convertible to a `pyarrow.Scalar` and add a detailed explanation of the difference between `pyarrow.scalar()` and `pyarrow.compute.scalar()`
### Are these changes tested?
via CI.
### Are there any user-facing changes?
No.
* GitHub Issue: #41985
Lead-authored-by: Chilin Chiou <chilin.chiou@gmail.com>
Co-authored-by: ChiLin Chiu <chilin.chiou@gmail.com>
Co-authored-by: Alenka Frim <AlenkaF@users.noreply.github.com>
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
Signed-off-by: AlenkaF <frim.alenka@gmail.com>
### Rationale for this change
While the `extract_regex` function returns substrings of the matching regex captures, `extract_regex_span` returns (index, length) pairs of these substrings relative to the original string values.
### Are these changes tested?
Yes, by dedicated unit tests.
### Are there any user-facing changes?
No, except a new compute function.
* GitHub Issue: #44615
Lead-authored-by: arash andishgar <arashandishgar1@gmail.com>
Co-authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Antoine Pitrou <antoine@python.org>
### What changes are included in this PR?
* Add "skew" and "kurtosis" scalar aggregate functions
* Add "hash_skew" and "hash_kurtosis" grouped aggregated functions
* Add support for decimal32/decimal64 to "variance", "stddev", "hash_variance" and "hash_stddev"
* Small optimization for "hash_variance" and "hash_stddev": avoid allocating the identity group id mapping when consuming an ExecBatch
### Are these changes tested?
Yes, by included unit tests.
### Are there any user-facing changes?
No, except new APIs.
* GitHub Issue: #45676
Authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Antoine Pitrou <antoine@python.org>
### Rationale for this change
Add "pivot wider" functionality such as [in Pandas](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.pivot.html), through two dedicated functions:
1. a "pivot_wider" scalar aggregate function returning a Struct scalar
2. a "hash_pivot_wider" grouped aggregate function returning a Struct array
Both functions take two arguments (the column of pivot keys and the column of pivot values) and require passing a `PivotWiderOptions` structure with the expected pivot keys, so as to determine the output Struct type.
### Are these changes tested?
Yes, by dedicated unit tests.
### Are there any user-facing changes?
No, just new APIs.
* GitHub Issue: #45269
Authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Antoine Pitrou <antoine@python.org>
### Rationale for this change
`RankQuantileOptions` are currently not exposed on Pyarrow and CI job breaks when `-W error` is used.
### What changes are included in this PR?
Expose `RankQuantileOptions` and test options and kernel from pyarrow.
It also includes some minor refactor for the unwrap sort keys logic to move it into a common function.
### Are these changes tested?
Yes
### Are there any user-facing changes?
The options for the new kernel are exposed on pyarrow.
* GitHub Issue: #45380
Lead-authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Co-authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Antoine Pitrou <antoine@python.org>
### Rationale for this change
In another project, I've been referring to the PyArrow Compute documentation quite a lot, and I believe having more documentation here would be helpful.
I added an example for Index, also to highlight the behavior of `start` and the -1 return if the value isn't found.
### What changes are included in this PR?
Adding an example that should appear on the [pyarrow.compute.index](https://arrow.apache.org/docs/python/generated/pyarrow.compute.index.html#) documentation page.
### Are these changes tested?
I ran this code using the latest version of PyArrow and ensured the outputs were as expected. I did not test building the documentation and rendering, although I would like to start doing this and learn more about this process for future documentation purposes.
### Are there any user-facing changes?
A new example would be available on the website.
Authored-by: Tyler White <50381805+IndexSeek@users.noreply.github.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
### Rationale for this change
Expose recursive flatten for logical lists on list_flatten kernel function and pyarrow bindings.
### What changes are included in this PR?
1. Expose recursive flatten for logical lists on `list_flatten` kernel function
2. Support [Large]ListView for some kernel functions: `list_flatten`,`list_value_length`, `list_element`
3. Support recursive flatten for pyarrow bindinds and simplify [Large]ListView's pyarrow bindings
4. Refactor vector_nested_test.cc for better support [Large]ListView types.
### Are these changes tested?
Yes
### Are there any user-facing changes?
Yes.
1. Some kernel functions like: list_flatten, list_value_length, list_element would support [Large]ListView types
2. `list_flatten` and related pyarrow bindings could support flatten recursively with an ListFlattenOptions.
* GitHub Issue: #41183
Lead-authored-by: ZhangHuiGui <hugo.zhang@openpie.com>
Co-authored-by: ZhangHuiGui <2689496754@qq.com>
Signed-off-by: Felipe Oliveira Carvalho <felipekde@gmail.com>
### Rationale for this change
In Arrow compute, there are four main types of functions: Scalar, Vector, ScalarAggregate and HashAggregate.
Some of the previous work added support for Scalar, ScalarAggregate(https://github.com/apache/arrow/issues/35515) and HashAggregate(https://github.com/apache/arrow/issues/36252). I think it makes sense to add support for vector function as well to complete all non-decomposable UDF kernel support.
Internally, we plan to extend Acero to implement a "SegmentVectorNode" which would use this API to invoke vector on a segment by segment basis, which will allow to use constant memory to compute things like "rank the value across all rows per segment using a python UDF".
### What changes are included in this PR?
The change includes is very similar to the support for aggregate function, which includes code to register the vector UDF, and a kernel that invokes the vector UDF on given inputs.
### Are these changes tested?
Yes. Added new test.
### Are there any user-facing changes?
Yes. This adds an user-facing API to register the vector function.
* Closes: #36672
Authored-by: Li Jin <ice.xelloss@gmail.com>
Signed-off-by: Li Jin <ice.xelloss@gmail.com>
### Rationale for this change
Add a `pairwise_diff` function similar to pandas' [Series.Diff](https://pandas.pydata.org/docs/reference/api/pandas.Series.diff.html), the function computes the first order difference of an array.
### What changes are included in this PR?
I followed [these instructions](https://github.com/apache/arrow/pull/12460#issuecomment-1057520554). The function is implemented for numerical, temporal and decimal types. Chuck arrays are not yet supported.
### Are these changes tested?
Yes. They are tested in vector_pairwise_test.cc and in python/pyarrow/tests/compute.py.
### Are there any user-facing changes?
Yes, and docs are also updated in this PR.
* Closes: #35786
Lead-authored-by: Jin Shang <shangjin1997@gmail.com>
Co-authored-by: Benjamin Kietzman <bengilgit@gmail.com>
Signed-off-by: Benjamin Kietzman <bengilgit@gmail.com>
### Rationale for this change
Implement cumulative prod, max and min compute functions
### What changes are included in this PR?
1. Add implementations, docs and tests for the three functions.
2. Refactor `CumulativeSumOptions` to `CumulativeOptions` for reusability.
3. Fix a bug where `GenericFromScalar(GenericToScalar(std::nullopt)) != std::nullopt`.
4. Remove an unnecessary Cast with the default start value.
5. Add tests to check behavior with `NaN`.
I'll explain some of the changes in comments.
### Are these changes tested?
Yes, in vector_accumulative_ops_test.cc and test_compute.py
### Are there any user-facing changes?
No. The data members of `CumulativeSumOptions` are changed, but the member functions behave as before. And std::optional<T> also can be constructed directly from T. So users should not feel any difference.
* Closes: #32190
Lead-authored-by: Jin Shang <shangjin1997@gmail.com>
Co-authored-by: Benjamin Kietzman <bengilgit@gmail.com>
Signed-off-by: Benjamin Kietzman <bengilgit@gmail.com>
### Rationale for this change
Non decomposable aggregation is aggregation that cannot be split into consume/merge/finalize. This is often when the logic rewritten with external python libraries (numpy, pandas, statmodels, etc) and those either cannot be decomposed or not worthy the effect (these are often one-off function instead of reusable one). This PR implements the support for non decomposable aggregation UDFs.
The major issue with non decomposable UDF is that the UDF needs to see all data at once, unlike scalar UDF where UDF only needs to see a batch at a time. This makes non decomposable not so useful as it is same as collect all the data to a pd.DataFrame and apply the UDF on it. However, one very application of non decomposable UDF is with segmented aggregation. To refresh, segmented aggregation works on ordered data and passed one logic chunk at a time (e.g., all data with the same date). With segmented aggregation and non decomposable aggregation UDF, the user can apply any custom aggregation logic over large stream of ordered data, with the memory overhead of a single segment.
### What changes are included in this PR?
This PR is currently WIP and not ready for review.
So far I have implemented the minimal amount of code to make a basic test working but needs clean up, error handling etc.
* [x] First round of self review
* [x] Second round of self review
* [x] Implement and test unary
* [x] Implement and test varargs
* [x] Implement and test Acero support with segmented aggregation
### Are these changes tested?
Added new test calling with compute and acero.
The compute tests calls the aggregation on the full array. The acero test callings the aggregation with segmented aggregation.
### Are there any user-facing changes?
* Closes: #35515
Lead-authored-by: Li Jin <ice.xelloss@gmail.com>
Co-authored-by: Weston Pace <weston.pace@gmail.com>
Signed-off-by: Li Jin <ice.xelloss@gmail.com>
### Rationale for this change
See the linked issue
### What changes are included in this PR?
C++:
* remove all compute/exec/* from libarrow
* rename compute/exec -> acero and make libarrow_acero
* add new ARROW_ACERO option, required if ARROW_DATASET is on
* libarrow_dataset now depends on libarrow_acero
c_glib: add the new libarrow_acero dependency - we disallow building glib without it
python: added PYARROW_BUILD_ACERO, set to on if DATASETS are built
### Are these changes tested?
All the standard tests do work properly.
I manually compiled C++ with:
* no ARROW_ACERO
* ARROW_ACERO and no ARROW_DATASET
* ARROW_ACERO and ARROW_DATASET and no ARROW_SUBSTRAIT
I manually compiled python without ACERO & DATASET and with ACERO and without DATASET
### Are there any user-facing changes?
If users include compute/exec files directly then they'll have to update their code.
* Closes: #15280
Lead-authored-by: Davide Pasetto <dpasetto69@gmail.com>
Co-authored-by: Li Jin <ice.xelloss@gmail.com>
Co-authored-by: Weston Pace <weston.pace@gmail.com>
Signed-off-by: Li Jin <ice.xelloss@gmail.com>
### Rationale for this change
Currently, `pyarrow.array` doesn't accept pyarrow Arrays and this PR adds a check to allow that.
* Closes: #34411
Authored-by: Alenka Frim <frim.alenka@gmail.com>
Signed-off-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
- [x] Add ability to pass 0 or more than 1 target fields via the Aggregate API
- [x] Add support for nullary `count` -- `count(*)`
- [x] Add a n-ary aggregate function to test changes `*`
`*` I implemented a `"covariant(y, x)"` aggregation function and used it to test the Aggregate API changes, but it's not present in this PR now that I intend to focus on passing the CI tests and get a final review
* Closes: #33566
Lead-authored-by: Felipe Oliveira Carvalho <felipekde@gmail.com>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
Co-authored-by: Weston Pace <weston.pace@gmail.com>
Signed-off-by: Weston Pace <weston.pace@gmail.com>
This adds the numpydoc checks GL10, PR04, PR05, RT03, and YD01 to CI and fixes the associated issues in docstrings.
These checks are:
- GL10: reST directives {directives} must be followed by two colons
- PR04: Parameter "{param_name}" has no type
- PR05: Parameter "{param_name}" type should not finish with "."
- RT03: Return value has no description
- YD01: No Yields section found
https://numpydoc.readthedocs.io/en/latest/validation.html
Lead-authored-by: Bryce Mecum <petridish@gmail.com>
Co-authored-by: Will Jones <willjones127@gmail.com>
Signed-off-by: Will Jones <willjones127@gmail.com>
Parent issue: ARROW-16755. Also resolves ARROW-16819
ArraySpan has no shared pointers at all and is much cheaper to pass around, copy, and basically eliminates the current significant overhead associated with ExecBatch ExecBatchIterator.
This PR isn't going to show meaningful performance gains in function or expression evaluation -- that will require implementing a more streamlined expression evaluator that is based on ArraySpan.
This is only an intermediate patch to try to limit the scope of work as much as possible and facilitate follow up PRs. I have a long list of things I would like to do pretty much right away in follow up patches
Some notes:
* The ArraySpan retains pointers to the buffers that were used to populate it because in many places in the existing scalar kernels, we have to "go back" to a `shared_ptr<ArrayData>`
* There are multiple places where having only `const DataType*` or `const Scalar*` would disallow the use of APIs that require either `shared_ptr<DataType>` or `shared_ptr<Scalar>`, so I added `std::enable_shared_from_this` on these classes. I don't know whether this increases the initialization cost of `make_shared<T>` if anyone knows, but I hope that in the future we can remove `std::enable_shared_from_this`. It would be better to have `Scalar::Copy` and `DataType::Copy` methods so this isn't necessary, but rather than try to hack this in this PR, I left this for follow on work
* A few kernels have been refactored to always write into preallocated memory
(IsIn, IndexIn, IsNull, IsValid)
* Some internal APIs were best refactored to use ArraySpan, such as
ArrayBuilder::AppendArraySlice, stuff in arrow/util/int_util.h
In the interest of getting this merged sooner rather than later, rather than trying to make everything perfect here let's try to fix any glaring / serious issues that you see otherwise leave many improvements for follow up patches, otherwise any work in the scalar kernels codebase will be blocked.
Authored-by: Wes McKinney <wesm@apache.org>
Signed-off-by: Wes McKinney <wesm@apache.org>
PR for Scalar UDF integration
This is the first phase of UDF integration to Arrow. This version only includes ScalarFunctions.
In future of PRs, Vector UDF (using Arrow VectorFunction), UDTF (user-defined table function)
and Aggregation UDFs will be integrated. This PR includes the following;
- [x] UDF Python Scalar Function registration and usage
- [x] UDF Python Scalar Function Examples
- [x] UDF Python Scalar Function test cases
- [x] UDF C++ Example extended from Compute Function Example
- [x] Added aggregation example (optional to this PR: if required can remove and push in a different PR)
Closes#12590 from vibhatha/arrow-15639
Lead-authored-by: Vibhatha Abeykoon <vibhatha@gmail.com>
Co-authored-by: Vibhatha Lakmal Abeykoon <vibhatha@users.noreply.github.com>
Co-authored-by: Antoine Pitrou <antoine@python.org>
Co-authored-by: Weston Pace <weston.pace@gmail.com>
Co-authored-by: David Li <li.davidm96@gmail.com>
Signed-off-by: Antoine Pitrou <antoine@python.org>
This implements the `Table.join` method and the underlying infrastructure.
It provides the `tables_join` function that wraps the `execplan` machinery and tests to verify that joins work as expected.
Closes#12452 from amol-/ARROW-14292
Lead-authored-by: Alessandro Molina <amol@turbogears.org>
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
Signed-off-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
Implement a Scalar Kernel to lookup a value for a given key in a `MapArray`, whose type is an alias for `List(Struct(<key>, <item>))`
Todo:
- [x] Add more tests and combinations (Do suggest more)
- [x] `null` maps
- [x] empty maps
- [x] `null` items
- [x] `null` non-empty items
Types tested:
- [x] PhysicalIntegralArrowTypes
- [x] DecimalArrowTypes
- [x] BaseBinaryArrowTypes
- [x] Boolean
- [x] FixedSizeString
- [x] MonthDayNanoInterval
- [x] Template Kernel
Things to consider:
- [x] Looking up `null` keys. (Null keys are not allowed)
Closes#12162 from dhruv9vats/arrow-15089-test
Lead-authored-by: Dhruv Vats <dhruv25vats@gmail.com>
Co-authored-by: Dhruv Vats <dhruv@Dhruvs-MacBook-Pro.local>
Signed-off-by: David Li <li.davidm96@gmail.com>
Add docstrings for parameters of the various function options classes.
Automatically ingest those parameter docstrings into the generated compute function docstrings
(this uses vendored code from `numpydoc`).
Example with the `min_max` parameter docs:
* before:
```
array : Array-like
Argument to compute function
skip_nulls : optional
Parameter for ScalarAggregateOptions constructor. Either `options`
or `skip_nulls` can be passed, but not both at the same time.
min_count : optional
Parameter for ScalarAggregateOptions constructor. Either `options`
or `min_count` can be passed, but not both at the same time.
options : pyarrow.compute.ScalarAggregateOptions, optional
Parameters altering compute function semantics.
memory_pool : pyarrow.MemoryPool, optional
If not passed, will allocate memory from the default memory pool.
```
* after:
```
array : Array-like
Argument to compute function.
skip_nulls : bool, default True
Whether to skip (ignore) nulls in the input.
If False, any null in the input forces the output to null.
min_count : int, default 1
Minimum number of non-null values in the input. If the number
of non-null values is below `min_count`, the output is null.
options : pyarrow.compute.ScalarAggregateOptions, optional
Alternative way of passing options.
memory_pool : pyarrow.MemoryPool, optional
If not passed, will allocate memory from the default memory pool.
```
Closes#12076 from pitrou/ARROW-10317-py-doc-function-options
Authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Antoine Pitrou <antoine@python.org>
Also change the underlying generator as successive seeds (e.g. 0, 1, 2, 3) could produce the same output.
Closes#12054 from pitrou/ARROW-15219-py-random
Authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Antoine Pitrou <antoine@python.org>
Some compute function have required options (because there is no reasonable default value).
Currently they would raise a cryptic error:
```
>>> pc.partition_nth_indices([1,2,3])
Traceback (most recent call last):
...
ArrowInvalid: Attempted to initialize KernelState from null FunctionOptions
```
With this PR, the error becomes slightly more insightful:
```
>>> pc.partition_nth_indices([1,2,3])
Traceback (most recent call last):
...
ArrowInvalid: Function 'partition_nth_indices' cannot be called without options
```
Closes#11961 from pitrou/ARROW-14853-compute-required-options
Authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Antoine Pitrou <antoine@python.org>
This makes compute functions easier to use, for example here the required "pattern" option doesn't need to be passed by name:
```
>>> pc.split_pattern("abacab", "a")
<pyarrow.ListScalar: ['', 'b', 'c', 'b']>
```
... and producing the following doc at the prompt:
```
split_pattern(strings, /, pattern, *, max_splits=-1, reverse=False, options=None, memory_pool=None)
Split string according to separator.
Split each string according to the exact `pattern` defined in
SplitPatternOptions. The output for each string input is a list
of strings.
The maximum number of splits and direction of splitting
(forward, reverse) can optionally be defined in SplitPatternOptions.
Parameters
----------
strings : Array-like or scalar-like
Argument to compute function
pattern : optional
Parameter for SplitPatternOptions constructor. Either `options`
or `pattern` can be passed, but not both at the same time.
max_splits : optional
Parameter for SplitPatternOptions constructor. Either `options`
or `max_splits` can be passed, but not both at the same time.
reverse : optional
Parameter for SplitPatternOptions constructor. Either `options`
or `reverse` can be passed, but not both at the same time.
options : pyarrow.compute.SplitPatternOptions, optional
Parameters altering compute function semantics.
memory_pool : pyarrow.MemoryPool, optional
If not passed, will allocate memory from the default memory pool.
```
Closes#11955 from pitrou/ARROW-10209-compute-pos-args
Authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Antoine Pitrou <antoine@python.org>
Indicate to the user, in the generated function signature, that the given arguments cannot be passed by name.
Before:
```
Signature: pc.add(x, y, *, memory_pool=None)
Docstring:
Add the arguments element-wise.
[...]
```
After:
```
Signature: pc.add(x, y, /, *, memory_pool=None)
Docstring:
Add the arguments element-wise.
[...]
```
Closes#11951 from pitrou/ARROW-1476-compute-positional-only
Authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
In this PR I added `StructFieldOptions` binding and two small typo corrections for `TypeError` message in `compute.py`.
What I haven't managed to solve and would ask for suggestions is the case when options are not specified. Example:
```python
import pyarrow as pa
import pyarrow.compute as pc
a = pa.array([4, 5, 6], type=pa.int64())
b = pa.array(["bar", None, ""])
c = pa.StructArray.from_arrays([a, b], ["a", "b"])
arr = pa.StructArray.from_arrays([a, c], ["a", "c"])
pc.struct_field(arr)
```
This error I get:
```python-traceback
>>> pc.struct_field(arr)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/alenkafrim/repos/arrow/python/pyarrow/compute.py", line 202, in wrapper
return func.call(args, options, memory_pool)
File "pyarrow/_compute.pyx", line 327, in pyarrow._compute.Function.call
result = GetResultValue(
File "pyarrow/error.pxi", line 143, in pyarrow.lib.pyarrow_internal_check_status
return check_status(status)
File "pyarrow/error.pxi", line 99, in pyarrow.lib.check_status
raise ArrowInvalid(message)
pyarrow.lib.ArrowInvalid: Attempted to initialize KernelState from null FunctionOptions
```
For which I tried to add
https://github.com/AlenkaF/arrow/blob/bec0a6beac3415e03400088a3269f3be3fc7150c/python/pyarrow/_compute.pyx#L977-L978
but it doesn't seem to help.
Closes#11765 from AlenkaF/ARROW-14765
Authored-by: Alenka Frim <frim.alenka@gmail.com>
Signed-off-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
Rok Mihevc