mirror of
https://github.com/apache/arrow.git
synced 2026-10-29 17:15:39 +08:00
Alessandro MolinaandRaúl Cumplido
e0ab40d379
Addresses some missing features and usability issues when using PyArrow with Substrait ExtendedExpressions * GitHub Issue: #41692 - [x] Allow passing `BoundExpressions` for `Scanner(columns=X)` instead of a dict of expressions. - [x] Allow passing `BoundExpressions` for `Scanner(filter=X)` so that user doesn't have to distinguish between `Expression` and `BoundExpressions` and can always just use `pyarrow.substrait.deserialize_expressions` - [x] Allow decoding `pyarrow.BoundExpressions` directly from `protobuf.Message`, thus allowing to use substrait-python objects. - [x] Return `memoryview` from methods encoding substrait, so that those can be directly passed to substrait-python (or more in general other python libraries) without a copy being involved. - [x] Allow decoding messages from `memoryview` so that the output of encoding functions can be sent back to dencoding functions. - [x] Allow to encode and decode schemas from substrait - [x] When encoding schemas return the extension types required for a substrait consumer to decode the schema - [x] Handle arrow extension types when decoding a schema - [x] Update docstrings and documentation --------- Co-authored-by: Raúl Cumplido <raulcumplido@gmail.com>
42 lines
1.4 KiB
ReStructuredText
42 lines
1.4 KiB
ReStructuredText
.. Licensed to the Apache Software Foundation (ASF) under one
|
|
.. or more contributor license agreements. See the NOTICE file
|
|
.. distributed with this work for additional information
|
|
.. regarding copyright ownership. The ASF licenses this file
|
|
.. to you under the Apache License, Version 2.0 (the
|
|
.. "License"); you may not use this file except in compliance
|
|
.. with the License. You may obtain a copy of the License at
|
|
|
|
.. http://www.apache.org/licenses/LICENSE-2.0
|
|
|
|
.. Unless required by applicable law or agreed to in writing,
|
|
.. software distributed under the License is distributed on an
|
|
.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
|
.. KIND, either express or implied. See the License for the
|
|
.. specific language governing permissions and limitations
|
|
.. under the License.
|
|
|
|
.. _integration:
|
|
|
|
********************
|
|
PyArrow Integrations
|
|
********************
|
|
|
|
Arrow is designed to be both a framework and an interchange format.
|
|
|
|
Developers can use Arrow to exchange data between various
|
|
technologies and languages without incurring in any extra cost of
|
|
marshalling/unmarshalling the data. The Arrow bindings and Arrow
|
|
native libraries on the various platforms will all understand Arrow data
|
|
natively without the need to decode it.
|
|
|
|
This allows to easily integrate PyArrow with other languages and technologies.
|
|
|
|
.. toctree::
|
|
:maxdepth: 2
|
|
|
|
integration/substrait
|
|
integration/python_r
|
|
integration/python_java
|
|
integration/extending
|
|
integration/cuda
|