mirror of
https://github.com/apache/arrow.git
synced 2026-10-29 17:15:39 +08:00
Bryce Mecum
0f7e9af437
### What changes are included in this PR? This adds developer tooling to the repo for linting the docs by adding the sphinx-lint tool to archery and our pre-commit hooks. In both locations, only two rules are enabled at the moment (Discussed in https://github.com/apache/arrow/pull/40006): `trailing-whitespace` and `missing-final-newline`. This PR also fixes the individual issues covered by the new tooling. ### Are these changes tested? Yes, though manually. I tested this works by running `archery lint --docs` and `pre-commit` without and without changes that should get caught by the rules. It works as expected. ### Are there any user-facing changes? Yes, 1. Developers that use pre-commit hooks will see a change in behavior when they modify docs 2. Developers using archery will see a new --docs option in `archery lint` 3. Developers working on the docs may see CI failures related to the new checks * Closes: #39990 * GitHub Issue: #39990 Authored-by: Bryce Mecum <petridish@gmail.com> Signed-off-by: Bryce Mecum <petridish@gmail.com>
77 lines
3.8 KiB
ReStructuredText
77 lines
3.8 KiB
ReStructuredText
.. Licensed to the Apache Software Foundation (ASF) under one
|
|
.. or more contributor license agreements. See the NOTICE file
|
|
.. distributed with this work for additional information
|
|
.. regarding copyright ownership. The ASF licenses this file
|
|
.. to you under the Apache License, Version 2.0 (the
|
|
.. "License"); you may not use this file except in compliance
|
|
.. with the License. You may obtain a copy of the License at
|
|
|
|
.. http://www.apache.org/licenses/LICENSE-2.0
|
|
|
|
.. Unless required by applicable law or agreed to in writing,
|
|
.. software distributed under the License is distributed on an
|
|
.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
|
.. KIND, either express or implied. See the License for the
|
|
.. specific language governing permissions and limitations
|
|
.. under the License.
|
|
|
|
Getting Involved
|
|
================
|
|
|
|
Right now the primary audience for Apache Arrow are the developers of data
|
|
systems; most people will use Apache Arrow indirectly through systems that use
|
|
it for internal data handling and interoperating with other Arrow-enabled
|
|
systems.
|
|
|
|
Even if you do not plan to contribute to Apache Arrow itself or Arrow
|
|
integrations in other projects, we'd be happy to have you involved:
|
|
|
|
* Join the mailing list: send an email to
|
|
`dev-subscribe@arrow.apache.org <mailto:dev-subscribe@arrow.apache.org>`_.
|
|
Share your ideas and use cases for the project or read through the
|
|
`Archive <http://mail-archives.apache.org/mod_mbox/arrow-dev/>`_.
|
|
* Follow our activity on `GitHub <https://github.com/apache/arrow/issues>`_
|
|
* Learn the `Format / Specification
|
|
<https://github.com/apache/arrow/tree/main/format>`_
|
|
|
|
|
|
.. _py_arch_overview:
|
|
|
|
PyArrow Architecture
|
|
--------------------
|
|
|
|
PyArrow is for the major part a wrapper around the functionalities that
|
|
Arrow C++ implementation provides. The library tries to take what's available
|
|
in C++ and expose it through a user experience that is more pythonic and
|
|
less complex to use. So while in some cases it might be easy to map what's
|
|
in C++ to what's in Python, in many cases the C++ classes and methods are
|
|
used as foundations to build easier to use entities.
|
|
|
|
.. image:: /python/images/py_arch_overview.svg
|
|
:alt: Four layers of PyArrow architecture: .py, .pyx, .pxd and low level C++ code.
|
|
|
|
* The ``*.py`` files in the pyarrow package are usually where the entities
|
|
exposed to the user are declared. In some cases, those files might directly
|
|
import the entities from inner implementation if they want to expose it
|
|
as is without modification.
|
|
* The ``lib.pyx`` file is where the majority of the core C++ libarrow
|
|
capabilities are exposed to Python. Most of the implementation of this
|
|
module relies on included ``*.pxi`` files where the specific pieces
|
|
are built. While being exposed to Python as ``pyarrow.lib`` its content
|
|
should be considered internal. The public classes are then directly exposed
|
|
in other modules (like ``pyarrow`` itself) by virtue of importing them from
|
|
``pyarrow.lib``
|
|
* The ``_*.pyx`` files are where the glue code is usually created, it puts
|
|
together the C++ capabilities turning it into Python classes and methods.
|
|
They can be considered the internal implementation of the capabilities
|
|
exposed by the ``*.py`` files.
|
|
* The ``includes/*.pxd`` files are where the raw C++ library APIs are declared
|
|
for usage in Cython. Here the C++ classes and methods are declared as they are
|
|
so that in the other ``.pyx`` files they can be used to implement Python classes,
|
|
functions and helpers.
|
|
* Apart from Arrow C++ library, which dependence is mentioned in the previous line,
|
|
PyArrow is also based on PyArrow C++, dedicated pieces of code that live in
|
|
``python/pyarrow/src/arrow/python`` directory and provide the low level
|
|
code for capabilities like converting to and from numpy or pandas and the classes
|
|
that allow to use Python objects and callbacks in C++.
|