mirrors/arrow
0
0 Fork 0
mirror of https://github.com/apache/arrow.git synced 2026-10-29 00:56:31 +08:00
Code Activity

Add more C++ docs

Browse Source
This commit is contained in:
Uwe L. Korn committed 2018-12-05 18:29:37 +01:00
1 parent 78a5eaf4b7
commit fbe99c96cc
7 files changed
+149 -58
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.gitignore
+1
View File
@@ -39,6 +39,7 @@ MANIFEST
cpp/.idea/
cpp/apidoc/xml/
python/.eggs/
python/doc/
.vscode
.idea/
.pytest_cache/
cpp/apidoc/index.md
-57
View File
@@ -41,60 +41,3 @@ Table of Contents
* [Convert a vector of row-wise data into an Arrow table](tutorials/row_wise_conversion.md)
* [Using the Plasma In-Memory Object Store](tutorials/plasma.md)
* [Use Plasma to Access Tensors from C++ in Python](tutorials/tensor_to_py.md)
Getting Started
---------------
The most basic structure in Arrow is an `arrow::Array`. It holds a sequence
of values with known length all having the same type. It consists of the data
itself and an additional bitmap that indicates if the corresponding entry of
array is a null-value. Note that for array with zero null entries, we can omit
this bitmap.
As Arrow objects are immutable, there are classes provided that should help you
build these objects. To build an array of `int64_t` elements, we can use the
`arrow::Int64Builder`. In the following example, we build an array of the range
1 to 8 where the element that should hold the number 4 is nulled.
Int64Builder builder;
builder.Append(1);
builder.Append(2);
builder.Append(3);
builder.AppendNull();
builder.Append(5);
builder.Append(6);
builder.Append(7);
builder.Append(8);
std::shared_ptr<Array> array;
builder.Finish(&array);
The resulting Array (which can be casted to `arrow::Int64Array` if you want
to access its values) then consists of two `arrow::Buffer`. The first one is
the null bitmap holding a single byte with the bits `0|0|0|0|1|0|0|0`.
As we use [least-significant bit (LSB) numbering](https://en.wikipedia.org/wiki/Bit_numbering)
this indicates that the fourth entry in the array is null. The second
buffer is simply an `int64_t` array containing all the above values.
As the fourth entry is null, the value at that position in the buffer is
undefined.
// Cast the Array to its actual type to access its data
std::shared_ptr<Int64Array> int64_array = std::static_pointer_cast<Int64Array>(array);
// Get the pointer to the null bitmap.
const uint8_t* null_bitmap = int64_array->null_bitmap_data();
// Get the pointer to the actual data
const int64_t* data = int64_array->raw_values();
In the above example, we have yet skipped explaining two things in the code.
On constructing the builder, we have passed `arrow::int64()` to it. This is
the type information with which the resulting array will be annotated. In
this simple form, it is solely a `std::shared_ptr<arrow::Int64Type>`
instantiation.
Furthermore, we have passed `arrow::default_memory_pool()` to the constructor.
This `arrow::MemoryPool` is used for the allocations of heap memory. Besides
tracking the amount of memory allocated, the allocator also ensures that the
allocated memory regions are 64-byte aligned (as required by the Arrow
specification).
docs/Dockerfile
+26
View File
@@ -0,0 +1,26 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
FROM arrow:python-3.6
ADD ci/conda_env_sphinx.yml /arrow/ci/
RUN conda install -c conda-forge \
--file arrow/ci/conda_env_sphinx.yml && \
conda clean --all
CMD arrow/ci/docker_build_cpp.sh && \
arrow/ci/docker_build_python.sh && \
arrow/ci/docker_build_sphinx.sh
docs/source/cpp/api.rst
+1
View File
@@ -24,3 +24,4 @@ API Reference
:caption: Getting Started
api/array
api/memory
docs/source/cpp/api/memory.rst
+57
View File
@@ -0,0 +1,57 @@
.. Licensed to the Apache Software Foundation (ASF) under one
.. or more contributor license agreements. See the NOTICE file
.. distributed with this work for additional information
.. regarding copyright ownership. The ASF licenses this file
.. to you under the Apache License, Version 2.0 (the
.. "License"); you may not use this file except in compliance
.. with the License. You may obtain a copy of the License at
.. http://www.apache.org/licenses/LICENSE-2.0
.. Unless required by applicable law or agreed to in writing,
.. software distributed under the License is distributed on an
.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
.. KIND, either express or implied. See the License for the
.. specific language governing permissions and limitations
.. under the License.
Memory (management)
===================
Basic containers
----------------
.. doxygenclass:: arrow::Buffer
:project: arrow_cpp
:members:
.. doxygenclass:: arrow::MutableBuffer
:project: arrow_cpp
:members:
.. doxygenclass:: arrow::ResizableBuffer
:project: arrow_cpp
:members:
.. doxygenclass:: arrow::BufferBuilder
:project: arrow_cpp
:members:
Memory Pools
------------
.. doxygenfunction:: arrow::default_memory_pool
:project: arrow_cpp
:outline:
.. doxygenclass:: arrow::MemoryPool
:project: arrow_cpp
:members:
.. doxygenclass:: arrow::LoggingMemoryPool
:project: arrow_cpp
:members:
.. doxygenclass:: arrow::ProxyMemoryPool
:project: arrow_cpp
:members:
docs/source/cpp/index.rst
+63
View File
@@ -23,3 +23,66 @@ C++ Implementation
:caption: Getting Started
api
Getting Started
---------------
The most basic structure in Arrow is an :cpp:class:`arrow::Array`. It holds a sequence
of values with known length all having the same type. It consists of the data
itself and an additional bitmap that indicates if the corresponding entry of
array is a null-value. Note that for array with zero null entries, we can omit
this bitmap.
As Arrow objects are immutable, there are classes provided that should help you
build these objects. To build an array of ``int64_t`` elements, we can use the
:cpp:class:`arrow::Int64Builder`. In the following example, we build an array of
the range 1 to 8 where the element that should hold the number 4 is nulled.
.. code::
Int64Builder builder;
builder.Append(1);
builder.Append(2);
builder.Append(3);
builder.AppendNull();
builder.Append(5);
builder.Append(6);
builder.Append(7);
builder.Append(8);
std::shared_ptr<Array> array;
builder.Finish(&array);
The resulting Array (which can be casted to :cpp:class:`arrow::Int64Array` if you want
to access its values) then consists of two :cpp:class:`arrow::Buffer`. The first one is
the null bitmap holding a single byte with the bits ``0|0|0|0|1|0|0|0``.
As we use `least-significant bit (LSB) numbering`_.
this indicates that the fourth entry in the array is null. The second
buffer is simply an ``int64_t`` array containing all the above values.
As the fourth entry is null, the value at that position in the buffer is
undefined.
.. code::
// Cast the Array to its actual type to access its data
std::shared_ptr<Int64Array> int64_array = std::static_pointer_cast<Int64Array>(array);
// Get the pointer to the null bitmap.
const uint8_t* null_bitmap = int64_array->null_bitmap_data();
// Get the pointer to the actual data
const int64_t* data = int64_array->raw_values();
In the above example, we have yet skipped explaining two things in the code.
On constructing the builder, we have passed :cpp:func:`arrow::int64()` to it. This is
the type information with which the resulting array will be annotated. In
this simple form, it is solely a :cpp:class:`std::shared_ptr<arrow::Int64Type>`
instantiation.
Furthermore, we have passed :cpp:func:`arrow::default_memory_pool()` to the constructor.
This :cpp:class:`arrow::MemoryPool` is used for the allocations of heap memory. Besides
tracking the amount of memory allocated, the allocator also ensures that the
allocated memory regions are 64-byte aligned (as required by the Arrow
specification).
.. _least-significant bit (LSB) numbering: https://en.wikipedia.org/wiki/Bit_numbering
docs/source/format/README.rst
+1 -1
View File
@@ -24,7 +24,7 @@ Currently, the Arrow specification consists of these pieces:
- Physical memory layout specification (see :doc:`Layout`)
- Logical Types, Schemas, and Record Batch Metadata (see Schema.fbs)
- Encapsulated Messages (see Message.fbs)
- Mechanics of messaging between Arrow systems (IPC, RPC, etc.) (see :doc:`IPC.md`)
- Mechanics of messaging between Arrow systems (IPC, RPC, etc.) (see :doc:`IPC`)
- Tensor (Multi-dimensional array) Metadata (see Tensor.fbs)
The metadata currently uses Google's `flatbuffers library`_ for serializing a