mirror of
https://github.com/apache/arrow.git
synced 2026-10-29 17:15:39 +08:00
Antoine ProuvostandZihan Qi
8509ca4d91
### Rationale for this change We want the ThreadPool default capacity to follow the CPU affinity set by the user, if any. For example: ```console $ python -c "import pyarrow as pa; print(pa.cpu_count())" 24 $ taskset -c 5,6,7 python -c "import pyarrow as pa; print(pa.cpu_count())" 3 ``` ### What changes are included in this PR? - Implement and expose CPU affinity detection as a utility function in `arrow/io_util.h`; on non-Linux platform, it returns `Status::NotImplemented` - Use CPU affinity count, if available, to choose the default ThreadPool capacity (note: based on original changes by Zihan Qi in PR #46034) ### Are these changes tested? By unit tests on CI, and by hand locally. ### Are there any user-facing changes? ThreadPool capacity now follows CPU affinity settings on Linux. * GitHub Issue: #45860 Lead-authored-by: AntoinePrv <AntoinePrv@users.noreply.github.com> Co-authored-by: Zihan Qi <zihan.qi@tum.de> Signed-off-by: Antoine Pitrou <antoine@python.org>
103 lines
4.6 KiB
ReStructuredText
103 lines
4.6 KiB
ReStructuredText
.. Licensed to the Apache Software Foundation (ASF) under one
|
|
.. or more contributor license agreements. See the NOTICE file
|
|
.. distributed with this work for additional information
|
|
.. regarding copyright ownership. The ASF licenses this file
|
|
.. to you under the Apache License, Version 2.0 (the
|
|
.. "License"); you may not use this file except in compliance
|
|
.. with the License. You may obtain a copy of the License at
|
|
|
|
.. http://www.apache.org/licenses/LICENSE-2.0
|
|
|
|
.. Unless required by applicable law or agreed to in writing,
|
|
.. software distributed under the License is distributed on an
|
|
.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
|
.. KIND, either express or implied. See the License for the
|
|
.. specific language governing permissions and limitations
|
|
.. under the License.
|
|
|
|
.. default-domain:: cpp
|
|
.. highlight:: cpp
|
|
|
|
.. _cpp_thread_management:
|
|
|
|
=================
|
|
Thread Management
|
|
=================
|
|
|
|
.. seealso::
|
|
:doc:`Thread management API reference <api/thread>`
|
|
|
|
Thread Pools
|
|
============
|
|
|
|
Many Arrow C++ operations distribute work across multiple threads to take
|
|
advantage of underlying hardware parallelism. For example, when :func:`reading a
|
|
Parquet file<parquet::arrow::FileReader::set_use_threads>` we can decode each
|
|
column in parallel. To achieve this we submit tasks to an executor of some kind.
|
|
|
|
Within Arrow C++ we use thread pools for parallel scheduling and an event loop
|
|
when the user has requested serial execution. It is possible for
|
|
users to provide their own custom implementation, though that is an advanced
|
|
concept and not covered here.
|
|
|
|
CPU vs. I/O
|
|
-----------
|
|
|
|
In order to minimize the overhead of context switches our default thread pool
|
|
for CPU-intensive tasks has a fixed size, defaulting to the process CPU affinity (on Linux) or
|
|
`std::thread::hardware_concurrency <https://en.cppreference.com/w/cpp/thread/thread/hardware_concurrency>`_.
|
|
This means that CPU tasks should never block for long periods of time because this
|
|
will result in under-utilization of the CPU. To achieve this we have a separate
|
|
thread pool which should be used for tasks that need to block. Since these tasks
|
|
are usually associated with I/O operations we call this the I/O thread pool. This
|
|
model is often associated with asynchronous computation.
|
|
|
|
.. _io_thread_pool:
|
|
|
|
The size of the I/O thread pool currently defaults to 8 threads and should
|
|
be sized according to the parallel capabilities of the I/O hardware. For example,
|
|
if most reads and writes occur on a typical HDD then the default of 8 will probably
|
|
be sufficient. On the other hand, when most reads and writes occur on a remote
|
|
filesystem such as S3, it is often possible to benefit from many concurrent reads
|
|
and it may be possible to increase I/O performance by increasing the size of the
|
|
I/O thread pool. The size of the default I/O thread pool can be managed with
|
|
the :envvar:`ARROW_IO_THREADS` environment variable or
|
|
with the :func:`arrow::io::SetIOThreadPoolCapacity` function.
|
|
|
|
Increasing the size of the CPU thread pool is not likely to have any benefit. In
|
|
some cases it may make sense to decrease the size of the CPU thread pool in order
|
|
to reduce the impact that Arrow C++ has on hardware shared with other processes or user
|
|
threads. The size of the default CPU thread pool can be managed with the
|
|
:envvar:`OMP_NUM_THREADS` environment variable or with the
|
|
:func:`arrow::SetCpuThreadPoolCapacity` function.
|
|
|
|
Serial Execution
|
|
----------------
|
|
|
|
Operations in Arrow C++ that may use threads can usually be configured to run serially
|
|
via some kind of parameter. In this case we typically replace the CPU executor with
|
|
an event loop operated by the calling thread. However, many operations will continue
|
|
to use the I/O thread pool. This means that some parallelism may still occur even when
|
|
serial execution is requested.
|
|
|
|
Jemalloc Background Threads
|
|
---------------------------
|
|
|
|
When using the :ref:`jemalloc allocator<cpp_memory_pool>` a small number of
|
|
background threads will be created by jemalloc to manage the pool. These threads
|
|
should have minimal impact but can show up as a memory leak when running analysis
|
|
tools like Valgrind. This is harmless and can be safely suppressed or Arrow C++ can
|
|
be compiled without jemalloc.
|
|
|
|
Asynchronous Utilities
|
|
======================
|
|
|
|
Future
|
|
------
|
|
|
|
Arrow C++ uses :class:`arrow::Future` to communicate results between threads. Typically
|
|
an :class:`arrow::Future` will be created when an operation needs to perform some kind
|
|
of long running task that will block for some period of time. :class:`arrow::Future`
|
|
objects are mainly meant for internal use and any method that returns an
|
|
:class:`arrow::Future` will usually have a synchronous variant as well.
|