Raúl Cumplido
4ecdfe046e
### Rationale for this change Trying to reduce the size of libarrow.so and remove google-cloud-cpp SDK on some builds. Allow for users to plug and play based on requirements and divide our functionality into cleaner modules. ### What changes are included in this PR? Unconditionally build GCS and the google-cloud-cpp SDK into a different module libarrow_gcs.so outside of libarrow.so. Update bindings to link against the new libarrow_gcs.so library. Update the Linux Package jobs to have the new module into a different package. ### Are these changes tested? Yes via CI ### Are there any user-facing changes? Yes, users will need to either link against libarrow_s3.so or register using LoadFileSystemFactories ### Was AI used for this PR? In accordance to the [AI generation guidelines](https://arrow.apache.org/docs/dev/developers/overview.html#ai-generated-code), please disclose below whether and how AI was used in this PR. **PR code and description written by:** - [x] Human - [x] AI **Reviewed before submission by:** - [x] Human - [ ] AI - [ ] Not reviewed * GitHub Issue: #54902 Authored-by: Raúl Cumplido <raulcd@apache.org> Signed-off-by: Raúl Cumplido <raulcd@apache.org>
Linux packages for Apache Arrow C++ and GLib
Requirements
- Ruby
- Docker
- Tools to build tar.gz for Apache Arrow C++ and GLib
How to build .deb packages for all supported platforms
cd dev/tasks/linux-packages/apache-arrow
rake version:update
rake apt:build
How to build only specific .deb packages for supported platforms
The following command line shows all supported platforms, bear in mind
to execute this command from your root arrow clone folder:
for x in dev/tasks/linux-packages/apache-arrow/apt/{debian,ubuntu}*; do basename $x; done
You can specify target platforms by setting APT_TARGETS:
cd dev/tasks/linux-packages/apache-arrow
rake version:update
rake apt:build APT_TARGETS=debian-bookworm,ubuntu-noble
How to debug .deb packages build
You can use apt:build:console task to debug .deb packages build:
cd dev/tasks/linux-packages/apache-arrow
rake version:update
rake apt:build:console APT_TARGETS=debian-bookworm
It will show a Bash prompt. You can start .deb build by /host/build.sh:
host$ rake apt:build:console APT_TARGETS=debian-bookworm
container$ /host/build.sh
You can keep the Bash session even when the .deb build failed. You can debug in the Bash session.
How to build .rpm packages for all supported platforms
cd dev/tasks/linux-packages/apache-arrow
rake yum:build
How to build only specific .rpm packages for supported platforms
The following command line shows all supported platforms, bear in mind
to execute this command from your root arrow clone folder:
for x in dev/tasks/linux-packages/apache-arrow/yum/{alma,amazon,centos}*; do basename $x; done
You can specify target platforms by setting YUM_TARGETS:
cd dev/tasks/linux-packages/apache-arrow
rake yum:build YUM_TARGETS=almalinux-9,amazon-linux-2023
How to debug .rpm packages build
You can use yum:build:console task to debug .rpm packages build:
cd dev/tasks/linux-packages/apache-arrow
rake yum:build:console YUM_TARGETS=almalinux-9
It will show a Bash prompt. You can start .rpm build by /host/build.sh:
host$ rake yum:build:console YUM_TARGETS=almalinux-9
container$ /host/build.sh
You can keep the Bash session even when the .rpm build failed. You can debug in the Bash session.