### Rationale for this change
Three links in the docs return 404. Each one is the reference target for advice the page is giving, so following it leaves the reader with nothing:
- the checkstyle config a Java contributor is told to follow,
- the conda install guide in the Crossbow setup instructions,
- the ROCm reference in the C device data interface spec.
### What changes are included in this PR?
Three URL strings.
- `developers/java/development.rst`: `apache/arrow/blob/main/java/dev/checkstyle/checkstyle.xml` becomes `apache/arrow-java/blob/main/dev/checkstyle/checkstyle.xml`. The file moved with the Java split and the `java/` path segment went with it; it is 10,851 bytes on `arrow-java@ main`. This was the last remaining `apache/arrow/.../java/...` link under `docs/source`.
- `developers/continuous_integration/crossbow.rst`: `conda.io/docs/user-guide/install/index.html` becomes `docs.conda.io/projects/conda/en/latest/user-guide/install/index.html`.
- `format/CDeviceDataInterface.rst`: the `_ROCm:` target becomes `amd.com/en/products/software/rocm.html`, since AMD retired `en/graphics/servers-solutions-rocm`.
### Are these changes tested?
Not by the test suite; nothing outside `docs/source` changed and no code is touched. Each old URL was requested and returns 404, each new one returns 200, and the two GitHub paths were checked through the contents API rather than the HTML pages, because github.com rate-limits a bulk sweep and a 429 is indistinguishable from a 404. I did not build the docs locally.
### Are there any user-facing changes?
Documentation only. Three links that were broken now resolve.
Disclosure: prepared with AI assistance (Claude, via Claude Code), per the AI-generated Code Guidance in the contributing overview. I reviewed the diff; it introduces no third-party material.
* GitHub Issue: #51148
Authored-by: Roshan Ramani <roshanramani.dev@gmail.com>
Signed-off-by: Nic Crane <thisisnic@gmail.com>
### Rationale for this change
The docs don't build cleanly for me locally and some warnings and errors that are consequential are being ignored. Some of these issues ended up being as severe as entirely missing tables.
### What changes are included in this PR?
- Fixes for each issue I ran into. Each change is a commit with whatever extra detail I needed included in the commit body.
- This also includes an auto-update of our Doxyfile which silences a number of deprecation warnings. I took a look at the diff and it seemed pretty safe and the docs look good to me locally. We can do a preview in CI first though.
### Are these changes tested?
Yes, each change was tested locally.
### Are there any user-facing changes?
No.
* GitHub Issue: #46520
Authored-by: Bryce Mecum <petridish@gmail.com>
Signed-off-by: Bryce Mecum <petridish@gmail.com>
### Rationale for this change
This is a follow-on to https://github.com/apache/arrow/pull/43566 which was merged earlier today. See https://github.com/apache/arrow/issues/43570 for more detail on the entire set of changes.
### What changes are included in this PR?
- Replace instances where we call or document calling `docker-compose` with `docker compose`.
### Are these changes tested?
Yes.
### Are there any user-facing changes?
This PR includes the require user-facing changes to document things.
* GitHub Issue: #43570
Lead-authored-by: Bryce Mecum <petridish@gmail.com>
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
### Rationale for this change
This change allows to use a different JDK version for tests than the one used to build the project.
### What changes are included in this PR?
Provided a new property `arrow.test.jdk-version` which specify a JDK version to be used by surefire/failsafe plugins instead of the version used to execute Maven.
As part of the change, also add a Java version for `TestOpens` to only be executed if Java runtime version is 16 or greater
Also add a Testing section to the Java developer documentation
### Are these changes tested?
via CI/CD
### Are there any user-facing changes?
New build property `arrow.test.jdk-version` allows developers to specify the JDK version used for tests
* GitHub Issue: #43380
Lead-authored-by: Laurent Goujon <laurent@apache.org>
Co-authored-by: Laurent Goujon <laurentgo@users.noreply.github.com>
Co-authored-by: David Li <li.davidm96@gmail.com>
Co-authored-by: Dane Pitkin <dpitkin.oss@gmail.com>
Signed-off-by: Dane Pitkin <dpitkin@apache.org>
### Rationale for this change
* The JAR staging location is not documented, let's fix that.
### What changes are included in this PR?
* Add Arrow Java Apache Staging URL to docs.
* Remove lingering JDK 8 references.
### Are these changes tested?
Yes - screenshot shared below
### Are there any user-facing changes?
Yes, improved documentation on the Arrow website.
Authored-by: Dane Pitkin <dpitkin@apache.org>
Signed-off-by: Dane Pitkin <dpitkin@apache.org>
### What changes are included in this PR?
* Remove support for Java 8 in Github actions and other CI/CD tasks and make Java 11 now the default version
* Make Java 11 the minimum version required to build and run Arrow by changing the Maven project configuration:
- Change minimum java version and source/target/release compiler properties to 11
- Remove `maven` modules
- Remove jdk11+ profiles and integrate their content into the main section
- Let maven-compiler-plugin process `module-info.java` files and address several declaration issues
- Exclude non modularized modules from javadoc aggregate tasks
- Exclude module-info.class files from shaded jars as it is not representative of the whole content and may actually directly coming from a 3rd party dependency.
* Update documentation
### Are these changes tested?
Through CI/CD.
### Are there any user-facing changes?
Yes. Java 11 is now required to run any Arrow code
**This PR includes breaking changes to public APIs.**
* GitHub Issue: #38051
Authored-by: Laurent Goujon <laurent@apache.org>
Signed-off-by: Dane Pitkin <dpitkin@apache.org>
### Rationale for this change
Adding code style and formatting options for Algorithm module.
### What changes are included in this PR?
Code formatting spotless plugin has been added.
### Are these changes tested?
Yes, but doesn't involve test cases, the plugin itself corrects.
### Are there any user-facing changes?
No
* GitHub Issue: #40819
Lead-authored-by: Vibhatha Abeykoon <vibhatha@gmail.com>
Co-authored-by: Vibhatha Lakmal Abeykoon <vibhatha@users.noreply.github.com>
Co-authored-by: David Li <li.davidm96@gmail.com>
Signed-off-by: David Li <li.davidm96@gmail.com>
### Rationale for this change
This PR is a continuation of errorprone warning fixes.
### What changes are included in this PR?
Fixing warnings suggested by the errorpone module.
- [x] Adapter
- [x] C
- [x] Format
- [x] Maven
- [x] Memory
- [x] Performance
### Are these changes tested?
The existing test cases will be covering them.
### Are there any user-facing changes?
No
* Closes: #39649
Lead-authored-by: Vibhatha Abeykoon <vibhatha@gmail.com>
Co-authored-by: Vibhatha Lakmal Abeykoon <vibhatha@gmail.com>
Co-authored-by: vibhatha <vibhatha@gmail.com>
Co-authored-by: Vibhatha Lakmal Abeykoon <vibhatha@users.noreply.github.com>
Co-authored-by: David Li <li.davidm96@gmail.com>
Signed-off-by: David Li <li.davidm96@gmail.com>
### Rationale for this change
https://github.com/apache/arrow/issues/41611
### What changes are included in this PR?
- Update to pre-commit config to enable all checks except `dangling-hyphen`, `line-too-long` by default
- Associated fix docs
### Are these changes tested?
Yes, by building and looking at the docs locally.
### Are there any user-facing changes?
Just docs.
* GitHub Issue: #41611
Authored-by: Bryce Mecum <petridish@gmail.com>
Signed-off-by: AlenkaF <frim.alenka@gmail.com>
### What changes are included in this PR?
This adds developer tooling to the repo for linting the docs by adding the sphinx-lint tool to archery and our pre-commit hooks. In both locations, only two rules are enabled at the moment (Discussed in https://github.com/apache/arrow/pull/40006): `trailing-whitespace` and `missing-final-newline`.
This PR also fixes the individual issues covered by the new tooling.
### Are these changes tested?
Yes, though manually. I tested this works by running `archery lint --docs` and `pre-commit` without and without changes that should get caught by the rules. It works as expected.
### Are there any user-facing changes?
Yes,
1. Developers that use pre-commit hooks will see a change in behavior when they modify docs
2. Developers using archery will see a new --docs option in `archery lint`
3. Developers working on the docs may see CI failures related to the new checks
* Closes: #39990
* GitHub Issue: #39990
Authored-by: Bryce Mecum <petridish@gmail.com>
Signed-off-by: Bryce Mecum <petridish@gmail.com>
### What changes are included in this PR?
To make sure that all pom.xml files follow the same convention regarding indentation and organization, add spotless plugin with sortPom module to check and format the files automatically.
Update Java developer documentation to provide the link to the convention and instructions on how to use spotless
Finally apply the convention to all `pom.xml` files
### Are these changes tested?
No new test as no code change (just formatting)
### Are there any user-facing changes?
None
* GitHub Issue: #41173
Authored-by: Laurent Goujon <laurent@apache.org>
Signed-off-by: David Li <li.davidm96@gmail.com>
### Rationale for this change
Adding documentation for debugging JNI-based Java modules.
### What changes are included in this PR?
Documentation update for developer docs for Java development.
### Are these changes tested?
Locally built the docs and it shows the expected content.
### Are there any user-facing changes?
N/A
* GitHub Issue: #40684
Lead-authored-by: Vibhatha Abeykoon <vibhatha@gmail.com>
Co-authored-by: Vibhatha Lakmal Abeykoon <vibhatha@gmail.com>
Signed-off-by: David Li <li.davidm96@gmail.com>
### Rationale for this change
Modularize remaining modules outside of memory modules, vector, and format.
### What changes are included in this PR?
### Are these changes tested?
Yes, existing unit tests now run with modules when using JDK9+.
### Are there any user-facing changes?
Yes. There are new command-line options that may be necessary. The way of specifying the output directory for
JNI native library builds differs. The flight-grpc module has been eliminated since it is now built into flight-core.
Documentation has been updated for these changes.
**This PR includes breaking changes to public APIs.**
There are a number of package structure changes and some modules now need additional command-line arguments.
* Closes: #39001
Authored-by: James Duong <james.duong@improving.com>
Signed-off-by: David Li <li.davidm96@gmail.com>
### Rationale for this change
### What changes are included in this PR?
Spelling fixes to docs/
### Are these changes tested?
### Are there any user-facing changes?
* Closes: #38950
Lead-authored-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
Signed-off-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
### Rationale for this change
Add a crossbow task group for java, since it is not intuitive to newcomers that "crossbow submit *java*" should be `crossbow submit *java*`.
Also, clarify what is supported/tested in CI under Java build documentation.
### What changes are included in this PR?
* Java crossbow task group added
* Java docs updated to clarify supported JDK versions
### Are these changes tested?
Yes, the crossbow task will be verified in this PR via a github actions request comment.
### Are there any user-facing changes?
No.
Authored-by: Dane Pitkin <dane@voltrondata.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
### Rationale for this change
Use the same JDK vendor for all linux CI jobs and upgrade maven to latest version.
### What changes are included in this PR?
* Eclipse Temurin image for Ubuntu is used for JDK 8 and 11
* maven is upgraded to 3.9.5 for all Ubuntu jobs
### Are these changes tested?
This will be tested in CI.
### Are there any user-facing changes?
No, just a developer change when using docker/archery. The image name is now `java` instead of `eclipse-java` and `debian-java`.
* Closes: #38439
Authored-by: Dane Pitkin <dane@voltrondata.com>
Signed-off-by: Jacob Wujciak-Jens <jacob@wujciak.de>
### Rationale for this change
To disable the debug log when running Java tests
### What changes are included in this PR?
Java Testing resource that configure SLF4J API and adapters.
### Are these changes tested?
Yes
### Are there any user-facing changes?
No
* Closes: #38005
Authored-by: David Susanibar Arce <dsusanibar@Voltron.local>
Signed-off-by: David Li <li.davidm96@gmail.com>
### Rationale for this change
gh-37537 added integration testing for the C Data Interface, but the documentation was not updated.
### What changes are included in this PR?
Add documentation for C Data Interface integration testing.
### Are these changes tested?
N/A, only doc changes.
### Are there any user-facing changes?
No.
* Closes: #37934
Authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Antoine Pitrou <antoine@python.org>
### Rationale for this change
To close: https://github.com/apache/arrow/issues/36927
### What changes are included in this PR?
- Enable MacOS Gandiva build as part of Java maven commands
- Enable Windows ORC build as part of Java maven commands
### Are these changes tested?
Yes:
- Gandiva: mvn generate-resources -Pgenerate-libs-jni-macos-linux -N
- ORC: mvn generate-resources -Pgenerate-libs-jni-windows -N
### Are there any user-facing changes?
No
* Closes: #36927
Lead-authored-by: david dali susanibar arce <davi.sarces@gmail.com>
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
### Rationale for this change
JSON support in the datasets module was recently introduced. Update the maven build profile and build documentation to reflect this. Similar to https://github.com/apache/arrow/pull/36899
### What changes are included in this PR?
* update maven build profile
* update build docs
### Are these changes tested?
Yes, tested locally on M1 Mac.
### Are there any user-facing changes?
No
* Closes: #36920
Authored-by: Dane Pitkin <dane@voltrondata.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
### Rationale for this change
The Java JNI dataset module recently included the Substrait module as a dependency. The dependency was added to the CI scripts, but not added to the build profiles and documentation yet.
### What changes are included in this PR?
- Update maven build profiles
- Update Java build documentation
### Are these changes tested?
I tested locally on MacOS and was able to reproduce + fix with this change.
### Are there any user-facing changes?
No
* Closes: #36885
Authored-by: Dane Pitkin <dane@voltrondata.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
### What changes are included in this PR?
Enable the docs to be built with sphinx 6.x.
- Update the pin in the `requirements.txt`
- Add `sphinxcontrib.jquery` to include jQuery which is used in the dropdown.
### Are these changes tested?
The Sphinx version update and the dropdown are tested locally.
### Are there any user-facing changes?
No.
* Closes: #35403
Lead-authored-by: AlenkaF <frim.alenka@gmail.com>
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
Signed-off-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
This PR adds corrections to the source docs to remove some of the warnings when building the documentation.
* Closes: #35438
Authored-by: Alenka Frim <frim.alenka@gmail.com>
Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com>
### Rationale for this change
We can't maintain Plasma.
### What changes are included in this PR?
These changes remove all Plasma related codes.
### Are these changes tested?
Yes.
### Are there any user-facing changes?
Yes.
* Closes: #33243
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
C Data Interface library (.so / .dylib) able to compile with mvn command:
$ mvn clean generate-resources -Pgenerate-cdata-dylib_so -N
Authored-by: david dali susanibar arce <davi.sarces@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
ASF policies state we shouldn't direct people to
unofficial/developer/nightly releases. Move the instructions to
the developer documentation to make this clearer.
Authored-by: David Li <li.davidm96@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
Add a basic documentation page for the arrow-jdbc adapter.
I would also like to add a Cookbook page, and then cross-link the two pages, as a follow-up.
Authored-by: David Li <li.davidm96@gmail.com>
Signed-off-by: David Li <li.davidm96@gmail.com>
Update os-maven-plugin to make it work with IntelliJ, and move a test file to the proper directory. Also add some more detailed instructions.
Closes#13017 from lidavidm/arrow-15959
Authored-by: David Li <li.davidm96@gmail.com>
Signed-off-by: David Li <li.davidm96@gmail.com>
Changes needed to support JDK17:
- There are some [errorprone](https://errorprone.info/docs/installation) suppress warnings needed to be able to continue using JSE17 with JSE1.8 source/target. This need to be migrated to new JSE X base on [ML discussion](https://lists.apache.org/thread/phpgpydtt3yrgnncdyv4qdq1gf02s0yj).
- JSE11+ is modular for that reason was needed to add `--add-exports` and `--add-opens`
- The --add-exports option allows code in the target module to access types in the named package of the source module if the target module reads the source module.
- If you have to allow code on the class path to do deep reflection to access nonpublic members, then use the --add-opens runtime option.
- To be able to running unit test was needed to configure `maven-surefire-plugin` with `<argLine>--add-opens=java.base/java.nio=ALL-UNNAMED</argLine>`
Closes#12941 from davisusanibar/java-ARROW-15755
Lead-authored-by: david dali susanibar arce <davi.sarces@gmail.com>
Co-authored-by: David Li <li.davidm96@gmail.com>
Signed-off-by: David Li <li.davidm96@gmail.com>
List and describe the environment variables which influence the behaviour of Arrow C++ at runtime.
Closes#12624 from pitrou/ARROW-15617-doc-env-vars-
Authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Antoine Pitrou <antoine@python.org>
Roshan Ramani