mirror of
https://github.com/apache/arrow.git
synced 2026-10-28 16:47:19 +08:00
Sutou Kouhei
ed37b14f40
### Rationale for this change There are some missing features in the Ruby implmentaion for now but we can pass them by skipping some tests in our integration tests. ### What changes are included in this PR? Archery: * Add `--with-ruby` to `archery integration` * Add `archery.integration.tester_ruby.RubyTester` * Add `no_map_field_names_validate` quirk for GH-49415 * Show environment variables too on external command failure because Ruby tester uses environment variables not command line arguments to pass information to integration tester * Use `ARCHERY_INTEGRATION_WITH_CPP=1` instead of `ARROW_INTEGRATION_CPP=ON` like other implementations such as `ARCHERY_INTEGRATION_WITH_GO` Ruby: * Add `red-arrow-format-integration-test` as the test driver * This is not included in `.gem` because this is only for development * Add `ruby/red-arrow-format/lib/arrow-format/integration/` as helpers of the test driver * This is not included in `.gem` because this is only for development * Add `ArrowFormat::Array#empty?` * Add `ArrowFormat::RecordBatch#empty?` * Add `ArrowFormat::NullArray#n_nulls` * `ArrowFormat::*Array#to_a`: Add support for empty case * Fix Apache Arrow decimal <-> `BigDecimal` conversion * `ArrowFormat::Bitmap#each`: Fix a bug that one bit is ignored * Move dictionary ID to `ArrowFormat::DictionaryType` from `ArrowFormat::Field` * Add support for V4 union that has validity bitmap * Add support for no continuation token message for backward compatibility * `ArrowFormat::StreamingReader`: Add support for reading schema without calling `#each` * `ArrowFormat::MapType`: Add support for keys sorted * `ArrowFormat::MapType`: Always use "key"/"value"/"entries" for field names ### Are these changes tested? Yes. ### Are there any user-facing changes? Yes. * GitHub Issue: #49422 Authored-by: Sutou Kouhei <kou@clear-code.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com>
77 lines
2.6 KiB
Bash
Executable File
77 lines
2.6 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
#
|
|
# Licensed to the Apache Software Foundation (ASF) under one
|
|
# or more contributor license agreements. See the NOTICE file
|
|
# distributed with this work for additional information
|
|
# regarding copyright ownership. The ASF licenses this file
|
|
# to you under the Apache License, Version 2.0 (the
|
|
# "License"); you may not use this file except in compliance
|
|
# with the License. You may obtain a copy of the License at
|
|
#
|
|
# http://www.apache.org/licenses/LICENSE-2.0
|
|
#
|
|
# Unless required by applicable law or agreed to in writing,
|
|
# software distributed under the License is distributed on an
|
|
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
|
# KIND, either express or implied. See the License for the
|
|
# specific language governing permissions and limitations
|
|
# under the License.
|
|
|
|
set -e
|
|
|
|
arrow_dir=${1}
|
|
build_dir=${2}
|
|
|
|
gold_dir=$arrow_dir/testing/data/arrow-ipc-stream/integration
|
|
|
|
# For backward compatibility.
|
|
: "${ARROW_INTEGRATION_CPP:=ON}"
|
|
: "${ARCHERY_INTEGRATION_WITH_CPP:=$([ "${ARROW_INTEGRATION_CPP}" = "ON" ] && echo "1" || echo "0")}"
|
|
export ARCHERY_INTEGRATION_WITH_CPP
|
|
: "${ARCHERY_INTEGRATION_WITH_RUBY:=1}"
|
|
export ARCHERY_INTEGRATION_WITH_RUBY
|
|
|
|
: "${ARCHERY_INTEGRATION_TARGET_IMPLEMENTATIONS:=cpp,ruby}"
|
|
export ARCHERY_INTEGRATION_TARGET_IMPLEMENTATIONS
|
|
|
|
. "${arrow_dir}/ci/scripts/util_log.sh"
|
|
|
|
github_actions_group_begin "Integration: Prepare: Archery"
|
|
pip install -e "$arrow_dir/dev/archery[integration]"
|
|
github_actions_group_end
|
|
|
|
github_actions_group_begin "Integration: Prepare: Dependencies"
|
|
# For C Data Interface testing
|
|
if [ "${ARCHERY_INTEGRATION_WITH_DOTNET}" -gt "0" ]; then
|
|
pip install pythonnet
|
|
fi
|
|
if [ "${ARCHERY_INTEGRATION_WITH_JAVA}" -gt "0" ]; then
|
|
pip install jpype1
|
|
fi
|
|
github_actions_group_end
|
|
|
|
export ARROW_BUILD_ROOT=${build_dir}
|
|
export ARROW_JS_ROOT=${build_dir}/js
|
|
|
|
# Get more detailed context on crashes
|
|
export PYTHONFAULTHANDLER=1
|
|
|
|
# Due to how Go reads environment variables, we have to set them from the calling
|
|
# process, or they would get ignored.
|
|
# (see https://forum.golangbridge.org/t/are-godebug-and-other-env-vars-ignored-when-loading-a-go-dll-from-foreign-code/33694)
|
|
export GOMEMLIMIT=200MiB
|
|
export GODEBUG=gctrace=1,clobberfree=1
|
|
|
|
# Rust can be enabled by exporting ARCHERY_INTEGRATION_WITH_RUST=1
|
|
time archery integration \
|
|
--run-c-data \
|
|
--run-ipc \
|
|
--run-flight \
|
|
--gold-dirs="$gold_dir/0.14.1" \
|
|
--gold-dirs="$gold_dir/0.17.1" \
|
|
--gold-dirs="$gold_dir/1.0.0-bigendian" \
|
|
--gold-dirs="$gold_dir/1.0.0-littleendian" \
|
|
--gold-dirs="$gold_dir/2.0.0-compression" \
|
|
--gold-dirs="$gold_dir/4.0.0-shareddict" \
|
|
--gold-dirs="$gold_dir/cpp-21.0.0" \
|