mirror of
https://github.com/apache/arrow.git
synced 2026-10-29 00:56:31 +08:00
### Rationale for this change See https://github.com/apache/parquet-format/issues/600 for rationale. ### What changes are included in this PR? This PR adds support for using `TimestampType` to annotate `FIXED_LEN_BYTE_ARRAY(12)` values. It also adds functionality to convert FLBA(12) values to Arrow INT64 timestamps with the following flags/logic to handle overflow: 1. The conversion is guarded behind the `convert_flba_timestamps` property (default true). If false, conversion fails regardless of value. 2. If the FLBA(12) value overflows max int64 or underflows min int64, the `flba_timestamp_clamp_on_overflow` property (default false) is consulted. If false, conversion fails. If true, the value is clamped to min/max int64. ### Are these changes tested? Yes, via unit tests and an e2e test that reads the file added in parquet-testing (https://github.com/apache/parquet-testing/pull/123). ### Are there any user-facing changes? No * GitHub Issue: #50915 Authored-by: Divjot Arora <div.arora@databricks.com> Signed-off-by: Antoine Pitrou <antoine@python.org>