mirror of
https://github.com/apache/arrow.git
synced 2026-10-29 00:56:31 +08:00
### Rationale for this change Closes #50801 As title, this exposes RecordBatchReaderSourceNodeOptions to Python. With it, a hash join can build its hash table once and stream the probe side through as a generator-backed RecordBatchReader, releasing each input chunk as it is consumed, instead of materializing the full probe table up front (table_source) or pinning all fragments for the plan's lifetime (dataset scan). ### Are these changes tested? Yes ### Are there any user-facing changes? Yeah, users can pass RecordBatchReaderSourceNodeOptions into Declaration, but its not a breaking change * GitHub Issue: #50801 Lead-authored-by: You-Cheng Lin <106612301+owenowenisme@users.noreply.github.com> Co-authored-by: You-Cheng Lin <mses010108@gmail.com> Signed-off-by: Rossi Sun <zanmato1984@gmail.com>