Our team would like to implement mutual tls authentication for a pyarrow client, hence modifications to the C++ and Python implementations to at a GrpcMTls scheme for connections.
Closes#7130 from erinleeryan/master
Lead-authored-by: Erin Ryan <erin.ryan@ccri.com>
Co-authored-by: David Li <li.davidm96@gmail.com>
Signed-off-by: David Li <li.davidm96@gmail.com>
The `get` command wouldn't use the adequate TLS root certs when fetching a Flight from its endpoints.
Also fix style in the Python examples and configure `archery lint` to check them.
Closes#6808 from pitrou/ARROW-8304-py-flight-client-tls
Lead-authored-by: Antoine Pitrou <antoine@python.org>
Co-authored-by: Ravindra Wagh <ravindra.wagh@cambridgesemantics.com>
Signed-off-by: Antoine Pitrou <antoine@python.org>
Patch by @ravindra-wagh
(had to redo a new branch and GitHub PR because of git UI crappiness, sorry)
Closes#6479 from pitrou/ARROW-7928-update-py-flight-examples and squashes the following commits:
d54dd7d77 <Antoine Pitrou> Some nits
269590fba <ravindra-wagh> Updated client to upload a csv file to the server
322dace54 <ravindra-wagh> Update server.py
Lead-authored-by: ravindra-wagh <57516622+ravindra-wagh@users.noreply.github.com>
Co-authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: David Li <li.davidm96@gmail.com>
Should fix the sporadic crashes in test_flight.py.
Author: Antoine Pitrou <antoine@python.org>
Closes#4891 from pitrou/ARROW-5930-core-flight and squashes the following commits:
842f1013c <Antoine Pitrou> ARROW-5930: Make Flight server init phase explicit
Also covers [ARROW-4627](https://issues.apache.org/jira/browse/ARROW-4627).
This is quite an enormous change, if preferred, I can do my best to try and separate changes.
Author: David Li <li.davidm96@gmail.com>
Closes#4282 from lihalite/arrow-4626-application-metadata and squashes the following commits:
6f1cd8db7 <David Li> Rework interface for accessing server-sent metadata during DoPut
8fd99cd9b <David Li> Inline CompletableFuture in Flight acceptPut
4cebc543a <David Li> Mark flaky Flight test
c551d8527 <David Li> Fix new CheckStyle violations
85e21699c <David Li> Fix Flight integration tests using metadata
eff22393c <David Li> Use FlightStreamChunk in Flight/C++
72c2a3fa0 <David Li> Try to always close FlightStream after acceptPut
1718d9b42 <David Li> Make FlightStream cancellable from acceptPut
7ac44df79 <David Li> Make Netty version consistent with gRPC
1225b67af <David Li> Use ArrowBuf instead of byte for Flight metadata
ccfef2d1e <David Li> Disable Flight cancellation tests in CI
0484c333c <David Li> Pass Flight context to ListActions in Python
b0f71d967 <David Li> Replace ARROW_EXPORT with ARROW_FLIGHT_EXPORT
fdaa76e99 <David Li> Add client-side cancelation of DoGet operations
b4dbc445e <David Li> Enable non-nested dictionary batches in Flight integration tests
f7631a2fd <David Li> Add basic Arrow Flight docs
a8ac27fb3 <David Li> Implement application metadata in Flight
86f4789ab <David Li> Add application metadata field to FlightData message
Also improve the example a bit.
Author: Antoine Pitrou <antoine@python.org>
Closes#4376 from pitrou/ARROW-5398-fix-py-flight-tests and squashes the following commits:
a15313f60 <Antoine Pitrou> ARROW-5398: Fix Flight tests
I haven't changed the client/server construction interfaces to really take advantage of this. I would rather follow up with creating builders instead, as proposed, as that would be cleaner than special-casing the logic in the current constructors.
Author: David Li <li.davidm96@gmail.com>
Closes#4047 from lihalite/flight-uris and squashes the following commits:
870f6ebb8 <David Li> Add more builder options for Java Flight servers
5c127631b <David Li> Make Python Flight bindings more complete
675acc9bc <David Li> Introduce builder for C++ Flight servers
474419667 <David Li> Use builder for Flight server in Java
460442cea <David Li> Use URIs for Flight locations
Also sets up the server-side APIs for [ARROW-5136](https://issues.apache.org/jira/browse/ARROW-5136).
I mostly mirrored the existing Java APIs in each language, except I didn't add the BasicAuth specialization.
Author: David Li <li.davidm96@gmail.com>
Author: Antoine Pitrou <antoine@python.org>
Closes#4125 from lihalite/arrow-5137-auth and squashes the following commits:
067ff3217 <Antoine Pitrou> Avoid making I/O methods const
48d42485f <David Li> Implement auth API for Flight
14774082f <David Li> Implement auth API for Flight
02a9663ae <David Li> Implement auth API for Flight
Author: Antoine Pitrou <antoine@python.org>
Closes#3770 from pitrou/ARROW-4687-python-flight-sigint and squashes the following commits:
5e548839f <Antoine Pitrou> Use std::atomic<>
8a28061be <Antoine Pitrou> ARROW-4687: Stop Flight server on incoming signals
Author: David Li <David.M.Li@twosigma.com>
Closes#3712 from lihalite/flight-python-server and squashes the following commits:
10dccb674 <David Li> Use 'except *' to handle exceptions in Cython
445eb87cd <David Li> Copy Flight out parameter values instead of moving in Python
9839e0f14 <David Li> Lint Python Flight code
f33b51f74 <David Li> Guard against Flight server methods not filling out parameter
422b4bcd5 <David Li> Bind Flight action endpoints to Python
c09b138de <David Li> Bind Flight ListFlights server call to Python
4a5cbaf52 <David Li> Guard against Flight server returning null listing
033734df6 <David Li> Bind Flight DoGet server call to Python
35c62f694 <David Li> Treat null FlightDataStream as error
- [ ] Add docs
- [ ] Format code
- [ ] Include Python in integration tests (requires binding the JSON reader/writer from C++)
- [ ] Validate performance?
- [ ] Complete server bindings if approach makes sense
Author: David Li <David.M.Li@twosigma.com>
Author: Wes McKinney <wesm+git@apache.org>
Closes#3566 from lihalite/flight-python and squashes the following commits:
ac29ab88 <David Li> Clean up to-be-implemented parts of Flight Python bindings
9d5442a0 <David Li> Clarify various RecordBatchStream{Reader,Writer} wrappers
e1c298ad <David Li> Lint CMake files
77644447 <Wes McKinney> Reformat cmake
c6b02aa9 <David Li> Add basic Python bindings for Flight
