mirror of
https://github.com/quickwit-oss/quickwit.git
synced 2026-10-28 10:45:05 +08:00
176f0b29f9
* Add metastore read replica role and config
Introduce the `metastore_read_replica` service role and the
`metastore_read_replica_uri` node config option: the foundation for
routing read-only metastore traffic to a PostgreSQL read replica.
- Add `QuickwitService::MetastoreReadReplica`, parsed from
`metastore_read_replica` / `metastore-read-replica`.
- Split `QuickwitService::default_services()` from `supported_services()`
so the new role is opt-in and never enabled implicitly on all-in-one
nodes.
- Add the optional `metastore_read_replica_uri` field (env
`QW_METASTORE_READ_REPLICA_URI`), redacted alongside `metastore_uri`.
- Validate that the role requires a PostgreSQL `metastore_read_replica_uri`
and cannot be co-located with the `metastore` role.
* Support resolving a read-only PostgreSQL metastore
Add the resolution plumbing for connecting to a PostgreSQL read replica
over a read-only connection.
- Add `MetastoreFactoryOptions { read_only }` and thread it through
`MetastoreFactory::resolve`.
- Add `MetastoreResolver::resolve_read_only`, which rejects any non-
PostgreSQL backend.
- Key the PostgreSQL factory cache on `(uri, options)` so the read-write
and read-only clients get distinct connection pools.
- Add `PostgresqlMetastore::new_read_only`: a read-only connection pool
with migrations skipped (the replica is migrated by the primary).
* Serve a read-only metastore for the read replica role
Resolve and expose a metastore gRPC server when the
`metastore_read_replica` role is enabled, backed by a read-only
connection to `metastore_read_replica_uri`.
- The read replica server reuses the metrics + load-shed layers but omits
the control-plane event layers, which only wrap write RPCs.
- A read-replica node is exempted from the control-plane connectivity
wait, like a primary metastore node, so dedicated replica pods start
independently.
- Extract `metastore_max_in_flight_requests` shared by both roles.
* Route search-path metastore reads to the read replica
Add `ReadReplicaRoutingMetastore`, a `MetastoreService` wrapper that
routes the stale-tolerant reads issued by the search and analytics paths
(`index_metadata`, `indexes_metadata`, `list_indexes_metadata`,
`list_splits`, `list_metrics_splits`, `list_sketch_splits`) to read
replica nodes when any are connected, and everything else (writes and
non-hot-path reads) to the primary.
- Routing is decided per request from the read replica balance channel's
live connection set, so it degrades to the primary when no replica is
deployed. The check is a synchronous `watch` read with no borrow held
across an await.
- Wire it into the searcher service and the DataFusion session builder, so
all search (REST, Elasticsearch, gRPC) and metrics analytics benefit,
while REST admin handlers keep read-your-writes against the primary.
* Document the metastore read replica and test read-only resolution
- Document `metastore_read_replica_uri` in the node config reference and
the example `quickwit.yaml`.
- Add a PostgreSQL-gated test that resolves a read-only metastore against
a real database and verifies it serves read RPCs.
* Use a read-only MetastoreReadService trait for read routing
Replace the full `MetastoreService` implementation on
`ReadReplicaRoutingMetastore` (which forced ~45 delegating methods, most
of them writes) with a narrow, read-only `MetastoreReadService` trait —
the read-only subset of the metastore RPCs (`index_metadata`,
`list_indexes_metadata`, `list_splits`, `list_metrics_splits`,
`list_sketch_splits`).
- `MetastoreServiceClient` implements `MetastoreReadService`;
`MetastoreReadServiceClient = Arc<dyn MetastoreReadService>`.
- `ReadReplicaRoutingMetastore` now implements only the 5-method trait, so
writes are excluded at the type level rather than delegated.
- The search and DataFusion read paths take `MetastoreReadServiceClient` /
`&dyn MetastoreReadService`; `list_parquet_splits_*` take
`&dyn MetastoreReadService`. `single_node_search` keeps its concrete
`MetastoreServiceClient` parameter and adapts internally.
Addresses review feedback that the wrapper should expose a Go-style
read-only interface instead of reimplementing the whole service.
* Avoid primary fallback for read replica routing
* Fix clippy warning in read replica config validation
* Add searcher metastore read replica setting
* Remove config URI validation for read replica
* Fix metastore gRPC mount comment
* Update metastore readiness checks
* Require metastore read replica standalone role
* Remove unnecessary unit tests
* Refine metastore read replica wiring
* Fix default service startup in tests
* Simplify metastore read replica validation
* Trim metastore read replica config tests
* Classify read-only Postgres errors
* Fix search test clippy warnings
* Simplify readiness test mock setup
* Remove metastore read service
* Default metastore read replica routing to false
* Update Quickwit env var index test
* Exclude read replica role from TLS test nodes
* Use ensure for metastore checks
* Simplify read replica URI assertion
* Test read-only Postgres writes
* Use service name in metastore connection log
* Use read replica metastore directly
* Fix read replica metastore readiness
* Refactor local metastore server handling
* Validate metastore read replica searcher config
* improve errors
Co-authored-by: Adrien Guillo <adrien.guillo@gmail.com>
* make test deterministic
* move metastore to own file
* Cache PostgreSQL metastore environment flags
* Make read replica readiness selection explicit
* Format PostgreSQL metastore options
* Fix read-only resolver error assertion
* add more logging
* Apply service overrides before config validation
* Label metastore metrics by kind
---------
Co-authored-by: PSeitz-dd <pascal.seitz@datadoghq.com>
Co-authored-by: Adrien Guillo <adrien.guillo@gmail.com>
Co-authored-by: Pascal Seitz <pascal.seitz@gmail.com>