mirror of
https://github.com/milvus-io/milvus.git
synced 2026-10-28 17:05:53 +08:00
Zhen Yeandsijie-ni-0214
3f5e2439df
issue: #40451 - SN query execution: add two-phase search/query RPCs, Growing/Transform MVCC waits, DataVersion-based segment selection, monotonic runtime admission and renewable Up-view leases (default 60s). Reuse QueryNode execution and boost scoring through query-owned growing handles, including the master Arrow retrieve path. - Query resources: atomically capture growing snapshots and register live delivery, replay historical Deletes and fence commit/sealed notifications before Ready. Add minimal Coord resource/load-info RPCs; maintain one shared SN BM25 aggregate, retain growing statistics in memory and read sealed statistics from object storage when needed. - View lifecycle: persist Up topology before reporting, recover views in version order, drain canceled builds before recovery-resource teardown and preserve retryable lifecycle/resource errors. Keep growing and shared IDF versions monotonic across replicas and expose top-level queryView configuration. - WALSummary and message consumption: introduce a manifest-aligned chunk cache populated by reads and writes, coalesce cold loads and scope live notifications while preserving coverage barriers. Retry transient TransformLog reads with cancellable backoff without advancing delivery cursors on failure. Retain every committed Insert summary, resume old subscriptions at the retained lower bound and share immutable Body decoding through singleflight and bounded LRU caching. Review scope: the foundational change titled `enhance: split walsummary, transformlog and vchannel view modules from WAL recovery` belongs to another PR and is excluded from review. Interactions introduced by SN query integration remain in scope. Integration boundaries: new Proxy orchestration, complete Coord scheduling/history, remote QN TransformLog subscriptions, Requery execution and DDL MVCC visibility remain outside this extraction. The minimal load-info RPC reads current metadata; LoadInfoVersion is advisory. Reconciliation of schema, index and load-scope changes when reusing a runtime remains follow-up work. Ready fences a captured event prefix; execution still waits for the request's MVCC. Follow-ups include preparation failure recovery and dependency lifetime across superseded Preparing views, slow-consumer buffering/backpressure, NodeScheduler task priorities and reliable WALSummary GC across restart, remote consumers and owners. Lower-bound resume does not reconstruct skipped Deletes; existing SN retention pins remain active. --------- Signed-off-by: chyezh <chyezh@outlook.com> Signed-off-by: sijie-ni-0214 <sijie.ni@zilliz.com> Co-authored-by: sijie-ni-0214 <sijie.ni@zilliz.com>