mirrors/quickwit
0
0 Fork 0
mirror of https://github.com/quickwit-oss/quickwit.git synced 2026-10-28 10:45:05 +08:00
Code Activity

cleanup CLAUDE.md (#6712)

Browse Source 
one AGENTS.md avoids extra steop

Small edit experiment:

Before:
 Messages
 Total: 20
 User: 1
 Assistant: 8
 Tools: 11 calls, 11 results

 Tokens
 Input: 169,738
   Cached: 136,704 (80.5%)
   Uncached: 33,034
 Output: 722
 Total: 170,460

 Cost
 Total: $0.255

After:
 Messages
 Total: 11
 User: 1
 Assistant: 5
 Tools: 5 calls, 5 results

 Tokens
 Input: 53,103
   Cached: 27,136 (51.1%)
   Uncached: 25,967
 Output: 373
 Total: 53,476

 Cost
 Total: $0.155

Co-authored-by: Pascal Seitz <pascal.seitz@gmail.com>
This commit is contained in:
PSeitz-ddandPascal Seitz authored and GitHub committed 2026-08-20 09:16:47 +00:00
1 parent 50f2b31cc6
commit 9ff4d362dd
3 files changed
+149 -405
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

AGENTS.md
+147
View File
@@ -0,0 +1,147 @@
# Quickwit repository instructions
Rust source code is under `quickwit/`.
## Project
Quickwit is a cloud-native distributed search engine for observability data (logs, traces).
## Repository layout
- `quickwit/`: Rust workspace and REST API tests
- `docs/internals/`: architecture, verification, and design documentation
- `config/`: runtime configurations
- `Makefile`: repository-level development commands
- `docker-compose.yml`: development test services
## Working principles
- Keep changes focused on the requested task.
- Do not add fallback behavior that hides failures in production paths.
- Do not work around incorrect behavior by weakening or manipulating tests.
- Treat unexpected states as errors or violated invariants; do not silently
pretend they succeeded.
- Update relevant documentation when changing public behavior, configuration,
protocols, or architecture.
## Testing
Repository-level commands include:
```bash
make docker-compose-up
make test-all
make docker-compose-down
make typos
```
# Rust workspace (`quickwit/`)
## Architecture Overview
Quickwit is organized as a ~38-crate Rust workspace.
### Key Layers
**Protocol & Types** — `quickwit-proto` defines all gRPC service contracts and message types via protobuf. Service traits are auto-generated.
**Actor System** — `quickwit-actors` is a custom lightweight actor framework. The indexing pipeline is fully actor-based:
```
Source → DocProcessor → Indexer → IndexSerializer → Packager → Uploader → Sequencer → Publisher
```
A parallel merge pipeline runs alongside.
**Search** — `quickwit-search` implements a root-leaf pattern: root servers parse queries and coordinate, leaf servers search their assigned splits in parallel, leaf results are merged at root.
**Storage** — `quickwit-storage` abstracts cloud storage (S3, Azure, GCS, local file, RAM) behind a `Storage` trait.
**Metastore** — `quickwit-metastore` manages index metadata with file-backed (dev) and PostgreSQL (production) backends.
**Cluster** — `quickwit-cluster` uses Chitchat gossip protocol for membership. `quickwit-control-plane` handles indexing task scheduling and placement.
**API Surface** — `quickwit-serve` hosts both REST and gRPC endpoints over the same service traits, plus serves the embedded React UI.
### Core Crates
| Crate | Purpose |
|-------|---------|
| `quickwit-cli` | CLI entry point and binary |
| `quickwit-serve` | REST/gRPC server |
| `quickwit-search` | Distributed search orchestration |
| `quickwit-indexing` | Actor-based indexing pipeline |
| `quickwit-ingest` | Distributed ingestion with replication |
| `quickwit-metastore` | Index metadata storage |
| `quickwit-storage` | Multi-cloud storage abstraction |
| `quickwit-config` | Configuration parsing/validation |
| `quickwit-doc-mapper` | Index schema and document mapping |
| `quickwit-query` | Query DSL parsing (ES-compatible) |
| `quickwit-cluster` | Cluster membership (Chitchat) |
| `quickwit-control-plane` | Indexing task scheduling |
| `quickwit-actors` | Actor framework |
| `quickwit-proto` | Protobuf definitions and gRPC traits |
| `quickwit-common` | Shared utilities and metrics |
| `quickwit-lambda-server` | AWS Lambda leaf search handler |
| `quickwit-lambda-client` | Lambda invocation with auto-deployment |
`quickwit-common` contains shared utilities for metrics, rate-limited logging, reading from environment variables, and more. It also contains `run_cpu_intensive`, which should be used to run CPU-intensive work from Tokio tasks.
When the client is unlikely to match on an error, you can rely on the crate-level error or `anyhow::Error`. If you need to introduce a new error type, use `thiserror`.
### Design Patterns
- **Trait-based services**: `SearchService`, `MetastoreService`, etc. — enables mocking and multiple implementations
- **Feature gates**: Cloud backends (`azure`, `gcs`), message sources (`kafka`, `kinesis`, `pulsar`, `sqs`, `gcp-pubsub`), `postgres` metastore, `multilang` tokenizers
- **Metrics**: `std::sync::LazyLock` statics with `quickwit_common::metrics::*` factories
### Key Dependencies
- **Tantivy**: Search engine library (with `quickwit` feature flag)
- **Tonic/Prost**: gRPC framework and protobuf
- **Tokio**: Async runtime
- **SQLx**: PostgreSQL metastore
## Quickwit Dependencies
When adding a new dependency, update licenses by running `make update-licenses`.
Prefer referring to the crate in the workspace and keep features minimal.
In other words, prefer:
```toml
zip = { workspace = true, default-features = false, features = ["deflate"] }
```
over:
```toml
zip = "2"
```
Run `cargo check` after editing `Cargo.toml` to update `Cargo.lock`.
## Coding Style
- Avoid single-letter variable names except for indices (`i`, `j`, `k`).
- Document all "hidden contracts" (implicit assumptions, invariants, preconditions).
- Try to avoid deep nesting. In particular, prefer early returns.
- Avoid abusing iterator chaining with complex constructs like `.transpose()`.
- Write type names explicitly when it aids readability.
## Checklist After Making Changes
**MUST**:
- [ ] Run `make fmt`.
- [ ] Keep new files under 500 lines (split by responsibility if larger).
- [ ] Ensure tests pass (see below).
- [ ] Update documentation for new public behavior, configuration, protocols, or architecture.
### Testing
- Single crate test: `cargo nextest run -p quickwit-search my_test_name`
- `make test-all` — starts Docker services (LocalStack S3, PostgreSQL, Pub/Sub emulator) and runs the full test suite with `cargo nextest run --all-features --retries 5`.
- `make test-failpoints` — runs failpoint tests only: `cargo nextest run --test failpoints --features fail/failpoints`.
- Docker services: `make docker-compose-up` / `make docker-compose-down` (subset: `DOCKER_SERVICES=kafka,postgres`).
- Integration tests are under `rest-api-tests`; compile and run the `quickwit-cli` binary to have an instance available to test against.
CLAUDE.md
+1 -280
View File
@@ -1,280 +1 @@
# Quickwit Development Guide
## About Quickwit
[Quickwit](https://github.com/quickwit-oss/quickwit) is a cloud-native search engine for observability data (logs, traces, metrics). Key components:
- **Tantivy + Parquet hybrid**: Full-text search via Tantivy, columnar analytics via Parquet
- **Parquet metrics pipeline** (`quickwit-parquet-engine`): DataFusion/Parquet-based analytics (under active development)
- **Three observability signals**: Metrics, traces, and logs — architectural decisions must generalize across all three
See `quickwit/CLAUDE.md` for architecture overview, crate descriptions, and build commands.
## Core Policies
- Execute given tasks fully, avoid TODOs or stubs.
- If TODOs or stubs are absolutely necessary, ensure user is made aware and they are recorded in any resulting plans, phases, or specs.
- Produce code and make decisions that are consistent across metrics, traces, and logs. Metrics is the current priority, then traces, then logs — but decisions should generalize to all three.
- Tests should be holistic: do not work around broken implementations by manipulating tests.
- Follow [CODE_STYLE.md](CODE_STYLE.md) for all coding conventions.
## Known Pitfalls (Update When Claude Misbehaves)
**Add rules here when Claude makes mistakes. This is a living document.**
| Mistake | Correct Behavior | Bug Reference |
|---------|------------------|---------------|
| Adds mock/fallback implementations | Use real dependencies, no error masking | User preference |
| Claims feature works without integration test | Run through the actual REST/gRPC stack | CLAUDE.md policy |
| Uses workarounds to avoid proper setup | **NEVER** workaround — follow the rigorous path (clone deps, fix env, run real tests) | User policy |
| Bypasses production path in tests | **MUST** test through HTTP/gRPC, not internal APIs | CLAUDE.md policy |
| Uses `Path::exists()` | Disallowed by `clippy.toml` — use fallible alternatives | clippy.toml |
| Uses `Option::is_some_and`, `is_none_or`, `xor`, `map_or`, `map_or_else` | Disallowed by `clippy.toml` — use explicit match/if-let instead | clippy.toml |
| Ignores clippy warnings | Run `cargo clippy --workspace --all-features --tests`. Fix warnings or add targeted `#[allow()]` with justification | Code quality |
| Uses `debug_assert` for user-facing validation | Use `Result` errors — debug_assert is silent in release | Code quality |
| Uses `unwrap()` in library code | Use `?` operator or proper error types | Quickwit style |
| File over 500 lines | Split into focused modules by responsibility | Code quality |
| Unnecessary `.clone()` in non-concurrent code | Return `&self` references or `Arc<T>` — cloning is OK in actor/async code for simplicity | Code quality |
| Raw String for new domain types | Prefer existing type aliases (`IndexId`, `SplitId`, `SourceId` from `quickwit-proto`) | Quickwit style |
| Shadowing variable names within a function | Avoid reusing the same variable name (see CODE_STYLE.md) | Quickwit style |
| Uses chained iterators with complex error handling | Use procedural for-loops when chaining hurts readability | Quickwit style |
| Uses `tokio::sync::Mutex` | **FORBIDDEN** — causes data corruption on cancel. Use actor model with message passing | GAP-002 |
| Uses `JoinHandle::abort()` | **FORBIDDEN** — arbitrary cancellation violates invariants. Use `CancellationToken` | GAP-002 |
| Recreates futures in `select!` loops | Use `&mut fut` to resume, not recreate — dropping loses data | GAP-002 |
| Holds locks across await points | Invariant violations on cancel. Use message passing or synchronous critical sections | GAP-002 |
| Silently swallows unexpected state | If a condition "shouldn't happen," return an error or assert — don't silently return Ok. Skipping optional/missing data is fine; pretending a bug didn't occur is not | Code quality |
| Replies to PR review comments as standalone inline comments | Use `gh api repos/.../pulls/{pr}/comments/{codex_comment_id}/replies` (or `in_reply_to_id` in the POST body) so the reply is **threaded under** the original review comment. Standalone inline comments at the same line are NOT replies; they show as separate threads. Verify with the GitHub API that `in_reply_to_id` is set on your reply. | Review hygiene |
## Engineering Priority
**Safety > Performance > Developer Experience**
| Pillar | Location | Purpose |
|--------|----------|---------|
| **Code Quality** | [CODE_STYLE.md](CODE_STYLE.md) + this doc | Coding standards & reliability |
> For formal specs (TLA+, Stateright) and DST pillars, see the verification docs below. These describe the target workflow — implementation is in progress.
## Reliability Rules
```rust
// 1. Use debug_assert! to document invariants
// (Quickwit CODE_STYLE.md endorses this — helps reviewers proofread)
debug_assert!(offset >= HEADER_SIZE, "offset must include header");
debug_assert!(splits.is_sorted_by_key(|s| s.time_range.end));
// 2. Validate inputs at API boundaries (Result, not debug_assert)
if duration.as_nanos() == 0 {
return Err(Error::InvalidParameter("duration must be positive"));
}
// 3. Define explicit limits as constants
const MAX_SEGMENT_SIZE: usize = 256 * 1024 * 1024;
if size > MAX_SEGMENT_SIZE {
return Err(Error::LimitExceeded(...));
}
// 4. No unwrap() in library code — propagate errors
let timestamp = DateTime::from_timestamp(secs, nsecs)
.ok_or_else(|| anyhow!("invalid timestamp: {}", nanos))?;
```
## Testing Through Production Path
**MUST NOT** claim a feature works unless tested through the actual network stack.
```bash
# 1. Start quickwit
cargo run -p quickwit-cli -- run --config ../config/quickwit.yaml
# 2. Ingest via OTLP
# (send logs/traces to localhost:4317)
# 3. Query via REST API
curl http://localhost:7280/api/v1/<index>/search -d '{"query": "*"}'
```
**Bypasses to AVOID**: Testing indexing pipeline without the HTTP/gRPC server, testing search without the REST API layer.
## Repository Layout
```
quickwit/ # Repository root
├── quickwit/ # Main Rust workspace (all crates live here)
│ ├── Cargo.toml # Workspace root
│ ├── CLAUDE.md # Build commands, architecture overview, crate guide
│ ├── Makefile # Build targets (fmt, fix, test-all, build)
│ ├── clippy.toml # Disallowed methods (enforced)
│ ├── rustfmt.toml # Nightly formatter config
│ ├── rust-toolchain.toml # Pinned Rust toolchain
│ ├── scripts/ # License header checks, log format checks
│ └── rest-api-tests/ # Python-based REST API integration tests
├── docs/
│ └── internals/ # Architecture docs
│ ├── adr/ # Architecture Decision Records
│ │ ├── README.md # ADR index
│ │ ├── gaps/ # Design limitations from incidents
│ │ └── deviations/ # Intentional divergences from ADR intent
│ └── specs/
│ └── tla/ # TLA+ specs for protocols and state machines
├── config/ # Runtime YAML configs (quickwit.yaml, etc.)
├── Makefile # Outer orchestration (delegates to quickwit/)
└── docker-compose.yml # Local services (localstack, postgres, kafka, jaeger, etc.)
```
## Architecture Evolution
Quickwit tracks architectural change through three lenses. See `docs/internals/adr/EVOLUTION.md` for the full process.
```
Architecture Evolution
│
┌────────────────────┼────────────────────┐
▼ ▼ ▼
Characteristics Gaps Deviations
(Proactive) (Reactive) (Pragmatic)
"What we need" "What we learned" "What we accepted"
```
| Lens | Location | When to Use |
|------|----------|-------------|
| **Characteristics** | `docs/internals/adr/` | Track cloud-native requirements |
| **Gaps** | `docs/internals/adr/gaps/` | Design limitation from incident/production |
| **Deviations** | `docs/internals/adr/deviations/` | Intentional divergence from ADR intent |
## Common Commands
All Rust commands run from the `quickwit/` subdirectory.
```bash
# Build
cd quickwit && cargo build
# Run all tests (requires Docker services)
# From repo root:
make docker-compose-up
make test-all
# Or from quickwit/:
cargo nextest run --all-features --retries 5
# Run tests for a specific crate
cargo nextest run -p quickwit-indexing --all-features
# Run failpoint tests
cargo nextest run --test failpoints --features fail/failpoints
# Clippy (must pass before commit)
cargo clippy --workspace --all-features --tests
# Format (requires nightly)
cargo +nightly fmt --all
# Auto-fix clippy + format
make fix # from quickwit/
# Check license headers
bash scripts/check_license_headers.sh
# Check log format
bash scripts/check_log_format.sh
# Spellcheck (from repo root)
make typos # or: typos
# REST API integration tests (Python)
cd quickwit/rest-api-tests
pipenv shell && pipenv install
./run_tests.py --engine quickwit
```
## Testing Strategy
### Unit Tests
- Run fast, avoid IO when possible
- Testing private functions is encouraged
- Property-based tests (`proptest`) are welcome — narrow the search space
- Not always deterministic — proptests are fine
### Integration Tests
- `quickwit-integration-tests/`: Rust integration tests exercising the full stack
- `rest-api-tests/`: Python YAML-driven tests for Elasticsearch API compatibility
### Required for CI
- `cargo nextest run --all-features --retries 5` (with Docker services running)
- Failpoint tests: `cargo nextest run --test failpoints --features fail/failpoints`
- `RUST_MIN_STACK=67108864` is set for test runs (64MB stack)
## Docker Services for Testing
```bash
# Start all services (localstack, postgres, kafka, jaeger, etc.)
make docker-compose-up
# Start specific services
make docker-compose-up DOCKER_SERVICES='jaeger,localstack'
# Tear down
make docker-compose-down
```
Environment variables set during test-all:
- `AWS_ACCESS_KEY_ID=ignored`, `AWS_SECRET_ACCESS_KEY=ignored`
- `QW_S3_ENDPOINT=http://localhost:4566` (localstack)
- `QW_S3_FORCE_PATH_STYLE_ACCESS=1`
- `QW_TEST_DATABASE_URL=postgres://quickwit-dev:quickwit-dev@localhost:5432/quickwit-metastore-dev`
## Key Entry Points
| Port | Protocol | Purpose |
|------|----------|---------|
| 7280 | HTTP | Quickwit REST API |
| 7281 | gRPC | Quickwit gRPC services |
| 4317 | gRPC | OTLP ingest |
## Checklist Before Committing
**MUST** (required for merge):
- [ ] `cargo clippy --workspace --all-features --tests` passes with no warnings
- [ ] `cargo +nightly fmt --all -- --check` passes (run `cargo +nightly fmt --all` to fix; applies to **all** changed `.rs` files including tests — CI checks every file, not just lib code)
- [ ] `debug_assert!` for non-obvious invariants
- [ ] No `unwrap()` in library code
- [ ] No silent error ignoring (`let _ =`)
- [ ] New files under 500 lines (split by responsibility if larger)
- [ ] No unnecessary `.clone()` (OK in actor/async code for clarity)
- [ ] Tests through production path (HTTP/gRPC)
- [ ] License headers present (run `bash quickwit/scripts/check_license_headers.sh` — every `.rs`, `.proto`, and `.py` file needs the Apache 2.0 header)
- [ ] Log format correct (run `bash quickwit/scripts/check_log_format.sh`)
- [ ] `typos` passes (spellcheck)
- [ ] `cargo machete` passes (no unused dependencies in Cargo.toml)
- [ ] `cargo doc --no-deps` passes (each PR must compile independently, not just the final stack)
- [ ] Tests pass: `cargo nextest run --all-features`
**SHOULD** (expected unless justified):
- [ ] Functions under 70 lines
- [ ] Explanatory variables for complex expressions
- [ ] Documentation explains "why"
- [ ] Integration test for new API endpoints
## Detailed Documentation
| Topic | Location |
|-------|----------|
| Code style (Quickwit) | [CODE_STYLE.md](CODE_STYLE.md) |
| Rust style patterns | [docs/internals/RUST_STYLE.md](docs/internals/RUST_STYLE.md) |
| Verification & DST | [docs/internals/VERIFICATION.md](docs/internals/VERIFICATION.md) |
| Verification philosophy | [docs/internals/VERIFICATION_STACK.md](docs/internals/VERIFICATION_STACK.md) |
| Simulation workflow | [docs/internals/SIMULATION_FIRST_WORKFLOW.md](docs/internals/SIMULATION_FIRST_WORKFLOW.md) |
| Benchmarking | [docs/internals/BENCHMARKING.md](docs/internals/BENCHMARKING.md) |
| Contributing guide | [CONTRIBUTING.md](CONTRIBUTING.md) |
| ADR index | [docs/internals/adr/README.md](docs/internals/adr/README.md) |
| Architecture evolution | [docs/internals/adr/EVOLUTION.md](docs/internals/adr/EVOLUTION.md) |
| Compaction architecture | [docs/internals/compaction-architecture.md](docs/internals/compaction-architecture.md) |
| Tantivy + Parquet design | [docs/internals/tantivy-parquet-architecture.md](docs/internals/tantivy-parquet-architecture.md) |
| Locality compaction | [docs/internals/locality-compaction/](docs/internals/locality-compaction/) |
| Runtime config | [config/quickwit.yaml](config/quickwit.yaml) |
## References
- [Quickwit](https://github.com/quickwit-oss/quickwit)
- [Tantivy search engine](https://github.com/quickwit-oss/tantivy)
- [Apache DataFusion](https://datafusion.apache.org/)
@AGENTS.md
quickwit/CLAUDE.md
+1 -125
View File
@@ -1,125 +1 @@
## Build & Test Commands
### Formatting & Linting
- **`make fmt`** — Format and validate code (requires nightly toolchain: `rustup toolchain install nightly`):
1. Runs `cargo +nightly fmt`
2. Checks license headers on `.rs`, `.ts`, `.proto` files
3. Enforces log format policy: no trailing punctuation, no uppercase first character in log and error messages
- **`make fix`** — Runs clippy with `--fix`, then `make fmt`, then `make unused-deps`
- **`make unused-deps`** — Detects unused dependencies via `cargo-machete`
Log messages (`info!`, `warn!`, `error!`, `debug!`) must:
- Start with a **lowercase** letter
- Have **no trailing punctuation**
### Testing
- **Single crate test**: `cargo nextest run -p quickwit-search my_test_name`
- **Single test**: `cargo test -p quickwit-common my_test_name`
- **`make test-all`** — Starts Docker services (LocalStack S3, PostgreSQL, Pub/Sub emulator) and runs the full test suite with `cargo nextest run --all-features --retries 5`
- **`make test-failpoints`** — Runs failpoint tests only: `cargo nextest run --test failpoints --features fail/failpoints`
- Docker services: `make docker-compose-up` / `make docker-compose-down` (subset: `DOCKER_SERVICES=kafka,postgres`)
### Building
- **`make doc`** — Generates docs with `cargo doc --all-features` (warnings as errors)
- Rust toolchain: **1.96**
## Code Conventions
### Clippy Disallowed Methods
These methods are banned (see `clippy.toml`):
- `Path::exists` — (use try_exists)
- `Option::is_some_and`, `Option::is_none_or`, `Option::xor`
- `Option::map_or`, `Option::map_or_else` — use `.map(..).unwrap_or(..)` or `let Some(..) else {..}` instead
### Formatting Shortcut
Use `/fmt` to automatically run format checks.
## Architecture Overview
Quickwit is a cloud-native distributed search engine for observability data (logs, traces). It's organized as a ~38-crate Rust workspace.
### Key Layers
**Protocol & Types** — `quickwit-proto` defines all gRPC service contracts and message types via protobuf. Service traits are auto-generated.
**Actor System** — `quickwit-actors` is a custom lightweight actor framework. The indexing pipeline is fully actor-based:
```
Source → DocProcessor → Indexer → IndexSerializer → Packager → Uploader → Sequencer → Publisher
```
A parallel merge pipeline runs alongside.
**Search** — `quickwit-search` implements a root-leaf pattern: root servers parse queries and coordinate, leaf servers search their assigned splits in parallel, leaf results are merged at root.
**Storage** — `quickwit-storage` abstracts cloud storage (S3, Azure, GCS, local file, RAM) behind a `Storage` trait.
**Metastore** — `quickwit-metastore` manages index metadata with file-backed (dev) and PostgreSQL (production) backends.
**Cluster** — `quickwit-cluster` uses Chitchat gossip protocol for membership. `quickwit-control-plane` handles indexing task scheduling and placement.
**API Surface** — `quickwit-serve` hosts both REST and gRPC endpoints over the same service traits, plus serves the embedded React UI.
### Core Crates
| Crate | Purpose |
|-------|---------|
| `quickwit-cli` | CLI entry point and binary |
| `quickwit-serve` | REST/gRPC server |
| `quickwit-search` | Distributed search orchestration |
| `quickwit-indexing` | Actor-based indexing pipeline |
| `quickwit-ingest` | Distributed ingestion with replication |
| `quickwit-metastore` | Index metadata storage |
| `quickwit-storage` | Multi-cloud storage abstraction |
| `quickwit-config` | Configuration parsing/validation |
| `quickwit-doc-mapper` | Index schema and document mapping |
| `quickwit-query` | Query DSL parsing (ES-compatible) |
| `quickwit-cluster` | Cluster membership (Chitchat) |
| `quickwit-control-plane` | Indexing task scheduling |
| `quickwit-actors` | Actor framework |
| `quickwit-proto` | Protobuf definitions and gRPC traits |
| `quickwit-common` | Shared utilities and metrics |
| `quickwit-lambda-server` | AWS Lambda leaf search handler |
| `quickwit-lambda-client` | Lambda invocation with auto-deployment |
quickwit-common contains shared utilities about metrics, rate limited logging, reading from environment variables, etc.
It also contains the `run_cpu_intensive` that should be use to run CPU-intensive tasks from tokio tasks.
When the client is unlikely to match on an error, you can rely on the crate level Error or anyhow::Error. If you need to introduce a new Error type, use thiserror.
### Design Patterns
- **Trait-based services**: `SearchService`, `MetastoreService`, etc. — enables mocking and multiple implementations
- **Feature gates**: Cloud backends (`azure`, `gcs`), message sources (`kafka`, `kinesis`, `pulsar`, `sqs`, `gcp-pubsub`), `postgres` metastore, `multilang` tokenizers
- **Metrics**: `std::sync::LazyLock` statics with `quickwit_common::metrics::*` factories
### Key Dependencies
- **Tantivy**: Search engine library (custom fork)
- **Tonic/Prost**: gRPC framework and protobuf
- **Tokio**: Async runtime
- **SQLx**: PostgreSQL metastore
# Quickwit Claude Guidelines
When adding a new dependency, update license by running `make update-licenses`.
Prefer referring to the crate in workspace.
Make sure to keep features minimal.
In other words, prefer
zip = { workspace = true, default-features = false, features=["deflate"] }
to
zip = "2"
Do not write Cargo.lock yourself.
Instead, just may run `cargo check`, if you have edited Cargo.toml.
It will update `Cargo.lock` with as little changes as possible.
You may also run `cargo update` to update dependencies provide you were explicitly asked to update dependencies.
## Code Formatting
### Quick Fix
Use `/fmt` to automatically run format checks and see issues.
## Coding Style
- Avoid single-letter variable names except for indices (i, j, k)
- Document all "hidden contracts" (implicit assumptions, invariants, preconditions)
- Try to avoid deep nesting. In particular, prefer early return style
- Avoid abusing iterator chaining with complex constructs like `.transpose()`
- Write type names explicitly when it aids readability
- Use `with_capacity` to hint container capacity when size is known
@AGENTS.md