trinity-1686a
c6549b50e8
* cargo update * bump lru to 0.19 * bump mockall to 0.15 * bump quanta to 0.13 * bump tikv-jemalloc-ctl and tikv-jemallocator to 0.7 * bump tabled to 0.22 * bump tower-http to 0.7 small breaking change: compression SizeAbove predicate threshold from u16 to u64 * bump ulid to 3.0 Ulid::new() renamed to Ulid::generate() * bump syn to 3.0 and prettyplease to 0.3 * bump vrl to 0.37 diagnostic feature no longer exists and is now the default there are some changes in the vrl language wrt to array and NaN handling * bump multiple opentelemetry deps to 0.33 Bumps opentelemetry, opentelemetry_sdk, opentelemetry-otlp, opentelemetry-appender-tracing to 0.33, tracing-opentelemetry to 0.34 and the pinned metrics-opentelemetry to =0.24033.1. retries are no longer experimental and are built-in by default * bump reqwest to 0.13, reqwest-middleware to 0.5, reqwest-retry to 0.9 there are a few breaking changes, feature renamed, function signature slighly modified reqwest now transitively depends on platform trust store instead of webpki-roots, and moved from ring to aws-lc-crypto * bump opendal and opendal-http-transport-reqwest to 0.59 small breaking change on bulk delete * dd-rust-license-tool * restore proper opendal+reqwest+tls test * use aws-lc instead of ring we were already doing that in tests, no reason to not unify * upgrade tokio again apparently 1.54.0 has an issue with tokio::time::Interval
Open-source search engine for observability (logs, traces, and soon metrics!).
Quickstart | Docs | Tutorials | Chat | Download
We just released Quickwit 0.8! Read the blog post to learn about the latest powerful features!
Quickwit is the fastest search engine on cloud storage. It's the perfect fit for observability use cases
- Log management
- Distributed tracing
- Metrics support is on the roadmap
🚀 Quickstart
- Search and analytics on Stack Overflow dataset
- Trace analytics with Grafana
- Distributed tracing with Jaeger
💡 Features
- Full-text search and aggregation queries
- Elasticsearch-compatible API, use Quickwit with any Elasticsearch or OpenSearch client
- Jaeger-native
- OTEL-native for logs and traces
- Schemaless or strict schema indexing
- Schemaless analytics
- Sub-second search on cloud storage (Amazon S3, Azure Blob Storage, Google Cloud Storage, …)
- Decoupled compute and storage, stateless indexers & searchers
- Grafana data source
- Kubernetes ready - See our helm-chart
- RESTful API
Enterprise ready
- Multiple data sources Kafka / Kinesis / Pulsar native
- Multi-tenancy: indexing with many indexes and partitioning
- Retention policies
- Delete tasks (for GDPR use cases)
- Distributed and highly available* engine that scales out in seconds (*HA indexing only with Kafka)
📑 Architecture overview
- Architecture overview
- Log management
- Distributed traces
📕 Documentation
📚 Resources
🙋 FAQ
How can I switch from Elasticsearch or OpenSearch to Quickwit?
Quickwit supports a large subset of Elasticsearch/OpenSearch API.
For instance, it has an ES-compatible ingest API to make it easier to migrate your log shippers (Vector, Fluent Bit, Syslog, ...) to Quickwit.
On the search side, the most popular Elasticsearch endpoints, query DSL, and even aggregations are supported.
The list of available endpoints and queries is available here, while the list of supported aggregations is available here.
Let us know if part of the API you are using is missing!
If the client you are using is refusing to connect to Quickwit due to missing headers, you can use the extra_headers option in the node configuration to impersonate any compatible version of Elasticsearch or OpenSearch.
How is Quickwit different from traditional search engines like Elasticsearch or Solr?
The core difference and advantage of Quickwit is its architecture built from the ground to search on cloud storage. We optimized IO paths, revamped the index data structures and made search stateless and sub-second on cloud storage.
How does Quickwit compare to Elastic in terms of cost?
We estimate that Quickwit can be up to 10x cheaper on average than Elastic. To understand how, check out our blog post about searching the web on AWS S3.
What license does Quickwit use?
Quickwit is open-source under the Apache License, Version 2.0 - Apache-2.0.
Is it possible to set up Quickwit for a High Availability (HA)?
HA is available for search, for indexing it's available only with a Kafka source.
🤝 Contribute and spread the word
We are always thrilled to receive contributions: code, documentation, issues, or feedback. Here's how you can help us build the future of log management:
- Start by checking out the GitHub issues labeled "Good first issue". These are a great place for newcomers to contribute.
- Read our Contributor Covenant Code of Conduct to understand our community standards.
- Create a fork of Quickwit to have your own copy of the repository where you can make changes.
- To understand how to contribute, read our contributing guide.
- Set up your development environment following our development setup guide.
- Once you've made your changes and tested them, you can contribute by submitting a pull request.
✨ After your contributions are accepted, don't forget to claim your swag by emailing us at hello@quickwit.io. Thank you for contributing!
💬 Join Our Community
We welcome everyone to our community! Whether you're contributing code or just saying hello, we'd love to hear from you. Here's how you can connect with us: