Quickwit
Sub-second search & analytics engine on cloud storage
- Project Website
- https://github.com/quickwit-oss/quickwit
- Platform
- License
- other
- Category
- Usecase
- Developer Tools
- Technology
- Python Rust Typescript javascript Shell Dockerfile Makefile HTML SCSS PLpgSQL
- Offers premium version?
- NO
- Proprietary?
- NO
- About
-
Sub-second search & analytics engine on cloud storage
Quickstart | Docs | Tutorials | Chat | Download
We just released Quickwit 0.6! Read the blog post to learn about the latest powerful features!
gharchive-demo.mp4
Quickwit is the fastest search engine on cloud storage. It is a great fit for:- Log management
- Distributed traces
- Any immutable data: conversational data (emails, texts, messaging platforms) & event-based analytics โฆ and more!
๐ก Features
- Full-text search and aggregation queries
- Elasticsearch query language support
- Sub-second search on cloud storage (Amazon S3, Azure Blob Storage, โฆ)
- Decoupled compute and storage, stateless indexers & searchers
- Schemaless or strict schema indexing
- Schemaless analytics
- Grafana data source
- Jaeger-native
- OTEL-native for logs and traces
- Kubernetes ready - See our helm-chart
- RESTful API
Enterprise ready
- Multiple data sources Kafka / Kinesis / Pulsar native
- Multi-tenancy: indexing with many indexes and partitioning
- Retention policies
- Delete tasks (for GDPR use cases)
- Distributed and highly available* engine that scales out in seconds (*HA indexing only with Kafka)
๐ Quickstart
For a quick guide on how to install Quickwit, start a server, add documents to index, and search them - check out our Quickstart guide.
๐ Architecture overview
- Architecture overview
- Log management
- Distributed traces
๐ Documentation
๐ Resources
๐ฎ Roadmap
-
- Distributed and replicated native ingestion
- Local storage caching
-
- Live tail
- SQL
- Security (TLS, authentication, RBAC)
- Alerting
- and more...
Quickwit has an Elasticsearch-compatible Ingest-API to make it easier to migrate your log shippers (Vector, Fluent Bit, Syslog, ...) to Quickwit. However, we only support ES aggregation DSL, the query DSL support is planned for Q2 2023.
How is Quickwit different from traditional search engines like Elasticsearch or Solr?
The core difference and advantage of Quickwit are its architecture built from the ground to search on cloud storage. We optimized IO paths, revamped the index data structures and made search stateless and sub-second on cloud storage.
How does Quickwit compare to Elastic in terms of cost?
We estimate that Quickwit can be up to 10x cheaper on average than Elastic. To understand how, check out our blog post about searching the web on AWS S3.
What license does Quickwit use?
Quickwit is open-source under the GNU Affero General Public License Version 3 - AGPLv3. Fundamentally, this means you are free to use Quickwit for your project if you don't modify Quickwit. However, if you do and you are distributing your modified version to the public, you have to make the modifications public. We also provide a commercial license for enterprises to provide support and a voice on our roadmap.
Is it possible to setup Quickwit for a High Availability (HA)?
HA is available for search, for indexing it's available only with a Kafka source.
What is Quickwit's business model?
Our business model relies on our commercial license. There is no plan to become SaaS soon.
๐ค Contribute and spread the word
We are always thrilled to receive contributions: code, documentation, issues, or feedback. Here's how you can help us build the future of log management:
- Start by checking out the GitHub issues labeled "Good first issue". These are a great place for newcomers to contribute.
- Read our Contributor Covenant Code of Conduct to understand our community standards.
- Create a fork of Quickwit to have your own copy of the repository where you can make changes.
- To understand how to contribute, read our contributing guide.
- Set up your development environment following our development setup guide.
- Once you've made your changes and tested them, you can contribute by submitting a pull request.
โจ After your contributions are accepted, don't forget to claim your swag by emailing us at hello@quickwit.io. Thank you for contributing!
๐ฌ Join Our Community
We welcome everyone to our community! Whether you're contributing code or just saying hello, we'd love to hear from you. Here's how you can connect with us:
-
๐Upstream landscape generation application244
-
AWS Lambda monitoring & debugging platform. Serverless observability & troubleshooting. Serverless monitoring.
-
Blazing fast, instant realtime GraphQL APIs on your DB with fine grained access control, also trigger webhooks on database events.30.5K
-
Lumigo provides Troubleshoot Serverless Applications at the speed of light.
-
Command line tool to locally run and deploy your node.js application to Amazon Lambda1.34K
-
Serverless Container-aware ARchitectures (e.g. Docker in AWS Lambda)589
-
๐ฅ๐ฅ๐ฅ Serverless Devs developer tool ( Serverless Devs ๅผๅ่ ๅทฅๅ ท )1.59K
-
Sigma is a serverless application developer tool; a cloud IDE, which helps you rapidly build, test and deploy serverless applications.
-
Stackery provides intuitive automation, predictable performance, and operational control over server-less applications and infrastructure.
-
Thundra helps answer your toughest questions about the health of your Serverless infrastructure by giving you invaluable insights into your AWS Lambda functions.
-
Argos is a visual testing solution that fits in your workflow to avoid visual regression. Takes screenshots on each commit and be notified if something changes.263
-
Modern and easy to use SQL client for MySQL, Postgres, SQLite, SQL Server, and more. Linux, MacOS, and Windows.14.1K
-
An Enterprise SAML single sign-on service designed as an OAuth 2.0 flow. Integrate SAML and OIDC SSO with just a few lines of code.1.27K
-
An online tool to quickly bundle & minify your projects, while viewing the compressed gzip/brotli bundle size, all running locally on your browser.659
-
Cloudogu Ecosystem is an open platform, which lets you choose how and where your team creates great software. Each service or tool is delivered as a Dลgu, a Docker container, that can be easily integrated in your environment just by pulling it from our registry.42
-
Exoframe is a self-hosted tool that allows simple one-command deployments using Docker1.08K
-
A scalable, fast and 100% open-source feature flag & experimentation management platform1.22K
-
Open Source Feature Flagging and Remote Config Service. Host on-prem or use our hosted version at https://flagsmith.com/3.33K
-
Hestia Control Panel | A lightweight and powerful control panel for the modern web.2.77K
-
Continuous Profiling Platform. Debug performance issues down to a single line of code8.85K
-
The data scientist's open-source choice to scale, assess and maintain natural language data. Treat training data like a software artifact.1.28K
-
A modified web browser that helps in responsive web development. A web developer's must have dev-tool.21.2K
-
Lightweight static analysis for many languages. Find bug variants with patterns that look like source code.9.12K
-
๐ฅ Sturdy is an open-source, real-time, version control platform for startups (https://getsturdy.com)532
-
Open-source IoT Platform - Device management, data collection, processing and visualization.14.7K
-
Developer & translator friendly web-based localization platform1.02K
-
binary releases of VS Code without MS branding/telemetry/licensing22.4K
-
Powerful and flexible web-based server management control panel3.29K
-
Open-source developer platform to turn scripts into workflows and UIs. Open-source alternative to Airplane and Retool.7.53K
Subscribe to Open Source Businees Newsletter
Twice a month we will interview people behind open source businesses. We will talk about how they are building a business on top of open source projects.
We'll never share your email with anyone else.