Vector

A high-performance observability data pipeline.

Stars
15K
Forks
1.26K
Open issues
1.68K
Closed issues
5.5K
Last release
8 months ago
Last commit
7 months ago
Watchers
15K
Total releases
70
Total commits
10.8K
Open PRs
91
Closed PRs
9.41K
Repo URL
Project Website
https://vector.dev/
Platform
License
mpl-2.0
Category
Offers premium version?
NO
Proprietary?
NO
About

Quickstart  •   Docs  •   Guides  •   Integrations  •   Chat  •   Download  •   Rust Crate Docs

What is Vector?

Vector is a high-performance, end-to-end (agent & aggregator) observability data pipeline that puts you in control of your observability data. Collect, transform, and route all your logs and metrics to any vendors you want today and any other vendors you may want tomorrow. Vector enables dramatic cost reduction, novel data enrichment, and data security where you need it, not where it is most convenient for your vendors. Additionally, it is open source and up to 10x faster than every alternative in the space.

To get started, follow our quickstart guide or install Vector.

Principles
  • Reliable - Built in Rust, Vector's primary design goal is reliability.
  • End-to-end - Deploys as an agent or aggregator. Vector is a complete platform.
  • Unified - Logs, metrics (beta), and traces (coming soon). One tool for all of your data.
Use cases
  • Reduce total observability costs.
  • Transition vendors without disrupting workflows.
  • Enhance data quality and improve insights.
  • Consolidate agents and eliminate agent fatigue.
  • Improve overall observability performance and reliability.
Community
  • Vector is relied on by startups and enterprises like Atlassian, T-Mobile, Comcast, Zendesk, Discord, Fastly, CVS, Trivago, Tuple, Douban, Visa, Mambu, Blockfi, Claranet, Instacart, Forcepoint, and many more.
  • Vector is downloaded over 100,000 times per day.
  • Vector's largest user processes over 30TB daily.
  • Vector has over 100 contributors and growing.
Documentation About Setup Reference Administration Resources Comparisons Performance

The following performance tests demonstrate baseline performance between common protocols with the exception of the Regex Parsing test.

Test Vector Filebeat FluentBit FluentD Logstash SplunkUF SplunkHF TCP to Blackhole 86mib/s n/a 64.4mib/s 27.7mib/s 40.6mib/s n/a n/a File to TCP 76.7mib/s 7.8mib/s 35mib/s 26.1mib/s 3.1mib/s 40.1mib/s 39mib/s Regex Parsing 13.2mib/s n/a 20.5mib/s 2.6mib/s 4.6mib/s n/a 7.8mib/s TCP to HTTP 26.7mib/s n/a 19.6mib/s <1mib/s 2.7mib/s n/a n/a TCP to TCP 69.9mib/s 5mib/s 67.1mib/s 3.9mib/s 10mib/s 70.4mib/s 7.6mib/s

To learn more about our performance tests, please see the Vector test harness.

Correctness

The following correctness tests are not exhaustive, but they demonstrate fundamental differences in quality and attention to detail:

Test Vector Filebeat FluentBit FluentD Logstash Splunk UF Splunk HF Disk Buffer Persistence ✓ ⚠ ✓ ✓ File Rotate (create) ✓ ✓ ✓ ✓ ✓ ✓ File Rotate (copytruncate) ✓ ✓ File Truncation ✓ ✓ ✓ ✓ ✓ ✓ Process (SIGHUP) ⚠ ✓ ✓ JSON (wrapped) ✓ ✓ ✓ ✓ ✓ ✓

To learn more about our correctness tests, please see the Vector test harness.

Features

Vector is an end-to-end, unified, open data platform.

Vector Beats Fluentbit Fluentd Logstash Splunk UF Splunk HF Telegraf End-to-end ✓ Agent ✓ ✓ ✓ ✓ Aggregator ✓ ✓ ✓ ✓ Unified ✓ Logs ✓ ✓ ✓ ✓ ✓ ✓ ✓ Metrics ⚠ ⚠ ⚠ ⚠ ⚠ ⚠ ✓ Traces 🚧 Open ✓ ✓ ✓ Open-source ✓ ✓ ✓ ✓ ✓ Vendor-neutral ✓ ✓ ✓ Reliability Memory-safe ✓ Delivery guarantees ✓ ✓ Multi-core ✓ ✓ ✓ ✓ ✓ ✓ ✓

⚠ = Not interoperable, metrics are represented as structured logs

Developed with ❤️ by Datadog - Security Policy - Privacy Policy

Alternative Projects

Subscribe to Open Source Businees Newsletter

Twice a month we will interview people behind open source businesses. We will talk about how they are building a business on top of open source projects.

We'll never share your email with anyone else.