DeepFlow

Application Observability using eBPF

Stars
1.53K
Forks
199
Open issues
101
Closed issues
213
Last release
3 months ago
Last commit
3 months ago
Watchers
1.53K
Total releases
44
Total commits
9.37K
Open PRs
29
Closed PRs
4.31K
Repo URL
Platform
License
apache-2.0
Category
Offers premium version?
NO
Proprietary?
NO
About

Instant Observability for Cloud-Native Applications

Zero Code, Full Stack, eBPF & Wasm

English | 简体中文

What is DeepFlow

The DeepFlow open-source project aims to provide deep observability for complex cloud infrastructures and cloud-native applications. DeepFlow implemented Zero Code data collection with eBPF for metrics, distributed tracing, request logs and function profiling, and is further integrated with SmartEncoding to achieve Full Stack correlation and efficient access to all observability data. With DeepFlow, cloud-native applications automatically gain deep observability, removing the heavy burden of developers continually instrumenting code and providing monitoring and diagnostic capabilities covering everything from code to infrastructure for DevOps/SRE teams.

Key Features

  • Universal Map for Any Service: DeepFlow provides a universal map with Zero Code by eBPF for production environments, including your services in any language, third-party services without code and all cloud-native infrastructure services. In addition to analyzing common protocols, Wasm plugins are supported for your private protocols. Full Stack golden signals of applications and infrastructures are calculated, pinpointing performance bottlenecks at ease.
  • Distributed Tracing for Any Request: Zero Code distributed tracing powered by eBPF supports applications in any language and infrastructures including gateways, service meshes, databases, message queues, DNS and NICs, leaving no blind spots. Full Stack network performance metrics and file I/O events are automatically collected for each Span. Distributed tracing enters a new era: Zero Instrumentation.
  • Continuous Profiling for Any Function: DeepFlow collects profiling data at a cost of below 1% with Zero Code, plots OnCPU/OffCPU function call stack flame graphs, locates Full Stack performance bottleneck in application, library and kernel functions, and automatically relates them to distributed tracing data. DeepFlow can even analyze code performance through network profiling under old version kernels (2.6+).
  • Seamless Integration with Popular Stack: DeepFlow can serve as storage backed for Prometheus, OpenTelemetry, SkyWalking and Pyroscope. It also provides SQL, PromQL and OLTP APIs to work as data source in popular observability stacks. It injects meta tags for all observability signals including cloud resource, K8s container, K8s labels, K8s annotations, CMDB business attributes, etc., eliminating data silos.
  • Performance 10x ClickHouse: SmartEncoding injects standardized and pre-encoded meta tags into all observability data, reducing storage overhead by 10x compared to ClickHouse String or LowCard method. Custom tags and observability data are stored separately, making tags available for almost unlimited dimensions and cardinalities with uncompromised query experience like BigTable.

Documentation

For more information, please visit the documentation website.

Quick start

There are three editions of DeepFlow:

  • DeepFlow Community: for developers
  • DeepFlow Enterprise: for organizations, solving team collaboration problems
  • DeepFlow Cloud: SaaS service, currently in beta

The DeepFlow Community Edition consists of the core components of the Enterprise Edition.

DeepFlow Community

Please refer to the deployment documentation.

At the same time, we have also built a complete DeepFlow Community Demo, welcome to experience it. Login account/password: deepflow/deepflow.

DeepFlow Enterprise

You can visit the DeepFlow Enterprise Demo, currently available in Chinese only.

Compile DeepFlow from Source

Software Architecture

DeepFlow Community Edition consists of two components, Agent and Server. An Agent runs in each K8s node, legacy host and cloud host, and is responsible for AutoMetrics and AutoTracing data collection of all application processes on the host. Server runs in a K8s cluster and provides Agent management, tag injection, data ingest and query services.

Milestones

Here is our future feature plan. Issues and Pull Requests are welcome.

Contact Us

  • Discord:Click here to join our discussion.
  • Twitter:DeepFlow
  • WeChat Group:

Acknowledgments

  • Thanks eBPF, a revolutionary Linux kernel technology.
  • Thanks OpenTelemetry, provides vendor-neutral APIs to collect application telemetry data.

Honors

Alternative Projects

Subscribe to Open Source Businees Newsletter

Twice a month we will interview people behind open source businesses. We will talk about how they are building a business on top of open source projects.

We'll never share your email with anyone else.