Cortex
A horizontally scalable, highly available, multi-tenant, long term Prometheus.
Stars
5.26K
Forks
772
Open issues
194
Closed issues
1.59K
Last release
5 months ago
Last commit
5 months ago
Watchers
5.26K
Total releases
69
Total commits
4.94K
Open PRs
27
Closed PRs
3.34K
- Project Website
- https://cortexmetrics.io/
- Platform
- License
- apache-2.0
- Category
- Offers premium version?
- NO
- Proprietary?
- NO
- About
-
Cortex: horizontally scalable, highly available, multi-tenant, long term storage for Prometheus.
Cortex provides horizontally scalable, highly available, multi-tenant, long term storage for Prometheus.
- Horizontally scalable: Cortex can run across multiple machines in a cluster, exceeding the throughput and storage of a single machine. This enables you to send the metrics from multiple Prometheus servers to a single Cortex cluster and run "globally aggregated" queries across all data in a single place.
- Highly available: When run in a cluster, Cortex can replicate data between machines. This allows you to survive machine failure without gaps in your graphs.
- Multi-tenant: Cortex can isolate data and queries from multiple different independent Prometheus sources in a single cluster, allowing untrusted parties to share the same cluster.
- Long term storage: Cortex supports S3, GCS, Swift and Microsoft Azure for long term storage of metric data. This allows you to durably store data for longer than the lifetime of any single machine, and use this data for long term capacity planning.
Cortex is a CNCF incubation project used in several production systems including Amazon Managed Service for Prometheus (AMP).
Cortex is primarily used as a remote write destination for Prometheus, with a Prometheus-compatible query API.
Chunk Storage Deprecation Notice
The chunks storage is deprecated since v1.10.0. You're encouraged to use the blocks storage.
Chunks storage was removed in release 1.14.0
Documentation
Read the getting started guide if you're new to the project. Before deploying Cortex with a permanent storage backend you should read:
- An overview of Cortex's architecture
- Getting started with Cortex
- Information regarding configuring Cortex
There are also individual guides to many tasks. Please review the important security advice before deploying.
For a guide to contributing to Cortex, see the contributor guidelines.
Further reading
To learn more about Cortex, consult the following talks and articles.
Talks and articles
- Apr 2023 KubeCon talk "How to Run a Rock Solid Multi-Tenant Prometheus" (video, slides)
- Oct 2022 KubeCon talk "Current State and the Future of Cortex" (video, slides)
- Oct 2021 KubeCon talk "Cortex: Intro and Production Tips" (video, slides)
- Dec 2020 blog post "How AWS and Grafana Labs are scaling Cortex for the cloud"
- Oct 2020 blog post "How to switch Cortex from chunks to blocks storage (and why you won’t look back)"
- Oct 2020 blog post "Now GA: Cortex blocks storage for running Prometheus at scale with reduced operational complexity"
- Sep 2020 blog post "A Tale of Tail Latencies"
- Sep 2020 KubeCon talk "Scaling Prometheus: How We Got Some Thanos Into Cortex" (video, slides)
- Aug 2020 blog post "Scaling Prometheus: How we’re pushing Cortex blocks storage to its limit and beyond"
- Jul 2020 blog post "How blocks storage in Cortex reduces operational complexity for running Prometheus at massive scale"
- Jul 2020 PromCon talk "Sharing is Caring: Leveraging Open Source to Improve Cortex & Thanos" (video, slides)
- Mar 2020 blog post "Cortex: Zone Aware Replication"
- Mar 2020 blog post "How we're using gossip to improve Cortex and Loki availability"
- Jan 2020 blog post "[The Future of Cortex: Into the Next Decade][https://grafana.com/blog/2020/01/21/the-future-of-cortex-into-the-next-decade/]"
- Nov 2019 KubeCon talks "Cortex 101: Horizontally Scalable Long Term Storage for Prometheus" (video, slides), "Configuring Cortex for Max Performance" (video, slides, write up) and "Blazin’ Fast PromQL" (slides, video, write up)
- Nov 2019 PromCon talk "Two Households, Both Alike in Dignity: Cortex and Thanos" (video, slides, write up)
- May 2019 KubeCon talks; "Cortex: Intro" (video, slides, blog post) and "Cortex: Deep Dive" (video, slides)
- Feb 2019 blog post & podcast; "Prometheus Scalability with Bryan Boreham" (podcast)
- Feb 2019 blog post; "How Aspen Mesh Runs Cortex in Production"
- Dec 2018 KubeCon talk; "Cortex: Infinitely Scalable Prometheus" (video, slides)
- Dec 2018 CNCF blog post; "Cortex: a multi-tenant, horizontally scalable Prometheus-as-a-Service"
- Nov 2018 CloudNative London meetup talk; "Cortex: Horizontally Scalable, Highly Available Prometheus" (slides)
- Nov 2018 CNCF TOC Presentation; "Horizontally Scalable, Multi-tenant Prometheus" (slides)
- Sept 2018 blog post; "What is Cortex?"
- Aug 2018 PromCon panel; "Prometheus Long-Term Storage Approaches" (video)
- Jul 2018 design doc; "Cortex Query Optimisations"
- Aug 2017 PromCon talk; "Cortex: Prometheus as a Service, One Year On" (videos, slides, write up part 1, part 2, part 3)
- Jun 2017 Prometheus London meetup talk; "Cortex: open-source, horizontally-scalable, distributed Prometheus" (video)
- Dec 2016 KubeCon talk; "Weave Cortex: Multi-tenant, horizontally scalable Prometheus as a Service" (video, slides)
- Aug 2016 PromCon talk; "Project Frankenstein: Multitenant, Scale-Out Prometheus": (video, slides)
- Jun 2016 design document; "Project Frankenstein: A Multi Tenant, Scale Out Prometheus"
If you have any questions about Cortex:
- Ask a question on the Cortex Slack channel. To invite yourself to the CNCF Slack, visit http://slack.cncf.io/.
- File an issue.
- Send an email to cortex-users@lists.cncf.io
Your feedback is always welcome.
For security issues see https://github.com/cortexproject/cortex/security/policy
Community Meetings
The Cortex community call happens every two weeks on Thursday, alternating at 1200 UTC and 1700 UTC. Meeting notes are held here.
To see meeting calendar:
- See the calendar in your browser (time zone will be UTC).
- If you use Google Calendar, add the Cortex's calendar to your own Google Calendar.
- You can also just download the .ics file.
Hosted Cortex (Prometheus as a service)
There are several commercial services where you can use Cortex on-demand:
Amazon Managed Service for Prometheus (AMP)
Amazon Managed Service for Prometheus (AMP) is a Prometheus-compatible monitoring service that makes it easy to monitor containerized applications at scale. It is a highly available, secure, and managed monitoring for your containers. Get started here. To learn more about the AMP, reference our documentation and Getting Started with AMP blog.
Emeritus Maintainers
- Peter Štibraný @pstibrany
- Marco Pracucci @pracucci
- Bryan Boreham @bboreham
- Goutham Veeramachaneni @gouthamve
- Jacob Lisi @jtlisi
- Tom Wilkie @tomwilkie
History of Cortex
The Cortex project was started by Tom Wilkie (Grafana Labs' VP Product) and Julius Volz (Prometheus' co-founder) in June 2016.
Alternative Projects
-
Sentry is cross-platform application monitoring, with a focus on error reporting.35.7K
-
ML powered analytics engine for outlier//anomaly detection and root cause analysis658
-
Amazon CloudWatch is a monitoring and management service built for developers, system operators, site reliability engineers (SRE), and IT managers. CloudWatch provides you with data and actionable insights to monitor your applications, understand and respond to system-wide performance changes, optimize resource utilization, and get a unified view of operational health.
-
AppDynamics develops application performance management solutions that deliver problem resolution for highly distributed applications.
-
A SaaS-based service that aims to improve the high availability of your applications. Application High Availability Service features automatic application architecture discovery, high availability assessment following principles of chaos engineering, application traffic control, and service fallback.
-
Applications Manager is a multi-platform monitoring tool built for IT admins and DevOps for tracking KPIs of 100+ technologies spanning hybrid clouds, containers as well as traditional environments (web applications, servers, VMs, application servers, databases, big data stores, middleware & messaging components, web servers, web services & ERP suites).
-
AppNeta operates as a SaaS-based application and network performance platform.
-
Simple, powerful infrastructure and application monitoring
-
AppSignal is a powerful APM that works out-of-the-box for Elixir, Ruby, and Node.js.
-
Cloud-native, scalable, high-definition application performance monitoring (APM) for next gen apps.
-
Collect, analyze, and act on telemetry data from your Azure and on-premises environments. Azure Monitor helps you maximize performance and availability of your applications and proactively identify problems in seconds.
-
🐠 Beats - Lightweight shippers for Elasticsearch & Logstash11.9K
-
Automated Monitoring for AWS, Kubernetes, and more
-
An app that helps you monitor your Kubernetes cluster, debug critical deployments & gives recommendations for standard practices1.94K
-
Catchpoint is a digital experience observability company.
-
Centreon is a network, system and application monitoring tool. Centreon is the only AIOps Platform Providing Holistic Visibility to Complex IT Workflows from Cloud to Edge.58
-
Checkmk - Best-in-class infrastructure & application monitoring1.19K
-
Chronosphere develops a scalable, reliable, and customizable monitoring solution built for cloud-native applications.
-
CloudHealth Technologies delivers cloud service management for enterprises to effectively optimize cost, usage, performance, and security.
-
Founded in 2009, Cloudwise is a leading full-stack Artificial Intelligence for IT Operations (AIOps) solution provider in China.
-
Application Observability using eBPF1.53K
-
Dynatrace delivers answers and intelligent automation from data.
-
Epsagon is a privately held, modern observability company with offices in New York and Tel Aviv.
-
An open-source and enterprise-level monitoring system.7.17K
-
Monitor 100% of your distributed application in real time with negligible overhead, no sampling, and no code changes.
-
Data first monitoring agent using (e)BPF, built on RedBPF410
-
Stackdriver aggregates metrics, logs, and events from infrastructure, giving developers and operators a rich set of observable signals that speed root-cause analysis and reduce mean time to resolution (MTTR).
-
Gradle Enterprise provides both acceleration and observability technologies to speed up the software build and test process. Data analytics make benchmarking and troubleshooting more efficient. It is a key enabling technology for the emerging discipline of Developer Productivity Engineering.
-
Grafana allows you to query, visualize, alert on and understand your metrics no matter where they are stored. Create, explore, and share beautiful dashboards with your team and foster a data driven culture.58.3K
-
Grafana Mimir lets you scale to 1 billion metrics and beyond, with high availability, multi-tenancy, durable storage, and blazing fast query performance over long periods of time.3.48K
-
Grafana Phlare is an open source database that provides fast, scalable, highly-available, and efficient storage and querying of profiling data. By doing so, it allows you to understand your application performance and optimize your infrastructure spend. It natively integrates with Grafana, making it possible to connect your profiling data to your metrics, logs, and traces, and get a comprehensive view of your entire stack.2.07K
-
A highly scalable real-time graphing system5.77K
-
Guance Cloud offers a full-stack observability platform for cloud-based workflows.
-
Exception, uptime, and cron monitoring for web developers
-
Hubble - Network, Service & Security Observability for Kubernetes using eBPF2.99K
-
The core of our monitoring platform with a powerful configuration language and REST API.1.91K
-
Scalable datastore for metrics, events, and real-time analytics26.8K
-
Enterprise observability platform for modern Dev+Ops cloud-native microservices Application Performance Monitoring
-
Circonus provides a unified platform for monitoring and analyzing all of your metrics, infrastructure, applications.
-
Kiali project, observability for the Istio service mesh3.24K
-
A Kubernetes operator for running synthetic checks as pods. Works great with Prometheus!1.84K
-
LeanIX Value Stream Management (VSM) is a SaaS solution to help development teams to create complete transparency across your software supply chain, identify known vulnerabilities and improve collaboration with the business.
-
LinDB is a scalable, high performance, high availability distributed time series database.2.75K
-
LogicMonitor provides IT observability and monitoring for physical, virtual, and cloud-based IT infrastructures.
-
Logz.io provides a scalable and intelligent machine data analytics platform built on ELK and Grafana for monitoring modern applications.
-
M3 monorepo - Distributed TSDB, Aggregator and Query Engine, Prometheus Sidecar, Graphite Compatible, Metrics Platform4.56K
-
The server monitoring platform we always wanted.
-
A toolset for network packet capture in Cloud/Kubernetes and Virtualized environment.936
-
New Relic is a digital intelligence company that delivers full-stack visibility and analytics to enterprises.
-
Metrics Pipeline for interoperability and Enterprise Prometheus561
-
An enterprise-level cloud-native monitoring system, which can be used as drop-in replacement of Prometheus for alerting and Grafana for visualization.8.03K
-
Building products focused on Node.js security and performance for the Enterprise.
-
Evolving the Prometheus exposition format into a standard.2.19K
-
A scalable, distributed Time Series Database.4.91K
-
Secure Open Source Observability Deployed in your own Network1.22K
-
Prevent rapid code changes from impacting customers.
-
Open source Kubernetes observability for developers5.01K
-
The Prometheus monitoring system and time series database.50.9K
-
The observability backend powered by SQL built on PostgreSQL and TimescaleDB.1.33K
-
The central source of truth for all IT infrastructure insights. Providing one central platform that uniquely combines technical and financial metrics to manage and optimize modern containerized IT infrastructure stacks successfully.
-
Collect data directly from your live code using non-breaking breakpoints. Debug, observe and understand anything, anytime, anywhere.
-
Simple. Scalable. Multi-cloud monitoring.931
-
Collect traces, exception stacks and generate logs on-demand without stopping & redeploying your applications.1.65K
-
Splunk is a software company that provides operational intelligence software that monitors, reports, and analyzes real-time machine data.
-
Simple Kubernetes real-time dashboard and management.1.19K
-
Sosivio is a non-intrusive autonomous cloud native platform delivering unmatched visibility into your cloud native environment, eventless failure prediction, and automatic resolution.
-
StackState is the only topology-powered observability & AIOps company.
-
sysdig is a simple tool with deep system visibility for exploration and troubleshooting, with native support for containers.7.44K
-
Highly available Prometheus setup with long term storage capabilities. A CNCF Incubating project.12.3K
-
Tingyun specializes in developing applications for companies to conduct algorithmic IT Operations on mobile, on servers, and the web.
-
Open Source HTTP Reverse Proxy Cache and Time Series Dashboard Accelerator1.92K
-
A high-performance observability data pipeline.15K
-
VictoriaMetrics: fast, cost-effective monitoring solution and time series database9.85K
-
Virtasant’s Cloud Optimization solution includes program setup, global cloud expertise, and a proprietary technology platform that offers complete end-to-end automation to manage Cloud FinOps programs and optimize cloud resources.
-
Wavefront provides real-time views of your code, full stack, at detail down to sub-seconds.
-
Monitoring, visualisation & management for Docker & Kubernetes5.79K
-
WhaTap Labs provides an integrated monitoring service for DevOps that analyzes application performance issue running on kubernetes in real time.
-
Real-time monitoring of IT components and services, such as networks, servers, VMs, applications and the cloud.3.47K
-
Alibaba Cloud develops highly scalable cloud computing and data management services.
-
Scalyr offers a cloud-based data analytics platform that enables high-speed log management and server monitoring.
-
Free and Open, Distributed, RESTful Search Engine
-
Fluentd: Unified Logging Layer (project under CNCF)12.3K
-
Loki is a horizontally-scalable, highly-available, multi-tenant log aggregation system inspired by Prometheus. It is designed to be very cost effective and easy to operate. It does not index the contents of the logs, but rather a set of labels for each log stream.20.7K
-
Humio provides a live observability platform that enables data aggregation, exploration, reporting, and analysis from a range of sources.
-
Loggie is a lightweight, high-performance, cloud-native log collection agent and aggregator based on Golang.1.12K
-
Loggly parses your log files, shows you the code in GitHub which caused the log errors. 10,000+ customers, including 1/3 of the Fortune 500.
-
Logiq.ai is a data platform for engineering teams.
-
Logstash - transport and process your logs, events, or other data13.8K
-
Mezmo, formerly LogDNA, is an observability platform to manage and take action on your data.
-
OpenSearch is a community-driven, Apache 2.0-licensed open source search and analytics suite that makes it easy to ingest, search, visualize, and analyze data.7.95K
-
Your unified data lake and analysis platform for 5G
-
Rizhiyi makes search analysis of professional and complex logs in enterprise information systems into simple and useful search engines.
-
Sematext is an all-in-one observability solution that provides key insights into front-end and back-end performance.
-
Sumo Logic is a provider of cloud-based machine data analytics that enables reliable and secure cloud-native applications.
-
Tencent is an internet service portal offering value-added internet, mobile, telecom, and online advertising services.
-
Trink.io is the first log shipping platform, built to replace fragmented solutions and workarounds with a single solution that takes care of the basics while offering visibility and control that only a managed solution can. Trink handles the entire operational data collection scope for you, connecting to all of your sources (e.g AWS) and enabling you to choose your preferred destinations (Datadog, Logz, etc.) in minutes instead of days.
-
Aspecto is an operator of an efficiency platform intended to eliminate the need for quality assurance.
-
EaseAgent is a Javaagent that can be integrated with the mainstream monitoring system, providing standard data formats that are fully compatible with OpenZipkin and Prometheus. EaseAgent is also very easy to extend through the Plugin Mechanism which only a minimum of three interfaces are required to be implemented to complete a plugin development.566
-
Grafana Tempo is an open source, easy-to-use and high-scale distributed tracing backend. Tempo is cost-efficient, requiring only object storage to operate, and is deeply integrated with Grafana, Prometheus, and Loki. Tempo can be used with any of the open source tracing protocols, including Jaeger, Zipkin, and OpenTelemetry.3.38K
-
Helios gives you the power to understand, troubleshoot, and test all your applications and services.
-
Honeycomb provides full-stack observability for engineering teams to understand, debug, and improve production systems.
-
CNCF Jaeger, a Distributed Tracing Platform18.7K
-
LightStep's mission is to deliver confidence at scale for those who develop, operate, and rely upon today’s powerful software applications.
-
OpenTracing API for Go. 🛑 This library is DEPRECATED! https://github.com/opentracing/specification/issues/1633.48K
-
APM, (Application Performance Management) tool for large-scale distributed systems.13.1K
-
APM, Application Performance Monitoring System22.7K
-
SOFATracer is a component for the distributed system call trace. And through a unified traceId logging the logs of various network calls in the invoking link. These logs can be used for quick discovery of faults, service governance, etc.1.08K
-
Distributed tracing for spring cloud1.74K
-
TelemetryHub is a simple, efficient and affordable observability tool provided by Scout APM, a leading performance monitoring platform.
-
Tracetest allows easy creation of end-to-end tests via a simple user interface. It leverages your current investment in OpenTelemetry based tracing to make deep integration and E2E testing simple.756
-
A Chaos Engineering Platform for Kubernetes.6.09K
-
Chaos Engineering Toolkit & Orchestration for Developers1.79K
-
An easy to use and powerful chaos engineering experiment toolkit.（阿里巴巴开源的一款简单易用、功能强大的混沌实验注入工具）5.6K
-
chaoskube periodically kills random pods in your Kubernetes cluster.1.69K
-
The world's first hosted Chaos Engineering service.
-
Litmus helps SREs and developers practice chaos engineering in a Cloud-native way. Chaos experiments are published at the ChaosHub (https://hub.litmuschaos.io). Community notes is at https://hackmd.io/a4Zu_sH4TZGeih-xCimi3Q3.96K
-
A powerful testing tool for Kubernetes clusters.1.91K
-
Steadybit goes beyond Chaos Engineering. Organizations from eCommerce, SaaS, financial services and global enterprises choose Steadybit.
-
BMC Software is a business service management platform offering cloud and IT management solutions for businesses.
-
CAST is an AI-driven cloud optimization platform that slashes cloud cost, optimizes DevOps, and automates disaster prevention.
-
CloudZero is a SaaS platform offering real-time cost anomaly and waste detection to optimize cloud costs and prevent accidental overspend.
-
Cost optimization and control for cloud and container infrastructure
-
Crane (FinOps Crane) is an opensource project which manages cloud resource on Kubernetes stack, it is inspired by FinOps concepts.1.65K
-
Granulate is an AI-powered optimization software creating a streamlined environment for any application.
-
Infracost shows engineers how their code changes will impact cloud costs. It does this by sitting in the CI/CD workflow and on changing IaC code, leaves a comment like “This change will increase your next month bill by 20%” with a detailed breakdown of resources changed and their cost.9.76K
-
Cross-cloud cost allocation models for Kubernetes workloads4.33K
-
mPLAT offers cloud-type service management functions necessary for businesses.
-
OpenCost provides visibility into current and historical Kubernetes spend and resource allocation.4.33K
-
Opsani uses the capabilities of AI to optimize applications in the cloud.
-
PerfectScale is an easy-to-use solution that helps DevOps teams ensure peak Kubernetes performance at the lowest possible cost.
-
Sedai is an autonomous cloud management platform to optimize cost & performance and remediate issues for Kubernetes & Serverless
-
Spot specializes in cloud cost analysis, reserved capacity management, auto scaling applications, and big data services.
-
StormForge brings together data scientists and software engineers to enable businesses to drive breakthrough IT and operations efficiency.
-
Turbonomic develops tools that optimize application performance, compliance, and cost in real-time.
-
Vamp is a smart and stress free canary testing and releasing for modern cloud platforms.
-
Zesty enables Dynamic Cloud Infrastructure that instantly adapts to your changing business needs.
-
📛 An open source status page system for everyone.13.5K
-
Real-time performance monitoring, done right! https://www.netdata.cloud66.2K
-
Open Source Data & Experience Management Platform (PIM, MDM, CDP, DAM, DXP/CMS & Digital Commerce)3.07K
-
SigNoz is an open-source APM. It helps developers monitor their applications & troubleshoot problems, an open-source alternative to DataDog, NewRelic, etc. 🔥 🖥. 👉 Open source Application Performance Monitoring (APM) & Observability tool15.5K
Subscribe to Open Source Businees Newsletter
Twice a month we will interview people behind open source businesses. We will talk about how they are building a business on top of open source projects.
We'll never share your email with anyone else.