orchestration and scheduling platform
Event-Driven Declarative Orchestrator
"Click on the image to get started in 4 minutes with Kestra."
Live Demo
Try Kestra using our live demo.
What is Kestra
Kestra is a universal open-source orchestrator that makes both scheduled and event-driven workflows easy. By bringing Infrastructure as Code best practices to data, process, and microservice orchestration, you can build reliable workflows and manage them with confidence.
In just a few lines of code, you can create a flow directly from the UI. Thanks to the declarative YAML interface for defining orchestration logic, business stakeholders can participate in the workflow creation process.
Kestra offers a versatile set of language-agnostic developer tools while simultaneously providing an intuitive user interface tailored for business professionals. The YAML definition gets automatically adjusted any time you make changes to a workflow from the UI or via an API call. Therefore, the orchestration logic is always managed declaratively in code, even if some workflow components are modified in other ways (UI, CI/CD, Terraform, API calls).
Key concepts
Flow
is the main component in Kestra. It's a container for your tasks and orchestration logic.Namespace
is used to provide logical isolation, e.g., to separate development and production environments. Namespaces are like folders on your file system — they organize flows into logical categories and can be nested to provide a hierarchical structure.Tasks
are atomic actions in a flow. By default, all tasks in the list will be executed sequentially, with additional customization options, a.o. to run tasks in parallel or allow a failure of specific tasks when needed.Triggers
define when a flow should run. In Kestra, flows are triggered based on events. Examples of such events include:
Inputs
allow you to pass runtime-specific variables to a flow. They are strongly typed, and allow additional validation rules.Extensible platform via plugins
Most tasks in Kestra are available as plugins, but many type of tasks are available in the core library, including a.o. script tasks supporting various programming languages (e.g., Python, Node, Bash) and the ability to orchestrate your business logic packaged into Docker container images.
To create your own plugins, check the plugin developer guide.
Rich orchestration capabilities
Kestra provides a variety of tasks to handle both simple and complex business logic, including:
disabled
to true
.id: getting_started namespace: devdescription: | # Getting Started Let's
write
some markdown - first flow 🚀labels: owner: rick.astley project: never-gonna-give-you-up
tasks:
id: hello type: io.kestra.core.tasks.log.Log message: Hello world! description: a very important task disabled: false timeout: PT10M retry: type: constant # type: string interval: PT15M # type: Duration maxDuration: PT1H # type: Duration maxAttempt: 5 # type: int warningOnRetry: true # type: boolean, default is false
triggers:
You can write workflows directly from the UI. When writing your workflows, the UI provides:
We release new versions every month. Give the repository a star to stay up to date with the latest releases and get notified about future updates.
Getting StartedFollow the steps below to start local development.
PrerequisitesMake sure that Docker is installed and running on your system. The default installation requires the following:
Launch KestraDownload the Docker Compose file:
curl -o docker-compose.yml https://raw.githubusercontent.com/kestra-io/kestra/develop/docker-compose.yml
Alternatively, you can use wget https://raw.githubusercontent.com/kestra-io/kestra/develop/docker-compose.yml
.
Start Kestra:
docker compose up
Open http://localhost:8080
in your browser and create your first flow.
Here is a simple example logging hello world message to the terminal:
id: getting_started namespace: dev tasks:
id: hello_world type: io.kestra.core.tasks.log.Log message: Hello World!
For more information:
Kestra is built on a plugin system. You can find your plugin to interact with your provider; alternatively, you can follow these steps to develop your own plugin.
For a full list of plugins, check the plugins page.
Here are some examples of the available plugins:
<a href="https://kestra.io/plugins/plugin-airbyte#cloudjobs" rel="nofollow">Airbyte Cloud</a>
<a href="https://kestra.io/plugins/plugin-airbyte#connections" rel="nofollow">Airbyte OSS</a>
<a href="https://kestra.io/plugins/plugin-aws#athena" rel="nofollow">Amazon Athena</a>
<a href="https://kestra.io/plugins/plugin-aws#cli" rel="nofollow">Amazon CLI</a>
<a href="https://kestra.io/plugins/plugin-aws#dynamodb" rel="nofollow">Amazon DynamoDb</a>
<a href="https://kestra.io/plugins/plugin-jdbc-redshift" rel="nofollow">Amazon Redshift</a>
<a href="https://kestra.io/plugins/plugin-aws#s3" rel="nofollow">Amazon S3</a>
<a href="https://kestra.io/plugins/plugin-aws#sns" rel="nofollow">Amazon SNS</a>
<a href="https://kestra.io/plugins/plugin-aws#sqs" rel="nofollow">Amazon SQS</a>
<a href="https://kestra.io/plugins/plugin-amqp" rel="nofollow">AMQP</a>
<a href="https://kestra.io/plugins/plugin-serdes#avro" rel="nofollow">Apache Avro</a>
<a href="https://kestra.io/plugins/plugin-cassandra" rel="nofollow">Apache Cassandra</a>
<a href="https://kestra.io/plugins/plugin-kafka" rel="nofollow">Apache Kafka</a>
<a href="https://kestra.io/plugins/plugin-jdbc-pinot" rel="nofollow">Apache Pinot</a>
<a href="https://kestra.io/plugins/plugin-serdes#parquet" rel="nofollow">Apache Parquet</a>
<a href="https://kestra.io/plugins/plugin-pulsar" rel="nofollow">Apache Pulsar</a>
<a href="https://kestra.io/plugins/plugin-spark" rel="nofollow">Apache Spark</a>
<a href="https://kestra.io/plugins/plugin-tika" rel="nofollow">Apache Tika</a>
<a href="https://kestra.io/plugins/plugin-azure/#batchjob" rel="nofollow">Azure Batch</a>
<a href="https://kestra.io/plugins/plugin-azure/#storage-blob" rel="nofollow">Azure Blob Storage</a>
<a href="https://kestra.io/plugins/plugin-azure/#storagetable" rel="nofollow">Azure Blob Table</a>
<a href="https://kestra.io/plugins/plugin-serdes#csv" rel="nofollow">CSV</a>
<a href="https://kestra.io/plugins/plugin-jdbc-clickhouse" rel="nofollow">ClickHouse</a>
<a href="https://kestra.io/plugins/plugin-compress" rel="nofollow">Compression</a>
<a href="https://kestra.io/plugins/plugin-couchbase" rel="nofollow">Couchbase</a>
<a href="https://kestra.io/plugins/plugin-databricks" rel="nofollow">Databricks</a>
<a href="https://kestra.io/plugins/plugin-dbt#cloud" rel="nofollow">dbt cloud</a>
<a href="https://kestra.io/plugins/plugin-dbt#cli" rel="nofollow">dbt core</a>
<a href="https://kestra.io/plugins/plugin-debezium-sqlserver" rel="nofollow">Debezium Microsoft SQL Server</a>
<a href="https://kestra.io/plugins/plugin-debezium-mysql" rel="nofollow">Debezium MYSQL</a>
<a href="https://kestra.io/plugins/plugin-debezium-postgres" rel="nofollow">Debezium Postgres</a>
<a href="https://kestra.io/plugins/plugin-jdbc-duckdb" rel="nofollow">DuckDb</a>
<a href="https://kestra.io/plugins/plugin-elasticsearch" rel="nofollow">ElasticSearch</a>
<a href="https://kestra.io/plugins/plugin-notifications#mail" rel="nofollow">Email</a>
<a href="https://kestra.io/plugins/plugin-fivetran" rel="nofollow">Fivetran</a>
<a href="https://kestra.io/plugins/plugin-fs#ftp" rel="nofollow">FTP</a>
<a href="https://kestra.io/plugins/plugin-fs#ftps" rel="nofollow">FTPS</a>
<a href="https://kestra.io/plugins/plugin-git" rel="nofollow">Git</a>
<a href="https://kestra.io/plugins/plugin-gcp#bigquery" rel="nofollow">Google Big Query</a>
<a href="https://kestra.io/plugins/plugin-gcp#pubsub" rel="nofollow">Google Pub/Sub</a>
<a href="https://kestra.io/plugins/plugin-gcp#gcs" rel="nofollow">Google Cloud Storage</a>
<a href="https://kestra.io/plugins/plugin-gcp#dataproc" rel="nofollow">Google DataProc</a>
<a href="https://kestra.io/plugins/plugin-gcp#firestore" rel="nofollow">Google Firestore</a>
<a href="https://kestra.io/plugins/plugin-gcp#cli" rel="nofollow">Google Cli</a>
<a href="https://kestra.io/plugins/plugin-gcp#vertexai/" rel="nofollow">Google Vertex AI</a>
<a href="https://kestra.io/plugins/plugin-gcp#gke" rel="nofollow">Google Kubernetes Engines</a>
<a href="https://kestra.io/plugins/plugin-googleworkspace#drive" rel="nofollow">Google Drive</a>
<a href="https://kestra.io/plugins/plugin-googleworkspace#sheets" rel="nofollow">Google Sheets</a>
<a href="https://kestra.io/plugins/plugin-script-groovy" rel="nofollow">Groovy</a>
<a href="https://kestra.io/plugins/plugin-fs#http" rel="nofollow">Http</a>
<a href="https://kestra.io/plugins/plugin-serdes#json" rel="nofollow">JSON</a>
<a href="https://kestra.io/plugins/plugin-script-julia" rel="nofollow">Julia</a>
<a href="https://kestra.io/plugins/plugin-script-jython" rel="nofollow">Jython</a>
<a href="https://kestra.io/plugins/plugin-kubernetes" rel="nofollow">Kubernetes</a>
<a href="https://kestra.io/plugins/plugin-jdbc-sqlserver" rel="nofollow">Microsoft SQL Server</a>
<a href="https://kestra.io/plugins/plugin-notifications#teams" rel="nofollow">Microsoft Teams</a>
<a href="https://kestra.io/plugins/plugin-mongodb" rel="nofollow">MongoDb</a>
<a href="https://kestra.io/plugins/plugin-mqtt" rel="nofollow">MQTT</a>
<a href="https://kestra.io/plugins/plugin-jdbc-mysql" rel="nofollow">MySQL</a>
<a href="https://kestra.io/plugins/plugin-script-nashorn" rel="nofollow">Nashorn</a>
<a href="https://kestra.io/plugins/plugin-nats" rel="nofollow">NATS</a>
<a href="https://kestra.io/plugins/plugin-neo4j" rel="nofollow">Neo4j</a>
<a href="https://kestra.io/plugins/plugin-script-node" rel="nofollow">Node</a>
<a href="https://kestra.io/plugins/plugin-openai" rel="nofollow">OpenAI</a>
<a href="https://kestra.io/plugins/plugin-crypto#openpgp" rel="nofollow">Open PGP</a>
<a href="https://kestra.io/plugins/plugin-jdbc-oracle" rel="nofollow">Oracle</a>
<a href="https://kestra.io/plugins/plugin-jdbc-postgres" rel="nofollow">PostgreSQL</a>
<a href="https://kestra.io/plugins/plugin-powerbi" rel="nofollow">Power BI</a>
<a href="https://kestra.io/plugins/plugin-script-powershell" rel="nofollow">PowerShell</a>
<a href="https://kestra.io/plugins/plugin-script-python" rel="nofollow">Python</a>
<a href="https://kestra.io/plugins/plugin-jdbc-rockset" rel="nofollow">Rockset</a>
<a href="https://kestra.io/plugins/plugin-script-powershell" rel="nofollow">RScript</a>
<a href="https://kestra.io/plugins/plugin-fs#sftp" rel="nofollow">SFTP</a>
<a href="https://kestra.io/plugins/plugin-servicenow" rel="nofollow">ServiceNow</a>
<a href="https://kestra.io/plugins/plugin-singer" rel="nofollow">Singer</a>
<a href="https://kestra.io/plugins/plugin-script-shell" rel="nofollow">Shell</a>
<a href="https://kestra.io/plugins/plugin-notifications#slack" rel="nofollow">Slack</a>
<a href="https://kestra.io/plugins/plugin-jdbc-snowflake" rel="nofollow">Snowflake</a>
<a href="https://kestra.io/plugins/plugin-soda" rel="nofollow">Soda</a>
<a href="https://kestra.io/plugins/plugin-fs#ssh" rel="nofollow">SSH</a>
<a href="https://kestra.io/plugins/plugin-notifications#telegram" rel="nofollow">Telegram</a>
<a href="https://kestra.io/plugins/plugin-jdbc-trino" rel="nofollow">Trino</a>
<a href="https://kestra.io/plugins/plugin-serdes#xml" rel="nofollow">XML</a>
<a href="https://kestra.io/plugins/plugin-jdbc-vertica" rel="nofollow">Vertica</a>
This list is growing quickly and we welcome contributions.
Community Support
If you need help or have any questions, reach out using one of the following channels:
Roadmap
See the open issues for a list of proposed features (and known issues) or look at the project board.
Contributing
We love contributions, big or small. Check out our contributor guide for details on how to contribute to Kestra.
See our Plugin Developer Guide for details on developing and publishing Kestra plugins.
License
Apache 2.0 © Kestra Technologies
Twice a month we will interview people behind open source businesses. We will talk about how they are building a business on top of open source projects.
We'll never share your email with anyone else.