orchest

Build data pipelines, the easy way πŸ› οΈ

Stars
3.99K
Forks
251
Open issues
127
Closed issues
367
Last release
over 1 year ago
Last commit
about 1 year ago
Watchers
3.99K
Total releases
201
Total commits
9.69K
Open PRs
15
Closed PRs
977
Repo URL
Platform
License
agpl-3.0
Category
Usecase
ELT / ETL
Offers premium version?
NO
Proprietary?
NO
About

Notice: we’re no longer actively developing Orchest. We could not find a way to make building a workflow orchestrator commercially viable. Check out Apache Airflow for a robust workflow solution.

Build data pipelines, the easy way πŸ™Œ

No frameworks. No YAML. Just write your data processing code directly in Python, R or Julia.

πŸ’‘ Watch the full narrated video to learn more about building data pipelines in Orchest.

Note: Orchest is in beta.

Features

  • Visually construct pipelines through our user-friendly UI
  • Code in Notebooks and scripts (quickstart)
  • Run any subset of a pipelines directly or periodically (jobs)
  • Easily define your dependencies to run on any machine (environments)
  • Spin up services whose lifetime spans across the entire pipeline run (services)
  • Version your projects using git (projects)

When to use Orchest? Read it in the docs.

πŸ‘‰ Get started with our quickstart tutorial or have a look at our video tutorials explaining some of Orchest's core concepts.

Roadmap

Missing a feature? Have a look at our public roadmap to see what the team is working on in the short and medium term. Still missing it? Please let us know by opening an issue!

Examples

Get started with an example project:

πŸ‘‰ Check out the full list of example projects.

Installation

Want to skip the installation and jump right in? Then try out our managed service: Orchest Cloud.

Slack Community

Join our Slack to chat about Orchest, ask questions, and share tips.

License

The software in this repository is licensed as follows:

  • All content residing under the orchest-sdk/ and orchest-cli/ directories of this repository are licensed under the Apache-2.0 license as defined in orchest-sdk/LICENSE and orchest-cli/LICENSE respectively.
  • Content outside of the above mentioned directories is available under the AGPL-3.0 license.

Contributing

Contributions are more than welcome! Please see our contributor guides for more details.

Alternatively, you can submit your pipeline to the curated list of Orchest examples that are automatically loaded in every Orchest deployment! πŸ”₯

Contributors

Alternative Projects

Subscribe to Open Source Businees Newsletter

Twice a month we will interview people behind open source businesses. We will talk about how they are building a business on top of open source projects.

We'll never share your email with anyone else.