Configuration

Data Orchestration Platform with python

dop

Design Concept

DOP is designed to simplify the orchestration effort across many connected components using a configuration file without the need to write any code.
We have a vision to make orchestration easier to manage and more accessible to a wider group of people.

Here are some of the key design concept behind DOP,

  • Built on top of Apache Airflow – Utilises it’s DAG capabilities with interactive GUI
  • DAGs without code – YAML + SQL
  • Native capabilities (SQL) – Materialisation, Assertion and Invocation
  • Extensible via plugins – DBT job, Spark job, Egress job, Triggers, etc
  • Easy to setup and deploy – fully automated dev environment and easy to deploy
  • Open Source – open sourced under the MIT license

Please note that this project is heavily optimised to run

 

 

 

To finish reading, please visit source site