This is an experiment to understand if we can write data pipelines in a way that you declare the supporting services that each step required, similar to how FastAPI’s Dependencies function. The manner in which each dependencies would be resolved would depend on its runtime environment, thus supporting pipelines that could run locally or in a cloud-based environment. Additionally, each pipeline step should be able to inform cloud providers of its permission requirements when deployed. Design Interfaces Interfaces serve as […]