Python client for using Prefect Cloud with Saturn Cloud
prefect-saturn
prefect-saturn is a Python package that makes it easy to run Prefect Cloud flows on a Dask cluster with Saturn Cloud. For a detailed tutorial, see “Fault-Tolerant Data Pipelines with Prefect Cloud “.
Installation
prefect-saturn is available on PyPi.
pip install prefect-saturn
prefect-saturn can be installed directly from GitHub
pip install git+https://github.com/saturncloud/[email protected]
Getting Started
prefect-saturn is intended for use inside a Saturn Cloud environment, such as a Jupyter notebook.
import prefect
from prefect import Flow, task
from prefect_saturn import PrefectCloudIntegration
@task
def hello_task():
logger = prefect.context.get("logger")
logger.info("hello prefect-saturn")
flow = Flow("sample-flow", tasks=[hello_task])
project_name = "sample-project"
integration = PrefectCloudIntegration(
prefect_cloud_project_name=project_name
)
flow = integration.register_flow_with_saturn(flow)
flow.register(
project_name=project_name,
labels=["saturn-cloud"]
)
Customize Dask
You can