Hi,
I’m creating ephemeral Dask Clusters on K8S (AWS EKS) to run some ETL jobs.
Unfortunately, the “dask-root” pods are never stopped and deleted.
Is there a config flag I can set for a full cleanup?
At the moment, my cluster accumulates a one idling dask-root pod per finished ETL run:
Code:
# see https://kubernetes.dask.org/en/latest/
def dask_pod_spec() -> V1Pod:
return make_pod_spec(
image="ghcr.io/dask/dask:latest",
memory_limit="1G",
memory_request="1G",
cpu_limit=1,
cpu_request=1,
extra_pod_config={"serviceAccountName": "prefect-agent-service-account"},
)
# https://kubernetes.dask.org/en/latest/kubecluster.html
dask_runner = DaskTaskRunner(
cluster_class=dask_kubernetes.KubeCluster,
adapt_kwargs={"minimum": 1, "maximum": 10},
cluster_kwargs={
"pod_template": dask_pod_spec(),
"env": {
"EXTRA_PIP_PACKAGES": "prefect==2.0b11"
}
},
)