Hello community, I’ve installed dask-gateway in cluster and can run my example code into it, it perfectly spins workers, do math and shutsdown. Now my task is to make “profiles” so different teams get different worker images, and also some teams will have kubernetes volumes attached to there container. Can I do this somehow?
Hi @melaz, welcome to Dask Discourse forum!
Dask Gateway propose some ways to be able to select and specify on the user side some configurations, this is explained here.
You should be able to give Options to override any configuration parameter available in the KubeClusterConfig.
For the Docker image, it should be pretty simple using the image configuration option. For the Kubernetes volumes, it’s probably a bit more complicated, maybe through worker_extra_pod_config?
It will be really itneresting to share your result if you achieve your goal!