It is possible to disable autoscaling behavior on a dask job

Deploying Dask
,
1

For some workloads the behaviour of starting to calculate before the cluster is “complete” (before it has the replicas size number of nodes) is causing to us some problems, would be possible to disable this behaviour on a daskjob? for instance having the option to set up the number minimum and maximum the replicas to be equal ?

thanks

2

Autoscaling is opt-in, so you may not even have it enabled.

It sounds like you want to wait until all workers are ready before you start your computation. In this case you should add client.wait_for_workers(n) to your code to explicitly block until your desired number of workers are online.

1 Like