Connect your scheduler and worker to a cluster

I have created a scheduler and cluster using dask python advance APIs. Now I want to scale the workers based on scheduler load but none of the APIs is there for that. The only way is from the cluster API.

Any idea how to join make scheduler join the cluster or any other way how we can scale workers while we are using dask python advance APIs.

@NamitPathak Welcome to Discourse!

In the Python API (advanced) doccs, I see that the Scheduler has functions like adaptive_target(), async add_worker, async close_worker, workers_to_close, etc., that can be used to scale up/down workers. Would these help, or are you looking for something else?

If you can share a minimal example of your workflow, I’d be happy to test out scaling. :smile: