Creating cluster from windows

Distributed
1

I have 4 windows machines and I would like to create a cluster.
I want to be able to use the cluster adapt option (ie automatically adjust the number of workers) based on a changing workload. Is there a way to go about this with Dask Python API? Would SSHCluster work with windows?

2

@jadeidev Thanks for this question! SSHCluster on Windows isn’t reliably supported yet, there’s an open issue here: Test SSHCluster on Windows CI · Issue #4509 · dask/distributed · GitHub

Also, tagging @jacobtomlinson here for visibility :smile:

3

Thank you for the info. Are there any other ways to achieve this for windows?

4

@jadeidev I’m not sure, but I’ll leave this topic open for the community to chime in!

@akbaritabar do you have thoughts on this? @scharlottej13 mentioned you might have tinkered with Windows-clusters :smile: