I try to create dask cluster with using daskhub on the local k8s cluster. I try to use custom image for the dask-gateway but I need to know that which versions compatible with each other for dask, distributed and dask gateway ?
Hi @menendes! Dask and Dask distributed are released at the same time, so they should have the same version, the most recent is 2021.11.2. Most recent versions of Dask gateway should work, but feel free to reach back out if you have problems.
@scharlottej13 Firstly thank you for your response. I try to install daskhub via helm on the k8s cluster. When I pull the related daskhub package from dask repository, I can deploy it on the k8s cluster and I can connect the gateway. Also I can create a cluster and scale it however when I try to connect related cluster could not connect it. I got an error like " OSError: Timed out during handshake while connecting to gateway". After getting this error I search a couple days and people comments says that the reason of the error is incompatible dask and distributed package and they suggest create custom image with compatible versions for the gateway image. I also try this solution but I stuck in there because in this solution I am able to connect to cluster but I could not create a cluster. To sum up I could not run dask gateway end to end and stuck in there.
@guillaumeeb yes same problem and resolved. Thank you !