Following on a discussion here I am wondering whether anyone is having ideas how to connect to dashboard of ad-hoc dask cluster that were created using KubeCluster.
Currently, the KubeCluster’s (or client) dashboard_link points to localhost which is necessarily incorrect.
In my setup, the KubeCluster is created by a job triggered from jenkins. What I am planning to do as a quick-fix is print out to the user instruction how to kubectl port-forward from her local computer to the scheduler pod so that the dashboard is accessible. This of course requires a local installation and proper environment of kubectl…
In the post mentioned, @jacobtomlinson suggest a few approaches - expose the dashboard as a k8s service/ingress, use a proxy or use dask-gateway. I guess the latter is a good option though I am not sure I’d be able to support it so I am looking to other approaches…
Any ideas / inputs would be appreciated. I’ll be happy to contribute code (assuming I have the skills to do so) once a clear solution is discussed…
Sorry for the delay in responding here. If this question is still relevant I would suggest going down the service/ingress route and printing out the URL to use to access the dashboard.
We ended up just writing to the log the relevant kubectl port-forward commend. In our setup using ingress/service would require registering a sub-domain with a dedicated IP address so we gave that option up.
Would using the dask operator help with this?
Yeah I think it would. The operator supports creating multiple instances of KubeCluster that all point to the same cluster. It also supports automatically port forwarding the dashboard.
So you could launch the cluster from Jenkins and then provide the user with the name of the cluster to connect to. This does expect the user has kubectl and k8s credentials to be able to list the resources.
That’s pretty much what we did with the “classic” Kubecluster. I initially tried to wrestle the name of the scheduler pod from within dask but eventually resorted using python’s kubernetes library.
Yeah that sounds a bit painful, the new implementation should reduce all the sharp edges around this.
1 Like