I am using Dask on different compute clusters (Slurm and HTCondor) and wanted to used the nice option of having a deeper look into things using the dashboard. Unfortunately the here (Interactive Use — Dask-jobqueue 0.7.3+15.g088ac4e.dirty documentation) described workflow via a Jupyter Notebook to have access to the dashboard link is no option. I therefore wanted to ask if there are alternatives to get a deeper insight into a deployed cluster whilst being on HPC? Thank you very much for the support!
@verakye Hi, thanks for the question!
I’m not aware of another way to connect to the dashboard, but I’ll keep looking into this.
described workflow via a Jupyter Notebook to have access to the dashboard link is no option
I also noticed that you can view the dashboard without Jupyter: Interactive Use — Dask-jobqueue 0.7.3+15.g088ac4e.dirty documentation, would that also not work?
@pavithraes Thank you for your reply! Unfortunately this also doesn’t work.
There are several possibilities when running on an HPC cluster to see the Dashboard:
- The HPC platform provides a Jupyterhub with the jupyter-server-proxy pakcage installed ont he ontebook: easy, but not the case for you.
- Using SSH tunneling. Interactive Use — Dask-jobqueue 0.7.3+15.g088ac4e.dirty documentation. The dask-jobqueue doc explain how to use ssh tunneling to connect to a Jupyterlab. But if you started the Dask Scheduler on the login node, you can also make an SSH tunnel towards the Scheduler Dashboard.
- The HPC platform provides access to nodes you can connect to into a “visualisation” mode, e.g. with remote desktop. This is usually the case. Through this desktop, open a browser and enter the Dashboard address.