Alternatives to dashboard link via Jupyter notebook on a HPC cluster

Distributed
1

Hello everyone,

I am using Dask on different compute clusters (Slurm and HTCondor) and wanted to used the nice option of having a deeper look into things using the dashboard. Unfortunately the here (Interactive Use — Dask-jobqueue 0.7.4+11.g96e39da.dirty documentation) described workflow via a Jupyter Notebook to have access to the dashboard link is no option. I therefore wanted to ask if there are alternatives to get a deeper insight into a deployed cluster whilst being on HPC? Thank you very much for the support!

2

@verakye Hi, thanks for the question!

I’m not aware of another way to connect to the dashboard, but I’ll keep looking into this.

described workflow via a Jupyter Notebook to have access to the dashboard link is no option

I also noticed that you can view the dashboard without Jupyter: Interactive Use — Dask-jobqueue 0.8.2+2.gef45c0e.dirty documentation, would that also not work?

3

@pavithraes Thank you for your reply! Unfortunately this also doesn’t work.

4

Hi all,

There are several possibilities when running on an HPC cluster to see the Dashboard:

  • The HPC platform provides a Jupyterhub with the jupyter-server-proxy pakcage installed ont he ontebook: easy, but not the case for you.
  • Using SSH tunneling. Interactive Use — Dask-jobqueue 0.7.4+11.g96e39da.dirty documentation. The dask-jobqueue doc explain how to use ssh tunneling to connect to a Jupyterlab. But if you started the Dask Scheduler on the login node, you can also make an SSH tunnel towards the Scheduler Dashboard.
  • The HPC platform provides access to nodes you can connect to into a “visualisation” mode, e.g. with remote desktop. This is usually the case. Through this desktop, open a browser and enter the Dashboard address.
1 Like