Setup dask dashboard link for user in the-littlest-jupyter-hub

Deploying Dask
1

I’m currently using the-littlest-jupyter-hub (tljh; The Littlest JupyterHub — The Littlest JupyterHub v0.1 documentation) as a simple way to lauanch/use jupyter lab on AWS EC2.

I’m using a plugin for configuration of tljh (tljh-plugin/tljh_plugin.py at main · raybellwaves/tljh-plugin · GitHub) and slowly expanding it to include more steps rather that manual steps after jupyterhub is launched.

One thing I would like to do is seamless get the dask dashboard working on launching.

There are a couple of manual options to do this which I made a note of in the dask docs (e.g. create config yaml file (Configuration — Dask documentation) or an environmental variable (Configuration — Dask documentation)).

I believe the approach I would like to do is go down the yaml file route and create a default yaml like

distributed:
  dashboard:
    # Locate the dashboard if working on a Jupyter Hub server
    link: /user/${JUPYTERHUB_USER}/proxy/8787/status

When a user logs in this file is copied to /home/JUPYTERHUB_USER and JUPYTERHUB_USER is replaced with the JUPYTERHUB_USER environmental variable value.

For example, I log in and my username is ray (jupyter-ray) and have a file ~/.config/dask/dask.yaml which looks like

distributed:
  dashboard:
    # Locate the dashboard if working on a Jupyter Hub server
    link: /user/jupyter-ray/proxy/8787/status

I can then see a link to the dashboard after doing

from dask.distributed import Client, LocalCluster
cluster = LocalCluster(local_directory="/tmp")
client = Client(cluster)
client

I don’t know enough sysadmin stuff where I think I could provision this i.e. write a skeleton file which is later filled in for the yaml approach

If it is easier I could provision an environmental variable as export DASK_DISTRIBUTED__DASHBOARD__LINK="/user/${JUPYTERHUB_USER}/proxy/8787/status"

I image I would drop the code in (tljh-plugin/tljh_plugin.py at main · raybellwaves/tljh-plugin · GitHub)

Relevant resources:

4

(post deleted by author)

5

Hi @raybellwaves, welcome! Your approach sounds reasonable to me, is it not working?

One point: I think you’ll also want to template in the {port} so that if your cluster starts on a different port than the default it will still work:

distributed:
  dashboard:
    # Locate the dashboard if working on a Jupyter Hub server
    link: /user/${JUPYTERHUB_USER}/proxy/{port}/status
6

Hi @raybellwaves I believe this works similarly to when you do it for binder, where you need this file in a .dask directory. Although I might be wrong (@ian please call me out on this), but whenever I need to get dashboard configurations right I follow this template repo, which might help GitHub - jrbourbeau/dask-binder-template

7

Thanks @ian and @ncclementi for your responses.

I believe i’m almost there.

I can confirm I am able to see the dashboard when logging in and writing the file:

/home/jupyter-ray.bell/.dask/config.yaml:

distributed:
  dashboard:
    link: "{JUPYTERHUB_BASE_URL}user/{JUPYTERHUB_USER}/proxy/{port}/status"

I am wondering where I should write this file on the disk when the machine launches? (/opt/tljh/user/share/ ?)

I would like the config.yaml to be copied to a users home directory (~/.dask.config.yaml) when they log in.

8

The search paths for a config file are documented here. If writing to the home directory is challenging at startup time, you might try putting it in /etc/dask. Though this might not work well in a shared environment such as a JupyterHub when the config involves usernames. Can you do a mkdir -p /home/jupyter-ray.bell/.config/dask on a new user login to ensure that the config directory exists?

The environment variable approach is also totally a fine way to go, it can just get a little unwieldy if you are doing a lot of configuration.

1 Like
9

@raybellwaves Hi! Just checking in, were you able to solve this with Ian’s latest suggestions?