Dask-gateway: Cluster Options on K8s Setup empty

Deploying Dask
1

Hi There,

We have deployed daskhub on our AWS K8s Cluster. All works fine after a version sync exercise. However, when I want to configure a dask cluster using

gateway = Gateway()
options = gateway.cluster_options()
options

then options is empty. According to the docs options should contain e.g. worker_cores.

dask-gateway version is 0.9.0.

EDIT: I should add that everything else works like a charm.
EDIT2: I get a weird 401 on the API service when asking for the options:
[W 2022-01-28 11:38:55.167 DaskGateway] 401 GET /api/v1/options 0.435ms

Any ideas?

Thanks heaps

H

2

@dzelge Hi and welcome to Discourse! I’m still looking into it, but a 401 error suggests some permission issues (see this implementation that makes an HTTP request) – maybe it’s related to how you’re implementing authentication? Could you please share some details about your authentication setup?

1 Like
3

Hi @dzelge. The 401 just indicates that your client hadn’t done an authentication cycle yet, it’s not a signal that things aren’t working properly (note that the client didn’t see any errors).

In the case of empty cluster options - note that no cluster options are configured by default. You need to manually expose them when configuring your dask-gateway-server process. See Exposing Cluster Options — Dask Gateway 2022.6.1 documentation for more information.

1 Like
4

Hi @jcristharif,

Thanks heaps for the answer. Seem to me, that you are on teh right track. Is there somewhere an example, how to configure this in a daskhub K8s setup?

Thanks

Helge

5

Hi @jcristharif,

Forget my last comment. I got it. It is working now. Thanks heaps again

Nga mihi

Helge

image

1 Like