Dask-gateway fails to create resource DaskCluster CRD on kubernetes

Deploying Dask
,
1

Hello community,
I deployed Dask-Hub and Dask-Operator on kubernetes. I created a daskcluster crd (via kubectl) but, I can’t ceate a Daskcluster via Gateway:

Exception ignored in: <function Gateway.__del__ at 0x7f4170716980>
Traceback (most recent call last):
  File "/srv/conda/envs/notebook/lib/python3.11/site-packages/dask_gateway/client.py", line 380, in __del__
    self.close()
  File "/srv/conda/envs/notebook/lib/python3.11/site-packages/dask_gateway/client.py", line 353, in close
    elif self.loop.asyncio_loop.is_running():
         ^^^^^^^^^
  File "/srv/conda/envs/notebook/lib/python3.11/site-packages/dask_gateway/client.py", line 330, in loop
    return self._loop_runner.loop
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/conda/envs/notebook/lib/python3.11/site-packages/distributed/utils.py", line 652, in loop
    )
RuntimeError: Accessing the loop property while the loop is not running is not supported
2

Hi @jihed, welcome to Dask Discourse.

I’m not sure if this stack trace is relevant, given it seems to happen when dask-gateway object is deleted.

Do you have other information you could share? When does the problem happen? What is your code, your setup?

3

Thanks for the reply :slight_smile: Here’s the full instructions, I just started tried to create a cluster using the dask-gw and problem happen when I invoked new_cluster().

Full code:

from dask_gateway import GatewayCluster
cluster = GatewayCluster()
from dask_gateway import Gateway
gateway = Gateway(
“http://-.us-west-2.elb.amazonaws.com/services/dask-gateway/”, auth=“jupyterhub”
)
gateway.list_clusters()

[ClusterReport<name=dask-hub.e2f95006c4ba413aa9bc22c87a46a44a, status=RUNNING>,
 ClusterReport<name=dask-hub.7491bb686f4744df8c77a5bedd3f7dcc, status=RUNNING>

cluster = gateway.new_cluster()

Exception ignored in: <function Gateway.__del__ at 0x7f9a960928e0>
Traceback (most recent call last):
  File "/srv/conda/envs/notebook/lib/python3.11/site-packages/dask_gateway/client.py", line 380, in __del__
    self.close()
  File "/srv/conda/envs/notebook/lib/python3.11/site-packages/dask_gateway/client.py", line 353, in close
    elif self.loop.asyncio_loop.is_running():
         ^^^^^^^^^
  File "/srv/conda/envs/notebook/lib/python3.11/site-packages/dask_gateway/client.py", line 330, in loop
    return self._loop_runner.loop
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/conda/envs/notebook/lib/python3.11/site-packages/distributed/utils.py", line 652, in loop
    raise RuntimeError(
RuntimeError: Accessing the loop property while the loop is not running is not supported
4

This first invocation creates a cluster instance not tied to any gateway (you didn’t connect to the gateway yet). You then reassign the cluster variable, causing this original cluster to be cleaned up - which fails because it doesn’t exist/wasn’t started.

1 Like