Get_worker in client.run: raise ValueError("No worker found") from None

Distributed
1

This simple code gives errors out with Exception: ValueError('No worker found') for me.

from distributed import Client, LocalCluster, get_worker

def return_one():
    worker = get_worker()
    return 1


def main():
    with LocalCluster(n_workers=2, processes=True) as cluster:
        client = Client(cluster)
        worker = client.scheduler_info()["workers"]
        one = client.run(return_one, workers=list(worker.keys()))
        print(one)

if __name__ == "__main__":
    main()
Full Traceback

/Users/christianroth/repos/playgrounds/yaml_inout/.pixi/envs/default/bin/python /Users/christianroth/repos/playgrounds/yaml_inout/dasky.py
2025-02-11 21:16:18,568 - distributed.worker - WARNING - Run Failed
Function: return_one
args: ()
kwargs: {}
Traceback (most recent call last):
File “/Users/christianroth/repos/playgrounds/yaml_inout/.pixi/envs/default/lib/python3.12/site-packages/distributed/worker.py”, line 3175, in run
result = function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File “/Users/christianroth/repos/playgrounds/yaml_inout/dasky.py”, line 4, in return_one
worker = get_worker()
^^^^^^^^^^^^
File “/Users/christianroth/repos/playgrounds/yaml_inout/.pixi/envs/default/lib/python3.12/site-packages/distributed/worker.py”, line 2746, in get_worker
raise ValueError(“No worker found”) from None
ValueError: No worker found
2025-02-11 21:16:18,568 - distributed.worker - WARNING - Run Failed
Function: return_one
args: ()
kwargs: {}
Traceback (most recent call last):
File “/Users/christianroth/repos/playgrounds/yaml_inout/.pixi/envs/default/lib/python3.12/site-packages/distributed/worker.py”, line 3175, in run
result = function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File “/Users/christianroth/repos/playgrounds/yaml_inout/dasky.py”, line 4, in return_one
worker = get_worker()
^^^^^^^^^^^^
File “/Users/christianroth/repos/playgrounds/yaml_inout/.pixi/envs/default/lib/python3.12/site-packages/distributed/worker.py”, line 2746, in get_worker
raise ValueError(“No worker found”) from None
ValueError: No worker found
2025-02-11 21:16:18,569 - distributed.protocol.pickle - ERROR - Failed to serialize No worker found.
Traceback (most recent call last):
File “/Users/christianroth/repos/playgrounds/yaml_inout/.pixi/envs/default/lib/python3.12/site-packages/distributed/worker.py”, line 3175, in run
result = function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File “/Users/christianroth/repos/playgrounds/yaml_inout/dasky.py”, line 4, in return_one
worker = get_worker()
^^^^^^^^^^^^
File “/Users/christianroth/repos/playgrounds/yaml_inout/.pixi/envs/default/lib/python3.12/site-packages/distributed/worker.py”, line 2746, in get_worker
raise ValueError(“No worker found”) from None
ValueError: No worker found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/Users/christianroth/repos/playgrounds/yaml_inout/.pixi/envs/default/lib/python3.12/site-packages/distributed/protocol/pickle.py”, line 60, in dumps
result = pickle.dumps(x, **dump_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: cannot pickle ‘_contextvars.ContextVar’ object

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/Users/christianroth/repos/playgrounds/yaml_inout/.pixi/envs/default/lib/python3.12/site-packages/distributed/protocol/pickle.py”, line 65, in dumps
pickler.dump(x)
TypeError: cannot pickle ‘_contextvars.ContextVar’ object

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/Users/christianroth/repos/playgrounds/yaml_inout/.pixi/envs/default/lib/python3.12/site-packages/distributed/protocol/pickle.py”, line 77, in dumps
result = cloudpickle.dumps(x, **dump_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/Users/christianroth/repos/playgrounds/yaml_inout/.pixi/envs/default/lib/python3.12/site-packages/cloudpickle/cloudpickle.py”, line 1537, in dumps
cp.dump(obj)
File “/Users/christianroth/repos/playgrounds/yaml_inout/.pixi/envs/default/lib/python3.12/site-packages/cloudpickle/cloudpickle.py”, line 1303, in dump
return super().dump(obj)
^^^^^^^^^^^^^^^^^
TypeError: cannot pickle ‘_contextvars.ContextVar’ object
2025-02-11 21:16:18,569 - distributed.protocol.pickle - ERROR - Failed to serialize No worker found.
Traceback (most recent call last):
File “/Users/christianroth/repos/playgrounds/yaml_inout/.pixi/envs/default/lib/python3.12/site-packages/distributed/worker.py”, line 3175, in run
result = function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File “/Users/christianroth/repos/playgrounds/yaml_inout/dasky.py”, line 4, in return_one
worker = get_worker()
^^^^^^^^^^^^
File “/Users/christianroth/repos/playgrounds/yaml_inout/.pixi/envs/default/lib/python3.12/site-packages/distributed/worker.py”, line 2746, in get_worker
raise ValueError(“No worker found”) from None
ValueError: No worker found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/Users/christianroth/repos/playgrounds/yaml_inout/.pixi/envs/default/lib/python3.12/site-packages/distributed/protocol/pickle.py”, line 60, in dumps
result = pickle.dumps(x, **dump_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: cannot pickle ‘_contextvars.ContextVar’ object

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/Users/christianroth/repos/playgrounds/yaml_inout/.pixi/envs/default/lib/python3.12/site-packages/distributed/protocol/pickle.py”, line 65, in dumps
pickler.dump(x)
TypeError: cannot pickle ‘_contextvars.ContextVar’ object

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/Users/christianroth/repos/playgrounds/yaml_inout/.pixi/envs/default/lib/python3.12/site-packages/distributed/protocol/pickle.py”, line 77, in dumps
result = cloudpickle.dumps(x, **dump_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/Users/christianroth/repos/playgrounds/yaml_inout/.pixi/envs/default/lib/python3.12/site-packages/cloudpickle/cloudpickle.py”, line 1537, in dumps
cp.dump(obj)
File “/Users/christianroth/repos/playgrounds/yaml_inout/.pixi/envs/default/lib/python3.12/site-packages/cloudpickle/cloudpickle.py”, line 1303, in dump
return super().dump(obj)
^^^^^^^^^^^^^^^^^
TypeError: cannot pickle ‘_contextvars.ContextVar’ object
Traceback (most recent call last):
File “/Users/christianroth/repos/playgrounds/yaml_inout/dasky.py”, line 16, in
main()
File “/Users/christianroth/repos/playgrounds/yaml_inout/dasky.py”, line 12, in main
one = client.run(return_one, workers=list(worker.keys()))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/Users/christianroth/repos/playgrounds/yaml_inout/.pixi/envs/default/lib/python3.12/site-packages/distributed/client.py”, line 3192, in run
return self.sync(
^^^^^^^^^^
File “/Users/christianroth/repos/playgrounds/yaml_inout/.pixi/envs/default/lib/python3.12/site-packages/distributed/utils.py”, line 363, in sync
return sync(
^^^^^
File “/Users/christianroth/repos/playgrounds/yaml_inout/.pixi/envs/default/lib/python3.12/site-packages/distributed/utils.py”, line 439, in sync
raise error
File “/Users/christianroth/repos/playgrounds/yaml_inout/.pixi/envs/default/lib/python3.12/site-packages/distributed/utils.py”, line 413, in f
result = yield future
^^^^^^^^^^^^
File “/Users/christianroth/repos/playgrounds/yaml_inout/.pixi/envs/default/lib/python3.12/site-packages/tornado/gen.py”, line 766, in run
value = future.result()
^^^^^^^^^^^^^^^
File “/Users/christianroth/repos/playgrounds/yaml_inout/.pixi/envs/default/lib/python3.12/site-packages/distributed/client.py”, line 3097, in _run
raise exc
File “/Users/christianroth/repos/playgrounds/yaml_inout/dasky.py”, line 4, in return_one
worker = get_worker()
^^^^^^^^^^^^^^^^^
File “/Users/christianroth/repos/playgrounds/yaml_inout/.pixi/envs/default/lib/python3.12/site-packages/distributed/worker.py”, line 2746, in get_worker
raise ValueError(“No worker found”) from None
^^^^^^^^^^^^^^^^^
Exception: ValueError(‘No worker found’)

This feels unintuitive: I requested to run on my two workers but it does not seem to have access to get_worker()?

Is there a way to access the worker from client.run?

EDIT: it seems like it did work at some point? client - How to get the worker name in dask cluster? - Stack Overflow

2

Hi @ChristianRothQC,

I think that the problem comes from the fact you are launching this function using Client.run and not Client.submit or other.

From the documentation:

Run a function on all workers outside of task scheduling system

Your code works correctly with Client.submit.

However, if you need Client.run, then you can follow this:

If your function takes an input argument named dask_worker then that variable will be populated with the worker itself.

1 Like