Get host:port of additional worker groups

I am currently deploying a dask cluster with multiple worker groups with helm. I deploy as
explained here: How to run different worker types with the Dask Helm Chart

I would like to determine which workers correspond to the gpu-worker group.

Is there a simple way to do this? @jacobtomlinson maybe you have some insight here? I can use annotations to determine which workers to run things on, but I would like to explicity handle the list of tcp://host:port associated with the additional worker group.

Then I could do something like:, workers=gpu_dask_workers)

How can I specify gpu_dask_workers?

1 Like

I think resources is what you’re looking for.

1 Like

Ok, this is what I have tried and no luck with any approach:

def get_name():
    worker = get_worker()

# doesn't work because only the first deployed worker gets the name with helm
# when I use the extraArgs and try to set the name
names =

def get_ip():
    worker = get_worker()
    return str(worker.ip)

# doesn't work because I am at the mercy of how the task was submitted
with dask.annotate(resources={'additional_units': 1}):
    r = client.submit(get_ip)
# doesn't work because run does not take annotations
with dask.annotate(resources={'additional_units': 1}):
    r =
# hacky, but still at the mercy of dask scheduling. All tasks gets scheduled to a single machine
futures = []
for i in range(100):
    with dask.annotate(resources={'additional_units': 1}):

ips = set(client.gather(futures))

Am I missing something @jacobtomlinson? I can’t see how to discover the host:port of the additional workers either using annotations with name or resources and can’t think of another way to leverage resources.

If you are using annotations you shouldn’t need to specify which worker to send it to. Everything in the context manager should be constrained to the workers with those resources.