Installing Dask Workers on Partcular Node Pool

Hi Team,

We would like to use Dask Hub with Gateway Cluster and have 2 Nodepools (one for CPU and one for GPU) and we want to deploy the Dask Workers on GPU nodes , we have tried different configuration to set the node selector and taint but nothing worked and dask workers are creating in CPU pool only , Please find the config file for your reference

backend:
      node_selector: {'app.kubernetes.io/name': 'gpupoc'}
      tolerations: [
        {
          'key': 'sku',
          'operator': 'Equal',
          'value': 'gpu',
          'effect': 'NoSchedule'
        }
      ]
      worker:
        extraPodConfig:
          node_selector: {'app.kubernetes.io/name': 'gpupoc'}
          tolerations: 
            - 'key': 'sku'
              'operator': 'Equal'
              'value': 'gpu'
              'effect': 'NoSchedule'
        node_selector: {'app.kubernetes.io/name': 'gpupoc'}
        tolerations: [
          {
            'key': 'sku',
            'operator': 'Equal',
            'value': 'gpu',
            'effect': 'NoSchedule'
          }
        ]
      cores:
          request: 2
        memory:
          request: "4 G"
        threads: 2

Code which we are using to create dask cluster

from dask_gateway import Gateway
gateway = Gateway()
clusters = gateway.list_clusters()
cluster = gateway.new_cluster(shutdown_on_close=False)
cluster.adapt(minimum=2, maximum=4)
client = cluster.get_client();

Can you please check and guide how we can select node selectors and taints for dask worker nodes and how to set no of threads in dask worker nodes as nthread configuration in helm config file not working .

@akash-goel Thanks for your question! I’m not familiar with Dask+GPUs, but @jacobtomlinson might have thoughts on this. :smile: