How do I avoid distributed.client - WARNING - Couldn't gather keys, rescheduling?

@crusaderky

The function client.get_worker_logs seems to give the dask-worker-logs, ( from this post). I see there is an argument named nanny in the function. Below is an example of the output using nanny = True, without running anything.

client.get_worker_logs(nanny=True):
{'tls://10.8.10.2:34213': (('INFO',
   "2023-09-01 18:14:36,446 - distributed.nanny - INFO -         Start Nanny at: 'tls://10.8.10.2:42197'"),),
 'tls://10.8.10.3:39955': (('INFO',
   "2023-09-01 18:14:36,881 - distributed.nanny - INFO -         Start Nanny at: 'tls://10.8.10.3:35365'"),),
 'tls://10.8.10.4:33253': (('INFO',
   "2023-09-01 18:14:39,111 - distributed.nanny - INFO -         Start Nanny at: 'tls://10.8.10.4:42165'"),),
 'tls://10.8.10.5:34085': (('INFO',
   "2023-09-01 18:14:37,257 - distributed.nanny - INFO -         Start Nanny at: 'tls://10.8.10.5:45371'"),),
 'tls://10.8.10.6:39941': (('INFO',
   "2023-09-01 18:14:37,684 - distributed.nanny - INFO -         Start Nanny at: 'tls://10.8.10.6:45223'"),),
 'tls://10.8.10.7:40643': (('INFO',
   "2023-09-01 18:14:38,613 - distributed.nanny - INFO -         Start Nanny at: 'tls://10.8.10.7:42967'"),),}

Is this the correct Nannies’ logs that I should be checking during the computation?