Best practice for shutting down a cluster

Distributed
#1

I’ve got a LocalCluster working well except for teardown. I’ve tried a number of functions, including:

  • client.close()
  • client.shutdown()
  • client.scheduler.shutdown()
  • client.retire_workers()
  • doing nothing

But I’ve found no combination that avoids a variety of errors on exit. All the examples I’ve found are interactive ipynb files that don’t ever terminate the cluster. So basic question:

What is the best practice for cleanly terminating a client / cluster?

Thanks.

#2

Hi @rjplevin,

This is also something that I’ve experienced, and I’ve no good advice. I’d say the better to use is client.shutdown(), but it’s true in some workload you can get random errors… In general, you can safely ignore them, but I agree this isn’t really clean.