I’m a relatively new user of dask. I’m using dask on a GPU cluster. On the CPU cluster I’ve been able to monitor CPU usage using the dask dashboard. I see no analog for GPU usage. I have been monitoring usage directly by logging into the nodes and using cuda/nvidia cmd line tools but if there is a graphical tool people are using or even a cmd line dask tool that routes to the correct nodes for monitoring, that would be very useful. Any advice?
Hi @liberabaci, welcome to Dask Discourse forum!
There is a great post about this by @jacobtomlinson, and certainly he also knows more than me about default monitoring of GPU included in Dask.
There is support for GPUs in the dashboard. It requires the nvidia-ml-py package to be installed. If you are using dask-cuda this will be installed automatically.
1 Like
@guillaumeeb @jacobtomlinson Thank you so much for the info. Exactly what I was looking for.
1 Like