Good afternoon, friends!
We are trying to measure the max memory consumption on each worker over time. (during the execution of certain graph). We tried
https://github.com/itamarst/dask-memusage and our homegrown solution using client.submit(). But then I’m wondering it might be impact to the performance because of the frequency. I see we are already reporting
bytes stored and
bytes stored per worker on the dashboard, so there must be similar things going on already.
Im just wondering if it’s possible to get access to the data used by the dashboard programmatically? If so, how?
Thanks in advance.