How can I programatically get profiled data?

Hello everybody,

I’m trying to profile the time stamp of all my dask tasks. Exactly the same thing that dashboard does when you access Profile tab. I know that I can generate HTML content to visualize after, but is there any way to do it using code instead of a HTML page?

A similar approach of dask-memusage. Using dask-memusage it is possible to export it as csv and load it using pandas. I’m looking for something equally easy and not visual. The reason why I need this is to improve my code and debug possible areas to increase the speedup.

@jcfaracco Welcome to Discourse!

client.profile() gives you a dictionary with all profiling data, would that help?

1 Like