Logging info available for dask DataFrame compute or to_parquet calls?

I’m trying to profile my single and multi-threaded dask calls (dask.config.set(scheduler='threads') etc.) for some simple DataFrame operations on a 10gb collection of files. I was hoping just to get some simple log info on when each operation begins and ends but my calls to logging.basicConfig(level=logging.INFO) or logging.basicConfig(level=logging.DEBUG) does not produce any dask-related output. Am I missing a step or is this just not supported. I’m using dask version 2023.10.0 on python 3.10.2.

Hi @spillz, welcome to Dask community!

I don’t think that Dask logs things like that by itself.

However, depending on what you want, you have some other tools available, see


in particular.

Many thanks! It looks like ProgressBar and Profiler can do what I need.