Disable warning messages

Distributed
,
1

How do I disable Dask warnings that are printed to the terminal? For example, I see the following message in the terminal when I run my Dask project:

distributed.utils_perf - WARNING - full garbage collections took 10% CPU time recently

The project works fine and I don’t see any problems with the results generated from the code. Therefore, I would like to disable the warning messages so they don’t clutter up my terminal window.

2

Hi @wigging, you should be able to configure the logging level using the Dask config system.

So in your case, you can open your ~/.config/dask/distributed.yaml and add:

logging:
  distributed: error

to increase the log level to only show errors.

3

Is this in the Dask docs? I don’t see anything about configuring the logging level.

4

I linked to it in my response

5

I used this and it seems to disable the warning messages. Thank you for the help.

import dask
dask.config.set({'logging.distributed': 'error'})
2 Likes
6

I have very large rechunking operations ( making use of rechunker ) that I’m attempting to run on HPC. I’m having an issue where these long jobs (~24 hours) are being killed due to excessively large standard output and error streams.

I can redirect my STDERR & OUT to files on my storage but it made me think about turning off DASK logging - which is 99% of the output content.

However when I test the above solution it doesn’t appear to work.

for example a LocalCluster with these settings:

import dask
import distributed

with dask.config.set({"distributed.scheduler.worker-saturation": 1.0,
                      "distributed.nanny.pre-spawn-environ.MALLOC_TRIM_THRESHOLD_": 0,
                    "logging.distributed'": "error"}):
    client = distributed.Client()

will still report warnings and info messages like:

2024-04-07 04:40:27,842 - distributed.utils_perf - WARNING - full garbage collections took 47% CPU time recently (threshold: 10%)

and

UserWarning: Sending large graph of size 2.02 GiB.
This may cause some slowdown.
Consider scattering data ahead of time and using futures.
  warnings.warn(

I’d ppreciate any thoughts - especially if I’m missing something silly or obvious on how to control DASK logging messages.

7

Hi @Thomas-Moore,

I just see a typo in your code snippet, with a quote in "logging.distributed'", but I’m not sure this really is your problem?

Did you try setting the configuration into Yaml config files?

1 Like
8

Leave it to me - the king of silly typos. :person_facepalming: Thanks @guillaumeeb for your time replying.

My current platform is batch PBS jobs on HPC. I’ll first try with this typo fixed and see if it changes the result. I guess this typo fails silently ( or the error was buried in all the other warning stderr&out. ) :thinking: