Adding Custom messages to Worker Logs

Distributed
1

Is there a way for a process add it’s own logging messages into the Dask worker log ? This way the messages can be viewed with the other messages the the worker writes in the DASK dashboard .

Thanks

2

Hi @larryverdils, welcome to Dask Discourse!

Did you try the code mentionned in Where does print go on a worker? · dask/dask · Discussion #7202 · GitHub

from dask.distributed import Client
from distributed.worker import logger

client = Client()

def log_it():    
    logger.info("This will output to the logs!")
    
client.run(log_it)
3

Just to be sure , will this work from withing a function that was called by submit ? The reason i ask is that we might have a situation of a scheduled process trying to call out to a child process

Thank you

4

Well, I just tried it with submit, and it worked like a charm:

2023-08-04 13:09:58,148 - distributed.worker - INFO - Start worker at: tcp://127.0.0.1:34458
2023-08-04 13:09:58,148 - distributed.worker - INFO - Listening to: tcp://127.0.0.1:34458
2023-08-04 13:09:58,148 - distributed.worker - INFO - Worker name: 2
2023-08-04 13:09:58,148 - distributed.worker - INFO - dashboard at: 127.0.0.1:46370
2023-08-04 13:09:58,148 - distributed.worker - INFO - Waiting to connect to: tcp://127.0.0.1:34692
2023-08-04 13:09:58,148 - distributed.worker - INFO - -------------------------------------------------
2023-08-04 13:09:58,148 - distributed.worker - INFO - Threads: 1
2023-08-04 13:09:58,148 - distributed.worker - INFO - Memory: 5.00 GiB
2023-08-04 13:09:58,148 - distributed.worker - INFO - Local Directory: /tmp/pbs.49493185.admin01/dask-worker-space/worker-no97p2fx
2023-08-04 13:09:58,148 - distributed.worker - INFO - -------------------------------------------------
2023-08-04 13:09:59,241 - distributed.worker - INFO - Registered to: tcp://127.0.0.1:34692
2023-08-04 13:09:59,241 - distributed.worker - INFO - -------------------------------------------------
2023-08-04 13:10:29,799 - distributed.worker - INFO - This will output to the logs!