Publish counters from Worker task to Prometheus

Distributed
1

Is there any way to publish custom “counters” from tasks executed by workers into Prometheus-style DB ?

2

Hi,

It’s true there are Prometheus exposed metrics in distributed, but I don’t think there is currently a way to provide custom ones.

Maybe @jacobtomlinson will prove me wrong?

1 Like
3

If you want to add custom metrics to prometheus I expect this could be done via a scheduler plugin.

You should be able to register more metrics in the prometheus collector registry like this. distributed/distributed/http/worker/prometheus/core.py at 5a588aee064e6733f5aa1d0245f1e409a8971133 · dask/distributed · GitHub

Then use the task transitions hook in a scheduler plugin to update the values.

1 Like