I have a multi-step process that manipulates dask arrays. I’m surprised by the fact that lots of time elapses and I see a lot of operations on the dashboard prior to the code hitting .compute. According to my log, when it it finally does hit compute, it executes instantly. Is this expected behavior?
I was under the impression that because of lazy evaluation, the code should get to the .compute stage really quickly.
If this is abnormal, then it seems like the reason would be operations that inadvertently trigger compute prematurely?
Some other pieces of information - my task graph is large (28 MB) - I get a warning about that when I start the client.