How to make sure that the parent function is not kicked off unless all the children functions are not completed?

1

In the attached Dask Flow how can we make sure that the “Merge Function” waits for all the following “Pivot Functions” to get completed. “Merge Function” should kick off only when the “Pivot Functions” are completed.

image
image1920×1080 137 KB

Thanks
Rohit

2

@GosainRohit Thanks for this question! I believe this will happen by default? In the Dask task graph, we can see that merge_data has a dependency on all the pivot_data outputs (denoted by arrows). This means Dask will make sure it gets all the pivot_data outputs before calling merge_data. Does this help answer your question?

In case I’m not understanding your questions correctly, could you please share a minimal, reproducible example, it’ll allow us to help you better. :smile:

1 Like