Hi @jadeidev, welcome back!
We’ve add some discussions in the past few weeks about the unmanaged memory, for example:
https://dask.discourse.group/t/memory-limits-reached-in-simple-etl-like-data-transformations
This is often a tricky problem, and it seems you already try some possible solutions.
Here, I’m thinking about two things:
- Did you try to del the Future Python object once you got the result?
- It looks like your Workers still have some room in term of Memory usage after the computation, so maybe Python or the OS just don’t bother to reclaim memory space. What happens if you submit new computations after that: does the memory keeps stacking up? Or is it released at some point?