Changes from 2024.7 to 2025.7

Distributed
1

Hello people,

I have a question about what changed between Dask 2024.7 uintil 2025.7.

I have an app which was dpeloyed in Kubernetes using Dask distributed version 2024.7. Everything was fine and working nicely. A client makes a request; in the app a Dask Client is created, connecting to the scheduler, and work is being done. Important to say that it was working in a parallel manner. I have Gunicorn workers, which are serving parallel requests. Now with the upgrade to 2025.7 (dask changing to Expressions as backend), suddenly all these parallel computations are not working. It seems that Dask is hanging. Executing requests sequentially is not a problem, but as soon as many requests get in, at some point it starts hanging and leads to timeout.It seems a bit slow. For example if i make a parallel requests, i will get Dask lost dependency error or just in the middle of the work it hangs and not finishes,

I would appreciate your help!

2

I will assume the problem may be related that when you have a parallel requests, which each one of them is building a big graphs, will lead to somehow less forgiving than Dask 2024.*

3

Hi @Teodor_Chakarov,

Would you be able to give some reproducer of this issue? I’m not sure of the Dask API you are using and don’t have a clear view of the context. It seems you are creating several Dask Client connected to the same Scheduler and each of them submitting complex graphs?

4

Hi,

Yes actually today I managed to solve the issue, and the problem was the following.

I have Flask app + connexion served via Gunicorn workers (Gunicorn can run multiple workers so it can serve many requests at once. Each worker runs your app code and responds to clients).

So lets say that I deploy the app using only 1 worker. When you initialize a request, within this 1 worker I create a Dask Client which connects to a scheduler, and then later i have client.submit().

When I give to this 1 worker sequential requests, no problem. Gets a request submits to dask cluster and then status 200.
But when I execute 2 parallel requests, within this Gunicorn process it looks like this:

CASE 1:
Process (Gunicorn Worker)
├─ Client #1 → connects to scheduler
├─ Client #2 → also connects to scheduler
└─ Both try to submit overlapping graphs → scheduler sees conflicting graph/session state

CASE 2:

Process (Gunicorn Worker)
├─ Client #1 → connects to scheduler

2 Grpahs are being build and sent using this client

Dask 2024.* seems more forgiving and was returning both as ok response, meaning it recomputed the request, which failed. In Dask 2025.*, first request comes in and then suddenly it happens that it has lost-dependency error, or it seems to hang, the client is waiting for a response but never gets one.

In order to make it work, I used from the library multiprocessing, Semaphore(1), which locks all threads within the Gunicorn process. Therefore, 1 Gunicorn worker can have 1 client and only 1 dask graph being sent at the time.

So i can have 8 Gunicorn workers => 8 Dask Clients, but 1 graph computation at the time per client. So there is concurrency but not parallelism.