# Is it possible to use mulitple clients concurrently?
**URL:** https://dask.discourse.group/t/is-it-possible-to-use-mulitple-clients-concurrently/488
**Category:** Distributed
**Created:** [March 24, 2022, 6:22pm UTC](https://dask.discourse.group/t/is-it-possible-to-use-mulitple-clients-concurrently/488 "2022-03-24T18:22:57Z")
**Posts on this page:** 2
**Page:** 1
### Author:  [@baltun](https://dask.discourse.group/u/baltun)
#### Post date: [March 24, 2022, 6:22pm UTC](https://dask.discourse.group/t/is-it-possible-to-use-mulitple-clients-concurrently/488/1 "2022-03-24T18:22:57Z")
Hi all,
I am new to the group. I was wondering if two or more dask clients can be used concurrently. Below I have an example code where a cluster is utilized 10 times to call ‘doTheJob’ 1000 times. I am wondering if the code can be further parallelized by starting another set of cluster/client that handles the for loop for x. I would appreciate any support.
Thanks.
Bilgin
import pandas as pd
import dask
from dask.distributed import Client
def doTheJob(i):
return pd.DataFrame(
{‘i’:i,
‘i^2’:i\*i
})
for x in range(10):
with Client(cluster) as client:
jobs = [dask.delayed(doTheJob)(i) for i in range(1000)]
data = dask.dataframe.from\_delayed(jobs).compute()
==========================================================
I have tried the following or versions of it unsuccessfully for the problem stated above.
def useClient():
with Client(cluster) as client:
jobs = [dask.delayed(doTheJob)(i) for i in range(1000)]
data = dask.dataframe.from\_delayed(jobs).compute()
return data
with Client(cluster1) as client1:
jobs1 = [dask.delayed(useClient)() for x in range(10)]
data = dask.dataframe.from\_delayed(jobs1).compute()
---
### Author:  [@scharlottej13](https://dask.discourse.group/u/scharlottej13)
#### Post date: [March 25, 2022, 10:24pm UTC](https://dask.discourse.group/t/is-it-possible-to-use-mulitple-clients-concurrently/488/2 "2022-03-25T22:24:44Z")
Hi @baltun and welcome to Discourse!
It is possible to run computations on two clusters simultaneously in Dask, however, it’d be great to know a little bit more about your setup as this is not usually necessary for improving efficiency.
If you do want to use two cluster instances, calling `dask.dataframe.from_delayed(jobs1).compute()` will use the most recently created client; [`distributed.get_client`](https://distributed.dask.org/en/stable/api.html#distributed.get_client) will find the correct one (see [this stack overflow answer](https://stackoverflow.com/a/62139031/17015034)). Also note that `compute` is a blocking operation (see [the docs on managing computation](https://distributed.dask.org/en/stable/manage-computation.html#dask-collections-to-concrete-values)), which may change when you want to use it.
The other options I’d recommend are:
1. run your computation in batches (see [these best practices on avoiding too many tasks](https://docs.dask.org/en/stable/delayed-best-practices.html#avoid-too-many-tasks)).
2. if the data can fit into memory on the client, could be to remove the `delayed` step and use `dask.dataframe` directly:
```python
import dask
import dask.dataframe as dd
from dask.distributed import Client, LocalCluster
import pandas as pd
def doTheJob(n):
return dd.from_pandas(
pd.DataFrame(
{'i':[i for i in range(n)],
'i^2': [i*i for i in range(n)]}
), npartitions=2)
cluster = LocalCluster()
with Client(cluster) as client:
jobs = [doTheJob(1000) for x in range(10)]
# returns a tuple of 10 Dask DataFrames
data = dask.compute(*jobs)
```
1. Compute on many computations at once, per [these best practices](https://docs.dask.org/en/stable/delayed-best-practices.html#compute-on-lots-of-computation-at-once) (this snippet may not be the best way necessarily, that will depend on exactly what you’re doing)
```python
import dask.dataframe as dd
import pandas as pd
import dask
from dask.distributed import Client, LocalCluster
def doTheJob(i):
return pd.DataFrame({'i':[i], 'i^2': [i*i]})
cluster = LocalCluster()
client = Client(cluster)
jobs = [[dask.delayed(doTheJob)(i) for i in range(1000)] for x in range(10)]
data = client.map(dd.from_delayed, jobs)
results = client.gather(data)
```