How is job submission to FargateClusters controlled?

Deploying Dask
1

I am looking at deploying Dask using dask-distributed’s FargateCluster.

I can see that a public IP address is assigned to the scheduler. As I understand it IAM permissions are required in order to submit jobs to the cluster and it would not be possible to arbitrarily start jobs if a bad actor knew the IP address of the scheduler.

Is there a part of the dask-cloudprovider documentation that discusses this? Is anyone able to explain how this works in more detail?

2

@spatula Welcome! I’m wondering if this bit in the docs can help: Security — Dask Cloud Provider 2021.6.0+41.gc1e91f3 documentation? I also think you can check out Dask Gateway, which can take care of this directly.

@jacobtomlinson might be able to share more details. :smile: