How should I configure Daskhub helm chart to deploy Daskhub in a fully private EKS cluster?

Deploying Dask
1

Hi,

I have EC2 machines deployed in a VPC with (almost) no access to the internet, where every user is supposed to connect to via VPN. A Windows Server provides an interface for those who work with Jupyter notebooks, but everthing happens behind a VPN.

I just deployed a fully private EKS cluster (there are no public endpoints), and wanted some help to deploy Daskhub in this scenario.

When I try it, I always get the chart to launch a internet-facing load balancer in the public subnet, but I don’t want any public endpoint to start. Every service should be available to internal access only.

Is there a way to make this load balancer an internal service instead in my VPC? Or use another type of Kubernetes service to achieve the same result, but in a private/local network?
What parameters should I deploy the chart with to get this result?

2

@bressanmarcos Hi and welcome to Discourse! Could you please share some more details about how you’ve tried deploying Dask? Some minimal code/example would also be valuable!

I’m wondering if Dask Gateway can help here?

3

So far, I’ve been struggling with running the chart:

helm upgrade --wait --install --render-subchart-notes     dhub dask/daskhub     --namespace=dhub     --values=secrets.yaml     --values=config.yaml

With the files:

# file: secrets.yaml
jupyterhub:
  hub:
    services:
      dask-gateway:
        apiToken: mytoken

dask-gateway:
  gateway:
    auth:
      jupyterhub:
        apiToken: mytoken

# config.yaml
jupyterhub.proxy.service.annotations:
        service.beta.kubernetes.io/aws-load-balancer-internal: 0.0.0.0/0

I did notice that even using this option to make the load balancer internal-facing, it will insist to create the LB in my public subnet, open to the world. My purpose is to make it an internal service only, for local access only.

4

Nevermind, i just noticed my mistake: the config yaml file should look like this:

# config.yaml
jupyterhub:
  proxy:
    service:
      type: LoadBalancer
      annotations:
        service.beta.kubernetes.io/aws-load-balancer-internal: 0.0.0.0/0

This just shows how noob I am in Helm, LOL
Thanks anyway!

2 Likes
5

Thanks @bressanmarcos for posting your resolution :slight_smile: I’m sure it will be helpful for others!

1 Like