Introduce yourself

Hi everyone, my name is Naeem. I am a research software engineer member of Research Software Engineering (RSE) at the FAS Research Computing (FASRC) department. I design, build, and optimize software applications for researchers across Harvard University.

3 Likes

Hi, I’m a Solutions Engineer at Prefect working from Berlin, Germany :de:

Prefect is an open-source workflow orchestration platform that integrates seamlessly with Dask. If you have any questions about Prefect and Dask integration, I’m happy to chat via Prefect Discourse.

Great to see that Coiled forms a team to support the Dask Community even more!

2 Likes

hey there everyone I am sophmore pursuing my BTech in Data Science and Artificial Intelligence. My go to language for most of the work is Python so Dask became a choice for me to do large scale ML and Data analysis. Eager to learn more about the nitty grityy details of how to use dask as efficiently as possible.

3 Likes

Hello Dask folks, I’m an oceanographer working in Tasmania with very large datasets from coupled climate models. I’m a “user” of Dask in a general sense as it helps power the Pangeo-style workflows, mainly at large HPC facilities, allowing me to explore and process large datasets many times the size of available memory.

I’m always wanting to better understand Dask in more depth and detail so I’m grateful this Discourse exists. :smiley: Thanks!

4 Likes

Hi, I’m Paul. Freelance data engineer, mostly working for small startups :wave:

4 Likes

Hi Everyone! I’m a geospatial data engineer at a small tech nonprofit called Radiant Earth Foundation. Been learning to use Dask to scale our dataset ingestion, etc.

3 Likes

Hi, I’m Jaime, I am an oceanographer/numerical modeler at the Virginia Institute of Marine Science. I am new to Dask, and I decided to give it a try when I found myself in the need to write a “driver” to distribute my jobs over the HPC cluster. The approach I had works, but I decided to give Dask a try, because it seems promising. So far, I have been able to port to Dask some of my tasks, but not all of them. Maybe because I don’t fully understand the Dask system or maybe because Dask doesn’t seem to support at the moment my needs (to use threading inside a delayed function). The tasks I have been able to port though, run more efficiently in Dask, that with my previous solutions, but I still need to figure out some details, and unexplained crashes I sometimes get with distributed.

In general, I like the Dask approach, and I’d like to use it more, but I find that it has a somewhat steep learning curve. I would be willing to pay for a 2-4 hour one-to-one with a Dask expert that may be able to answer some of my questions. If anyone is willing, I’d be very grateful.

Thanks for developing and maintaining all of this awesomeness!

4 Likes

Welcome @jreniel! If you haven’t already, I’d start with the Dask tutorial or the getting started videos on the Dask youtube channel. Coiled also offers some live Dask trainings. Feel free to post your specific questions on this forum!

A post was split to a new topic: How to make JobQueueCluster launch and supervise an arbitrary job file

Hi everyone
I’m Tierry, I am a computer scientist (just finished my Masters) and just started working at MeteoSwiss. I’ll be working intensively with dask for at least the next few months and so I’ll probably be around this forum from time to time.

3 Likes

Hi there,

I am Alex Braun. I am a computer vision scientist working for a biotech company. I specialize in 3D microscopy and procedurally generated 3D models of tissue. Before biotech, I worked in the visual effects industry for film and television. I have built my own Kubernetes cluster (Harvester OS) and am in the process of automating my computer vision workflows with Dask. In my free time, I work on a 3D visual programming language which involves Kubernetes, Dask, Houdini, Blender and Category Theory (git repo is currently private).

On of the K8s apps I am working on is called Hidebound. It uses Dask, Plotly’s Dash and Schematics to ingest massive (100+ GB) image datasets.

3 Likes

Hi there,
I am Tibor Kiss. I am working on non-destructive investigation data such as Ground Penetration Radar datasets (each > 300GB). I am currently doing a proof-of-concept using Dask.

3 Likes

:wave: I’m Gabriel. I’m with the Development Data Group at the World Bank, where I work as a Data Scientist/Evangelist with the World Bank Data Lab and the Development Data Partnership. I’ve taken advantage of Dask intensely, mainly for orchestration/data engineering and scaling up geospatial operations.

4 Likes

Hi! I’m Eugene - here as a hobbyist, and very keen on uncovering the capabilities of dask. I’ve been working primarily with basic pandas, and it’s just so cool as a concept to be able to scale up / out data processing.

2 Likes

Hi! I am Li, a researcher. Different from most people, I am not from computer-science-related background. I am totally green hand in the distributed computing. Thanks for Dask who gives me a oppoertunity to do distributed computing!

1 Like

Hi, I am Mauro, a software Engineer at CRS4 in Italy. Currently I am using Dask for processing whole slide images for Digital Pathology related projects.

1 Like

Hello All, my name is Sorin! :wave:
I am currently working in Research for WithSecure, Finland and I am just starting to look into Dask and how I could use it in my use cases.
I will most probably ask a lot of noob questions regarding Dask, and I hope the conversations will be helpful also for others who are getting started with Dask. :slight_smile:

3 Likes

Hi All. I am Marc.

As a day job I work for Orsted, the global leader in Offshore Wind, as a Lead Trading Analyst. Developing or “owning” analytics solutions and platforms to enable us to better trade energy and power in particular.

In my spare time I contribute to HoloViz Panel and run awesome-panel.org.

I have for a long time been wanting to look more into using Dask as the execution engine for Panel. For many reasons including they both run on the Tornado/ Bokeh, server, support async, work great with Datashader and share lots of visions, I believe they are close to the best match for creating data apps at scale.

3 Likes

Hi all! I’m Alex, I’m a research scientist at Novo Nordisk, hoping to use dask to accelerate some of our high-throughput workflows! :wave:

2 Likes

Hello everyone,
my name is Anna, and I work as a researcher in the field of environmental science/engineering, currently at the University of Illinois at Urbana Champaign. I have used dask a few times to parallelize simulations, and got into contact with it because it is used in the background by xarray. But there are still many aspects of dask that I don’t yet know about or do not understand very well.

3 Likes