Using dask-awkward to speed up dask-awkward
Summary
We released the first non-pre-release version of dask-awkward a couple of months ago, but the project is far from done! Something that has taken a lot of our focus over the last few weeks has been improving what we call the “necessary columns” optimization. The goal of the optimization is to avoid wasting compute and memory on unnecessary disk reads. This post will describe how the optimization works.