Dask.array.reduction instead of da.map_blocks?

Dask Array
1

Hi! :slight_smile:

I’m new to this forum so I hope this is the right place to ask this question, if not feel free to redirect me.

I would like to reproduce the following numpy functionality : np.logical_and.reduce([array_1, array_2,...]) for an arbitrary number of arrays.

I figured this is possible via:


import dask.array as da
import numpy as np

data_1 = da.from_array(
    np.array(
        [
            [False, True, True, True],
            [True, True, True, True],
            [False, False, False, False],
            [True, True, True, True],
        ]
    ),
    chunks=2,
)
data_2 = da.from_array(
    np.array(
        [
            [True, True, True, True],
            [True, True, True, True],
            [False, False, False, False],
            [True, True, True, True],
        ]
    ),
    chunks=2,
)

data = [data_1, data_2]

output = da.map_blocks(lambda *arrays: np.logical_and.reduce(arrays), *data, chunks=data_1.chunks).compute()
output

However, I’m curious if I can do the same thing via dask.array.reduction(). I haven’t found many examples using dask.array.reduction so I’m curious if this is possible and if so, how :slight_smile:

thanks in advance!

2

Hi @ameliefro, welcome to Dask community!

Actually, reduce() in the Numpy API is a short cut for a reduce functionnal programming operation, whereas reduction in Dask Array is to apply such a reduction on a given Array itself, not on multiple ones.

This reduce() functionnality is not implemented in Dask. Your solution is a good one, there are other as you can see below:

import dask.array as da
import numpy as np

np_1 = np.array(
        [
            [False, True, True, True],
            [True, True, True, True],
            [False, True, True, False],
            [True, True, True, True],
        ]
    )

np_2 = np.array(
        [
            [True, True, True, True],
            [True, False, False, True],
            [False, False, False, False],
            [True, True, True, False],
        ]
    )

np_3 = np.array(
        [
            [True, True, True, False],
            [True, False, False, True],
            [False, False, False, False],
            [True, True, True, False],
        ]
    )

np.logical_and.reduce([np_1, np_2, np_3])
# equivalent
np_1 & np_2 & np_3

da_1 = da.from_array(np_1, chunks=2)
da_2 = da.from_array(np_2, chunks=2)
da_3 = da.from_array(np_3, chunks=2)
data = [da_1, da_2, da_3]

# possibilities
result = da_1 & da_2 & da_3
result.compute()

result = data[0]
for da_i in data[1:]:
    result = result & da_i
result.compute()

import functools 
functools.reduce(da.logical_and, data).compute()
1 Like
3

Hey @guillaumeeb

Thanks for the quick reply!

Ah ok I see! Thanks for the explanation and the examples! :slight_smile:

1 Like