"'coroutine' object is not iterable" trying to read_parquet from S3

Dask DataFrame
1

Using this code to read multiple parquet files from a private s3 bucket, not from AWS

import dask as dd
dd.read_parquet(
    's3://ns1/data/key',
    storage_options={
        'key': 'key',
        'secret': 'secret',
        'client_kwargs': {'endpoint_url': 'https://s3.sample-private-cloud.com'}
    }
)

Why am I getting an error:

TypeError: ‘coroutine’ object is not iterable

I was able to download the file using boto3 client but unable to read it using dask.

2

Hi @locorecto, welcome to Dask Discourse!

Could you give us the complete stack trace of the Error you get?

Also, could you try to access the data using s3fs Python library?

3

I was able to fix the issue. In my code, I replaced

's3://ns1/data/key'

with

's3://ns1/data/key/*'

as I was trying to retrieve multiple parquet file parts using the prefix data/key but did not specify the correct pattern.

This is the full stack trace btw:

tempsnip
tempsnip978×966 212 KB

1 Like