I’m wondering if it’s possible to use a customized plotting function in the map_partition function?
Currently, my implementation (parsed) looks something like:
def plotting_function(pandas_df, column, kwargs):
# Some stuff
f, t, Sxx = scipy.signal.spectogram(pandas_df[column], kwargs)
fig = matplotlib.pyplot.figure()
fig.gca().pcolormesh(t, f, tmp)
matplotlib.pyplot.show()
return []
I would like to apply this function on the partitions (n=10) of a dask dataframe created with a set of parquet files:
dask_df = dd.read_parquet(base_path+'/*.parquet')
dask_df.map_partitions(plotting_function, kwargs, meta=[])
There is no error when I run this. However, there is no plots that appear in the jupyter cell output either. It’s just an empty cell output. Any suggestions? Am I completely misinterpreting the intended use of this function? If so, my apologies, I am quite new to Dask.