I’m wondering if it’s possible to use a customized plotting function in the map_partition function?
Currently, my implementation (parsed) looks something like:
def plotting_function(pandas_df, column, kwargs): # Some stuff f, t, Sxx = scipy.signal.spectogram(pandas_df[column], kwargs) fig = matplotlib.pyplot.figure() fig.gca().pcolormesh(t, f, tmp) matplotlib.pyplot.show() return 
I would like to apply this function on the partitions (n=10) of a dask dataframe created with a set of parquet files:
dask_df = dd.read_parquet(base_path+'/*.parquet') dask_df.map_partitions(plotting_function, kwargs, meta=)
There is no error when I run this. However, there is no plots that appear in the jupyter cell output either. It’s just an empty cell output. Any suggestions? Am I completely misinterpreting the intended use of this function? If so, my apologies, I am quite new to Dask.