Jupyter notebook demonstrating use of Dask and Xarray for lightning fast 3-D interpolation of DNS data.
Dask provides parallelism and array chunking, while Xarray provides convienient writing and compression of NETCDF4 files.
Spline interpolation function is translated from IDL by Swike from StakedOverflow
Prerequisites:
Python
- Tested on Python 3.11.x
Jupyter Notebook
pip install jupyter notebook
Dask
pip install dask[complete]
Xarray
pip install xarray
NETCDF4-python
pip install netcdf4
Due to my needs specifically, the notebook reads coordinate data from:
- course mesh: (
'geometry_old.nc'
) - fine mesh: (
'geometry.nc'
) - course flow field data: (
'flowXXXX.nc'
)
and attempts to interpolate data (for instance the temperature field 'temp'
) from the coarse grid to the fine grid.
Arrays are loaded using the Dask.array.from__array
function to ensure large arrays do not overrun available RAM on system.
Final flowfile is written using Xarray
with zlib compression (specified in the encoding dict
)