xarray NetCDF LRU cache replaced autoclose

January 1, 2019

Python code using xarray.open_dataset() or xarray.open_dataarray() or similar functions reading from NetCDF4 use an LRU cache that automatically closes unneeded files.

As of xarray 0.11.0, the obsolete autoclose=True option should no longer be used.

Problems fixed by LRU cache: the LRU cache used to open NetCDF4 files with xarray fixes these problems, and gives high performance:

• random segmentation fault while reading NetCDF4 .nc files, where the same file is reopened in the program.
• OSError from too many open files, where even increasing ulimit doesn’t help