Madrigal GNSS line-of-sight data load in Python

June 9, 2023

Madrigal data repositories give access to numerous types of geospace data spanning multiple decades. One kind of data (3505) is GNSS line of sight (LOS) data. Loading the 10+ GB files is best done by choosing slices of data, typically in time at least. This example Python script may help get started. Plotting the data also requires knowing the location of the receiver and corresponding satellite.

The Madrigal data for GNSS LOS is stored under “Data / Table Layout” as a huge 1D unordered vector of HDF5 compound data. h5py can read slices of the compound data to avoid wasted reads and overusing RAM.

Note that Matlab “h5read()” currently can read only the entire compound dataset as a struct, which may use a lot of RAM.