Switch from IDL to Python

March 14, 2018

Python-based data analysis is dominating many fields. In astronomy and geoscience applications, Python dominance is increasing. Consider a widely valued langauge like Python inside of a niche, virtually unknown language outside a few specialties.

Installing Python for the first time: Miniconda Python is a 100 MB download–it won’t initially take several gigabytes of hard drive space like Matlab and IDL.

Package distribution can occur via a website or preferably, a centralized conda or pip repo. For example, to install popular data analysis modules:

conda install pandas scipy matplotlib spyder

IDL vs. Python syntax gives NumPy/Python equivalents to common IDL commands

IDL is a bit distinctive in its syntax. IDL and Matlab syntax remind me a bit of Fortran, while Python is C-based, including that Python has 0-based indexing and C-order array axes. Mayavi also gives advanced 3-D visualization.

Jupyter Notebook makes an IDE in your web browser, and you can make remarkable animated, interactive 3-D plots as well with Ipyvolume.

conda install jupyter
pip install ipyvolume

web browser 3-D animation flow field — Ipyvolume interactive flow field.

HDF5 can handle large and small datasets quickly and easily. Assume HDF5 file terabyte.h5 with double-precision float variable X of size 100000x2048x2048 (3.4 TB). Let’s load the first frame of the 2048x2048 image data, and write it variable first to image.h5. The with syntax uses Python’s context manager, which closes the file upon exiting the indented code section under with.

import h5py

with h5py.File('terabyte.h5') as f:
    img = f['X'][0,...]

with h5py.File('image.h5') as f:
    f['first'] = img

IDL supports Python

Call GDL from Python: GDL Gnu Data Language is free open-source alternative to IDL, compatible with most IDL syntax.

from Python, call IDL/GDL functions by simply import GDL. See PYTHON.TXT

Call Matlab from Python and Python from Matlab:

Ipyvolume interactive flow field.