Why use Python context manager for file I/O?

One should almost always use a Python context manager when working with file I/O in Python. Context managers for Python I/O resources help avoid exceeding system resource limits. For long running jobs, context managers help avoid random crashes due to excess file I/O resource utilization from files left hanging open. There are edge cases where you do need to keep the handle open without context manager–for example, inside a for loop. In many cases, it may be better and easier to let the file open and close with the context manager.

It is also possible to create your own content managers with Python contextlib, which we use in georinex for example.

Context Manager examples: assuming:

from pathlib import Path

fn = Path('~/mydir/myfile').expanduser()

simple file I/O:

with fn.open('r') as f:
    line = f.readline()

Note, if just reading a whole file, consider pathlib.Path methods like:

txt = fn.read_text()

b = fn.read_bytes()

h5py:

import h5py

with h5py.File(fn, 'r') as f:
    data = f['myvar'][:]

NetCDF4:

import netCDF4

with netCDF4.Dataset(fn, 'r') as f:
    data = f['myvar'][:]