Matlab exit return code for CI

Continuous integration (CI) systems generally rely on an integer return code to detect success (== 0) or failure (!= 0). The error() function of Matlab / GNU Octave returns non-zero status that works well with CI systems. To specifically keep compatibility for a customer that needs outdated Matlab, we run Matlab CI tests using this script pair in the next section.

Modern Matlab tests instead of the “matlab_runner.m” below should use simply

matlab -batch "assertSuccess(runtests)"

matlab_runner.m

This script is called by matlab_runner.py

function matlab_runner()
% for non-interactive use only (from system terminal)
% avoids command line quote escape issues
% fprintf() and exit() to be compatible with Matlab < R2019a

r = runtests;

if isempty(r)
  fprintf(2, 'no tests were discovered\n')
  exit(1)
end

if any(cell2mat({r.Failed}))
  fprintf(2, 'Failed with Matlab %s\n', version)
  exit(1)
end

exit(0)

end

matlab_runner.py

This calls matlab_runner.m. We use Python since it manages the command line much better than Matlab. Edit the variable “wanted_matlab” to test the required Matlab versions.

The exact method for switching Matlab versions may be different on your CI system.

#!/usr/bin/env python3
"""
Tests several versions of Matlab using Lmod module (for Linux)
relies on tests/version_runner.m
"module" requires shell=True
"""
import subprocess
import sys
import platform

if platform.system() != "Linux":
    raise SystemExit("This script for Linux only")

# the tests take several minutes, so we didn't test every possible version
wanted_matlab = ['2019b', '2022b']

failed = 0

for w in wanted_matlab:
    k = f"matlab/{w}"
    ret = subprocess.run(f"module avail {k}", stderr=subprocess.PIPE, text=True, shell=True)
    if k not in ret.stderr:
        print(f"SKIP: {k} not available", file=sys.stderr)
        continue

    mod_cmd = f"module load {k}"
    bat = "matlab -batch"

    ret = subprocess.run(mod_cmd + " && " + bat + " version_runner", text=True, shell=True, cwd='tests')
    if ret.returncode != 0:
        failed += 1


if failed == 0:
    print("OK:", wanted_matlab)
else:
    print(failed, " Matlab versions failed", file=sys.stderr)

Notes

Matlab -batch command line option makes error() return code 1. matlab -batch is so much more robust than matlab -r that users should generally switch to commands like:

matlab -batch test_myscript