Matlab + Octave unit tests quick-start

November 18, 2020

Matlab unit test framework is well-established and xUnit-like. The Matlab unit test framework is incompatible with the GNU Octave BIST unit testing framework. The Matlab unit test syntax is completely different from the Octave unit test syntax. Nonetheless, a common ground between Matlab and GNU Octave can be found in the easy to use Matlab script-based tests. A Matlab script-based test can be run as an ordinary script with GNU Octave.

A key limitation of Matlab script-based tests is they can’t use the Qualifiable functions (verify*, assert*, assume*). This also means you cannot mark a test as skipped or Incomplete (using assume*) with script-based tests. Basic tests can work without those functions, but more advanced tests suites do strongly benefit from those functions.

Matlab or Octave can run a script-based test as a plain script. The downsides of running a script-based test as a plain script are:

there is no decorated TestResult
the first failure ends the whole test

It’s best to put the script-based test scripts in the same directory as the code they’re testing to avoid Matlab path issues.

When using Matlab on this Octave-compatible script-based test, a richer result comes from using Matlab-only runtests(). Matlab runtests() will search all subdirectories under the current working directory and run all scripts with case-insensitive “test” in the filename. For example, a script testing HDF5 function of a program might be named TestHDF5.m or HDF5test.m. This is similar to PyTest filename-filtering.

Denote each test in the script-based test file with a double percent sign like:

% test_example.m
% setup code goes up top, as each test's variables are isolated from each other
% when using Matlab runtests()

A = magic(2);

%% test_dummy
B = A*2;  % only visible to this test, when using Matlab runtests()
assert(isequal(B, A*2), 'dummy example')

%% test_four
C = A*4; % note that "B" is not visible under runtests()
assert(isequal(C, A*4), 'dummy four')

Run Matlab runtests() from CI command like:

matlab -batch "assertSuccess(runtests)"

Just doing matlab -batch runtests will NOT fail in CI, even with failing test.

Octave runtests('.') only runs tests in specially formatted comments that Matlab ignores. For projects that need to have tests supporting Octave and Matlab, we generally recommend writing Matlab script-based tests, and manually running each file from Octave.