Scientific Computing

Git can be configured to not prompt for passwords, instead failing with an error message. This is useful for scripts that use Git, so that the script can handle the error instead of hanging indefinitely. Two distinct environment variables are used to control this behavior.

• GIT_TERMINAL_PROMPT
• GIT_SSH_COMMAND

Resolve the full path of Git using shutil.which() for better subprocess security / robustness.

Example of Git clone from Python or CMake without credential prompt–prints error instead:

A software package may have Matlab and Python functions. Plain Matlab code can be included via Pytest for Matlab and GNU Octave. To test the Matlab functions from Pytest, create test_matlab.py:

This assumes the Matlab test script resides in the same directory as “test_matlab.py”.

NOTE: Be sure that Matlab “runtests()” actually picks up the desired tests. We do not want the Matlab output to say

Totals:
   0 Passed, 0 Failed, 0 Incomplete.
   0 seconds testing time.

Generally, Python __file__ is not robust for packages but one might relax that restriction for test files where a clear error will result.

The CMake FindMatlab module adds several helper CMake helper functions including matlab_add_mex. When running CTest on MEX targets, the test script needs to add the path of the MEX target to the Matlab path. The matlab-cmake-mex project demonstrates this.

The Matlab runtests optional Name= parameter is the name of the test function in the test script. It allows making a single Matlab test function correspond to a single CTest test.

TARGET_FILE_DIR CMake generator expression tells Matlab addpath where the compiled MEX target is located. The generator expression handles single and multi config generators that may place the target binary under additional subdirectories depending on the build configuration. That is, CMAKE_CURRENT_BINARY_DIR is in general NOT correct for this application. This technique of using a CMake generator expression applies in general whenever a target path is passed to another script or program by CMake add_test.

CMake uses CMakeConfigureLog.yaml to log configure output. CMakeConfigureLog.yaml replaces CMakeOutput.log and CMakeError.log from older CMake. message(CONFIGURE_LOG) also writes to CMakeConfigureLog.yaml.

CMake can do a variety of configure-time checks that help avoid confusing build errors. The fundamental functions for these checks are try_compile and try_run. Higher level CMake functions like check_source_compiles, check_source_runs, check_symbol_exists, etc. wrap these functions and add logic to only run the check once.

In general for CMake try_compile() and try_run(), include directories are specified like:

try_compile(...
CMAKE_FLAGS -DINCLUDE_DIRECTORIES=${CMAKE_CURRENT_SOURCE_DIR}
)

To preserve the scratch directories, use option:

cmake -Bbuild --debug-trycompile

which will tell the scratch directory used for try_compile() and try_run().

Example of reading CMakeConfigureLog.yaml in Python.

CMake file-api is the canonical way to programmatically interact with CMake from external build systems and tools. CMake file-api uses JSON files to communicate CMake state and target graph to external tools. CMake file-api replaces the removed cmake-server API. CTest can output JSON describing the test graph, especially useful to understand complex test dependencies including test fixtures.

A complete standalone example of using CMake file-api from Python helps illustrate these ideas.

CTest runs can take an arbitrarily long time depending on the number and duration of tests configured in a CMake project. Whether one accidentally hit Ctrl+C in the Terminal window or intentionally stopped a CTest run with options like --stop-on-failure, it’s possible to resume from where CTest left off with ctest -F. If CTest ran all tests (whether or not there were failures of tests), then CTest will ignore “-F” and run all tests again or as specified by other CTest command line options.

NOTE: using git rebase in general can wipe out a lot of work in difficult or impossible to recover ways. Before doing operations like this, create a copy of the Git branch and push it to recover from undesired outcomes.

Git rebase or merge operations on a feature branch with numerous commits can be fatiguing when it’s known that all of either the local or remote changes should be accepted in conflicts. This auto-accept of changes can be done on some or all changed files.

Git rebase auto-accept operations have the reverse sense of merge.

Rebase on “main”, auto-accept all local changes in conflicts:

git rebase -X theirs main

Rebase on “main”, auto-accept all remote changes in conflicts:

git rebase -X ours main

Git merge auto-accept operations have the reverse sense of rebase.

Merge “main”, auto-accept all local changes in conflicts:

git merge -X ours main

Merge “main”, auto-accept all remote changes in conflicts:

git merge -X theirs main

Reference

CMake FetchContent and ExternalProject bring remote projects into the top-level project to avoid making a monorepo or vendoring code into the top-level project. For those who desire more control over the remote project download process for FetchContent and ExternalProject, a method demonstrated in this script can be used:

CMake is a meta build system that generates build files such as Makefile, build.ninja etc. consumed by the build system such as GNU Make, Ninja, etc. corresponding to the CMake Generator selected. The input files for the build system generated by CMake themselves require CMake. They are not directory portable.

The most general approach uses CMake’s build command after configuration like:

cmake -B build
cmake --build build

An equivalent alternative is to directly invoke the build tool like:

cmake -B build -G "Unix Makefiles"
make -C build

or

cmake -B build -G Ninja
ninja -C build

Even when directory invoking the build tool, observe that editing the CMake script then invoking the build tool directory reconfigures CMake before commencing the build.

Example: Suppose

main.c:

int main(void) { return 0; }

CMakeLists.txt:

project(hello LANGUAGES C)

add_executable(main main.c)

• cmake -G Ninja generates a 17 kB build.ninja.
• cmake -G "Unix Makefiles" generates 15 kB of three distinct Makefiles.

Numeric-focused libraries and programs may wise to “raise” and/or “catch” warnings generated by Numpy operations. By default for Numpy data types operations that raise with plain Python types may only warn with Numpy types.

Use numpy.errstate context manager or decorator to raise Numpy arithmetic warnings as errors. We do this INSTEAD of “numpy.seterr()” that changes the global Numpy settings. That is, scripts or packages that call this module would be affected by numpy.seterr() in surprising ways, while numpy.errstate as decorated or context manager limits the scope of the change to the function or context.

Make Numpy arithmetic warnings raise ArithmeticError as seen in this example.