Scientific Computing

CMake --graphviz and graphviz configure preset can generate GraphViz dependency graphs for CMake-supported project code languages including C, C++, and Fortran. Fortran executables and modules are shown in the directed dependency graph. Fortran submodule are not shown in the graph.

The “dot” GraphViz program converts the .dot files to PNG, SVG, etc. dot program is available by installing the “graphviz” program via the package manager.

Generating the dependency graph requires CMake configure and generate. Thus, the compiler and generator needed by the CMake project must be working. The project does not need to be compiled before generating the dependency graph. However, the user should select the same CMake configure options as they would for compiling the project.

Example: h5fortran HDF5 object-oriented Fortran dependency graph is below. SVG vector graphics can be zoomed arbitrarily large in a web browser. The “gfx/” directory is to avoid making files in the source directory.

cmake -B build --graphviz=gfx/block.dot

cd gfx

dot -Tpng -o block.png block.dot

dot -Tsvg -o block.svg block.dot

The “dependers” files show only the nodes depending on a node.

Scripts and Viewing output

CMakeUtils graph.py converts a directory of CMake dot diagrams to SVG or PNG and collects them in an HTML document for easy viewing:

python cmakeutils/graph.py ~/myprog/gfx

To open a web browser from the Terminal:

python -m webbrowser -t file:///$HOME/myprog/gfx/index.html

Note that “file:///” has three slashes and the file path must be absolute.

Related: Dependency graphs are also easily created in Python and Matlab.

The full path to executables on the system Path (and cwd on Windows) are discovered by Python shutil.which. On Windows, environment variable PATHEXT is used to search filename suffixes if not specified at the input to shutil.which().

Shell aliases are not found by shutil.which() since the shell is not invoked. Instead append the directory of the desired executable to environment variable PATH, or specify it in shutil.which(..., path="/path/to/exe").

import shutil

# None if executable not found
exe = shutil.which('ls')

Since shutil.which() returns None for non-found executable it is convenient for pytest.mark.skipif

For programs not on PATH where the executable path is known:

shutil.which('myexe', path="/path/to/myexe")

Homebrew can install Fortran compilers including GCC and LLVM Flang.

Gfortran

Gfortran comes with GCC Homebrew package:

brew install gcc

As a complete C / C++ / Fortran compiler package, Gfortran doesn’t require additional flags or environment variables.

To use GCC compilers, source a script like:

p=$(brew --prefix gcc)/bin
v=14

export CC=$p/gcc-$v CXX=$p/g++-$v FC=$p/gfortran-$v

# to avoid GCC include errors -- MacOSX15.sdk incompatable at the moment with Homebrew GCC
export SDKROOT=/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk/

where v=14 is the major version number of the GCC compiler installed. The SDKROOT line may be necessary when Homebrew GCC package hasn’t yet enabled the latest SDK–adjust to suite the system.

LLVM Flang

LLVM Flang is a separate package from the LLVM C/C++ compilers:

brew install flang

To use Flang compiler (works with Clang, AppleClang, GCC C-C++ compilers), source a script like:

export FC=$(brew --prefix flang)/bin/flang-new

To use LLVM Clang with Flang, source a script like:

p=$(brew --prefix llvm)/bin
export CC=$p/clang CXX=$p/clang++

export FC=$(brew --prefix flang)/bin/flang-new

Troubleshooting

When a new compiler version or macOS version or Xcode SDK is released, it may be necessary to adjust the environment variables or flags temporarily until Homebrew updates the package.

Some examples to try if needed:

• LIBRARY_PATH for libSystem.tbd and libc++.tbd

  export LIBRARY_PATH=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib

• LDFLAGS to the C++ standard library

  export LDFLAGS=-lc++

CMake find_program does not generally consider NAMES parameter to have file suffixes unless manually specified. For Windows, .com and .exe file suffixes are considered, with search order:

1. .com
2. .exe
3. no suffix

If on Windows and an executable “hello.exe” or “hello.com” exists, then CMake will find it. CMake would NOT find “hello.exe” on non-Windows platforms, where no file suffix is expected.

The full path to executables on the system Path (and cwd on Windows) are found by find_program(). Shell aliases are not found since the shell is not invoked. Instead specify find_program(... HINTS /path/to/exe).

NOTE: CMAKE_EXECUTABLE_SUFFIX ONLY affects find_program() in CMake role PROJECT

find_program(v NAMES hello)

Shell scripts of any file suffix on any operating system are found iff:

1. (non-Windows) script file executable flag is set (find_program general requirement)
2. script file suffix is specified as part of find_program(… NAMES) parameter

A complete standalone example:

The CMAKE_GENERATOR cache variable records the CMake generator used to configure the CMake project in the CMakeCache.txt CMake cache file. Surprisingly, the cmake -LA option does not emit the CMAKE_GENERATOR value.

Thus, parsing CMakeCache.txt will give the previously used CMake generator. This is relevant in automated processes such as CI/CD systems that may build for numerous configurations and generators. This parsing can be trivially done in scripts in many coding languages. Here we give an example in CMake script “detect_gen.cmake”:

cmake_minimum_required(VERSION 3.21)

file(REAL_PATH ${bindir} bindir EXPAND_TILDE)

file(READ "${bindir}/CMakeCache.txt" _cache)

if(_cache MATCHES "CMAKE_GENERATOR:INTERNAL=([^ \n]+)")
  set(CMAKE_GENERATOR ${CMAKE_MATCH_1})
  message(STATUS "Detected CMake generator: ${CMAKE_GENERATOR}")
else()
  message(FATAL_ERROR "Failed to detect CMake generator")
endif()

This script is used like:

cmake -Dbindir=/path/to/build -P detect_gen.cmake

CMake directory property CMAKE_CONFIGURE_DEPENDS can be used to specify additional dependencies for the configuration step. For example, if a JSON file is used to specify source files, CMake wouldn’t detect if a source file was added, removed, or modified without CMAKE_CONFIGURE_DEPENDS.

Sometimes, the situation is too complicated to specify all dependencies manually. If a change is made that requires CMake to rebuild the cache, two equivalent ways to do this without modifying previously set options are:

cmake -Bbuild
# preserves prior options

or

cmake --build build -t rebuild_cache

These techniques work with any versioned program or library. Here we use Matlab as an example. CMake find_package with a version range would be used to simply select from a known-working version range.

Many Matlab codes require a modern version of Matlab. It’s possible to select from an arbitrary min…max range of Matlab versions with CMake FindMatlab as follows. This technique works with other versioned programs and libraries as well.

foreach(v IN ITEMS 23.2 24.1 24.2)
  find_package(Matlab ${v} EXACT COMPONENTS MAIN_PROGRAM)
  if(Matlab_FOUND)
    add_test(NAME matlab-${v}
      COMMAND ${Matlab_MAIN_PROGRAM} -batch "buildtool"
      WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
      )
  endif()
endforeach()

Git typically works for HTTPS URLs using the default configuration. If certificates are not working, try configuring Git with the system SSL certificate location.

git config --global http.sslCAInfo /path/to/cert.crt

Git environment variable “GIT_SSL_CAINFO” can override this. This can be used to override missing or incorrect SSL certificate locations for Git operations in programs like Matlab.

To disable SSL certificate verification, set environment variable “GIT_SSL_NO_VERIFY” to “1”, but be aware of the security implications e.g. MITM.

Several modern, currently-supported compiler families are free-to-use for C, C++ and Fortran.

GCC has broad support of modern standards on a very wide range of computing platforms. GCC’s downside in some cases can be slower runtime performance than compilers having less broad language and platform support.

Intel oneAPI compilers are free to use for any user. The Intel performance libraries like MKL, IPP, and TBB are available at no cost.

LLVM Clang and Flang have significant industry support, including from Nvidia, and are known for high performance.

AMD AOCC LLVM compiler is tuned for AMD CPUs.

NVIDIA HPC SDK is free to use. A key feature of the HPC SDK compilers is intrinsic support for CUDA Fortran.

IBM OpenXL LLVM-based compilers are currently for POWER CPUs only e.g. ppc64le. IBM OpenXL compilers do not work with a typical x86-based computer.

Related: GCC / Clang header clash on macOS

When macOS, Xcode, or Command Line Tools upgrades, build directories (including for CMake-based projects) often need to be refreshed. If the user has set custom environment variables concerning Xcode, they may need to be updated as well. Here are some important environment variables and CMake variables to check if problems occur after upgrading.

For a simple CMake project on macOS, CMakeCache.txt might include:

CMAKE_OSX_DEPLOYMENT_TARGET:STRING=

CMAKE_OSX_SYSROOT:PATH=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX*.sdk

• environment variable MACOSX_DEPLOYMENT_TARGET helps set CMAKE_OSX_DEPLOYMENT_TARGET
• Environment variable SDKROOT helps set CMAKE_OSX_SYSROOT

Having multiple directories under the following is fine:

• /Library/Developer/CommandLineTools/SDKs
• /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs

CMake can select the appropriate SDK by the user specifying environment variable SDKROOT or by selecting the SDK:

xcode-select --switch

The XCode versions that Homebrew targets can be found in xcode.rb.

Workarounds

If Homebrew GCC breaks after upgrading Xcode or Command Line Tools, try specifying an older SDK. For example, if g++-14 main.cpp -v shows a different (older) SDK than CMake and it works, try specifying that SDK in environment variable SDKROOT. Note that the SDK version corresponds to macOS version, not the XCode version. For example, if the latest SDK is MacOSX14.4.sdk, try using MacOSX13.3.sdk in “~/gcc.sh”:

export CC=gcc-14 CXX=g++-14 FC=gfortran-14

export SDKROOT=/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk/

and then source ~/gcc.sh before running cmake with a fresh build directory.

If a CMake build step fails, try copy-pasting the command and removing the -isysroot portion of the command. This is a clear clue the older SDK is (at least temporarily) needed till Homebrew updates its GCC formula.

GCC will tell where included files are coming from by adding the gcc -H flag. This tells what to specify for environment variable SDKROOT.

Example minimal project