Scientific Computing

CI systems typically set the environment variable CI as a de facto standard for easy CI detection. Here are details of several popular CI services:

• GitHub Actions
• Travis-CI
• AppVeyor
• CircleCI
• GitLab
• Bitbucket Pipeline

In general, across programming languages, test frameworks allow behavior changes based on the environment variables set by the CI system.

Python Pytest

Pytest handles conditional tests well. This allows skipping a subset of tests on CI by detecting the CI environment variable:

import os
import pytest

CI = os.environ.get('CI') in ('True', 'true')


@pytest.mark.skipif(CI, reason="not a test for CI")
def test_myfun():
    ...

CMake CTest

CTest can also use CI environment variables to adjust test behavior.

if("$ENV{CI}")
  set(NO_GFX on)
endif()

add_test(NAME menu COMMAND menu)
set_tests_properties(menu PROPERTIES DISABLED $<BOOL:${NO_GFX}>)

Always enclose "$ENV{}" in quotes in case the environment variable is empty or not defined, or the configuration step may syntax error.

Fortran compilers typically use 4 bytes for logical while C compilers usually use 1 byte for bool.

oneAPI flag -fpscomp logicals ensures integer values 0,1 corresponding to .false., .true. are used. Otherwise by default unexpected values may cause programs to fail at runtime with impossible boolean values like 255.

In CMake this flag is applied like:

if(CMAKE_Fortran_COMPILER_ID MATCHES "^Intel")
  add_compile_options("$<$<COMPILE_LANGUAGE:Fortran>:-fpscomp;logicals>")
elseif(CMAKE_Fortran_COMPILER_ID STREQUAL "NVHPC")
  add_compile_options("$<$<COMPILE_LANGUAGE:Fortran>:-Munixlogical>")
endif()

For C interoperability, Fortran can use:

use, intrinsic :: iso_c_binding

logical(kind=C_BOOL) :: L
logical :: Q

c_sizeof(L) == 1
c_sizeof(Q) == 4
! typically

while C uses:

#include <stdbool.h>
#include <stdio.h>

int main(void) {
bool L;
printf("%zu\n", sizeof(L));
}

and likewise C++ bool is typically 1 byte:

#include <iostream>

int main(){
  bool L;
  std::cout << sizeof(L) << "\n";
}

Always use iso_c_binding when using C or C++ with Fortran modules to produce cross-platform compatible projects.

See “bool” examples for interfacing between C, C++ and Fortran.

Do not use

The Intel oneAPI compiler -standard-semantics flag to use C_BOOL values correctly between C, C++ and Fortran has a BIG DOWNSIDE in that it breaks linking with system libraries–including IntelMPI! All projects and libraries linking to a Fortran library that used oneAPI -standard-semantics must also use -standard-semantics.

GCC 5 enabled the __has_include macro that checks if a source file exists. __has_include() was made language standard syntax in C++17 and C23. __has_include("file.h") checks if “file.h” exists as a file on the compiler includes paths, but not if “file.h” can be included with the specified compiler language standard.

Watch out for buggy __has_include() in GCC < 10. We found that __has_include("linux/magic.h") with quotes "" instead of the usual carets __has_include(<linux/magic.h>) was needed specifically for header linux/magic.h. Other systems libraries with carets worked as expected even with GCC < 10.

#if __has_include("linux/magic.h")  // quotes needed for GCC < 10
#  include <linux/magic.h>
#endif

Examples

Compile and run the example number.cpp code.

% c++ -std=c++20 numbers.cpp

% ./a.out

3.14159

numbers.cpp:

#if __has_include(<numbers>)
#  define HAVE_NUMBERS
#  include <numbers>
#endif

#include <iostream>
#include <cstdlib>

int main(){
    // numbers example:
#ifdef HAVE_NUMBERS
    std::cout << std::numbers::pi << std::endl;
#else
    std::cout << "std::numbers::pi not available" << std::endl;
#endif
    return EXIT_SUCCESS;
}

CMake FetchContent is useful to incorporate subprojects at configure time. FetchContent subproject cache variables can override the top-level project cache, which can be confusing.

Projects may wish to use a default install prefix when cmake –install-prefix is not specified. Environment variable CMAKE_INSTALL_PREFIX can set a default install prefix across projects.

Some projects force a default install prefix if not specified by the top level project:

if(PROJECT_IS_TOP_LEVEL AND CMAKE_INSTALL_PREFIX_INITIALIZED_TO_DEFAULT)
  set_property(CACHE CMAKE_INSTALL_PREFIX PROPERTY VALUE ${CMAKE_BINARY_DIR}/local)
endif()

However, it is more a canonical approach to check the install prefix write permissions.

message(STATUS "CMAKE_INSTALL_PREFIX: ${CMAKE_INSTALL_PREFIX}")
file(MAKE_DIRECTORY ${CMAKE_INSTALL_PREFIX})
if(CMAKE_VERSION VERSION_GREATER_EQUAL 3.29)
  if(NOT IS_WRITABLE ${CMAKE_INSTALL_PREFIX})
    message(FATAL_ERROR "CMAKE_INSTALL_PREFIX is not writable: ${CMAKE_INSTALL_PREFIX}")
  endif()
else()
  file(TOUCH ${CMAKE_INSTALL_PREFIX}/.cmake_writable "")
endif()

CMake Presets install prefix

CMakePresets.json can set a default install prefix:

{
  "version": 3,

"configurePresets": [
{
  "name": "default",
  "binaryDir": "${sourceDir}/build",
  "installDir": "${sourceDir}/build/local"
}
]
}

C23 specification added C++-like attribute specifiers to C. C attribute specifiers add metadata to declarations and definitions. The metadata can be used by the compiler and by the developer to help understand the code intent.

Commonly used attributes such as [[maybe_unused]] and [[fallthrough]] suppress compiler warnings.

Compilers including GCC ≥ 11 have __has_c_attribute() to check if an attribute is supported by the compiler–even if the command line specified standard is older.

maybe_unused attribute

To enable code to fallback to no attribute with older compilers, use logic in header file like:

#if !defined(__has_c_attribute)
#  define __has_c_attribute(x) 0
#endif

#if __has_c_attribute(maybe_unused)
#  define MAYBE_UNUSED [[maybe_unused]]
#else
#  define MAYBE_UNUSED
#endif

then in the definition:

int windows_fun(int x, MAYBE_UNUSED int y) {

#ifdef _WIN32
  return y;
#else
  return x;
#endif
}

On non-Windows systems, the compiler would have issued a warning about y being an unused argument. The [[maybe_unused]] attribute suppresses that warning.

fallthrough attribute

The [[fallthrough]] attribute is used to indicate that a fall-through in a switch statement is intentional. This attribute can be used to suppress warnings about missing break statements in a switch block.

In declaration (header) file do like:

#ifndef __has_c_attribute
#define __has_c_attribute(x) 0
#endif
#if __has_c_attribute(fallthrough)
// GCC >= 11
#define FALLTHROUGH [[fallthrough]]
#elif defined(__GNUC__) && __GNUC__ >= 7 || defined(__clang__) && __clang_major__ >= 12
#define FALLTHROUGH __attribute__((fallthrough))
#else
#define FALLTHROUGH
#endif

then in the definition:

switch (x) {
  case 1:
    x++;
    FALLTHROUGH;
  case 2:
    x--;
}

C++ attribute specifiers add metadata to declarations and definitions. The metadata can be used by the compiler and by the developer to help understand the code intent.

Commonly used attributes such as [[maybe_unused]] and [[fallthrough]] suppress compiler warnings. [[likely]] and [[unlikely]] may be used by the compiler to optimize compilation, and also provide human code readers insight into the intent of the algorithm.

GCC ≥ 5 feature test macro __has_cpp_attribute() checks if an attribute is supported by the compiler–even if the command line specified standard is older.

maybe_unused attribute

To enable code to fallback to no attribute with older compilers, use logic in header file like:

#if !defined(__has_cpp_attribute)
#  define __has_cpp_attribute(x) 0
#endif

#if __has_cpp_attribute(maybe_unused)
#  define MAYBE_UNUSED [[maybe_unused]]
#else
#  define MAYBE_UNUSED
#endif

then in the definition:

int windows_fun(int x, MAYBE_UNUSED int y) {

#ifdef _WIN32
  return y;
#else
  return x;
#endif
}

On non-Windows systems, the compiler would have issued a warning about y being an unused argument. The [[maybe_unused]] attribute suppresses that warning.

fallthrough attribute

The [[fallthrough]] attribute is used to indicate that a fall-through in a switch statement is intentional. This attribute can be used to suppress warnings about missing break statements in a switch block.

In declaration (header) file do like:

#ifndef __has_cpp_attribute
#define __has_cpp_attribute(x) 0
#endif
#if __has_cpp_attribute(fallthrough)
#define FALLTHROUGH [[fallthrough]]
#else
#define FALLTHROUGH
#endif

then in the definition:

switch (x) {
  case 1:
    x++;
    FALLTHROUGH;
  case 2:
    x--;
}

Python tempfile is generally robust.

TemporaryDirectory(ignore_cleanup_errors=True) fixes the Windows corner case on exiting a tempfile context manager such as cloning a Git repo into the TemporaryDirectory.

import tempfile

with tempfile.TemporaryDirectory(ignore_cleanup_errors=True) as tmpdir:
    # create subdirectories, make files, git clone, etc.
    # the context manager attempts to recursively delete "tmpdir" on exit

If there is a PermissionError, the temporary directory remains until the operating system cleans up the temporary directory.

Python tempfile.NamedTemporaryFile defaults for Python ≥ 3.12 handle file cleanup on Windows.

Since 2015, NVIDIA has supported Flang: a Fortran compiler frontend to LLVM. The first Flang was based on the PGI compiler, with proprietary optimizations removed. However, the C-based decade-plus of cruft led Flang developers and management to a ground-up rewrite in modern C++. This new Flang f18 targets Fortran 2018 syntax and is implemented in modern C++ internally. Flang became part of LLVM 11.

Building the LLVM Flang Fortran compiler from source is a resource-intensive process.

Pacman can download packages in parallel to speed up the process. The number of parallel download threads can be controlled in the “/etc/pacman.conf” file. This can be useful on slower internet connections to install packages without disrupting other network activities.

To set the number of parallel download threads, edit “/etc/pacman.conf” file. Find the “ParallelDownloads” option in the file. If it is not present, add it under [options] section. Set the number of download threads. The ordering of options doesn’t matter. For example, to use 3 parallel download threads:

[options]
ParallelDownloads = 3

For very slow networks, consider ParallelDownloads = 1 and pacman --disable-download-timeout to avoid timeouts.

Olivier Giroux chairs the ISO C++ subgroup for Concurrency and Parallelism. Olivier continues to present very useful interactive talks on C++ concurrency and parallelism. This talk is about the C++ Forward Progress Guarantee in light of concurrency and parallelism.

C++ Proposal P2809R3 is to allow trivial infinite loops as defined behavior in C++.

The definition Olivier uses for concurrency at 16:50 in the video above is:

Concurrent tasks eventually observe each other’s effects.

Previous talks on this subject by Olivier follow: