Scientific Computing

Coccinelle allows specifying desired matches and transformations in C code. It is a powerful tool for making large-scale changes to programs. Alternatives include regex matching with “sed”, “awk”, IDEs or AI-assisted tools.

Consider Coccinelle examples and AI-based tools before deciding which tool to use. In any case, manual checking will be necessary to ensure that the changes are correct.

Many Git fixup commits are for forgetting to check project code standards like:

• PEP8 compliance
• type annotation checks
• clang-format

Mitigate git commit fixup clutter by making a Git pre-commit hook that applies to all repositories. The pre-commit hook also does a simple check for YaML syntax. This procedure works across operating systems since Python script is used.

Tell Git the directory for Git hooks:

git config --global core.hooksPath ~/.git/hooks

Create executable file ~/.git/hooks/pre-commit from this Python pre-commit script.

This global pre-commit check hook does several things:

1. get a list of Python file names that changed.
2. check PEP8, mypy type hinting and that Python breakpoint debug statements are not present.
3. check YaML syntax
4. clears IPython notebook output cells
5. Checks for trailing whitespaces in any code language

One can easily extend the concept of lint-checking for other programming languages.

These checks can be bypassed at any time for a commit by:

git commit -n

Override this global pre-commit check, substituting a per-repo .git/hooks/pre-commit by in that repo directory typing:

git config core.hooksPath .git/hooks

For example, a website made of Markdown files may wish to run a local-link check via Linkchecker-Markdown.

Troubleshooting

The Python pre-commit script uses the typical Python shebang:

#!/usr/bin/env python3

If on Windows failure occurs like:

cannot spawn .git/hooks/pre-commit: exec format error

Try changing the shebang to:

#!/usr/bin/env python

especially if the command “python3” is not found when typing it in Terminal.

References:

• Git dry run
• Git precommit hooks docs

For macOS developers using Homebrew (possibly other package managers too) there is a TLS v1.3 bug where system curl was used by build servers and curl had a bug. One package impacted is CMake, though it’s not a CMake bug as Kitware CMake binaries using newer curl are fine.

The bug causes some TLS v1.3 download/upload internet operations to fail with code 35. This was due to a bug in macOS system curl. macOS has already updated curl for end users. Updating Homebrew CMake fixes this issue.

If stuck, one can set an environment variable per command to disable TLS v1.3 as a temporary workaround:

CURL_SSL_BACKEND=SecureTransport cmake ...

However, it is better to upgrade Homebrew CMake to a fixed curl version:

brew upgrade cmake

This is not a CMake bug, as the CMake binaries downloaded from Kitware work since they are built with a more recent non-macOS-system curl. This is a Homebrew packaging bug that can impact other programs distributed by Homebrew (or Macports) due to the cloud build servers using not-yet-updated macOS for the new curl.

Diagnose: the version of curl compiled into CMake may be obtained by UserAgent.cmake:

cmake -P UserAgent.cmake

If the curl version is less than 8.4.0 this macOS SSL 35 bug may exist with TLS v1.3 connections.

To help debug CMake SSL operations, I use BadSSL and TLS check.

The Cisco VPN Secure Client is a small app that runs on the user device to establish and monitor a VPN connection. It requires background services to be allowed to run at login/startup to function. Otherwise, the connection attempt will fail with a message like:

VPN connect capability is unavailable because the VPN service is unavailable

For example, on macOS under settings → Login Items, there may be about three Cisco items that need to be enabled. After enabling, reboot the computer and try the VPN connection again.

CMake presets allow simple or complex JSON configurations for configuring, building, and testing projects. A project maintainer may with to enforce the use of presets, which may be accomplished by using sentinel environment variable:

Assuming the console shell supports color output, it can be pleasant for users to have color build system (Make) and compiler output. This is easily set by setting environment variable CMAKE_COLOR_DIAGNOSTICS.

Set environment variable CMAKE_COLOR_DIAGNOSTICS=1 to have CMake colorize the build system and compiler output.

CTest by design combines stdout and stderr into stdout for each test. For programs that emit a lot of text to stdout and put only diagnostic content to stderr, the combined text stream can be burdensome when debugging a test failure. There is a proposal to add a CTest property to make optional separate stdout and stderr, but it is not yet implemented as of this writing.

Example

Many IDEs create per-project cache directories with metadata relevant to the IDE configuration for that project. Similar to user global .gitattributes, instead of editing the .gitignore file for each repository, ignore directories in Git for all repos a particular user has as follows, for any operating system.

git config --global core.excludesfile ~/.gitignore

Edit ~/.gitignore file to contain items like the following.

*.orig

git merge conflict files. To not generate git merge conflict .orig files:

git config --global mergetool.keepBackup false

.vs/

Visual Studio cache

.vscode/

Visual Studio Code cache

.idea/

PyCharm cache

pip-wheel-metadata/

pip cache

.mypy_cache/

mypy cache

.DS_Store

macOS Finder cache

*.pyc

Python compiled files

*.orig

.vs/
.vscode/
.idea/

CMake can print all cache variables using the cmake -LAH command. This will print all cache variables, including their help text.

cmake -LAH -B build

The -B build option is used to specify the build directory as usual.

cmake -LH omits the advanced variables that may be of less interest to general users.

cmake -L omits the help text.

Assuming the project has already been configured once, the cmake -N option added will load the existing cache without regenerating.

This command does not print non-cache variables, such as variables set in the CMakeLists.txt file or cmake -Dopt flags that are not cached.

Example CMakeLists.txt

project(ex LANGUAGES NONE)

option(my1 "option 1")

Running this command will print only “my1” value and not “my2” value because “my2” is not cached.

cmake -LAH -B build -Dmy1=on -Dmy2=on

On Windows with Python subprocess, an executable may be missing DLL from PATH. This environment error can be detected and a message given to the user.

The return code of the subprocess can be checked for the error code 3221225781, which corresponds to hex error code C0000135.

import subprocess
import os
import logging

ret = subprocess.run('myprog.exe')

if ret.returncode == 3221225781 and os.name == 'nt':
    # Windows 0xc0000135, missing DLL
    logging.error('missing DLL detected. Ensure DLL directory is in environment variable PATH')
    path = os.environ['PATH']
    print("PATH:", path)
    raise SystemExit(ret.returncode)

If putting DLLs on PATH is problematic, another possible approach is statically compiling the executable, but this may not be feasible for all programs.