Scientific Computing

Measuring the peak RAM usage of a process and all its children can be done using various tools and techniques. OS-dependent tools may be the most accurate, but they can be complex to use. A simpler approach is to periodically sample the RAM usage of the process and its children, like this scripts for Linux and macOS using ps.

It is also possible though less accurate on macOS or Linux to use /usr/bin/time, but this only measures the peak RAM usage of the largest child process, not the total of all children, so this is unsuitable for multiprocess applications like “mpiexec”.

For Linux, a more accurate method is the Cgroup v2, such as implemented by cgmemtime. For macOS, the Instruments tool can be used to measure the RAM usage of a process and its children, but it requires a ‘codesign’d application and is more complex to set up.

xcrun xctrace record --template "Game Memory" --launch -- /path/to/application --output bench_game.trace --time-limit 30s

open bench_game.trace

Valgrind is a dynamic analysis tool that can detect memory leaks and other problems in programs including C, C++, and Fortran. Valgrind is available on Linux and BSD for x86 and ARM CPU architectures.

For macOS on Intel x86 CPUs, Valgrind is available in Homebrew.

For macOS on Apple Silicon ARM64 CPU, Valgrind can be used from an aarch64 Linux virtual machine in native mode for best performance. There is a development fork of Valgrind for Apple Silicon, but it may not yet ready for production use.

On macOS, when started from the Applications icon, Matlab does NOT source shell configuration files. Any environment variables set in these files such as those added by package managers like Homebrew or Conda will not be available in the Matlab environment.

On macOS, only if Matlab is started from the Terminal like /Applications/MatlabR20*.app/bin/matlab will Matlab source the shell configuration file, leading to the expected environment variables as default in an interactive login shell also available in Matlab.

• .matlab7rc.sh,
• ~/.zshrc or ~/.bashrc.

Instead of depending on how the user launched Matlab, a more uniform approach is to have users set environment variables in the Matlab startup.m script, which will be sourced regardless of how Matlab is launched. Set desired environment variables in the Matlab startup.m script like:

edit(fullfile(userpath,'startup.m'))

Add lines like

setenv('PATH', ['/opt/homebrew/bin/', pathsep, getenv('PATH')])

Where ‘/opt/homebrew/bin’ was obtained from the system command brew --prefix. We recommend not running system() scripts in startup.m unless truly necessary to avoid delaying Matlab startup time or affecting startup stability.

Then programs installed by Homebrew like CMake, GCC, etc. will be on Path environment variable in Matlab.

Note that the Matlab commands below only affect commands within that same command line:

!source ~/.zshrc && ls

system("source ~/.zshrc && ls")

In larger projects, one might not remember if they or a colleague has opened or commented on a prior Issue topic. To search the issues by a username, use this search syntax in the GitHub Issues search bar:

• Issue created by a user:

is:issue author:username

• Issue commented on by a user:

is:issue commenter:username

Note that the search term is not user:.

This maps into the GitHub API as a query parameter:

?q=is:issue+author:username

For example at the Bootstrap project for one’s self “@me”:

https://github.com/twbs/bootstrap/issues?=q=is%3Aissue+author%3A%40me

C has long had the __FILE__ macro, which expands to a string literal containing the name or path to the current source file. C++ provides a much more powerful mechanism to programmatically determine the source file location via source_location in the <source_location> header introduced in C++20. This capability is particularly powerful for logging and debugging, as shown in the example linked above that a function’s caller can be located by source file and line number without the caller needing to explicitly pass that information.

Sometimes when opening the Matlab Desktop, the Command Window where one types commands might not be visible. This can be restored from the “Home” tab on the top ribbon, click the “Layout” button on the Environment section, and click “Two Column (default)”. Notice that one can also “Save Layout” to store the current layout for future use and easily select between layout suitable for different tasks.

At the the writing, UTM didn’t have a known way to work with RISC-V, so we use QEMU directly on macOS (or Linux). Obtain QEMU - on macOS via brew install qemu or on Linux via the package manager (e.g. apt install qemu). We need a uboot.elf. One way to get it is from a Linux machine or VM if on macOS by apt install u-boot-qemu and then copy the file from /usr/lib/u-boot/qemu/riscv64/uboot.elf to the machine where you want to run QEMU.

cp /usr/lib/u-boot/qemu-riscv64_smode/uboot.elf ~/uboot.elf

Then obtain the Ubuntu LTS RISCV image.

Extract the image like:

unxz ubuntu-24.04.4-preinstalled-server-riscv64.img.xz

It’s important to increase the disk size, otherwise the image will run of space quickly when installing compilers etc. Add say 10 GB of virtual HD space by:

qemu-img resize -f raw ubuntu-*-preinstalled-server-riscv64.img +10G

Then run QEMU like (perhaps make a little script):

img=$HOME/riscv_vm/ubuntu-24.04.4-preinstalled-server-riscv64.img
uboot=$HOME/riscv_vm/uboot.elf

qemu-system-riscv64 \
-machine virt -nographic -m 2048 -smp 4 \
-kernel ${uboot} \
-device virtio-net-device,netdev=eth0 -netdev user,id=eth0 \
-device virtio-rng-pci \
-drive file=${img},format=raw,if=virtio

This should boot the Ubuntu image and in a few minutes get to the login prompt. The default username is ubuntu and the password is ubuntu.

When using Git pre-commit hooks with the wide range of files typical of scientific computing, it may occur that the hooks modify files that should be excluded, such as FITS files that have a text header and a binary data part. Exclude such files by using case-insensitive regex patterns in the .pre-commit-config.yaml file by top level exclude key, for example to case-insensitively exclude files with the .fit or .fits extension, use:

exclude: (?i)\.fits?$

Because it’s considered an anti-pattern to have a global pre-commit config, such exclusions and all other pre-commit configuration on a per-repo basis.

To compile legacy Fortran code, certain compiler flags can be used to enable non-standard Fortran syntax that was common before the Fortran 95 standard became widely adopted. It’s hard to pin an exact year for when developers transitioned to more standard Fortran code, but the mid-2000s is a reasonable estimate for when Fortran codebases started to modernize in significant numbers. Gfortran took over from g77 as the default Fortran compiler circa 2005 and was the first widely used free Fortran compiler capable of the Fortran 95 standard. Here are legacy-enabling flags for currently maintained Fortran compilers.

In addition to the flags below, it may be necessary to provide default real and / or integer precision flags to compile old code that relies on the default precision being different than the modern default of 4 bytes for real and 4 bytes for integer.

GNU GFortran

• -std=legacy enables pre-Fortran 77 arbitrary length arrays, where A(1) was declared instead of A(*) in Fortran 77 or A(:) in modern Fortran.
• -fdec- options enable various non-standard extensions in the DEC Fortran style.
• see also Gfortran runtime options for issues reading binary files.

LLVM Flang

Flang compiler flags have a “-std” option that might be checked to see if the project sets it too restrictively.

Intel oneAPI

• -nostand does not change compiler behavior–it disables standard-based warnings.
• -f66 apples Fortran 66 semantic rules, including Do loops that always run at least once.

IBM OpenXL

• -qlanglvl=extended disables compiler language level checks, and is the default.
• several more options may be employed if needed for compatibility

Cray Fortran

The ftn compiler can disable some checks with flags like -d C

Git uses per-user ~/.gitconfig, per-repository .git/config configuration files, and environment variables to customize Git behavior. The system gitconfig file is not covered here. A common pattern is a developer makes most of their settings in the per-user Git config file, occasionally overriding them in the per-repository config file.

We have several articles on Git configuration, leading to a per-user Git config file with sections like:

pull HTTPS for speed, push SSH for security

[url "ssh://github.com/"]
	pushInsteadOf = https://github.com/
[url "ssh://gitlab.com/"]
	pushInsteadOf = https://gitlab.com/
[url "ssh://gist.github.com/"]
	pushInsteadOf = https://gist.github.com/scivision/
[url "ssh://gitlab.kitware.com/"]
    pushInsteadOf = https://gitlab.kitware.com/

Use Meld for diff and merge

[diff]
	tool = meld
[merge]
	tool = meld

Maintain linear Git history with fast-forward only by default with pull.ff only

[pull]
    ff = only

Check upon “git push” if all Git submodules have first been “git push” before the consuming repository, to avoid broken builds for others with push.recurseSubmodules check

[push]
	recurseSubmodules = check

For any Git operation (except “git clone”) that changs references to Git submodules commit hashes, automatically update the Git submodules to match the new commit hashes with option submodules.recurse true

[submodule]
	recurse = true