Scientific Computing

macOS/Windows/Linux get/set sudo/admin users from Terminal

Each of macOS, Windows, and Linux has a way to get and set which users have administrative privileges (i.e. can use sudo or are in the Administrators group) from Terminal.

macOS get / set sudo users from Terminal

On macOS, the users who are sudo -capable (i.e. administrators) can be discovered from Terminal using the Directory Service command line utility dscl:

dscl . -read /Groups/admin GroupMembership

To add a user to the sudoers list, use the following command:

dscl . -append /Groups/admin GroupMembership <username>

Windows get Administrator users from Terminal

On Windows, the users who are Administrator capable can be discovered from PowerShell using:

Get-LocalGroupMember -Group "Administrators"

Query a specific user to see if they are an Administrator:

$user = "myUsername"
# put the desired username in the $user variable, then run the following command:

Get-LocalGroupMember -Group "Administrators" | Where-Object { $_.Name -like "*\$user" }

Make a user an Administrator:

Add-LocalGroupMember -Group "Administrators" -Member "myUsername"

For Windows Subsystem for Linux (WSL), use the Linux commands below.

Linux get / set sudo users from Terminal

On Linux, the users who are sudo -capable (i.e. administrators) can be discovered from Terminal using:

getent group sudo

Add a user to the sudo group (allowing them to use sudo):

sudo usermod -aG sudo <username>

List connected hard drives on macOS

List connected hard drives on macOS using the Terminal:

diskutil list

The “NAME” column shows the names of the hard drives under “/Volumes”. The following command gives a more detailed view of the connected hard drives:

system_profiler SPStorageDataType

Keep computer running disable suspend

Long-running Terminal programs might not be detected by the operating system on a laptop or other device that has power-saving sleep or suspend modes. This can lead to disappointment as the long-running program hasn’t completed when the user checks back. This caffeinate Python script works with:

to keep the computer awake while a command line program is running, using features built into the operating system utilities.

To avoid a computer going to sleep / suspend in general persistently is OS-dependent.

macOS power management from Terminal

For macOS, check and save to a text file the current power management settings with pmset:

pmset -g

To disable sleep / standby while on AC power (charger), which is also suitable for fixed devices like the Mac Mini:

pmset -a sleep 0 standby 0

Linux power management from Terminal

For Linux using systemd, check the current power management settings with:

systemctl list-unit-files | grep -E 'sleep|suspend|hibernate|hybrid'

look for any that show “enabled”.

To disable sleep while on AC power (charger), which is also suitable for fixed devices like a desktop, use systemctl mask to disable the sleep, suspend, hibernate, and hybrid-sleep targets:

systemctl mask sleep.target suspend.target hibernate.target hybrid-sleep.target

Windows power management from Command Prompt

For Windows, check and save to a text file the current power management settings with powercfg:

powercfg /query

Look for “Current AC Power Setting Index” in the output for “Sleep” and “Hibernate” to see the current settings.

To disable sleep while on AC power (charger), which is also suitable for fixed devices like a desktop:

powercfg /change standby-timeout-ac 0

To set Hibernate to never while on AC power (charger), while allowing hibernate to be enabled for battery power (important for laptop tucked away for the weekend), run the following command:

powercfg /change hibernate-timeout-ac 0

Note that powercfg /hibernate off completely disables hibernate, which is not necessarily recommended for laptops that may be stored without power when not manually shut down each time.

Build CMake from dev sources

While CMake binaries can be downloaded for most platforms, there are certain cases where one wishes to build CMake from source. For example, when preparing a merge request to fix or enhance CMake. Usually the computer will have at least an older version of CMake that can be used. If so, we recommend using the existing CMake to build the newer CMake from the CMake source directory. We recommend using Ninja in general for faster build and rebuild for any CMake project.

cmake -B build -DBUILD_TESTING:BOOL=OFF -DCMAKE_BUILD_TYPE=Release --install-prefix=$HOME/cmake-dev -G Ninja

cmake --build build

cmake --install build

A one-step download-build script is the scripts/build_cmake.cmake.


If an old CMake isn’t available on the computer, then use CMake bootstrap:

./bootstrap --prefix=$HOME/cmake-dev --generator=Ninja -- -DBUILD_TESTING:BOOL=OFF -DCMAKE_BUILD_TYPE:STRING=Release

ninja -C Bootstrap.cmk/ install

This puts the compiled CMake under ~/cmake-dev, without disturbing the primary CMake install. Upon making any CMake code changes, simply recompile the minimum needed bits by:

ninja -C Bootstrap.cmk/ install

Find Windows App install location with winget

Regardless of how an App is installed, whether through WinGet, the Microsoft Store, or a direct download, WinGet can be used to find the program install location on disk. For example, to find the install location of the Microsoft Edge browser:

winget list msedge --details

(one can also use the names Microsoft.Edge or Edge)

The results include “Installed Location” which for this example may be examined like:

ls ${Env:ProgramFiles(x86)}/microsoft/edge/application

with results including the Edge executable msedge.exe.

Matlab .NET on Linux, macOS, and Windows

Matlab external language interfaces includes .NET on Windows, Linux, and macOS. This allows efficiently calling .NET assemblies and using .NET libraries directly from Matlab.

The Matlab function dotnetenv is used to set up and check the active .NET environment in Matlab. Environment variable DOTNET_ROOT is vital for Matlab to detect the .NET installation, particularly on Linux and macOS. If Matlab is having issues detecting the .NET installation NET.isNETSupported is false, determine the value for DOTNET_ROOT from system Terminal:

dotnet --info

If “dotnet” command is not found, install .NET SDK:

  • macOS: brew install dotnet
  • Windows: winget search Microsoft.DotNet.SDK
  • Linux: from Microsoft
  • Matlab Online (Linux): if .NET isn’t detected, use .NET install script to install .NET SDK.

On macOS, determine environment variable DOTNET_ROOT by $(brew --prefix dotnet)/libexec.

If this path is not defined, look for the Base Path, and pick the directory that contains the “dotnet” executable.

Set the DOTNET_ROOT environment variable in Matlab by adding to Matlab startup.m so that future Matlab sessions have the correct DOTNET_ROOT set.

edit(fullfile(userpath,'startup.m'))

Then add the following line to startup.m:

setenv('DOTNET_ROOT', '/opt/homebrew/opt/dotnet/libexec')

Where the path was obtained from $(brew --prefix dotnet)/libexec on macOS, or from the .NET install location on Linux.

Restart Matlab. Do the following one-time command to finish setting up .NET in Matlab:

dotnetenv("core", Version="10")

Where version number “10” must match the major version of .NET in DOTNET_ROOT as shown by dotnet --list-sdks in Terminal.

In future Matlab sessions, dotnetenv() will be empty until the first .NET command is run, for example

NET.isNETSupported

then (optionally) running dotnetenv() shows the current .NET environment.

Fortran stop return codes

Summary:

  • stop (1956 Fortran I): return integer on stderr (recommendation)
  • stop (Fortran 77): return integer or constant character, if character it may or may not be printed, but return code is 0 (no error)
  • error stop (Fortran 2008): constant character with error code
  • error stop (Fortran 2018): variable character with error code, also allowed inside pure procedure.

Fortran 2018 finally brought the needed behavior for convenient error messages and continuous integration.

CMake and Meson handle automatic detection of compiler supported features like error stop.

Fortran 2008 error stop with constant code:

  • Gfortran ≥ 5
  • NAG ≥ 6.0
  • Intel oneAPI
  • Nvidia HPC SDK

Fortran 2018 error stop with variable code

  • Gfortran ≥ 7
  • NAG ≥ 6.2
  • Intel oneAPI
  • Nvidia HPC SDK

Fortran 2018 error stop,QUIET=.true./.false.

  • NAG ≥ 6.2

This feature was promoted by Steve Lionel, but has not yet been widely adopted. From the Fortran 2018 standard, the quiet= parameter not only suppresses any console output but also may suppress the error code? I would prefer to have the error return code, without the console text.

Since Fortran I in 1956, the stop statement has generally displayed a return code to indicate an error if an integer value was provided. Over time, stop statement behavior has changed to allow more refined signaling on program stop.

Since Fortran I in 1956, stop without return code to stop execution normally has been supported, along with stop with integer return code to indicate abnormal termination.

stop 1

The Fortran 2008 and 2018 standards recommend that the error code be returned on iso_fortran_env: error_unit, which was first defined in Fortran 2003 standard. The Fortran 77 standard defines the character string as “accessible” but doesn’t define where it goes. A best practice if desired to print a message when stopping a program is with explicit “print” or “write” statement. stop with integer code is still normal program termination in modern Fortran.

Since Fortran 77, stop may instead return a constant scalar character like “goodbye”. This generally sets return code to 0, that is, no error is indicated.

For continuous integration, having a reliable way to indicate error behavior is crucial. For HPC, indicating abnormal conditions to the shell is also vital to avoid taking resources on runs that suffered a computational error.

Fortran 2008 brought the long overdue error stop statement. stop still indicates normal program termination, and can for example stop individual images in a parallel executing program. Say an individual cell in a 3-D simulation did not find a stable solution. Depending on the simulation, that can be OK, perhaps set that value to NaN and stop with an error code on stderr, while letting the other images continue running.

However, in other types of simulations, an early failure to converge to a solution in a cell may invalidate the entire simulation taking a month of CPU time. Instead of writing cumbersome external checking code, the programmer can instead use error stop to reliably terminate all images when a critical failure is detected. Fortran 2008 error stop with constant string or integer code: both return non-zero exit status on stderr.

use, intrinsic:: iso_fortran_env, only: stderr=>error_unit

write(stderr,*) 'the failure was in ' // failedmod

error stop

Fortran 2018 added error stop with variable scalar string or variable integer code. A vital addition of Fortran 2018 is that error stop can be used within pure procedures, a very commonly needed use case. Fortran 2018 error stop variable character string allows for cleaner syntax, for example:

error stop 'the failure was in ' // failedmod

Conda per-environment channels & cross-arch install

“conda” is a package manager commonly used for Python distributions that can manage both Python and non-Python packages. Conda channels are repositories of packages that conda can install from. Conda channels can provide CPU architecture-specific optimized packages. For systems with CPU emulations like Windows Prism, channels for distinct CPU architectures like “win-64” on a “win-arm64” system can allow using packages build for non-native architectures when native versions are unavailable. Conda prioritized channels can help resolve version conflicts, helping mitigate Python package dependency hell

Conda channel priority order is ordered by which channel appears first (highest) in .condarc. It’s generally recommended to add per-environment channels rather than modifying the global configuration to avoid corrupting multiple environments with incompatible packages. In general “strict” channel priority is recommended to mitigate compatibility problems.

conda config --set channel_priority strict

# check with
conda config --get channel_priority

Add a conda per-environment channel:

conda activate <env-name>

conda config --env --add channels <channel-name>

Get the current channel list for an environment:

conda activate <env-name>

conda config --env --show channels

Example of conda cross-architecture install

Windows ARM64 channels may not have older Python versions that are available on x86-64 channels. One might be able to install an older EOL (End of Life) Python version from the “win-64” channel on a “win-arm64” system.

conda search --subdir win-64 python

may show more and additional older Python versions than

conda search --subdir win-arm64 python

Suppose Python 3.9 is available on “win-64” but not on “win-arm64”. Use the “win-64” channel to install Python 3.9 on a “win-arm64” system:

conda create -n py39 --subdir win-64 python=3.9

This will be visible in the environment’s channel list:

conda env list

and as usual used by:

conda activate py39

JPEG to DICOM simple conversion

Some patient medical image viewing systems (e.g. for X-ray, MRI, CT) only export JPEG. Other patient systems only allow uploading DICOM images, which contain metadata about the imaging study. If one needs to quickly convert JPEG images to DICOM, omitting the metadata, the following simple method can be used.

First, install DCMTK:

  • macOS: brew install dcmtk
  • Windows: Download and install from GitHub Releases *-win64.zip or use WSL
  • Linux (or Windows Subsystem for Linux): apt install dcmtk

Convert a single JPEG file to DICOM using the img2dcm command:

img2dcm img1.jpg img1.dcm

This will create a DICOM file img1.dcm from the image data of img1.jpg, but without any metadata. The resulting DICOM file can be uploaded to the patient system that only accepts DICOM, and the image will be viewable, but without any metadata about the imaging study.

Convert multiple JPEG files in a folder with a simple loop in the Unix-like terminal:

for img in *.jpg; do
    img2dcm "$img" "${img%.jpg}.dcm"
done

Or in PowerShell:

Get-ChildItem -Filter *.jpg | ForEach-Object {
    $img = $_.FullName
    $dcm = [System.IO.Path]::ChangeExtension($img, ".dcm")
    img2dcm $img $dcm
}

If the input file isn’t JPEG, first convert it to JPEG using ImageMagick, which can be installed via:

  • macOS: brew install imagemagick
  • Windows: winget install --id=ImageMagick.ImageMagick
  • Linux (or Windows Subsystem for Linux): apt install imagemagick

Then convert the image to JPEG:

magick img1.png img1.jpg

CMake file(ARCHIVE_EXTRACT) syntax

CMake file(ARCHIVE_EXTRACT), is more robust and easy to use than the command-line cmake -E tar syntax when within a CMakeLists.txt file. The option PATTERNS_EXCLUDE saves time and disk space by skipping the extraction of files that are not needed, such as documentation files.

cmake_minimum_required(VERSION 3.18)

if(CMAKE_VERSION VERSION_GREATER_EQUAL 4.5)
  set(_pat_excl_docs PATTERNS_EXCLUDE "docs/*")
endif()

file(ARCHIVE_EXTRACT
  INPUT ${archive}
  DESTINATION ${out_dir}
  ${_pat_excl_docs}
)