On Windows with Python subprocess, an executable may be missing DLL from PATH.
This environment error can be detected and a message given to the user.
The return code of the subprocess can be checked for the error code 3221225781, which corresponds to hex error code C0000135.
import subprocess
import os
import logging
ret = subprocess.run('myprog.exe')
if ret.returncode ==3221225781and os.name =='nt':
# Windows 0xc0000135, missing DLL logging.error('missing DLL detected. Ensure DLL directory is in environment variable PATH')
path = os.environ['PATH']
print("PATH:", path)
raiseSystemExit(ret.returncode)
If putting DLLs on PATH is problematic, another possible approach is
statically compiling the executable,
but this may not be feasible for all programs.
Most shells (including Bash) don’t have a straightforward way to get the directory of the script being executed. Here’s a common
one-liner
to get the script directory:
sdir="$( cd "$( dirname "${BASH_SOURCE[0]}")" >/dev/null 2>&1&& pwd )"
However, for some HPC batch systems, the above may not work as it copies this script to a temporary directory.
This fallback put after the line above works if “qsub” batch command was run from this script’s directory–supposing a script file “gcc.sh” also exists in the same directory as the original script.
CMake environment variable
CMAKE_BUILD_PARALLEL_LEVEL
can be manually set to control default number of build parallel threads.
Parallel builds are virtually always desired to save build and rebuild time.
As a starting point, perhaps set CMAKE_BUILD_PARALLEL_LEVEL environment variable to be equal to the number of physical or logical CPU cores by setting it in the user profile:
#!/bin/bash
if[[ x"${CMAKE_BUILD_PARALLEL_LEVEL}"== x ]]; thenn=8;
case"$OSTYPE" in
linux*)n=$(nproc);;
darwin*)n=$(sysctl -n hw.physicalcpu);;
bsd*)n=$(sysctl -n hw.ncpu);;
esacexport CMAKE_BUILD_PARALLEL_LEVEL=${n}fi
Or for Windows, in environment variable settings:
CMAKE_BUILD_PARALLEL_LEVEL=%NUMBER_OF_PROCESSORS%
If the computer runs out of RAM, reduce the specific command parallelism with the cmake --build --parallel N command line option.
For Ninja build systems, specific targets can control the number of workers with
job pools.
The purpose of compiler flag checks is to test if a flag is supported.
Metabuild system (such as CMake and Meson) compiler flag checks must not test the “-Wno-” form of a warning flag.
This is because several compilers including Clang, GCC, Intel oneAPI emit a “success” return code 0 for the “-Wno-” form of an unsupported flag.
From time to time, the topic of why meta-build systems like CMake and Meson are single-threaded sequentially executing processes is brought up.
With desktop workstations (not to mention build farms) having 32, 64, and even 128+ CPU cores increasingly widespread, and the configure / generation step of meta-build systems taking tens of seconds to a few minutes on large projects, developers are understandably frustrated by the lack of parallelism.
A fundamental issue with CMake, Meson and equivalently for other meta-build systems is that the user’s CMake scripts would then have to be dataflow / declarative versus imperative.
This would require reworking of script syntax and meta-build internal code radically.
In Meson, Python threading (single thread executes at one time) is used in subprojects, giving a speed boost in download time of subproject source code.
There is no Python multiprocessing or ProcessPoolExecutor in Meson configure step.
Meson parallel execution is for build (Ninja) and test.
Both build and test are also done in parallel in CMake.
For CMake, the ExternalProject steps can already be run in parallel (including download) via the underlying build system.
A way to speed up meta-build configure time–here specific to CMake–is to stuff the CMakeCache.txt file with precomputed values and/or use CMake Toolchain to do likewise, skipping configure tests when the host build system is static.
CMakeCache.txt stuffing is a technique Meson uses to speed up configure time of CMake-based subprojects from Meson projects.
Flake8
is a Python code style checker that can detect unexecuted syntax errors.
Python 3.12 with flake8 detects PEP8 formatting errors
inside f-strings
that are
not yet handled by Black.
Currently these formatting errors must be corrected by hand.
CI jobs should test with Python < 3.12 and Python >= 3.12 to ensure the f-string syntax is valid in older and newer Python versions.
However, it is often more convenient (if using care) to use terse variables that are not as specific:
if(APPLE)# macOS
elseif(BSD)# FreeBSD, NetBSD, OpenBSD, etc.
elseif(LINUX)# Linux
elseif(UNIX)# Linux, BSD, etc. (including macOS if not trapped above)
elseif(WIN32)# Windows
else() message(FATAL_ERROR"Unsupported system: ${CMAKE_SYSTEM_NAME}")endif()