Scientific Computing

Build GPSTest app with Android Studio

The GPSTest app is currently (December 2024) not available to most devices on the Google Play store due to the app temporarily being configured for a too-old minimum Android version. This is a security measure enforced by Google Play store to all apps to preserve security of devices. App stores of other operating systems enforce similar restrictions for security.

A workaround until the GPSTest app authors update the app configuration is to build and install the app using Android Studio. Android Studio is free to use and works on Windows, macOS, or Linux equally well. Default settings were used, including setting up an optional emulator. Any reasonably current Android Studio version should be suitable.

This method doesn’t make any changes to the app, but bypasses the Google Play store restrictions by building and installing GPSTest directly from the laptop onto an Android phone. An inconvenience is lack of easy app distribution as every phone needs to one-time plug into a laptop with USB-C to install the GPSTest app. If the phone isn’t detected, try another USB cable as some cables are power-only and don’t have data lines. A message will popup on the phone asking to authorize the laptop to connect in USB Debugging mode. The app can be used indefinitely after the one-time plugin and install.

Download and Build GPSTest

Use Git as in any other project from the Terminal:

git clone https://github.com/barbeau/gpstest

From Android Studio, open that “gpstest” directory from “File > Open” menu.

Try to build the app with the “Build > Make Project” menu – or click the hammer icon in the upper right toolbar of Android Studio. Failures like

Unknown Kotlin JVM target: 21

may result upon building. This stems from the default JVM version in Android Studio. For repeatability, it is typical across code languages and platforms to have a project manifest specifying exact library versions. Android apps manifests are typically “build.gradle” file(s). We don’t edit the manifest at this time, because maybe we’ll break something unexpectedly in the app. Instead, set Android Studio to use JVM 17 as specified by manifest file “GPSTest/build.gradle”:

compileOptions {
   sourceCompatibility JavaVersion.VERSION_17
   targetCompatibility JavaVersion.VERSION_17
}
kotlinOptions {
   jvmTarget = JavaVersion.VERSION_17
}

This is accomplished under Android Studio’s Settings icon (geared wheel) in the upper right corner of the Android Studio main window. Select “Build, Execution, Development > Build Tools > Gradle” in the left hand Settings panel. Under “Gradle Projects” in the right panel, “Gradle JDK” may be set to 21 or so. Select in that menu to “Download JDK and selection “Amazon Coretto 17” or JetBrains 17 or similar. This will take a couple minutes to download and install, then click OK.

Again try to build the app with the “Build > Make Project” menu – or click the hammer icon. There will be a number of warnings, but the app should build successfully. If the optional emulator is present, clicking “Run > Run GPSTest” or clicking the green right arrow in the top Android Studio menu bar will run the app in the emulator.

Connecting Phone

The phone needs to have Developer Mode enabled to upload apps from Android Studio. Under Developer Mode settings, USB Debugging needs to be enabled, at least for the brief time the phone is connected to the laptop to upload the app. If this is a personal phone, consider disabling USB Debugging at all other times to help the security of the phone from physical hacking (someone steals/takes the phone and wants to break into it). In Android Studio Device Manager, clicking “+” will add the phone, or it may appear automatically. Clicking the Android Studio “Run” green arrow will install the app automatically if the physical phone is selected in Android Studio Device Manager. After the app is installed, unplug USB at any time and turn off USB Debugging under the phone’s Developer Mode settings if desired.

GPSTest Caveats

Android 14 and newer will not have the “Share” icon at the top of GPSTest app due to enhanced security in Android 14. The GPSTest maintainers will need to update the GPSTest app to work fully with newer Android versions.

CTest stdin pipe

CTest can be used to run test executables that require input from stdin. This is accomplished by having add_test call a CMake script that uses execute_process to run the test executable with the input piped in from a file. Null stdin is also possible.

Suppose CMakeLists.txt contains:

enable_testing()

add_executable(main main.c)

add_test(NAME stdin_null
  COMMAND ${CMAKE_COMMAND} -Dexe:FILEPATH=$<TARGET_FILE:main> -P ${CMAKE_CURRENT_SOURCE_DIR}/stdin_null.cmake
)

add_test(NAME stdin_pipe
  COMMAND ${CMAKE_COMMAND} -Dexe:FILEPATH=$<TARGET_FILE:main> -P ${CMAKE_CURRENT_SOURCE_DIR}/stdin_pipe.cmake
)

In the non-CMakeLists.txt *.cmake script files, we set a reasonable cmake_minimum_required(VERSION) to avoid issues with defaulting to pre-CMake 3.0 behavior for strings, especially for regular expressions.

The null input to stdin may be required for some executables that may hang forever waiting for stdin.

cmake_minimum_required(VERSION 3.19)

if(UNIX)
  set(devnull INPUT_FILE /dev/null)
elseif(WIN32)
  set(devnull INPUT_FILE NUL)
endif()

execute_process(COMMAND ${exe}
${devnull}
COMMAND_ERROR_IS_FATAL ANY
)

A file (typically a text file) may be piped to stdin to replace interactive input.

cmake_minimum_required(VERSION 3.19)

execute_process(COMMAND ${exe}
INPUT_FILE ${CMAKE_CURRENT_LIST_DIR}/cli_exercise.txt
COMMAND_ERROR_IS_FATAL ANY
OUTPUT_VARIABLE out
ERROR_VARIABLE err
)

if(out MATCHES "TRACE")
  message(FATAL_ERROR "TRACE found in stdout")
endif()

if(err MATCHES "TRACE")
  message(FATAL_ERROR "TRACE found in stderr")
endif()

if(NOT err STREQUAL "")
  message(FATAL_ERROR "stderr output is not empty:
  ${err}")
endif()

message(STATUS "stdout:
${out}")

In this example, the developer wishes to ensure the program ran with no unexpected stdout text and no stderr text at all. This can be helpful to catch debugging output that should not be in the final program output.

Hash of empty file

Checking the hash checksum of downloaded files can help indicate if a file has been tampered with. Hash collisions are possible by intentionally manipulating a harmful file to have the same hash as the expected file. The simpler the hash function, the more likely hash collisions are. Hash collisions have been demonstrated for MD5 and SHA-1.

SHA-256 is a popular SHA-2 hash function for which it takes longer to generate collisons.

Example empty file hash

We use CMake command tool as a platform-independent command line tool to generate and compute hashes. The results are the same regardless of the tool used.

In general, the hash length is fixed for a given hash function. The input file size does not affect the hash length.

First create an empty file:

cmake -E touch empty-file

SHA-512 hash of an empty file:

cmake -E sha512sum empty-file
cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e  empty-file

SHA-256 hash of an empty file:

cmake -E sha256sum empty-file
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855  empty-file

SHA-1 hash of an empty file:

cmake -E sha1sum empty-file
da39a3ee5e6b4b0d3255bfef95601890afd80709  empty-file

MD5 hash of an empty file:

cmake -E md5sum empty-file
d41d8cd98f00b204e9800998ecf8427e  empty-file

Clang -Wunsafe-buffer-usage tips

Clang C++ flag -Wunsafe-buffer-usage enables a heuristic that can catch potentially unsafe buffer access. However, this flag is known to make warnings that are unavoidable, such as accessing elements of argv beyond argv[0], even via encapsulation such as std::span.

This flag could be used by occasionally having a human (or suitably trained AI) occasionally review the warnings. For example, in CMake:

option(warn_dev "Enable warnings that may have false positives" OFF)

if(warn_dev)
  add_compile_options("$<$<COMPILE_LANG_AND_ID:CXX,AppleClang,Clang,IntelLLVM>:-Wunsafe-buffer-usage>")
endif()

argv general issues

General issues with argv are discussed in C++ proposal P3474R0 std::arguments. An LLVM issue proposed an interim solution roughly like the following, but at the time of writing, this still makes a warning with -Wunsafe-buffer-usage.

#if __has_include(<span>)
#include <span>
#endif
#if defined(__cpp_lib_span)
#  if __cpp_lib_span >= 202311L
#    define HAVE_SPAN_AT
#  endif
#endif

int main(int argc, char* argv[]) {

#ifdef HAVE_SPAN_AT
const std::span<char*> ARGS(argv, argc);
#endif

int n = 1000;

if(argc > 1) {
  n = std::stoi(
#ifdef HAVE_SPAN_AT
  ARGS.at(1)
#else
  argv[1]
#endif
  );
}

return 0;
}

MSVC __cplusplus macro

The __cplusplus macro indicates the version of the C++ standard that the compiler claims to implement given the current compiler flags. Some later language standard features like __has_include are available despite earlier compiler standard settings, which is a great convenience. C++ projects regularly use the __cplusplus macro to conditionally compile code based on the C++ standard version implemented by the compiler in use. This allows adding new optional features, which still working with older compilers that do not support them.

Surprisingly, Visual Studio MSVC defines __cplusplus as 199711L by default, which is the C++98 standard. Visual Studio 2017 15.7 added the flag /Zc:__cplusplus to define __cplusplus as the correct value like other compilers.

Intel oneAPI 2023.1 release uniformly adds the MSVC flag /Zc:__cplusplus. To see the note, scroll down to the text “oneAPI 2023.1, Compiler Release 2023.1 New in this release” and click the down caret.

Added /Zc:__cplusplus as a default option during host compilation with MSVC.

In CMake, add this flags as needed by deciphering the MSVC compiler version.

if(CMAKE_CXX_COMPILER_ID STREQUAL "MSVC" AND CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 19.14)
  # MSVC has __cpluscplus = 199711L by default, which is C++98!
  # oneAPI since 2023.1 sets __cplusplus to the true value with MSVC by auto-setting this flag.
  add_compile_options("$<$<COMPILE_LANGUAGE:CXX>:/Zc:__cplusplus>")
endif()

Meson build system adds this flag automatically.

Matlab system stdin pipe

Matlab or GNU Octave can call programs and handle arbitrarily large and complex inputs and outputs via stdin, stderr, and stdout command line pipes as in matlab-stdlib subprocess_run that works for Matlab or GNU Octave across operating systems.

stdlib.subprocess_run() overcomes limitations of factory system. and works like Python subprocess. “stdout” and “stderr” are returned from stdlib.subprocess_run() separately, and “stdin” can be passed as a string.

stdlib.subprocess_run() can be faster than using temporary files. stdlib.subprocess_run() helps avoid filesystem clashes when running many external processes in parallel or asynchronously.

Across programming languages, calling an external program with pipes avoids the need to write additional code directly interfacing memory between Fortran or C/C++ by using file-based or pipe-based API for data streaming.

Reference: Python or Java pass stdin from Matlab to executable.

HDF5 / NetCDF4 in GNU Octave

Open data file formats such as HDF5 and NetCDF4 are excellent way to share and store archival data across computing platforms and software languages. Numerical software such as Matlab, GNU Octave, Python, and many more support these data file formats.

The syntax in the code examples below is exactly the same for Matlab and GNU Octave. Omit the pkg load and pkg install statements in Matlab.

HDF5

HDF5 files in GNU Octave are accessed via hdf5oct in similar fashion to Matlab.

From Octave prompt, install the package:

pkg install -forge hdf5oct

Octave program that writes an array to an HDF5 file “example.h5” dataset “/m”:

pkg load hdf5oct

fn = 'example.h5';

h5create (fn, '/m', [3 3]);
h5write (fn, '/m', magic (3));

Observe the file “example.h5” has been created. If the HDF5 command line tools are installed, the contents can be printed from system Terminal:

h5ls -v example.h5

In Octave or Matlab, the HDF5 file can be read to an array:

x = h5read (fn, '/m')
8   1   6
3   5   7
4   9   2

NetCDF4

NetCDF4 files in GNU Octave are accessed via Octave NetCDF4 package. Install the package from Octave prompt:

pkg install -forge netcdf

Write an array to a NetCDF4 file “example.nc” dataset “m”:

pkg load netcdf

fn = 'example.nc';

nccreate (fn, 'm', "Dimensions", {"x", 3, "y", 3});
% must include dimensions or a scalar dataset will be created

ncwrite (fn, 'm', magic (3));

Read the NetCDF4 file “example.nc” to an array:

x = ncread (fn, 'm')

Reference:

  • oct-hdf5 package: Octave low-level access to HDF5 files.

Eliminate old C-style casts in C++

The C++ named casts such as static_cast, dynamic_cast, and reinterpret_cast are preferred in C++ Core Guideline ES.49 over ambiguous old C-style casts. C++ named casts can help provide type safety by making the intention of the cast explicit / readable. C++ compilers can detect and warn about improper or unsafe casts when using named casts. C-style cast mistakes are more difficult to detect by humans or automated tools.

static_cast is used for conversions between compatible types, such as converting an int to a float or a pointer to a base class to a pointer to a derived class. Another common static_cast use case is interfacing with C functions such as Windows API functions that require specific types less common in pure C++ code.

int a = 10;
float b = static_cast<float>(a);

reinterpret_cast is used for low-level reinterpreting of bit patterns. It casts a type to a completely different type. This cast is not type safe and should be used with caution to avoid undefined behavior. reinterpret_cast is commonly used in low-level programming, such as interfacing with hardware or converting between pointers and integers.

int a = 10;
char* b = reinterpret_cast<char*>(&a);

dynamic_cast is used for safe downcasting of pointers or references to classes in a class hierarchy. It performs a runtime or RTTI check to help ensure that the cast is valid. dynamic_cast is used when you need to convert a pointer or reference to a base class to a pointer or reference to a derived class. static_cast is more common and faster than dynamic_cast, but dynamic_cast is safer when downcasting in a class hierarchy.

Detecting Old-Style Casts with GCC or Clang

To ensure that old C-style casts are not used in a codebase, consider the -Wold-style-cast flag with GCC or Clang. This flag generates warnings for any old-style casts found in the code.

In CMake, this flag is applied like:

dd_compile_options("$<$<COMPILE_LANG_AND_ID:CXX,AppleClang,Clang,GNU>:-Wold-style-cast>")

If the CMake variable CMAKE_COMPILE_WARNING_AS_ERROR is set true, the old-style cast warnings (and other compile warnings) will be treated as errors.