Install Matlab Engine API in Python

Matlab Engine API allows calling Matlab functions from Python code.

These commands are executed from Terminal, not from Matlab. Go to the Matlab Engine directory to setup Matlab Engine, where “python” starts the desired Python executable.

r=$(matlab -batch "disp(matlabroot)" | tail -n1)

python -m pip install -e ${r} --user

Root / admin permissions are NOT needed.


A simple example of using Matlab from Python:

import matlab.engine
eng = matlab.engine.start_matlab('-nojvm')
y = eng.asin(1.)

eng.quit()

The Matlab Engine should take about 1 second for Matlab Engine to start when called from Python. For Matlab functions requiring JVM remove the “-nojvm” option.

Many Matlab numeric classes (single, double, logical) can be converted to Python types like:

numpy.asarray(x)

Python floats pass into Matlab Engine by including a period . after the number.

  • asin(1) fails
  • asin(1.) works

Python can pass N-dimensional arrays to Matlab.

Matlab Engine provides asynchronous call with background=True


References:

Related:

PDCurses build with CMake

PDCurses is a long-standing Curses terminal graphics library for which we provide CMake build script that works across operating systems including Windows, Linux, macOS and DOS. X11, SDL2 and Windows console backends are supported.

CTest parallel run by default

CMake environment variable CTEST_PARALLEL_LEVEL controls default test parallellism to save test run wallclock time. CTEST_PARALLEL_LEVEL=0 uses unbounded test parallelism. If the computer runs out of memory or has conflicts with parallel tests, use fixtures and resource locks to control test run parallelism on a per-test basis.

CTest parallel somewhat randomizes the order of the tests. ctest –schedule-random randomizes the order of tests even for serial test runs.

This example run on a 4-core machine shows that no extra command line parameters are needed to use CTEST_PARALLEL_LEVEL:

CMakeLists.txt:

cmake_minimum_required(VERSION 3.12)

project(par LANGUAGES NONE)

enable_testing()

message(STATUS "ENV{CTEST_PARALLEL_LEVEL}: $ENV{CTEST_PARALLEL_LEVEL}")

foreach(t RANGE 8)
  add_test(NAME sleep${t} COMMAND ${CMAKE_COMMAND} -E sleep 1)
endforeach()
export CTEST_PARALLEL_LEVEL=0

ctest --test-dir build

or

ctest --test-dir build --parallel
Internal ctest changing into directory: C:/temp/build
Test project C:/temp/build
    Start 2: sleep1
    Start 5: sleep4
    Start 3: sleep2
    Start 7: sleep6
1/8 Test #5: sleep4 ...........................   Passed    1.05 sec
    Start 6: sleep5
2/8 Test #7: sleep6 ...........................   Passed    1.04 sec
    Start 1: sleep0
3/8 Test #2: sleep1 ...........................   Passed    1.06 sec
    Start 8: sleep7
4/8 Test #3: sleep2 ...........................   Passed    1.06 sec
    Start 4: sleep3
5/8 Test #1: sleep0 ...........................   Passed    1.06 sec
6/8 Test #6: sleep5 ...........................   Passed    1.06 sec
7/8 Test #8: sleep7 ...........................   Passed    1.05 sec
8/8 Test #4: sleep3 ...........................   Passed    1.05 sec

100% tests passed, 0 tests failed out of 8

Total Test time (real) =   2.14 sec

CMake version recommendations and install

CMake ≥ 3.21 is strongly recommended for general users for more robust and easy syntax. For project developers, we recommend CMake ≥ 3.28 for C++20 modules and IDE integration.

Downloading the latest release of CMake is usually easy. Admin / sudo is NOT required.

For platforms where CMake binaries aren’t easily available, use scripts/build_cmake.cmake.

To see the merge requests for a certain release, use a URL like: https://gitlab.kitware.com/cmake/cmake/-/merge_requests?milestone_title=3.28.4&scope=all&state=all

CMake 3.29 adds cmake_language(EXIT code) to exit CMake script mode with a specific return code. This is useful when using CMake as a platform-agnostic scripting language instead of shell script. Environment variable CMAKE_INSTALL_PREFIX is used to set the default install prefix across projects–it can be overridden as typical by cmake -DCMAKE_INSTALL_PREFIX= option. Test property TEST_LAUNCHER allows specifying a test launcher e.g. for MPI program this allows deduplicating or making more programmatic test runner scripts. Linker information variables including CMAKE__COMPILER_LINKER_ID have been added to allow programmatic logic like setting target_link_options() based on the particular linker. ctest --parallel without a number or 0 will use unbounded test run parallelism.

CMake 3.28 changes PATH behavior for Windows find_{library,path,file}() to no longer search PATH. This may break some projects that rely on PATH for finding libraries. The MSYS2-distributed CMake is patched to include PATH like earlier CMake, which can be confusing for CI etc. not using MSYS CMake with that patch. Windows CI/user may need to specify environment variable CMAKE_PREFIX_PATH like

$Env:SYSTEMDRIVE/msys64/mingw64/lib

Support for C++20 modules is considerably improved and most users will want at least CMake 3.28 to make C++ modules usable. Generator expressions $<IF> $<AND> $<OR> now short circuit. Test properties now have a DIRECTORY parameter, useful for setting test parameters from the project’s top level CMakeLists.txt. CMake 3.28.4 fixed a long-standing bug in Ninja Fortran targets that use include statements.

CMake 3.27 emits warning for cmake_minimum_required(VERSION) < 3.5. CTest test properties TIMEOUT_SIGNAL_NAME and TIMEOUT_SIGNAL_GRACE_PERIOD specify a POSIX signal to send to a timed out test process. Interactive CMake debugger added by cmake --debugger is used with an IDE such as Visual Studio. CMake script command cmake_file_api() allows querying CMake File API from within CMake. NOTE: Fortran + Ninja was broken for OBJECT libraries in CMake 3.27.0..3.27.8 and fixed in 3.27.9.


Older CMake changelog

X11 on macOS

X11 can be accessed using XQuartz as available via Homebrew.

brew install libx11 xquartz

To finish the one-time setup of XQuartz, logout/login or reboot. Check that X11 server is available by

echo $DISPLAY

which should show a temporary directory ending like “org.xquartz:0”.

Demo and troubleshooting

See that X11 is working with the classic Xeyes test GUI.

xeyes

A useful image browsing program on Linux, macOS etc. is feh:

brew install feh

Troubleshooting: XQuartz GitHub Issues

Specifically, be sure XQuartz is enabled in Login Items to fix problems like the XQuartz window won’t open.

XQuartz will open automatically when any X11 program is started.


Related: PulseAudio on macOS

Cleanup unused files in Linux

Keep at least 10% of drive space to avoid:

  • SSD wear
  • HDD fragmentation

Determine free space on Linux / macOS / Windows Subsystem for Linux with “ncdu”. ncdu uses Ncurses terminal graphics to quickly show the biggest files in the Linux filesystem tree. ncdu is very handy to find large files or directories that may be unneeded.

df -h

gives a drive-level summary of disk usage.

Package managers cache installed files in case of need to reinstall, but the packages can be redownloaded if needed to save disk space by clearing the cache. Clear the package cache–for APT (common in Debian-based systems):

apt autoclean

or for DNF (Fedora, RHEL, CentOS):

dnf clean dbcache

Remove unwanted packages

TeX Live documentation can consume a lot of disk space. To cleanup the documentation, consider removing packages matching texlive-*doc. This also removes texlive-full but with no detriment to TeX Live working.

Synaptic list of files to remove for texlive-doc to save disk space

Packages removed for texlive-doc to save over 1 GB of disk space.


Related:

CMake FindOpenSSL hints

For all CMake find_*() commands including FindOpenSSL, the package path can be hinted by setting an appropriate environment variable or CMake variable. This examples supposes a Homebrew package manager has installed OpenSSL 1.1, which the user wishes to use in a CMake project. To hint the package path when configuring a CMake project, either specify OpenSSL_ROOT by environment variable:

export OpenSSL_ROOT=$(brew --prefix openssl@1.1)

or directly in the CMake configure command:

cmake -B build -DOpenSSL_ROOT=$(brew --prefix openssl@1.1)

The example CMakeLists.txt:

cmake_minimum_required(VERSION 3.16)

project(f LANGUAGES NONE)

find_package(OpenSSL REQUIRED)

Use the –debug-find CMake option to see the paths CMake is searching.

To disable various search paths, consider the following CMake variables. These are normally only used for debugging or special cases.

set(CMAKE_FIND_USE_CMAKE_PATH false)
set(CMAKE_FIND_USE_CMAKE_SYSTEM_PATH false)
set(CMAKE_FIND_USE_CMAKE_ENVIRONMENT_PATH false)
set(CMAKE_FIND_USE_SYSTEM_ENVIRONMENT_PATH false)

The OpenSSL world is gradually transitioning from OpenSSL 1.1 to 3, and Homebrew uses subdirectory to isolate the OpenSSL installs. CMake does not recursively search as that would in general not have a stopping condition and at least significantly slow down the search performance.

GitHub Actions Apple Silicon CPU

GitHub Actions macOS runners can use Apple Silicon CPU, which is what most Apple users have. Some build issues including the linker have historically had Apple Silicon-specific issues. Generally it’s good to test on the same CPU architecture as the target platform.

We sometimes find it necessary to select the Xcode version compatible with Homebrew GCC if build errors occur that are not present on a physical Apple Silicon laptop.

jobs:

  mac:
    runs-on: macos-14

    strategy:
      matrix:
        cxx: [g++-13, clang++]

    env:
      HOMEBREW_NO_AUTO_CLEANUP: 1
      CXX: ${{ matrix.cxx }}

    steps:
    - uses: actions/checkout

    - name: Ninja install
      run: brew install ninja

    - run: sudo xcode-select --switch /Applications/Xcode_15.1.app

    - run: cmake --workflow --preset debug

    - run: cmake --workflow --preset release

In this example Ninja enables quick testing of builds in Debug and Release mode, which is important to catch bugs.

Matlab on macOS doesn't source ~/.zshrc

On macOS, Matlab does not source ~/.zshrc. This issue has existed at least since macOS started using ZSH as the default shell.

To workaround this issue, particularly when programs from package managers like Homebrew are needed, add a setup script to the Matlab project containing like:

if ~ismac
  return
end

% Add Homebrew to the PATH
[ret, homebrew_prefix] = system('brew --prefix');
if ret == 0
  p = fullfile(strip(homebrew_prefix), "bin");
  if isfolder(p)
    setenv('PATH', append(p, pathsep, getenv('PATH')))
  end
end

Then programs installed by Homebrew like CMake, GCC, etc. will be on Path environment variable in Matlab.

Note that the Matlab commands below do not help:

!source ~/.zshrc

system("source ~/.zshrc")

CMake macOS Xcode Environment

When macOS, Xcode, or Command Line Tools upgrades, build directories (including for CMake-based projects) often need to be refreshed. If the user has set custom environment variables concerning Xcode, they may need to be updated as well. Here are some important environment variables and CMake variables to check if problems occur after upgrading.

For a simple CMake project on macOS, CMakeCache.txt might include:

CMAKE_OSX_DEPLOYMENT_TARGET:STRING=

CMAKE_OSX_SYSROOT:PATH=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX*.sdk

Having multiple directories under /Library/Developer/CommandLineTools/SDKs and /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs is fine, CMake can select the appropriate one. The user can select the SDK:

xcode-select --switch

Workarounds

If Homebrew GCC breaks after upgrading Xcode or Command Line Tools, try specifying an older SDK. For example, if the latest SDK is MacOSX14.4.sdk, try using MacOSX13.3.sdk in “~/gcc.sh”:

export CC=gcc-13 CXX=g++-13 FC=gfortran-13

export SDKROOT=/Library/Developer/CommandLineTools/SDKs/MacOSX13.3.sdk/

and then source ~/gcc.sh before running cmake with a fresh build directory.

If a CMake build step fails, try copy-pasting the command and removing the -isysroot portion of the command. This is a clear clue the older SDK is (at least temporarily) needed till Homebrew updates its GCC formula.

GCC will tell where included files are coming from by adding the gcc -H flag. This tells what to specify for environment variable SDKROOT.