Ninja job pools for low memory CMake builds

An increasing number of systems have multiple CPUs, say four, six or eight but may have modest RAM of 1 or 2 GB. An example of this is the Raspberry Pi. Ninja job pools allow specifying a specific limit on number of CPU processes used for a CMake target. That is, unlike GNU Make where we have to choose one CPU limit for the entire project, with Ninja we can select CPU limits on a per-target basis. That’s one important benefit of Ninja for speeding up builds of medium to large projects, and why we see increasing adoption of Ninja in prominent projects including Google Chrome. This is another reason why we generally strongly encourage using Ninja with CMake.

Specifically, CMake + Ninja builds can limit CPU process count via target properties:

The global JOB_POOLS property defines the pools for the targets.

Upon experiencing build issues such as SIGKILL due to excessive memory usage, inspect the failed build step to see if it was a compile or link operation, to determine which to limit on a per-target basis.

Example

Suppose that 500 MB of RAM are needed to compile a target and we decide to ensure at least 1 GB of RAM is available to give some margin. Thus we constrain the number of CPU processes for that target based on CMake-detected available physical memory. The appropriate parameters for your project are determined by trial and error. If this method still is not reliable even with a single CPU process, then a possible solution is to cross-compile, that is to build the executable on a more capable system for this modest system.

CMakeLists.txt includes:

set_property(GLOBAL PROPERTY JOB_POOLS one_jobs=1 two_jobs=2)

cmake_host_system_information(RESULT _memfree QUERY AVAILABLE_PHYSICAL_MEMORY)

add_library(big big1.c big2.f90)
if(_memfree LESS 1000)
  set_property(TARGET big PROPERTY JOB_POOL_COMPILE one_jobs)
endif()

Related: tell CMake to use Ninja