Setup Docker Fortran image for CI

Install Docker image: add docker group and your username to this group, to avoid needing sudo for every docker command:

addgroup docker

adduser $(whoami) docker

then reboot (not just logout, an actual system restart)

Install Docker on the laptop. After install, sudo is no longer required.

Try the Hello World images, which should auto-download and print a welcome message

docker run hello-world

Search for a desired image. Consider those images listed as “official”. You might need to widen your terminal to 155 columns or so to render properly. Let’s use ubuntu as it’s quite popular for Docker and in general.

docker search ubuntu

Get the desired image

docker pull ubuntu

Verify the images on your computer

docker images

ubuntu takes just under 100 MB to start.

Start the Docker container on your laptop

docker run -it ubuntu
-it
interactive session

Verify the version running, it will have a # superuser prompt:

cat /etc/lsb-release

manage containers

These commands are issued NOT from within the Docker container itself (open a new terminal)

  • where are containers stored: docker info | grep "Root Dir"

  • How much space is Docker using altogether: du -sh /var/snap/docker

    • use previous command to find where your Docker stuff is

  • list images: docker images -a

  • list containers (running and stopped): docker ps -a

  • stop a Docker container: docker stop container_name

  • start a Docker container: docker start container_name

  • login to a running Docker container: docker exec -it container_name bash

  • get container environment variables: docker exec container_name env

  • cleanup unused containers docker system prune

  • delete image: docker image rm ubuntu:19.04

Configure Docker image

CI and other online, server and computer resources) can use Docker images. You can make a Docker image setup with everything you need, which is useful if the setup is cumbersome. For research reproducability, a Docker image gives a known software state that should be usable for many years.

Interactive setup method tends to make the final image larger than using RUN commands. So let’s start off by using a Dockerfile as that helps maintain smaller images with repeatable builds. The Docker container can be turned into a new image that is usable from CI or other suitable Docker host.

We do not claim this Dockerfile to be optimal. Please let us know of suggested improvements.

# based on https://github.com/naftulikay/docker-bionic-vm/blob/master/Dockerfile

FROM ubuntu:18.04
ENV container=docker TERM=xterm LC_ALL=en_US LANGUAGE=en_US LANG=en_US.UTF-8

# stop some interactive prompts, also need "-yq" on apt-get
ENV DEBIAN_FRONTEND=noninteractive

# locale
RUN apt-get update -q > /dev/null && \
  apt-get install --no-install-recommends -yq apt-utils locales language-pack-en dialog > /dev/null && \
  locale-gen $LANGUAGE $LANG

# add sudo commmand
RUN apt-get -yq install sudo > /dev/null

# create and switch to a non-priviliged (but sudo-enabled) user, arbitrary name
RUN echo "nonprivuser ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers
RUN useradd --no-log-init --home-dir /home/nonprivuser --create-home --shell /bin/bash nonprivuser && adduser nonprivuser sudo
USER nonprivuser
WORKDIR /home/nonprivuser

# Git/Curl -- don't disable recommends here or you won't have Certification Authority certificates and will fail
RUN sudo apt-get install -yq git curl > /dev/null

# packages specific to the needs
RUN sudo apt-get install --no-install-recommends -yq make gfortran libcoarrays-dev libopenmpi-dev open-coarrays-bin > /dev/null && \
  sudo apt-get clean -q

# latest cmake
RUN git clone --depth 1 https://github.com/scivision/cmake-utils && \
  mkdir -v /home/nonprivuser/.local && \
  cd cmake-utils && PREFIX=/home/nonprivuser/.local ./cmake_setup.sh > /dev/null  && \
  mv -v /home/nonprivuser/.local/cmake* /home/nonprivuser/.local/cmake

ENV PATH=$PATH:/home/nonprivuser/.local/cmake/bin

# other optional installs

# RUN sudo apt-get install --no-install-recommends -yq octave

Create the file above named Dockerfile in your Git repo.

Build the Docker image:

docker build -t opencoarrays_fortran .

assuming you’re in the same directory as Dockerfile

Check the image exists. For me it was about 350 MB:

docker images

Before committing for upload, you must invoke the container.

docker run opencoarrays_fortran

This will almost immediately start and stop, as you didn’t give it a command to run or persist.

Get the hexadecimal container ID by:

docker ps -a

the container will have an auto-assigned name like foo_bar. Note the hexadecimal “container ID”.

upload Docker image

Once you have configured a Docker container on your laptop suitable for your purposes, you may wish to share this container and use it on other hosts such as CI. This can be done with DockerHub.

Once you are ready to upload the image, note the “container ID”, which is the hexadecimal number at the terminal prompt of the Docker container, under docker ps. The container hex ID must appear under docker ps, just being in docker images is not enough.

docker commit -m "basic fortran OpenCoarrays setup" hex_id dockerhub_username/opencoarrays_fortran

The changes to the image the container made are gathered into a new image. It may take a minute or two if your image is large. Ideally with a small image it will take only a couple seconds. The new image has not yet left your computer, it will show up under

docker images

Once uploaded, your Docker image is visible to the world by default.

Login to DockerHub

docker login -u dockerhub_username

Push (upload) this image to DockerHub. Note this image is world-visible by default!

docker push dockerhub_username/opencoarrays_fortran

If your image is very large > 1 GB, be aware it will take a while to upload, and for the CI system to download. This is a telltale that it’s important to keep Docker images small.

Each docker exec command is a new shell instance. So changing directories in one docker exec has no effect on subsequent commands for example.