Skip to content

Using Containers

This chapter describes how to use containers.

In the Data Analysis System, Apptainer, which is a container environment, can be used for work servers and LSF batch jobs.

Note

Please refer to the following links for general usage of Apptainer.
Link to "Apptainer User Guide"

Preparing the container images creation environment

Administrator Privilege (root) is required for creating contaier images. Therefore, you need to create images on another system where Apptainer was installed and transfer images to an work server.

Note

See the link below for instructions on how to install Apptainer. It can be installed on Linux, Windows and Mac.
Link to "Installing Apptainer"

Creating container images

Create a container image on the system where Apptainer is installed. It is convenient to use a definition file to create images. The following is an example of a definition file (file name: almalinux9.def) that creates an image that can be used by the development environment and the "module" command based on the latest CentOS image downloaded from Docker Hub.

Note

See this page for the "module" command.

# cat almalinux9.def
Bootstrap: docker
From: almalinux:9

%post
    dnf groupinstall -y 'Development Tools'
    dnf -y install \
    environment-modules \
    which

Create a container image (file name: almalinux9.sif) with the following command.

# apptainer build almalinux9.sif almalinux9.def

The contents of SIF format image files cannot be changed. If you need to add files, etc. instead, use the following command to change to sandbox format.

# apptainer build --sandbox --fix-perms almalinux9 almalinux9.def

In this example, a directory called "almalinux9" will be created in the current directory. Add files under that directory. You can also start a shell in the container with the following command.

# apptainer shell --writable centos7
Apptainer>

"Apptainer>" is the shell prompt started in the container. You can add packages with the yum command.

Note

The commands that can be used depend on the Linux distribution of the container.

After changing the content, exit from the container shell and convert the sandbox format to SIF format with the following command.

# apptainer build almalinux9.sif almalinux9

Transfer the created SIF file to the work server using the scp command, etc.

Compile a program to run in a container

The following is an example of compiling a program using the container image prepared in the above procedure. Here we compile the Intel MPI benchmark using the Intel compiler and Intel MPI.

Note

The source for the Intel MPI Benchmark is available at the link below.
Link to "Intel MPI Benchmarks"

Extract the obtained archive under the home directory of the work server.

$> tar xf IMB-v2019.6.tar.gz
$> cd mpi-benchmarks-IMB-v2019.6

Start the container shell with the following command.

$> apptainer shell --bind /opt,/opt/Modules/modulefiles /path/to/almalinux9.sif

In the container shell started by "apptainer shell", the following can be inherited and used as is.

  • user ID
  • home directory
  • current directory

You can also make the specified directory available to the container using the "--bind" option. In the above example, /opt is specified to use the Intel compiler and Intel MPI, and /opt/Modules/modulefiles is specified to use the module command.

Note

The following notification appears at container startup. This is displayed because gocryptfs is not installed in the KEK container environment. It does not affect the use of the container, so you can safely ignore it.

INFO: gocryptfs not found, will not be able to use gocryptfs

Compile Intel MPI Benchmarks by executing the following command in the container shell.

Apptainer> source /usr/share/Modules/init/bash
Apptainer> module load intel/2020
Apptainer> export LANG=C
Apptainer> make IMB-MPI1
Apptainer> ls IMB-MPI1
IMB-MPI1
Apptainer> exit