Singularity Containers¶

Introduction to Singularity¶

Singularity is an open-source container platform. It allows users to pack an application and all of its dependencies into a single image (file). Developed at Lawrence Livermore National Laboratory, it has become the most popular container platform among HPC user communities.

Availability¶

Singularity is available on all compute nodes of our HPC clusters, thus user can run applications via the singularity container through either interactive or batch jobs. Singularity is not installed on the login nodes for security purposes, nor available as a software module key.

Running applications through singularity container¶

Basic singularity syntax¶

The general usage form of singularity is:

singularity <command> <options> <arguments>

To shell into the singularity image on HPC clusters, use the below command syntax:

singularity shell -B /work /path/to/the/singularity/image

To run an application through a singularity image, use the below command syntax:

singularity exec -B /work /path/to/the/singularity/image name_of_exe <exe_arguments>

Explanation of arguments after the command singularity:

shell # Run a shell within the container image
exec # Run a command within the container image

-B,-bind strings # a user-bind path specification, on HPC clusters, it is necessary to bind the /work directory so singularity is able to access the /work directory on the cluster

Running singularity images under default cluster system directories¶

Images are located under /project/containers/images/ and available as modules.

These are provided by HPC staff and typically include solutions for commonly requested software that are difficult to install on the cluster’s host operating system.

Users can run their own singularity images (there is no longer the need to be added to the singularity user group), and we strongly encourage users to build and run their own images.

Below example shows the command to run an application named maker through the singularity image available under path /project/containers/images/maker-2.31.11-ubuntu-14.04.mpich2-1.5.simg.

This image is built by HPC user services staff:

[example_01_basic]$ singularity exec -B /work /project/containers/images/maker-2.31.11-ubuntu-14.04.mpich2-1.5.simg maker -h
MAKER version 2.31.11
Usage:
     maker [options] <maker_opts> <maker_bopts> <maker_exe>
...

For applications that need to use Nvidia's GPU on the compute node:

Make sure you are running the application on a GPU node.
pass the --nv option to the singularity command that run (singularity shell), or otherwise execute containers (singularity exec), which will setup the container’s environment to use an NVIDIA GPU and the basic CUDA libraries to run a CUDA enabled application. Detailed explanation of the --nv flag can refer to this link.

Below example shows the command to run a python script gcn.py using the pytorch image:

[examples]$ singularity exec --nv -B /work,/usr/lib64 /project/containers/images/singularity/pytorch-1.5.1-dockerhub-v4.simg python gcn.py
Epoch: 001, Train: 0.2000, Val: 0.0740, Test: 0.0740
...
Epoch: 199, Train: 1.0000, Val: 0.8020, Test: 0.8240
Epoch: 200, Train: 1.0000, Val: 0.8020, Test: 0.8240

Running user's own singularity images¶

Users are encouraged to build their own singularity image on their own desktop/laptop that they have the "root" permission, upload to /home, /work (note that files in /work are subject to purge) or /project directory and then run them. There are many resources and tutorials on how to build singularity images, you can refer to singularity official website for more details.

Once your singularity image is successfully built, upload your own image to the cluster using either scp or rsync:

localhost $ scp /path/to/local/singularity/image username@remote.host:/home/or/work/dir