Docker container

The E3SM model can be run within the E3SM Docker container. This page describes how to install Docker on your laptop/workstation, build or download the Docker container, and run the E3SM model within the container.

Note: the container is limited to the resources of the single node or workstation it runs on so keep your case sizes in mind.

Install Docker

Install Docker on Linux, macOS or Windows as described at Get Docker.

Download the E3SM container

The latest version of the E3SM container can be downloaded from the docker repository:

$ docker pull lukaszlacinski/e3sm:2021.03.19

$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
lukaszlacinski/e3sm 2021.03.19 12d20f57cd94 10 months ago 702 MB

Example

By default, Docker containers do not use the host filesystem. Any host system directories that, a user wants to make accessible from the container environment, must be mounted when the container is started. A directory with SSH private keys will be needed in the container, if you plan to ssh from the container (clone the E3SM repository recursively). If you have the E3SM repository already cloned on your host system, and you want to work on it in the container environment, you need to mount it as well. Also do mkdir -p $HOME/projects/e3sm/cesm-inputdata (this directory is assumed to exist by the machine entry above. Input data will be downloaded here. (could get large, THE CONTAINER DOES NOT HAVE INPUT DATA)).

To start a Docker container with the E3SM image, run:

$ docker run -it \

--mount type=bind,source=$HOME/.ssh,target=/home/e3sm/.ssh \

--mount type=bind,source=$HOME/projects/e3sm,target=/home/e3sm/projects/e3sm \

--hostname singularity \

--shm-size=512m \

lukaszlacinski/e3sm:2021.03.19 /bin/su - e3sm

The command starts a Docker container with e3sm:2021.03.19 image, mounts your .ssh/ directory to a corresponding directory of the ‘e3sm’ user in the container, mounts your e3sm project data directory projects/e3sm to a corresponding directory in the container, and run a command to switch to the ‘e3sm’ user in the container. By default, Docker sets a memory usage limit for containers on macOS and Windows systems to 2 GB. The limit may trigger the OOM Killer that will kill a running case with 9 signal. If needed, a memory limit can be increased with the -m option when starting a docker container, e.g. -m 16g.

If you want to use ssh with public-key authentication without being asked for your private key passphrase, you can set up an ssh-agent:

e3sm@singularity:~$ eval `ssh-agent -s`

e3sm@singularity:~$ ssh-add

Now, you can clone the E3SM repo:

e3sm@singularity:~$ git clone --recursive git@github.com:E3SM-Project/E3SM.git

If you mounted your github repository from the host filesystem, you can skip the step above and to go directory to the repo, and run a case. By default, it assumes 16 threads/cores available on the machine, <GMAKE_J>16</GMAKE_J>. If you run it on less powerful workstation/laptop, you may run into an error when building the model, and need to change it (in the example below to 2).

e3sm@singularity:~$ cd E3SM/cime/scripts/
e3sm@singularity:~$ ./create_newcase --case container.A_WCYCL1850.ne4_oQU240.baseline --compset A_WCYCL1850 --res ne4_oQU240
e3sm@singularity:~$ cd container.A_WCYCL1850.ne4_oQU240.baseline/
e3sm@singularity:~$ ./case.setup
e3sm@singularity:~$ ./xmlchange GMAKE_J=2
e3sm@singularity:~$ ./case.build
e3sm@singularity:~$ ./case.submit

There is no batch system so ./case.submit launches an interactive job and you get to watch the output from mpirun and will have to wait until the model finishes. So instead launch it like this:

e3sm@singularity:~$ ./case.submit >& case.out &

You can then cd to the rundir and look at the log files as they are made or do other things inside the container. If you exit the container, the job will not finish.