...
Directories: (set these variables based on your configuration/preferences and target machine):
e3sm = /path/to/github/code/checkout/E3SM
homme = $e3sm/components/homme
wdir = /path/to/working/directory/for/compiling/and/running
mach = $homme/cmake/machineFiles/cori-knl.cmake
(see directory for other supported machines)
...
Load the same modules that a full E3SM case would load: (should be done once each time you login)
E3SM’s CIME provides a script which will give the csh output needed to load the correct modules and other configuration settings:
eval `$e3sm/cime/scripts/Tools/get_case_env`
Most users will be using the bash shell, in which case examine the output of
$e3sm/cime/
scriptsscript/Tools/get_case_env
and modify to work with bash
Run
cmake
from your$wdir
Code Block language bash cd $wdir
cmake -C $mach $homme
make -j4 theta-l # optional step to see if code will compile
(optional step to see if code will compile)
For additional CMAKE options, see
$homme/README.cmake
. Options include number of vertical levels (default 26), type of output, native or lat/lon or disabled, ability to enable/disable energy diagnostics, ability to disable kokkos components (SL transport).
After step 2 is complete, one can build many different targets. The generic targets are:
preqx
,preqx_kokkos.
E3SM v1 hydrostatic dycore (fortran Fortran and C++ version)theta-l
,theta-l_kokkos
. E3SM v2 NH dycoresweqx
. shallow water versionThen there are a host of specific targets such as “theta
theta-l-nlev20-
native”native
that are used by various test cases.
Run DCMIP2012 Test 3-1 (simple gravity wave)
Code Block language bash cd $wdir/dcmip_tests/dcmip2012_test3.1_nh_gravity_waves/theta-l
install (copies test case related files from $homme)make
(build the correct executable with correct compile time options for this test)install # Copies test case related files from $homme ./build.sh
ls jobscript-sh*.
look# Build the correct executable with correct compile time options for this test
Look for a suitable jobscript for your machine
. mostwith
“jobscriptls jobscript-sh*
. Most test cases include
sh”jobscript-snl.
neccessarysh
which runs on E3SM institutional clusters, andjobscript-knl.sh
for NERSC Cori. Some editing may benecessary.
The jobscript usually runs included NCL scripts for creating standard test case plots from the output. This may require the user to load an
ncl
module.
All the DCMIP2012 and DCMIP2016 test cases can be run similar to the above example for Test 3-1.
Other standalone HOMME test cases of interest. These usually contain a single script that will configure and build the model (assuming step1 above is done)
Held-Suarez: see
$homme/test/held_suarez0
and the script “hshs.job
.job”NGGPS and V1 benchmarks: see
$homme/test/benchmark
and associated scripts.
HOMME doubly periodic test cases
...
Here we include two example Bash scripts, setup-homme.sh
and jobscript-coriknl.sh
, which execute the instructions above and submit a job to execute Test 3-1 (simple gravity wave).
...
Note: The setup can be a little finicky if you have been doing other work during your current session. It is recommended that you run this script from a ‘fresh’ session (i.e., logout and login).
jobscript-
...
knl.sh
Code Block | ||
---|---|---|
| ||
#!/bin/bash # # Jobscript for launching dcmip2012 test3-1 on Cori KNL # #SBATCH --job-name=d31-theta #SBATCH --output=%j.out #SBATCH --error=%j.err #SBATCH --qos=debug #SBATCH --nodes=4 #SBATCH --constraint=knl #SBATCH --account=e3sm #SBATCH --mail-user=jason.torchinsky@wisc.edu #SBATCH --mail-type=all #SBATCH --time=0-00:30:00 OMP_NUM_THREADS=1 EXEC=../../../test_execs/theta-l-nlev20/theta-l-nlev20 ############################################################################# # theta (hydrostatic) ############################################################################# namelist=./namelist-h.nl cp -f $namelist input.nl srun -K -c 1 -N $SLURM_NNODES $EXEC < input.nl ncl plot_omega.ncl ncl plot_theta.ncl mv -f test31_omega.pdf hydro_test31_omega.pdf mv -f dcmip2012_test3_theta_diff.pdf hydro_test3_theta_diff.pdf mv -f dcmip2012_test3_theta_diff_last.pdf hydro_test3_theta_diff_last.pdf mv -f movies/dcmip2012_test31.nc movies/hydro_dcmip2012_test31.nc ############################################################################# # theta-nh (non-hydrostatic) ############################################################################# namelist=./namelist-nh.nl cp -f $namelist input.nl srun -K -c 1 -N $SLURM_NNODES $EXEC < input.nl ncl plot_omega.ncl ncl plot_theta.ncl mv -f test31_omega.pdf nonhydro_test31_omega.pdf mv -f dcmip2012_test3_theta_diff.pdf nonhydro_test3_theta_diff.pdf mv -f dcmip2012_test3_theta_diff_last.pdf nonhydro_test3_theta_diff_last.pdf mv -f movies/dcmip2012_test31.nc movies/nonhydro_dcmip2012_test31.nc |
...