...
Load the same modules that a full E3SM case would load: (put in your .login or .bashrc )
E3SM’s CIME provides a script which will give the csh output needed to load the correct modules and other configuration settings:
eval `$e3sm/cime/scriptsCIME/Tools/get_case_env`
Most users will be using the bash shell, in which case examine the output of
$e3sm/cime/scriptCIME/Tools/get_case_env
and modify to work with bash
Run
cmake
from your$wdir
. (only done once for each git checkout)Code Block language bash cd $wdir cmake -C $mach $homme make -j4 theta-l # optional step to see if code will compile
Should only need to be redone if source code dependencies or compiler options change. To rerun from an existing $wdir (keeping the cmake cache file), run “cmake .”
For additional CMAKE options, see
$homme/README.cmake
. Options include number of vertical levels (default 26), type of output, native or lat/lon or disabled, ability to enable/disable energy diagnostics, ability to disable kokkos components (SL transport).
After step 2 is complete, one can build many different targets. The generic targets are:
preqx
,preqx_kokkos.
E3SM v1 hydrostatic dycore (Fortran and C++ version)theta-l
,theta-l_kokkos
. E3SM v2 NH dycoresweqx
. shallow water versionThen there are a host of specific targets such as
theta-l-nlev20-native
that are used by various test cases.
Run DCMIP2012 Test 3-1 (simple gravity wave)
Code Block language bash cd $wdir/dcmip_tests/dcmip2012_test3.1_nh_gravity_waves/theta-l make install # Copies test case related files from $homme ./build.sh # Build the correct executable with correct compile time options for this test
Look for a suitable jobscript for your machine with
ls jobscript-sh*
. Most test cases includejobscript-snl.sh
which runs on E3SM institutional clusters, andjobscript-knl.sh
for NERSC Cori. Some editing may be necessary.The jobscript usually runs included NCL scripts for creating standard test case plots from the output. This may require the user to load an
ncl
module.
All the DCMIP2012 and DCMIP2016 test cases can be run similar to the above example for Test 3-1.
Other standalone HOMME test cases of interest. These usually contain a single script that will configure and build the model (assuming step1 above is done)
Held-Suarez: see
$homme/test/held_suarez0
and the scripths.job
.NGGPS and V1 benchmarks: see
$homme/test/benchmark
and associated scripts.
HOMME doubly periodic test cases
...
Code Block | ||
---|---|---|
| ||
#!/bin/bash # Sets file paths for running the dynamical core (HOMME) of the E3SM echo '-- Setting file paths...' e3sm=/global/homes/j/jlturner/practicum-2021/E3SM homme=$e3sm/components/homme wdir=/global/cscratch1/sd/jlturner/practicum-2021/homme mach=$homme/cmake/machineFiles/cori-knl.cmake test3_1=$wdir/dcmip_tests/dcmip2012_test3.1_nh_gravity_waves/theta-l export e3sm export homme export wdir export mach export test3_1 # Load the necessary modules echo '-- Loading necessary modules...' eval `$e3sm/cime/scriptsCIME/Tools/get_case_env` # Compile HOMME cwd=$(pwd) echo '-- Compiling HOMME...' cd $wdir cmake -C $mach $homme make -j4 theta-l cd $cwd # Compile Test 3.1 cd $test3_1 make install ./build.sh |
...
Note: The time limit on jobscript-knl.sh
may need to be adjusted. We have observed execution times ranging between 7 minutes and 23 minutes.
Below is a more generalized script for compiling DCMIP test cases, using a commend-line argument -t
to indicate test number
Code Block | ||
---|---|---|
| ||
#!/bin/bash # Compiles standalone HOMME for DCMIP 2016 test cases. # Must be adjust for each user, and depends on machine. # Currently set up for account jlturner on Cori. # Author: Jason Torchinsky # Parse command-line input while getopts t: flag do case "${flag}" in t) test=${OPTARG};; esac done # Set up file paths e3sm=/global/homes/j/jlturner/practicum-2022/E3SM homme=$e3sm/components/homme wdir=/global/cscratch1/sd/jlturner/practicum-2022/homme mach=$homme/cmake/machineFiles/cori-knl.cmake case "${test}" in 12_1_1) testname='DCMIP 2012 Test 1.1 - 3D Deformational Flow' testdir=$wdir/dcmip_tests/dcmip2012_test1.1_3d_deformational_flow/preqx ;; 12_1_2) testname='DCMIP 2012 Test 1.2 - Hadley-Like Meridional Circulation' testdir=$wdir/dcmip_tests/dcmip2012_test1.2_hadley_meridional_circulation/preqx ;; 12_1_3) testname='DCMIP 2012 Test 1.3 - Thin Clouds Over Orography' testdir=$wdir/dcmip_tests/dcmip2012_test1.3_thin_clouds_over_orography/preqx ;; 12_2_0) testname='DCMIP 2012 Test 2.0 - Steady State with Orography' testdir=$wdir/dcmip_tests/dcmip2012_test2.0_steady_state_with_orography/theta-l ;; 12_2_1) testname='DCMIP 2012 Test 2.1 - Non-Sheared Background Flow with Orography' testdir=$wdir/dcmip_tests/dcmip2012_test2.1_nh_mountain_waves_no_shear/theta-l ;; 12_2_2) testname='DCMIP 2012 Test 2.2 - Sheared Background Flow with Orography' testdir=$wdir/dcmip_tests/dcmip2012_test2.2_nh_mountain_waves_with_shear/theta-l ;; 12_3) testname='DCMIP 2012 Test 3 - Non-Hydrostatic Gravity Waves' testdir=$wdir/dcmip_tests/dcmip2012_test3.1_nh_gravity_waves/theta-l ;; 12_4_1) testname='DCMIP 2012 Test 4.1 - Baroclinic Instability' testdir=$wdir/dcmip_tests/dcmip2012_test4.1_baroclinic_instability/theta-l ;; 16_1) testname='DCMIP 2016 Test 1 - Moist Baroclinic Wave' testdir=$wdir/dcmip_tests/dcmip2016_test1_baroclinic_wave/theta-l ;; 16_2) testname='DCMIP 2016 Test 2 - Tropical Cyclone' testdir=$wdir/dcmip_tests/dcmip2016_test2_tropical_cyclone/theta-l ;; 16_3) testname='DCMIP 2016 Test 3 - Supercell' testdir=$wdir/dcmip_tests/dcmip2016_test3_supercell/theta-l ;; esac echo "-- Setting up test: $testname" export e3sm export homme export wdir export mach export testdir # Load the necessary modules echo "-- Loading necessary modules..." eval `$e3sm/cime/CIME/Tools/get_case_env` # Compile HOMME cwd=$(pwd) echo "-- Compiling HOMME..." cd $wdir cmake -C $mach $homme make -j4 theta-l cd $cwd # Compile Test cd $testdir make install ./build.sh |