...
These are the test types available with CIME.
From See https://github.com/ESMCI/cime/blob/master/scripts/Testing/Testcases/READMEconfig/config_tests.xml for latest.
The following are the test functionality categories:
1) smoke tests
2) restart tests
3) threading/pe-count modification tests
4) sequencing (layout) modification tests
5) multi-instance tests
6) ESMF interface tests
7) archiving (short-term and long-term) tests
8) performance tests
9) spinup tests (TODO)
10) compiler flag assumptions (e.g. auto-promotion)
NOTES:
- unless otherwise noted everything is run in one executable directory
- suffix: denotes the component history file suffixes that are added as part of the test
- IOP test is done along with regular tests - not as a separate test
- IOP test is only currently valid for SMS, ERS and PET
======================================================================
Smoke Tests
======================================================================
SMS smoke startup test (default length)
do a 5 day initial test (suffix: base)
if $IOP_ON is set then suffix is base_iop
success for non-iop is just a successful coupler
SBN smoke build-namelist test (just run preview_namelist and check_input_data)
======================================================================
Restart Tests
======================================================================
ERS exact restart from startup (default 6 days + 5 days)
do an 11 day initial test - write a restart at day 6 (suffix: base)
if $IOP_ON is set then suffix is base_iop
do a 5 day restart test starting from restart at day 6 (suffix: rest)
if $IOP_ON is set then suffix is rest_iop
compare component history files ".base" and ".rest" at day 11
ERP pes counts hybrid (open-MP/MPI) restart bfb test from startup, default 6 days + 5 days (previousy PER)
initial pes set up out of the box
do an 11 day initial test - write a restart at day 6 (suffix base)
half the number of tasks and threads for each component
do a 5 day restart test starting from restart at day 6 (suffix rest)
this is just like an ERS test but the pe-counts/threading count are modified on retart
ERI hybrid/branch/exact restart test, default (by default STOP_N is 22 days)
(1) ref1case
do an initial for ${STOP_N}/6 writing restarts at ${STOP_N}/6
ref1 case is a clone of the main case (by default this will be 4 days)
short term archiving is on
(2) ref2case
do a hybrid for ${STOP_N}-${STOP_N}/6 running with ref1 restarts from ${STOP_N}/6
and writing restarts at ( ${STOP_N} - ${STOP_N}/6 )/2 +1
(by default will run for 18 days and write a restart after 10 days)
ref2 case is a clone of the main case
short term archiving is on
(3) case
do a branch run starting from restart written in ref2 case
and run for ??? days
(4) case do a restart run from the branch case
======================================================================
Threading/PE-Counts/Pe-Sequencing Tests
======================================================================
PET modified threading openmp bfb test (seq tests)
do an initial run where all components are threaded by default (suffix: base)
do another initial run with nthrds=1 for all components (suffix: single_thread)
compare base and single_thread
PEM modified pe counts mpi bfb test (seq tests)
do an initial run with default pe layout (suffix: base)
do another initial run with half tasks, same threads (suffix: modpes)
compare base and modpes
PMT modified-task and modified-thread count bfb test (previousy OEM)
do an initial run (suffix: base)
do a second run with half tasks, twice threads (suffix: modpes)
(***note that PMT_script and PEM_script are the same - but PEM_build.csh and PMT_build.csh are different***)
PEA single pe bfb test
do an initial run on 1 pe with mpi (suffix: base)
do the same run on 1 pe with mpiserial (suffix: mpiserial)
======================================================================
Sequencing (layout) Tests (smoke)
======================================================================
SEQ different sequencing bfb test
do an initial run test with out-of-box PE-layout (suffix: base)
do a second run where all root pes are at pe-0 (suffix: seq)
compare base and seq
======================================================================
Multi-Instance Tests (smoke)
======================================================================
NCK multi-instance validation vs single instance - sequential PE for instances (default length)
do an initial run test with NINST 1 (suffix: base)
do an initial run test with NINST 2 (suffix: multiinst for both _0001 and _0002)
compare base and _0001 and _0002
NCR multi-instance validation vs single instance - concurrent PE for instances (default length)
do an initial run test with NINST 1 (suffix: base)
do an initial run test with NINST 2 (suffix: multiinst for both _0001 and _0002)
compare base and _0001 and _0002
(***note that NCR_script and NCK_script are the same - but NCR_build.csh and NCK_build.csh are different***)
NOC multi-instance validation for single instance ocean (default length)
do an initial run test with NINST 2 (other than ocn), with mod to instance 1 (suffix: inst1_base, inst2_mod)
do an initial run test with NINST 2 (other than ocn), with mod to instance 2 (suffix: inst1_base, inst2_mod)
compare inst1_base with inst2_base
compare inst1_mod with inst2_mod
======================================================================
ESMF Tests (smoke)
======================================================================
CME compare mct and esmf interfaces (default is 10 days for each)
do a MCT run (suffix: base)
do an ESMF run (suffix: esmf)
compare base with esmf
======================================================================
Performance Tests
======================================================================
PFS system performance test
ICP cice performance test
OCP pop performance test
======================================================================
SPINUP tests
======================================================================
SSP smoke CLM spinup test (only valid for CLM compsets with CLM45 and CN or BGC) (TODO - change to SPL)
do an initial spin test (setting CLM_BLDNML_OTPS to -bgc_spinup_on)
write restarts at the end of the run
short term archiving is on
do a hybrid non-spinup run run from the restart files generated in the first phase
SPO smoke spinup-ocean test
======================================================================
Archiving Tests
======================================================================
STA Short-term archiving restart test
LAR long term archive test
======================================================================
Compiler flag assumptions
======================================================================
APT auto promotion test ...