Available Tests
==========================================================================
APT (auto promotion test (default length))
CME (compare mct and esmf interfaces (10 days))
ERB (branch/exact restart test)
ERH (hybrid/exact restart test)
ERI (hybrid/branch/exact restart test, default 3+19/10+9/5+4 days)
ERS (exact restart from startup, default 6 days + 5 days)
ERT (exact restart from startup, default 2 month + 1 month (ERS with info dbug = 1))
ICP (cice performance test)
LAR (long term archive test)
NCK (multi-instance validation vs single instance (default length))
NOC (multi-instance validation for single instance ocean (default length))
OCP (pop performance test)
P4A (production branch test b40.1850.track1.1deg.006 year 301)
PEA (single pe bfb test (default length))
PEM (pes counts mpi bfb test (seq tests; default length))
PET (openmp bfb test (seq tests; default length))
PFS (performance test setup)
PRS (pes counts hybrid (open-MP/MPI) restart bfb test from startup, default 6 days + 5 days)
SBN (smoke build-namelist test (just run preview_namelist and check_input_data))
SEQ (sequencing bfb test (10 day seq,conc tests))
SMS (smoke startup test (default length))
SSP (smoke CLM spinup test (only valid for CLM compsets with CLM45 and CN or BGC))
The following modifiers can be used in the test name
_CG = gregorian calendar
_D = debug
_E = esmf interfaces
_IOP*= PnetCDF IO test where * is
A(atm), C(cpl), G(glc) , I(ice),
L(clm), O(ocn), W(wav) or blank (all components)
_L* = set run length y, m, d, h, s, n(nsteps) plus integer (ie _Lm6 for 6 months)
_M* = set the mpilib to *, where * is default, mpi-serial, mpich, etc
_N* = set NINST_ env value to *, where * is an integer
_P* = set pecount to *, where * are specific values which include
T,S,M,L,X,1,1x1,16,16x1,4x4, etc
_R* = PTS_MODE test case, valid values are LA, LB, OA, OB
See also https://github.com/ESMCI/cime/blob/master/scripts/Testing/Testcases/README reproduced below.
...
Table of Contents |
---|
Post-CIME
These are the test types available with CIME.
See https://github.com/ESMCI/cime/blob/master/CIME/SystemTests/README for latest.
The following are the test functionality categories: 1) smoke tests 2) basic reproducibility tests 3) restart tests 4) threading/pe-count modification tests 5) sequencing (layout) modification tests 6) multi-instance tests 7) performance tests 8) spinup tests (TODO) 9) other component-specific tests Some tests not yet implemented in python. They can be found in cime/scripts/Testing/Testcases NOTES: - IOP is currently not functional ====================================================================== Smoke Tests ====================================================================== SMS smoke startup test (default length) do a 5 day initial test (suffix: base) if $IOP_ON is set then suffix is base_iop success for non-iop is just a successful coupler ====================================================================== Basic reproducibility Tests ============================================================================== REP reproducibility: do two identical runs give the same results? ====================================================================== SmokeRestart Tests ====================================================================== SMSERS smokeexact restart from startup test (default length6 days + 5 days) do aan 511 day initial test - write a restart at day 6 (suffix: base) if $IOP_ON is set then suffix is base_iop successdo fora non-iop is5 just aday successfulrestart couplertest starting from SBNrestart at day 6 smoke build-namelist test (just run preview_namelist and check_input_data) ====================================================================== Restart Tests ====================================================================== ERS exact restart from startup ((suffix: rest) if $IOP_ON is set then suffix is rest_iop compare component history files ".base" and ".rest" at day 11 ERP pes counts hybrid (open-MP/MPI) restart bfb test from startup, default 6 days + 5 days (previousy PER) initial pes set up out of the box do an 11 day initial test - write a restart at day 6 (suffix: base) half the ifnumber $IOP_ONof istasks setand thenthreads suffixfor iseach base_iopcomponent do a 5 day restart test starting from restart at day 6 (suffix: rest) ifthis $IOP_ON is setjust thenlike suffixan isERS rest_ioptest but the pe-counts/threading count are modified on compareretart component historyERI files ".base" and ".rest" at day 11 ERP pes counts hybrid (open-MP/MPI) restart bfb test from startup, default 6 days + 5 days (previousy PER) initial pes set up out of the box hybrid/branch/exact restart test, default (by default STOP_N is 22 days) (1) ref1case do an initial for ${STOP_N}/6 writing restarts at ${STOP_N}/6 do anref1 11case dayis initiala testclone -of writethe amain restartcase at(by daydefault 6this will be 4 (suffix basedays) half the number ofshort tasksterm andarchiving threadsis foron each component (2) doref2case a 5 day restart test starting from restart at daydo 6a (suffixhybrid rest) this is just like an ERS test but the pe-counts/threading count are modified on retart ERI hybrid/branch/exact restart test, defaultfor ${STOP_N}-${STOP_N}/6 running with ref1 restarts from ${STOP_N}/6 and writing restarts at ( ${STOP_N} - ${STOP_N}/6 )/2 +1 (by default STOP_N is 22will run for 18 days) and write a restart after 10 (1days) ref1case ref2 docase anis initiala for ${STOP_N}/6 writing restarts at ${STOP_N}/6clone of the main case short ref1term casearchiving is aon clone of the main(3) case (by default this will be 4 days) do a branch run starting from restart written in shortref2 case term archiving is on (2) ref2caseand run for ??? days (4) case do a restart hybridrun for ${STOP_N}-${STOP_N}/6 running with ref1 restarts from ${STOP_N}/6 and writing restarts at ( ${STOP_N} - ${STOP_N}/6 )/2 +1 (by default will run for 18 days and write a restart after 10 days) ref2 case is a clone of the main case short term archiving is on (3) case do a branch run starting from restart written in ref2 case and run for ??? days (4) case do a restart run from the branch case =======from the branch case ERT Similar to ERS but longer. 2 months + 1 month ====================================================================== Restart and Archive Tests ====================================================================== ERR does an ERS test except that after the initial run the short term archive tool is run which moves model output out of the run directory into the short-term archive directory then the restart run is staged from the short term archive directory. In batch mode there are four submitted jobs for this test (mira excepted) these are run1, sta1, run2 and sta2 run1 and sta1 are submitted together with RESUBMIT=1. sta1 has a batch system dependancy on successful completion of run1, when sta1 is completed it uses the cime resubmit capabilty to submit run2. ====================================================================== Threading/PE-Counts/Pe-Sequencing Tests ====================================================================== PET modified threading openmp bfb test (seq tests) do an initial run where all components are threaded by default (suffix: base) do another initial run with nthrds=1 for all components (suffix: single_thread) compare base and single_thread PEM modified pe counts mpi bfb test (seq tests) do an initial run with default pe layout (suffix: base) do another initial run with modified pes (NTASKS_XXX => NTASKS_XXX/2) (suffix: modpes) compare base and single_thread PMTPEA modified-tasksingle and modified-thread count pe bfb test (previousy OEM) do an initial run on 1 pe with mpi (suffix: base) do the same run on (suffix: base) do a second run with half tasks, twice threads 1 pe with mpiserial (suffix: modpes) (***note that PTM_script and PEM_script are the same - but PEM_build.csh and PTM_build.csh are different***) PEA single pe bfb test do an initial run on 1 pe with mpi (suffix: base) do the same run on 1 pe with mpiserial (suffix: mpiserial) mpiserial) ====================================================================== Sequencing (layout) Tests (smoke) ============================================================================================= Sequencing (layout) Tests (smoke) === SEQ different sequencing bfb test do an initial run test with out-of-box PE-layout (suffix: base) do a second run where all root pes are at pe-0 (suffix: seq) compare base and seq ====================================================================== SEQ Multi-Instance differentTests sequencing bfb test (smoke) ====================================================================== NCK multi-instance validation vs single doinstance an- initialsequential runPE testfor with out-of-box PE-layout (suffix: baseinstances (default length) do aan secondinitial run wheretest allwith root pes are at pe-0 NINST 1 (suffix: seqbase) do comparean baseinitial andrun seqtest ====================================================================== Multi-Instance Tests (smoke) ====================================================================== NCK multi-instance validation vs single instance - sequential PE for instances (with NINST 2 (suffix: multiinst for both _0001 and _0002) compare base and _0001 and _0002 NCR multi-instance validation vs single instance - concurrent PE for instances (default length) do an initial run test with NINST 1 (suffix: base) do an initial run test with NINST 2 (suffix: multiinst for both _0001 and _0002) compare base and _0001 and _0002 NCR multi-instance validation vs single instance - concurrent PE for instances (default length) do an initial run test with NINST 1 (suffix: base(***note that NCR_script and NCK_script are the same - but NCR_build.csh and NCK_build.csh are different***) NOC multi-instance validation for single instance ocean (default length) do an initial run test with NINST 2 (suffix: multiinst for both _0001 and _0002other than ocn), with mod to instance 1 (suffix: inst1_base, inst2_mod) do comparean baseinitial andrun _0001test andwith _0002NINST 2 (other than ocn), with mod to (***note that NCR_script and NCK_script are the same - but NCR_build.csh and NCK_build.csh are different***) NOC multi-instance validation for single instance ocean (default length) do an initial run test with NINST 2 (other than ocn), with mod to instance 1 (suffix: inst1_base, inst2_mod) do an initial run test with NINST 2 (other than ocn), with mod to instance 2 (suffix: inst1_base, inst2_mod) compare inst1_base with inst2_base compare inst1_mod with inst2_mod ==instance 2 (suffix: inst1_base, inst2_mod) compare inst1_base with inst2_base compare inst1_mod with inst2_mod ====================================================================== Performance Tests ================================================================================= PFS ESMF Tests (smoke)system performance test. Do 20 day run, no restarts ICP cice performance test ====================================================================== CME SPINUP compare mct and esmf interfaces (default is 10 days for each) do a MCT run (suffix: basetests ====================================================================== SSP smoke CLM spinup test (only valid for CLM compsets with CN or BGC) (TODO - change to SPL) do an ESMFinitial spin runtest (suffix: esmf)setting CLM_BLDNML_OTPS to -bgc_spinup_on) write restarts at comparethe baseend withof esmfthe ========run short term archiving is on do a hybrid non-spinup run run from the restart files generated in the first phase ============================================================== Performance Tests ======== Other component-specific tests ====================================================================== PFSLII systemCLM performanceinitial test ICP cice performance test OCP pop performance test condition interpolation test
Pre-CIME
Available Tests before CIME integration.
=========================================================================
...
=
APT (auto promotion test (default length))
CME (compare mct and esmf interfaces (10 days))
ERB (branch/exact restart test)
ERH (hybrid/exact restart test)
ERI (hybrid/branch/exact restart test, default 3+19/10+9/5+4 days)
ERS (exact restart from startup, default 6 days + 5 days)
ERT (exact restart from startup, default 2 month + 1 month (ERS with info dbug = 1))
ICP (cice performance test)
LAR (long term archive test)
NCK (multi-instance validation vs single instance (default length))
NOC (multi-instance validation for single instance ocean (default length))
OCP (pop performance test)
P4A (production branch test b40.1850.track1.1deg.006 year 301)
PEA (single pe bfb test (default length))
PEM (pes counts mpi bfb test (seq tests; default length))
PET (openmp bfb test (seq tests; default length))
PFS (performance test setup)
PRS (pes counts hybrid (open-MP/MPI) restart bfb test from startup, default 6 days + 5 days)
SBN (smoke build-namelist test (just run preview_namelist and check_input_data))
SEQ (sequencing bfb test (10 day seq,conc tests))
SMS (smoke startup test (default length))
SSP (smoke CLM spinup test (only valid for CLM compsets with CLM45 and CN or BGC))
Test Modifiers
The following modifiers can be used in the test name
_CG = gregorian calendar
_D = debug
_E = esmf interfaces
_IOP*= PnetCDF IO test where * is
A(atm), C(cpl), G(glc) , I(ice),
L(clm), O(ocn), W(wav) or blank (all components)
_L* = set run length y, m, d, h, s, n(nsteps) plus integer (ie _Lm6 for 6 months)
_M* = set the mpilib to *, where * is default, mpi-serial, mpich, etc
_N* = set NINST_ env value to *, where * is an integer
_P* = set pecount to *, where * are specific values which include
T,S,M,L,X,1,1x1,16,16x1,4x4, etc
_R* = PTS_MODE test case, valid values are LA, LB, OA, OB