Versions Compared

Old Version 1

changes.mady.by.user Robert Jacob

Saved on

compared with

New Version 2

changes.mady.by.user Robert Jacob

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: distinquish pre and post-cime.

 

Available Tests
==========================================================================
APT (auto promotion test (default length)) 
CME (compare mct and esmf interfaces (10 days)) 
ERB (branch/exact restart test) 
ERH (hybrid/exact restart test) 
ERI (hybrid/branch/exact restart test, default 3+19/10+9/5+4 days) 
ERS (exact restart from startup, default 6 days + 5 days) 
ERT (exact restart from startup, default 2 month + 1 month (ERS with info dbug = 1)) 
ICP (cice performance test) 
LAR (long term archive test) 
NCK (multi-instance validation vs single instance (default length)) 
NOC (multi-instance validation for single instance ocean (default length)) 
OCP (pop performance test) 
P4A (production branch test b40.1850.track1.1deg.006 year 301) 
PEA (single pe bfb test (default length)) 
PEM (pes counts mpi bfb test (seq tests; default length)) 
PET (openmp bfb test (seq tests; default length)) 
PFS (performance test setup) 
PRS (pes counts hybrid (open-MP/MPI) restart bfb test from startup, default 6 days + 5 days) 
SBN (smoke build-namelist test (just run preview_namelist and check_input_data)) 
SEQ (sequencing bfb test (10 day seq,conc tests)) 
SMS (smoke startup test (default length)) 
SSP (smoke CLM spinup test (only valid for CLM compsets with CLM45 and CN or BGC))

The following modifiers can be used in the test name
_CG = gregorian calendar
_D = debug
_E = esmf interfaces
_IOP*= PnetCDF IO test where * is
A(atm), C(cpl), G(glc) , I(ice),
L(clm), O(ocn), W(wav) or blank (all components)
_L* = set run length y, m, d, h, s, n(nsteps) plus integer (ie _Lm6 for 6 months)
_M* = set the mpilib to *, where * is default, mpi-serial, mpich, etc
_N* = set NINST_ env value to *, where * is an integer
_P* = set pecount to *, where * are specific values which include
T,S,M,L,X,1,1x1,16,16x1,4x4, etc
_R* = PTS_MODE test case, valid values are LA, LB, OA, OB

 

See also https://github.com/ESMCI/cime/blob/master/scripts/Testing/Testcases/README reproduced below.

...

Post-CIME

These are the test types available with CIME.

From https://github.com/ESMCI/cime/blob/master/scripts/Testing/Testcases/README

The following are the test functionality categories:
  1) smoke tests
  2) restart tests
  3) threading/pe-count modification tests
  4) sequencing (layout) modification tests
  5) multi-instance tests
  6) ESMF interface tests    
  7) archiving (short-term and long-term) tests
  8) performance tests
  9) spinup tests (TODO)
 10) compiler flag assumptions (e.g. auto-promotion)


NOTES: 
- unless otherwise noted everything is run in one executable directory
- suffix: denotes the component history file suffixes that are added as part of the test
- IOP test is done along with regular tests - not as a separate test 
- IOP test is only currently valid for SMS, ERS and PET

======================================================================
    Smoke Tests
======================================================================

SMS    smoke startup test (default length) 
       do a 5 day initial test (suffix: base)
       if $IOP_ON is set then suffix is base_iop
       success for non-iop is just a successful coupler 

SBN    smoke build-namelist test (just run preview_namelist and check_input_data) 

======================================================================
    Restart Tests
========================================
    Smoke Tests
======================================================================

SMS    smoke startup test (default length

ERS    exact restart from startup (default 6 days + 5 days) 
       do aan 511 day initial test - write a restart at day 6     (suffix: base)
       if $IOP_ON is set then suffix is base_iop
       success for non-iop is just a successful coupler 

SBN    smoke build-namelist test (just run preview_namelist and check_input_data) 

======================================================================
    Restart Tests
======================================================================

ERS    exact restart from startup (do a  5  day restart test starting from restart at day 6 (suffix: rest)
       if $IOP_ON is set then suffix is rest_iop
       compare component history files ".base" and ".rest" at day 11 

ERP    pes counts hybrid (open-MP/MPI) restart bfb test from startup, default 6 days + 5 days (previousy PER)
       initial pes set up out of the box
       do an 11 day initial test - write a restart at day 6     (suffix: base) 
       half the ifnumber $IOP_ONof istasks setand thenthreads suffixfor iseach base_iopcomponent
       do a  5  day restart test starting from restart at day 6 (suffix: rest)
       ifthis $IOP_ON is setjust thenlike suffixan is rest_iop
       compare component history files ".base" and ".rest" at day 11 

ERP    pes counts hybrid (open-MP/MPI) restart bfb test from startup, default 6 days + 5 days (previousy PER)
       initial pes set up out of the box
       do an 11 day initial test - write a restart at day 6     (suffix base) 
       half the number of tasks and threads for each componentERS test but the pe-counts/threading count are modified on retart

ERI    hybrid/branch/exact restart test, default (by default STOP_N is 22 days)
       (1) ref1case
           do an initial for ${STOP_N}/6 writing restarts at ${STOP_N}/6
           ref1 case is a clone of the main case (by default this will be 4 days)
           short term archiving is on
       (2) ref2case
           do a hybrid 5  day restart test starting from restart at day 6 (suffix rest)for ${STOP_N}-${STOP_N}/6 running with ref1 restarts from ${STOP_N}/6
           and writing restarts thisat is just like an ERS test but the pe-counts/threading count are modified on retart

ERI    hybrid/branch/exact restart test, default (by default STOP_N is 22 days)( ${STOP_N} - ${STOP_N}/6 )/2 +1 
	   (by default will run for 18 days and write a restart after 10 days)
           ref2 (1)case ref1caseis a clone of the main case 
    do an initial for ${STOP_N}/6 writing restarts at ${STOP_N}/6
    short term archiving is on
       ref1(3) case is
a clone of the main case (by default this will be 4do days)a branch run starting from restart written in ref2 case 	  short 
 term archiving is on        (2) ref2caseand run for ???  days
       (4) case do a hybridrestart for ${STOP_N}-${STOP_N}/6 running with ref1 restarts from ${STOP_N}/6
           and writing restarts at ( ${STOP_N} - ${STOP_N}/6 )/2 +1 
	   (by default will run for 18 days and write a restart after 10 daysrun from the branch case

======================================================================
    Threading/PE-Counts/Pe-Sequencing Tests
======================================================================

PET    modified threading openmp bfb test (seq tests)
       do an initial run ref2where caseall iscomponents aare clonethreaded ofby thedefault main case(suffix: base)
       do another initial run with shortnthrds=1 termfor archivingall iscomponents on        (3(suffix: single_thread)
case       compare base and single_thread

PEM    modified pe counts mpi bfb test (seq tests)
       do aan branchinitial run startingwith fromdefault restartpe writtenlayout in ref2 case 	               and run for ???  days       (suffix: (4base)
case do a restart run from the branch case

======================================================================
    Threading/PE-Counts/Pe-Sequencing Tests
======================================================================

PET    modified threading openmp bfb test (seq tests)do another initial run with modified pes (NTASKS_XXX => NTASKS_XXX/2)  (suffix: modpes)
       compare base and single_thread

PMT    modified-task and modified-thread count bfb test (previousy OEM)
       do an initial run                   do an initial run where all components are threaded by default (suffix: base)
       do anothera initialsecond run with nthrds=1 for all components        half tasks, twice threads (suffix: single_threadmodpes)
       compare base and single_thread

PEM    modified pe counts mpi(***note that PTM_script and PEM_script are the same - but PEM_build.csh and PTM_build.csh are different***)

PEA    single pe bfb test (seq
tests)        do an initial run withon default1 pe layoutwith mpi     (suffix: base)
       do the same run on 1 pe with         mpiserial (suffix: base)
       do another initial run with modified pes (NTASKS_XXX => NTASKS_XXX/2)  (suffix: modpes)mpiserial)

======================================================================
    Sequencing (layout) Tests compare base and single_thread

PMT    modified-task and modified-thread count bfb test (previousy OEM)(smoke)
======================================================================

SEQ    different sequencing bfb test  
       do an initial run test with out-of-box PE-layout (suffix: base)
       do a second run where all root pes are at   pe-0    (suffix: baseseq)
       docompare abase secondand runseq
with half tasks, twice threads (suffix: modpes)
       (***note that PTM_script and PEM_script are the same - but PEM_build.csh and PTM_build.csh are different***)

PEA    single pe bfb test 
       
======================================================================
    Multi-Instance Tests (smoke)
======================================================================

NCK    multi-instance validation vs single instance - sequential PE for instances (default length)
       do an initial run on 1 petest with mpiNINST 1    (suffix: base)
       do thean sameinitial run ontest 1with peNINST with2 mpiserial (suffix: mpiserial)

======================================================================
    Sequencing (layout) Tests (smoke)
======================================================================

SEQ    different sequencing bfb test  multiinst for both _0001 and _0002)
       compare base and _0001 and _0002

NCR    multi-instance validation vs single instance - concurrent PE for instances  (default length)
       do an initial run test with NINST 1 (suffix: base)
       do an initial run test with out-of-box PE-layoutNINST 2 (suffix: base)multiinst for both _0001 and   _0002)
do a second run where all root pes arecompare atbase pe-0and _0001 and (suffix: seq)_0002
       (***note comparethat baseNCR_script and seq

======================================================================
    Multi-Instance Tests (smoke)
======================================================================

NCK    multi-instance validation vs single instance - sequential PE for instances (default lengthNCK_script are the same - but NCR_build.csh and NCK_build.csh are different***)

NOC    multi-instance validation for single instance ocean (default length)
       do an initial run test with NINST 2 (other than ocn), with mod to instance 1 (suffix: inst1_base, inst2_mod)
       do an initial run test with NINST 12 (suffix: base)
       do an initial run test with NINST 2 (suffix: multiinst for both _0001 and _0002)other than ocn), with mod to instance 2 (suffix: inst1_base, inst2_mod)
       compare inst1_base with inst2_base 
        compare baseinst1_mod and _0001with and inst2_0002mod  NCR    multi-instance validation vs single instance - concurrent PE for instances  (default length)
       do an initial run test with NINST 1 (suffix: base)


======================================================================
    ESMF Tests (smoke)
======================================================================

CME    compare mct and esmf interfaces (default is 10 days for each) 
       do ana initial run testMCT with NINST 2run (suffix: multiinstbase)
for both _0001 and _0002)   do an ESMF run (suffix: esmf) compare
base and _0001 and _0002   compare base with   (***note that NCR_script and NCK_script are the same - but NCR_build.csh and NCK_build.csh are different***)

NOC    multi-instance validation for single instance ocean (default length)
       do an initial run test with NINST 2 (other than ocn), with mod to instance 1 (suffix: inst1_base, inst2_mod)
       do an initial run test with NINST 2 (other than ocn), with mod to instance 2 (suffix: inst1_base, inst2_mod)
       compare inst1_base with inst2_base 
       compare inst1_mod  with inst2_mod  


esmf

======================================================================
    Performance Tests
======================================================================

PFS    system performance test
ICP    cice performance test 
OCP    pop performance test 

======================================================================
    SPINUP tests
======================================================================

 SSP  ESMF Tests (smoke)
======================================================================

CME    compare mct and esmf interfaces (default is 10 days for each)  CLM spinup test (only valid for CLM compsets with CLM45 and CN or BGC)  (TODO - change to SPL)
       do aan initial MCTspin test  run (suffix: base(setting CLM_BLDNML_OTPS to -bgc_spinup_on)
       do an ESMFwrite run (suffix: esmf) restarts at the end of the run
         short term archiving is on
       comparedo basea withhybrid esmfnon-spinup 
============================run run from the restart files generated in the first phase

SPO    smoke spinup-ocean test

======================================================================
    PerformanceArchiving Tests
======================================================================

PFSSTA    system performanceShort-term archiving restart test ICP
LAR   cice performancelong testterm 
OCP    pop performance archive test 

======================================================================
    SPINUPCompiler flag assumptions tests
======================================================================

SSPAPT    smokeauto CLM spinuppromotion test (only valid for CLM compsets with CLM45 and CN or BGC)  (TODO - change to SPL)
       do an initial spin test (setting CLM_BLDNML_OTPS to -bgc_spinup_on)
         write restarts at the end of the run
         short term archiving is on
       do a hybrid non-spinup run run from the restart files generated in the first phase

SPO    smoke spinup-ocean test

======================================================================
    Archiving Tests
======================================================================

STA    Short-term archiving restart test 
LAR    long term archive test 

======================================================================
    Compiler flag assumptions 
======================================================================

APT    auto promotion test

Pre-CIME

Available Tests before CIME integration.
==========================================================================
APT (auto promotion test (default length)) 
CME (compare mct and esmf interfaces (10 days)) 
ERB (branch/exact restart test) 
ERH (hybrid/exact restart test) 
ERI (hybrid/branch/exact restart test, default 3+19/10+9/5+4 days) 
ERS (exact restart from startup, default 6 days + 5 days) 
ERT (exact restart from startup, default 2 month + 1 month (ERS with info dbug = 1)) 
ICP (cice performance test) 
LAR (long term archive test) 
NCK (multi-instance validation vs single instance (default length)) 
NOC (multi-instance validation for single instance ocean (default length)) 
OCP (pop performance test) 
P4A (production branch test b40.1850.track1.1deg.006 year 301) 
PEA (single pe bfb test (default length)) 
PEM (pes counts mpi bfb test (seq tests; default length)) 
PET (openmp bfb test (seq tests; default length)) 
PFS (performance test setup) 
PRS (pes counts hybrid (open-MP/MPI) restart bfb test from startup, default 6 days + 5 days) 
SBN (smoke build-namelist test (just run preview_namelist and check_input_data)) 
SEQ (sequencing bfb test (10 day seq,conc tests)) 
SMS (smoke startup test (default length)) 
SSP (smoke CLM spinup test (only valid for CLM compsets with CLM45 and CN or BGC))

Test Modifiers

The following modifiers can be used in the test name
_CG = gregorian calendar
_D = debug
_E = esmf interfaces
_IOP*= PnetCDF IO test where * is
A(atm), C(cpl), G(glc) , I(ice),
L(clm), O(ocn), W(wav) or blank (all components)
_L* = set run length y, m, d, h, s, n(nsteps) plus integer (ie _Lm6 for 6 months)
_M* = set the mpilib to *, where * is default, mpi-serial, mpich, etc
_N* = set NINST_ env value to *, where * is an integer
_P* = set pecount to *, where * are specific values which include
T,S,M,L,X,1,1x1,16,16x1,4x4, etc
_R* = PTS_MODE test case, valid values are LA, LB, OA, OB