Versions Compared

Old Version 3

changes.mady.by.user Wade Burgess

Saved on

compared with

New Version 4

changes.mady.by.user James Foucar

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Test result output looks like the following:


Code Block
PASS ERS.
f19
f09_g16_
rx1
g.
A.edison_intelPASS ERS.f19
MPASLISIA.melvin_gnu CREATE_NEWCASE
PASS ERS.f09_g16_
rx1.A.edison_intel.memleak
PASS ERS.f19_g16_rx1.A.edison_intel.generate./scratch2/scratchdirs/johnson/acme-baseline-testcases
FAIL ERS_IOP4c.f19_g16_rx1.A.edison_intel
BFAIL ERS_IOP4c.f19_g16_rx1.A.edison_intel.generate./scratch2/scratchdirs/johnson/acme-baseline-testcases
RUN PEA_P1_M.f45_g37_rx1.A.edison_intel.G.acme_dev
PEND SMS.ne30_f19_g16_rx1.A.edison_intel
g.MPASLISIA.melvin_gnu XML
PASS ERS.f09_g16_g.MPASLISIA.melvin_gnu SETUP
PASS ERS.f09_g16_g.MPASLISIA.melvin_gnu SHAREDLIB_BUILD time=164
PASS ERS.f09_g16_g.MPASLISIA.melvin_gnu NLCOMP
PASS ERS.f09_g16_g.MPASLISIA.melvin_gnu MODEL_BUILD time=95
PASS ERS.f09_g16_g.MPASLISIA.melvin_gnu SUBMIT
PASS ERS.f09_g16_g.MPASLISIA.melvin_gnu RUN time=17
PASS ERS.f09_g16_g.MPASLISIA.melvin_gnu COMPARE_base_rest
PASS ERS.f09_g16_g.MPASLISIA.melvin_gnu BASELINE
PASS ERS.f09_g16_g.MPASLISIA.melvin_gnu MEMCOMP
PASS ERS.f09_g16_g.MPASLISIA.melvin_gnu TPUTCOMP
PASS ERS.f09_g16_g.MPASLISIA.melvin_gnu MEMLEAK

Briefly, PASS means the test passed, FAIL means that it failed (and SFAIL, CFAIL, BFAIL, etc are all descriptions of failure in the script generation, configuration, and baseline generation stages), RUN means the test is running, and PEND means the test is waiting to be run or currently running.

 

The following table, assembled from the Automated Test System slides presented by CSEG, describes the result codes of tests that have passed, failed, or are still running.

(1) Batch log files in case directory: determine if it simply ran out of wall-clock time

(2) TestStatus.out file in case directory

(3) Log files and core files in run directory

Note: Some tests (e.g. ERI) create multiple run directories, with .ref1, .ref2 extensions; you may need to check all of them, e.g. check:GEN
Result CodeMeaningWhere to Look First
Good results
PASSTest passed 
Bad results
BFAILCompare test could not find base result 
CHECKManual review of data required 
ERRORtest checker failed. Test may or may not have passed. 
EXPFAILexpected failure 
TFAILTest setup errorOutput from create_test script (stdout)
SFAILGeneration of test failed in scriptsOutput from create_test script (stdout)
CFAILcheck_case script failed. Env variable or Build errorOutput from *.test_build script (will generally point you to a build log file)
RUNRun timed out or exited abnormally

ERI.f10_f10.ICLM45BGC.yellowstone_intel.t01.ref1/run

FAILTest failed (either due to run failure or, e.g., no-exact restarts for an ERS test)

(1) TestStatus.out file in case directory: this will help you see the cause of failure – e.g., run didn’t complete vs. test requirements (such as exact restart) weren’t met. Look for FAIL in this file, and any messages above the FAIL line.

(2) If run didn’t complete, check log files and core files in run directory

(3) If requirements of test weren’t met,TestStatus.out will generally refer to differences in coupler log files and/or coupler history files. History file differences can be seen in the cprnc.out file in the run directory.

Test not yet completeTest has been generated 
BUILDBuild succeeded, not yet submitted 
PENDTest submitted, waiting in queue RUNTest is , or currently running(Check queues or log files to see whether still running or run failed)