Versions Compared

Old Version 7

changes.mady.by.user Robert Jacob

Saved on

compared with

New Version 8

changes.mady.by.user Robert Jacob

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
A test launched by the CIME Case Control System "create_test" script goes through all the phases of configuring, building and running a case.   Each phase is checked for a PASS/FAIL condition and the results are output in the TestStatus.log file in the case directory.   The   Within the E3SM nightly test system, the TestStatus.log for each test is uploaded to cdash for examination.


Code Block
PASS ERS.f09_g16_g.MPASLISIA.melvin_gnu CREATE_NEWCASE
PASS ERS.f09_g16_g.MPASLISIA.melvin_gnu SETUP
PASS ERS.f09_g16_g.MPASLISIA.melvin_gnu SHAREDLIB_BUILD time=164
PASS ERS.f09_g16_g.MPASLISIA.melvin_gnu NLCOMP
PASS ERS.f09_g16_g.MPASLISIA.melvin_gnu MODEL_BUILD time=95
PASS ERS.f09_g16_g.MPASLISIA.melvin_gnu SUBMIT
PASS ERS.f09_g16_g.MPASLISIA.melvin_gnu RUN time=17
PASS ERS.f09_g16_g.MPASLISIA.melvin_gnu COMPARE_base_rest
PASS ERS.f09_g16_g.MPASLISIA.melvin_gnu BASELINE
PASS ERS.f09_g16_g.MPASLISIA.melvin_gnu MEMCOMP
PASS ERS.f09_g16_g.MPASLISIA.melvin_gnu TPUTCOMP
PASS ERS.f09_g16_g.MPASLISIA.melvin_gnu MEMLEAK

Briefly, PASS means the test passed, FAIL means that it failed and PEND means the test is waiting to be run or currently running.

The following table, assembled from the Automated Test System slides presented by CSEG, describes the result codes of tests that have passed, failed, or are still running.

...

(1) TestStatus.out file in case directory: this will help you see the cause of failure – e.g., run didn’t complete vs. test requirements (such as exact restart) weren’t met. Look for FAIL in this file, and any messages above the FAIL line.

(2) If run didn’t complete, check log files and core files in run directory

(3) If requirements of test weren’t met,TestStatus.out will generally refer to differences in coupler log files and/or coupler history files. History file differences can be seen in the cprnc.out file in the run directory.

...


...

A test will not be PASS unless ALL the phases are PASS.  If even one phase is FAIL, the entire test fails.

Diagnosing test fails

If a test end result is FAIL (on the cdash dashboard the Status for the test will be "Failed" in white letters over  red background) you will have to look at the TestStatus.log output to determine how it failed (click on the testname on cdash to see the TestStatus.log).

FAIL messageFAIL meaningSettingControl




memleak detected, memory went from X to Y in Z days
(note that this will cause the test to fail even though the word "FAIL" does not appear in the message".

The driver computes highwater memory mark and outputs it to the coupler log.  The log is parsed and if the highwater memory increased by more then TEST_MEMLEAK_TOLERANCE which is env_run.xml.Always onTurn off by adding --ignore-memleak to create_test.  TEST_MEMLEAK_TOLERANCE is defined per machine in config_machines.xml.  Can be changed in testcase directory with xmlchange.