Step-by-step guide to running the tests
Questions? Ask them in Slack: Chat for E3SM
Choose the test category you want to run, in most cases, it will be one of the
...
e3sm-supported test categories
- acme
- e3sm_integration test suite - A substantial set of tests run nightly on "next" and "master" on a few major platforms or workstations that establishes the correctness of the
ACME - E3SM code base. Used for verifying merged code. The set of target machines should cover all supported compilers.
acme- e3sm_developer test suite - A more minimal set of tests that can be run by
develpers - developers on all supported platforms to instill a modicum of confidence that a set of changes does not break
ACME - E3SM before issuing a PR, without incurring the computational cost of running
acme_full or acme_integration everywhere- e3sm_integration everywhere. e3sm_developer is included within e3sm_integration. On some machines, we only run e3sm_developer nightly because it takes less time.
- component level suite. Some components have defined their own tests suites. e.g. e3sm_land_developer, e3sm_atm_developer, e3sm_mosart_developer.
To run a given test suite on a given machine, use the create_test command, issued from the cime/scripts
...
directory in your
...
E3SM source tree:
- cd
...
- E3SM/cime/scripts
- ./create_test test
...
- _suite [--compiler compiler --
...
- baseline-root e3sm_baseline_dir -t test_id -r test_root_dir] [-
...
- (c|g) [-b baseline_name ]] [--project project_or_account ]
- machine - The
- (c|g) [-b baseline_name ]] [--project project_or_account ]
...
- E3SM scripts will probe your machine name from the current nodename
- compiler - The compiler toolset you wish to use, examples are: gnu, intel, pgi. You do not need to provide this if you want the default compiler (the first compiler listed in the COMPILERS parameter for your machine)
- test_suite - The name of the test category you want to run, like
...
- e3sm_developer. You can also provide a list of test names. Instead a test suite, you can specify one specific test or set of tests.
- test_root_dir - The path where your test cases will be dumped
...
- . Default will be CIME_OUTPUT_ROOT for the machine you are using.
- e3sm_baseline_dir - You only need to specify this if you want to use a different baseline area than is specified
...
- for your machine by the BASELINE_ROOT parameter
- -c compare against baselines you generated in Generating new baseline files for a topic branch or the system default baselines which are usually from latest master.
- If you generate new baselines, especially for shared master and next branches, make them group-writable by others:
- chmod -R g+w $BASELINE_ROOT/$branch_name
- baseline_subdir - The name of the baselines (usually the major release you're on) you want to use.
...
- The default will be <current-branch-name>. Note that the tool will always prepend the compiler to the baseline subdir. Our baselines are always segregated by compiler. Don't bother with this option unless you provided -c or -g.
- project_or_account - The id that lets you run batch jobs on this machine
- test_id - a string appended to end of test name. default is a date and sha such as ".20210708_184630_c657sy"
- for a complete list of options, do "./create_test --help"
- Once the tests are running, you'll want to see test results
- Case 1: Simple
- cd test_root_dir
- you will find a script named cs.status.(testid)
- Case 1: Simple
...
- You can run this script from the test root directory to see the status of tests being run within this suite
- Case 2: We have a more sophisticated script called wait_for_tests that provides extra capabilites like: waiting for tests to finish, converting results to CTest and submitting to a CDash dashboard
- cd test_root_dir
...
- <E3SM>/
...
- cime/
...
- CIME/wait_for_tests [ -d buildname] */TestStatus
- Example 1 Testing without baselines: to run the '
...
- e3sm_developer' suite on
...
- Chrysalis using the Intel compiler
...
- , you would run:
- (make sure the clone is checked out on your development branch)
- cd
- , you would run:
...
- E3SM/cime/scripts
- ./create_test
...
- e3sm_
...
cd $SCRATCH/acme_dev
...
./cs.status.acme_dev.edison
...
Test result output from looks like the following:
PASS ERS.f19_g16_rx1.A.edison_intel
PASS ERS.f19_g16_rx1.A.edison_intel.memleak
PASS ERS.f19_g16_rx1.A.edison_intel.generate./scratch2/scratchdirs/johnson/acme-baseline-testcases
FAIL ERS_IOP4c.f19_g16_rx1.A.edison_intel
BFAIL ERS_IOP4c.f19_g16_rx1.A.edison_intel.generate./scratch2/scratchdirs/johnson/acme-baseline-testcases
RUN PEA_P1_M.f45_g37_rx1.A.edison_intel.G.acme_dev
PEND SMS.ne30_f19_g16_rx1.A.edison_intel
...
- developer -r mytestrun (the script will detect its on Chrysalis and use the default compiler, intel)
- cd /lcrc/group/e3sm/$USERID/scratch/chrys/mytestrun
- ./cs.status.(testid).
- Example 2: Testing against baselines:
See Interpreting test results.
See the CSEG presentation on testing for more information on the testing system.
...
...
Related articles
Filter by label (Content by label) | ||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Page Properties | |||
---|---|---|---|
| |||
|