Step-by-step guide to running the tests
Questions? Ask them in /wiki/spaces/SE/pages/19464303
Choose the test category you want to run, in most cases, it will be one of the acme-supported test categories
- acme_integration test suite - A substantial set of tests run nightly on "next" on a few major platforms or workstations that establishes the correctness of the ACME code base. Used for verifying merged code. The set of target machines should cover all supported compilers.
- acme_developer test suite - A more minimal set of tests that can be run by develpers on all supported platforms to instill a modicum of confidence that a set of changes does not break ACME before issuing a PR, without incurring the computational cost of running acme_full or acme_integration everywhere.
To run a given test suite on a given machine, use the create_test command, issued from the scripts-acme directory in your ACME source tree:
- cd ACME/cime/scripts-acme
- ./create_test test_suite [--compiler compiler --baseline-root acme_baseline_dir -t test_label -r test_root_dir ] [-(c|g) [-b baseline_subdir ]] [--project project_or_account ]
- machine - The ACME scripts will probe your machine name from the current nodename
- compiler - The compiler toolset you wish to use, examples are: gnu, intel, pgi. You do not need to provide this if you want the default compiler (the first compiler listed in the COMPILERS parameter for your machine)
- test_suite - The name of the test category you want to run, like acme_developer. You can also provide a list of test names.
- test_root_dir - The path where your test cases will be dumped. Default will be CESMSCRATCHROOT
- acme_baseline_dir - You only need to specify this if you want to use a different baseline area than is specified for your machine by the CCSM_BASELINE parameter
- -c/-g - Do a baseline comparison/generation. You do not have to do any baseline operations.
- baseline_subdir - The name of the baselines (usually the major release you're on) you want to use. The default will be <current-branch-name>. Note that the tool will always prepend the compiler to the baseline subdir. Our baselines are always segregated by compiler. Don't bother with this option unless you provided -c or -g.
- project_or_account - The id that lets you run batch jobs on this machine
- test_label - label applied to each case name directory. default is a date such as .20160201_223053.
- Once the tests are running, you'll want to see test results
- Case 1: Simple
- cd test_root_dir
- you will find a script named cs.status.(testid) You can run this script from the test root directory to see the status of tests being run within this suite
- Case 2: We have a more sophisticated script called wait_for_tests that provides extra capabilites like: waiting for tests to finish, converting results to CTest and submitting to a CDash dashboard
- cd test_root_dir
- <ACME>/scripts/acme/wait_for_tests [ -d buildname] */TestStatus
- Case 1: Simple
- Example: to run the 'acme_developer' suite on Edison using the Intel compiler (assuming everything has been set up and this is not the first run) and compare against v0.1 baselines, you would run:
- cd ACME/cime/scripts-acme
- ./create_test acme_developer -t acme_dev -c -b v0.1
- cd <CESMSCRATCHROOT>
- ./cs.status.acme_dev
See Interpreting test results.
See the CSEG presentation on testing for more information on the testing system.
Related articles