Page Comparison

Every ACME developer should be able to run the acme_developer test suite on at least one machine to which they have convenient access so that they can test their changes before submitting a pull request.

Step-by-step guide

1. Ensure you have the correct tools in your PATH
   1. A full list of software requirements can be found at the top of this page
   2. IF your machine is listed on that page, then most of the environment issues will be taken care of for you, however, you'll still need the following before you can even get started:
      1. git 1.7 or newer (so you can clone repository)
      2. Perl (5.16+), plus the following modules (so you can run ACME scripts):
         1. LibXML
         2. XML::SAX
         3. XML::SAX::Exception
         4. Switch (Switch.pm)
2. Clone the ACME repository
   1. If you are unsure how to do this, go here
3. Make sure the machine you want to test on is supported (should be listed here)
   
   1. If the machine is an institutional cluster or LCF, please contact the SE team and inform them of your needs
   2. If the machine is a personal machine, you'll need to setup configuration for it yourself, go here for help.
4. You'll need to set up your ACME scratch area
   1. Find the configuration for your machine
      1. Found here <ACME>/scripts/ccsm_utils/Machines/config_machines.xml
      2. Search for your machine
      3. Look for the CESMSCRATCHROOT element
   2. Make sure this directory exists and you have read/write access to it

You should now be ready to run tests.

...

You have to first follow the instructions in Installing the ACME Model

Adding more restart files.

The tests needs restart files not normally used in model runs.

1. Populate restart files in to the inputdata directory (these do not get downloaded automatically by the tools as of beta10)  
   1. cd $ACMEHOME/inputdata * svn co https://svn-ccsm-inputdata.cgd.ucar.edu/trunk/inputdata/ccsm4_init

Installing individual tests

1. Once the model is installed, you can immediately run individual tests.
   
   1. cd $ACMEHOME/scripts
   2. ./create_test -testid t01 -testname ERS_D.f10_f10.ICLM45BGC.edison_intel

Installing the acme test suites

Add tests for your machine, starting with the acme_developer tests.  This is code development and should be done on a branch following the Development Quick Guide

1. Add <ACME>/scripts/acme to your PATH
2. Cd to <ACME>/scripts/ccsm_utils/Testlistxml
3. Run: update_acme_tests acme_developer ./testlist.xml -p <machine>,<compiler>

Installing test baselines with first run of a test suite.

Baselines are similar to the input data directory in that there should ideally be one of them per machine that all the ACME developers on that machine can read from.  There only needs to be, for example, one baseline made from ACME v0.2 and the pgi compiler on Titan.   The data sizes  can get large because history files are kept for all components for dozens of (short) runs.  All ACME developers on Titan can then compare against those baselines if they expect their changes to be BFB.

Baselines should be updated when an answer-changing or namelist-changing modification has been made to master.  That should be obvious to anyone following github comments but we may need another way to broadcast it.  Local developers then have to know the new path for the baseline to compare against.  How this will be managed is all still being worked out.

Note: Baselines will accumulate over time so there needs to be some rule for how many to keep.  CSEG has settled on the 5 most recent (beta) tags.

The first run of acme_developer can generate baselines, all subsequent runs should compare against these baselines

1. cd <ACME>/scripts
2. ./create_test -xml_category acme_developer -xml_mach <machine> -xml_compiler <compiler> -testroot <any-path> -project <batch project> -generate <some-name-for-these-baselines>
3. cd <testroot>
4. <ACME>/scripts/acme/wait_for_tests */TestStatus
   1. Note: The wait_for_tests script requires a minimum python version number of 2.7.
5. All tests should pass

Related articles

...