When you begin work on a new topic branch, it is helpful to generate baselines to use in testing the correctness of your changes. It is recommended that you use the create_tests script to generate baselines from the origin of your topic branch so that you can check your results against these baselines without concern that they will be changed by concurrent integrations to the master branch.
To create baselines for your topic branch, you can invoke create_tests with the -generate flag. To generate baselines for all tests in a test suite:
cd projects/ACME/scripts
./create_test -machine machine -compiler compiler -category suite -testname new-test-case.machine.compiler -testroot test-root-directory -generate new-test-baseline-dir -project account
or, to generate a baseline for an individual test:
cd projects/ACME/scripts
./create_test -testname new-test-case.machine.compiler -testroot test-root-directory -generate new-test-baseline-dir -project account
The arguments (and their supplied values) are:
- machine: the name of the machine for which baselines are generated (test suite generation only)
- compiler: the name of the compiler for which baselines are generated (test suite generation only)
- category: the name of the test suite for which baselines are generated (test suite generation only)
- testname: the name used to identify the single test for which the baselines will be generated (single test generation only)
- testroot: the name of a directory in which the test case will be generated and run. Should probably be a new or empty directory.
- generate: the directory in which the baseline files will be stored. This should be the directory you use for baseline comparisons in your topic branch
- project: the name of the account to charge for compute time
If your code development changes an answer in a test, you will need to regenerate your baselines for your tests to pass, both in testing your own topic branch, and when the branch is integrated to next (and then to master).