2022/4/8: Original
2022/6/17: Revised to stress importance of not bundling features in a single PR

This document describes the policy for bringing in new features and changing existing code in the E3SM model. This new policy requires a significant amount of work improving the E3SM diagnostics, CIME test suites and developing documentation, collected /wiki/spaces/CNCL/pages/3315138566

Terminology

Code changes are divided into 4 categories

• BFB (bit-for-bit)

  • New code produces results which are bit-for-bit identical with old code

• Roundoff

  • New code would match old code in exact arithmetic, but will diverge exponentially fast when using finite precision

  • Climate of new and old code will converge as the averaging time goes to infinity

• Climate changing

  • New feature is brought into the model turned on

  • Other model configuration change (parameters, resolution, forcing data)

• Stealth Feature

  • BFB code change that includes a new feature, turned off by default

Reference Solutions

The E3SM project will maintain reference solutions reflecting the current state of the ‘main’ branch

• Coupled model: B-case runs (~100 years) for WC, cryosphere and BGC configurations

• Component models: F, I and G cases, typically much shorter (e.g. 5 year F case with cyclic year 2010 forcing)

• Updated periodically when climate changes are integrated, or monthly ( to check for unintended changes)

• New features will be evaluated with respect to these reference solutions, with documentation on the import metrics

Note: E3SM Reference solutions and related documentation not yet ready!

Section 1: New Feature Request Process and Documentation Requirements

...

1. Before writing code: Complete the New Feature Overview Document. Include, as appropriate:

   1. High-level description of code changes and/or new code, an overview of the design, infrastructure changes

   2. Expected improvements and how these will be demonstrated

   3. Describe needed updates to E3SM documentation

   4. Expected impacts on computational performance and mass/energy budgets.

   5. If relevant: describe papers that will be published

2. Before writing code: Document Review:

   1. Reviewed by component lead or their delegate. Reviewer should:

      1. Review document for completeness

      2. Determine if there is sufficient benefit to E3SM to justify the E3SM integration and future maintenance costs.

         1. For features not needed by the E3SM SFA, but which may be needed to support other DOE BER missions, the component lead should consult with E3SM leadership.

      3. Determine if review by performance and infrastructure groups if needed and ensure they are done.

         1. discourage and/or flag use of advanced language features or unnecessary complexity that may not be supported well by compilers or may cause performance degradation

      4. Create one confluence page per document to allow for discussion during the review process

         1. Confluence space for all new feature overview documents and their approval status.

3. After new feature is approved, begin work in an E3SM fork, following Development Getting Started Guide

   1. Features which are not approved but are needed for other reasons can be maintained by the developer on their E3SM fork.

4. When work is completed:

   1. Revise overview document to show improvements as originally proposed in overview document, and document simulation results from new feature PR guidelines. Include links to as yet to be constructed results archiving system

   2. Submit PR with links to documentation, follow new feature PR process.

Section 2: Collect documentation and simulation results needed for the Pull Request (PR)

PR’s are divided into 4 categories which require different levels of documentation and simulation results before a PR can be submitted.

1. BFB: Bit-for-bit (BFB) changes which do not include stealth features.

2. Roundoff: These are usually infrastructure and code cleanup changes, compiler changes and system updates which introduce changes for which strong evidence can be provided that the changes do not effect the simulated climate.

3. Stealth Features: These are new features that are turned off by default and BFB with the previous code when turned off.

   1. Stealth features that are not BFB when turned off should be split into two PRs.

4. Climate Changing: PRs that result in larger than roundoff changes. These include new features turned on by default and also changes to default configurations which turn on stealth features.

...