The Big Picture of E3SM Model Development
All E3SM developers should be familiar with the basics of development with git including cloning the E3SM repo, making a branch, committing changes and submitting a Pull Request on Github. Also writing good commit messages and pull request descriptions. For more info see, the Git Tutorial, Development Getting Started Guide, Development Reference and Commit and PR message template.
Group Leads and Epic leads coordinate and schedule development relevant to their group and/or epic. They should have developers assigned to work on features in the order necessary to meet deliverables in the Roadmaps (the leads might also be the developers). Epic leads should make sure developers are finishing their work and getting it merged to the E3SM "master" branch.
Mailing Lists: Note that getting a confluence account does not automatically add you to the relevant mailing lists. All users should periodically review the list of mailing lists and join any that may be relevant: Email Lists
Integrators: every group has 2 or 3 Integrators who take completed feature branches and, if they meet the requirements, merge them to master. See Integrator Guide OLD for more info.
The Coupled Model Group established the timeline for major new features (especially from two or more groups) that are needed for the model, relevant testing criteria, and when the master should be tagged (using the Branch, Tag, and Version name conventions) all timed to meet E3SM deliverables as defined in the Roadmaps. Integrators should be aware of the Roadmap for their component.
A branch for every feature: Group and task leads and developers should define units of code development that are big enough to be relevant to others, but simple enough that it can be called one feature (or perhaps a couple of highly-interdependent features) and make a branch for each unit of work. These branches are called "feature branches" in our documentation. The developer should make commits that are minimal atomic units of work to complete the feature that the branch seeks to implement. This separation makes it easier to review and debug later. New development should generally start as a branch from "master". Avoid creating large branches that contain many features and touch dozens of files. If a large branch is needed for some major development, the Software Engineering hub should be involved in planning.
Bug fix branches: When a bug is found on master, the branch for developing a fix should start from the commit that introduced the bug. This is so that any feature branch which also contains the bug can easily merge in the bug fix without having to merge in many other changes on that may have occurred while the feature branch was being developed.
Focus on your branch: While working on their branch, developers should focus on the work necessary to complete the branch and not worry about what else is going on. In particular, there is almost never a need in git to "merge from master" or "stay current" with development going on in master or other branches to make the eventual merge easier. If a developer thinks a merge from master or another branch is necessary to complete their work, they should check with an Integrator first.
Developing with externals: some code in E3SM has its primary development in another repo. See Externals in the E3SM code
"master" is always stable: Our code development workflow ensure that master is always stable and can be used to start new development.
Finishing development by Testing and Merging: When the feature branch is done, the developer makes sure the acme_developer test suite passes (or the failures are expected). When the test results are in order, the developer issues a "pull request (PR)", and designates an Integrator. The Integrator conducts the code review, merges the feature branch into the "next" integration branch and tests the feature within the latest version of E3SM using the acme_integration suite. The integrator and developer may need to work together to resolve any issues that arise from the code review or testing in the "next" branch. The integrator will then merge the branch into master. Once merged to master, the branch is complete and the developer can move on to other topics. See the Development Reference for more information. In summary:
- One feature per git branch (ideally)
- One pull request (PR) per branch (absolutely)
- Nothing gets added to "next" or "master" except by merging a branch through a PR (absolutely)
Coding with performance in mind: Fortran Performance Best Practices