This document provides details on E3SM development conventions and practices.
...
important -- Items colored in red mean they are important, and should not be ignored.
one time -- Items colored in green are commands to be issued once per machine.
repo once -- Items colored in orange are commands to be issued once per local repository.
...
git checkout -b github-username/component/feature master
Utilizing the repository history
Seeing as all of the E3SM developers are stewards of the repository history, it would be useful for developers to understand how to make use of the history they are maintaining.
The most useful command for making use of the history is the git log command. This command has an abundance of optional arguments and this second can be used to get an idea of what you can do with git log.
The basic usage of this command gives the equivalent of an svn log. Where you get a list of all commits and their commit messages. By default, git log only shows commits that are reachable in the history of the HEAD. Optional arguments can be used to look at different (or all) branches.
One extremely useful version of git log gives a command line graph of the history.
git log --oneline --graph --decorate
git log can also only show local branches that match a certain naming convention. i.e.
git log --branches=*pattern*
The --branches option can be replaced with --remotes to only show remote tracking branches that match a pattern.
A specific commit has three pieces of information. The first is the commit’s tree (i.e. the files and directories contained in the commit), the second is the commit’s message (the commit message issued when creating the commit), and the third is a list of the commit’s parents. There can be multiple parents for a single commit, which would imply the commit was a merge commit. Git stores the order of the parents, with the first parent always being the commit the merge was initiated from. This is useful to note, because within our workflow first parent commits on master should always be merge commits bringing in features or bug fixes. git log can be used to narrow your view to only see first parent commits as well, via:
git log --first-parent
As you develop a new feature, you might migrate files from one place to another, or even rename the file. In order to have git show you all renames of a file, you can use the following command:
git log --name-only --follow -- path/to/file
Any of these options can be combined to give more flexibility to exploring the history of a git repository and make the history more useful.
Committing new files and changes as you go
While working on a feature or a bug fix, you will be editing source code files. After editing a file, you might want to commit those changes to your local repository in order to checkpoint your work, or track the change you made. By default, git doesn’t track any files in the repository unless they are explicitly added to the repository. In order to add a file to the repository, you can use the following command:
git add path/to/file/in/repo
In addition to causing git to track the file, this will also stage all changes in the file. The next commit that is created will contain all changes in the stage by default. In order to commit only the changes in the staging area, the following command can be used:
git commit
NOTE: The git add command will only stage the changes in the file at the time you issue the command. If the file is subsequently modified you need to run git add again if you want those changes to be committed. Use git status frequently to see the status of staged and un-staged files.
...
Committing new files and changes as you go
While working on a feature or a bug fix, you will be editing source code files. After editing a file, you might want to commit those changes to your local repository in order to checkpoint your work, or track the change you made. By default, git doesn’t track any files in the repository unless they are explicitly added to the repository. In order to add a file to the repository, you can use the following command:
git add path/to/file/in/repo
In addition to causing git to track the file, this will also stage all changes in the file. The next commit that is created will contain all changes in the stage by default. In order to commit only the changes in the staging area, the following command can be used:
git commit
NOTE: The git add command will only stage the changes in the file at the time you issue the command. If the file is subsequently modified you need to run git add again if you want those changes to be committed. Use git status frequently to see the status of staged and un-staged files.
git commit will open up an editor where a commit message can be written. Please refer to the Commit and PR message template for more instructions on the E3SM commit message guidelines.
Utilizing the repository history
Seeing as all of the E3SM developers are stewards of the repository history, it would be useful for developers to understand how to make use of the history they are maintaining.
The most useful command for making use of the history is the git log command. This command has an abundance of optional arguments and this second can be used to get an idea of what you can do with git log.
The basic usage of this command gives the equivalent of an svn log. Where you get a list of all commits and their commit messages. By default, git log only shows commits that are reachable in the history of the HEAD. Optional arguments can be used to look at different (or all) branches.
One extremely useful version of git log gives a command line graph of the history.
git log --oneline --graph --decorate
git log can also only show local branches that match a certain naming convention. i.e.
git log --branches=*pattern*
The --branches option can be replaced with --remotes to only show remote tracking branches that match a pattern.
A specific commit has three pieces of information. The first is the commit’s tree (i.e. the files and directories contained in the commit), the second is the commit’s message (the commit message issued when creating the commit), and the third is a list of the commit’s parents. There can be multiple parents for a single commit, which would imply the commit was a merge commit. Git stores the order of the parents, with the first parent always being the commit the merge was initiated from. This is useful to note, because within our workflow first parent commits on master should always be merge commits bringing in features or bug fixes. git log can be used to narrow your view to only see first parent commits as well, via:
git log --first-parent
As you develop a new feature, you might migrate files from one place to another, or even rename the file. In order to have git show you all renames of a file, you can use the following command:
git log --name-only --follow -- path/to/file
Any of these options can be combined to give more flexibility to exploring the history of a git repository and make the history more useful.
Advanced: Incorporating another branch on to your branch.
...
Once you think development is finished on your branch, you should push the branch to the shared repository (if you haven't already), and submit a pull request (PR) for the integration of your feature into master.
A pull request initiates a process to merge your branch into master. The process will be documented with the subsequent code review via discussion and comments on the PR.
...
t already), and submit a pull request (PR) for the integration of your feature into master.
A pull request initiates a process to merge your branch into master. The process will be documented with the subsequent code review via discussion and comments on the PR.
- Commit all your changes locally.
- Clean up any "checkpoint" commits. After development is done, may have commits that look like:
f8ef2b5f75 fix
We do NOT want these in our commit history. Use git rebase (see https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History) to rewrite the history of your branch and group these in to substantive commits that follow our Commit and PR message template
d2e3616d55 progress
e58c74745f more progress
0be335ead0 almost working - Push the cleaned-up branch to the github repo.
- go to https://github.com/E3SM-Project/E3SM/branches and find your branch. Click on "New pull request".
- You should verify base and compare of your pull request are correct. The "base" is master and "compare" is your branch.
- Enter a PR Title: Make sure the title is 70 characters or less and explains the PR in a "verb noun" format like "add cool new feature". Do not just use the branch name or what is filled in by default. (If your branch has a single commit, the title of that commit will be the default, but if your branch has multiple commits, the branch name will be the default.) You should edit the title to make sure it is descriptive enough. DO NOT include github issue #'s or JIRA tasks in the title. Hyperlinking does not work from the title.
- Write a PR Description.
- In the "Write" field, provide a descriptive message of what all the commits in the PR will do together. This description will be used as the commit message when the branch is merged so follow the Commit and PR message template guidelines.
- If a bug fix: after the description, close any Github issue numbers for the bugs this commit fixes using keywords like "Fixes #123". See https://help.github.com/articles/closing-issues-using-keywords/
- After the description and any issue numbers, add a keyword indicating how the PR might change answers.
- [BFB] or [non-BFB] or [CC]: Add ONE of these keys to indicate if this commit will affect testing results to roundoff [non-BFB] or climate changing [CC]. Use [BFB] if-and-only-f commit is bit-for-bit and you know all the tests will pass without regenerating baselines.
- [FCC]: Add [FCC] if the commit will change climate if a flag is activated
- [NML]: Add [NML] if the commit introduces changes to the namelist.
- The reviewer will review the description as part of the pull request. The description will be used in the commit message used when the PR is merged to next and master. The integrator may work with you to edit the PR description. Do not include Confluence URL's in the PR description.
- Give PR label(s) ('Label' pull down menu). Add a label for the component this PR involves. Add the "bug fix" label if this is a bug fix. Add the label(s) for BFB, non-BFB, CC, FCC, NML as appropriate.
- Assign a single Integrator to manage the PR ( use the 'Assignee' pull down menu).
- If you want additional reviewers, add them using the Reviewer pull-down menu. Anyone can be asked to review.
- When complete, click on "Create pull request". This will start the code review and the process of moving this feature to master.
- After pull request is created, add the following in the Comments section:
- In the first comment, provide a link to the Design Document governing this PR. See /wiki/spaces/CNCL/pages/25231511.
- In subsequent comments.
- Provide information to aid the Integrator in running, testing, and validating the feature (but that is too specific to be included in the general PR description).
- Say how the feature was tested. Example: "e3sm_developer on Titan passed". If a test is expected to fail and required redoing baselines, state that and list the tests that fail.
Your PR is not finished until it has been merged to master by the Integrator.
This document can be used to help with pull request related issues.
...
- Check the github entry for the PR and make sure it has a good title and description, correct labels and a comment with a link to the Design Document. A PR can not be merged to next or master unless it has a Design Document with Phase 1 and 2 completed. See /wiki/spaces/CNCL/pages/25231511.
- Look at the code changes either on github or using: git log --reverse -p master.. on your checked out copy of the branch.
- Does new code hold up to visual inspection for code quality? Look over code changes for glaring mistakes or code style issues (e.g. useful comments, reasonable subroutine lengths, new code in an existing file follows conventions of that file).
- Check to see if the description of the code changes in the PR match the actual changes. Make sure nothing unrelated to the PR was committed accidentally.
- Although they can't be changed, see if commit messages on the branch follow the Commit and PR message template and let the developer know if they can not. Consider asking the developer to squash commits to clean up the history.
- Have tests been added or suggested that exercise this feature?
- Does code run on all platforms after integration into next?
...
Before communicating with remotes, you might want to add or remove remotes. In order to add and remove remotes the git remote command can be used. The two uses are as follows:
git remote add remote-name protocol:address/to/repo # Creates a remote
git remote remove remote-name # Removes a remote
In order to communicate with remotes, there are three actions. push, pull, and fetch.
...
- "frequent pulling of the \[master] into a development branch will add a certain amount of randomness to that branch; this randomness is not particularly helpful for somebody who is trying to get a feature working. It also increases the chances that another developer who ends up in the middle of the series while running a bisect operation will encounter unrelated bugs."
- A branch has a specific purpose. A topic branch 'add-frotz' would be about adding a new 'frotz' feature and shouldn't do anything else. When you merge from master, you declare that all the other unrelated changes done on 'master' in preparation for the next release somehow bring 'add-frotz' closer to the goal of the 'frotz' topic. That is usually not true
- Unnecessary merges and similar repository clutter reduces the ability to summarize, audit, notice bugs in review, and find bugs after the fact. Keeping clean history is not difficult. It requires a little bit of discipline on the part of integrators and developers, but it's a small price to pay for the time saved and improved quality/reliability.
There are some cases where merging from master (or better, a tagged version of master) can make sense. The rule of thumb for any merge is that if you can clearly describe what you are merging and why that merge is necessary for your branch to be completed, then it's fine to merge. For example, you might need a feature on master (some crucial functionality, not just a build system updates) to complete the feature on your branch. Integrators may ask you to merge from master to help resolve conflicts during a PR integration. But before merging from master, try one of the other ways to get features from other peoples development described in Incorporatinganotherbranchontoyourbranch.
...