This document provides details on ACME development conventions and practices.
All ACME developers should consider themselves stewards of the repository history. Our goal with defining a workflow is to improve the utility of the repository and create a useful history that can provide a tangible benefit to other developers.
As always, it’s a good idea to understand what you’re doing before you do it. This document should not take the place of understanding, but can be used to learn and remember.
NOTE:
This document does not discuss the use of forks. If a developer understands and prefers to use a fork, it is recommended that they make use of forks.
How to use this document
Within this document there are sections you can refer to when attempting to perform repository actions. The ordering is intended to closely resemble a typically feature development cycle, though some tasks may or may not exist in the development of all features.
Specific pieces of information will be colored as follows:
important -- Items colored in red mean they are important, and should not be ignored.
one time -- Items colored in green are commands to be issued once per machine.
repo once -- Items colored in orange are commands to be issued once per local repository.
common -- Items colored in bold black are commands that will be commonly used.
Project Life Cycle
Before getting into the git commands related to our workflow, here is an overview of the life cycle of a feature. This can be used to get a big picture that will be broken up in the following steps.
In the above image, red dots represent merge commits of features into an integration branch. Orange dots represent commits a developer makes as part of a feature branch. Blue commits represent merge commits incorporating a completed feature into the master branch. Purple commits represent tagged versions of the full model.
NOTE:
Within this diagram, blue dots have associated feature branches that are omitted for the sake of readability. However, each of them followed the same development cycle as the feature that is being focused on in the diagram. Also, red commits are made to next. Both next and master should only be modified by integrators or gatekeepers.
Creating a new branch
New projects should be carried out on a branch. New projects include the addition of a new feature (github-username/component/feature) or fixing a bug (github-username/component/bug-fix).
When created a branch, please name it based on guidelines contained in Branch, Tag, and Version name conventions.
NOTE:
As seen above, ACME utilizes a naming convention for branches. Branches have compound names separated by a “/” to describe what large scale part of ACME the branch changes will modify. The / does not denote a directory, it just is used in naming the branch. This lets other developers know what these branches are developing and what parts of the model they should be modifying. This practice will be referred to as “namespacing a branch”
When creating a branch, multiple levels of namespacing are allowed. In ACME, a branch should be namespaced first by github user in charge of the branch. Similarly, directories contained within tools and scripts can be used to namespace within those directories. This pattern continues into each subdirectory. For example, a branch that is implementing a new parameterization within cam would be named:
joeuser/cam/new-parameterization
And a branch that is modifying the HOMME dynamics would be:
janeuser/homme/new-se-feature
Branch names should be as long as needed to clearly convey what the developer is working on, but they shouldn’t be overly long and descriptive. The following example is a branch name that is too long:
joeuser/cam/this-is-my-awesome-new-feature-that-does-something-cool-and-we-might-not-use-in-the-future
The following diagram depicts two branches that are created. One to fix a bug (github-usernam/component/bug-fix) and another to master a new feature (github-username/component/feature).
The following commands can be used to create these branches:
feature branch: git branch github-username/component/feature acme-vXX
bug fix branch: git branch github-username/component/bug-fix commit-with-bug
The last argument (acme-vXX, or commit-with-bug in this example) is a tag name. If it is omitted, git creates a new branch from the current HEAD (the commit you are currently on).
When creating a new feature branch, best practice suggests you should branch from a tested version of the code. In the previous example, acme-vXX represents a tag of the ACME model. A branch should be created from a tag that has baselines you can compare with to ensure you are not introducing bugs to the model.
Advanced developers can branch off of alternate places if they know what they are doing.
NOTE:
Creating a new branch will not change the current branch that is being worked on in the current working directory.
Changing Branches
Because the git branch command does not modify the current branch in the current working directory, new commits will not be made to the new branch. In order to change the branch that is being worked on, the git checkout command is used. For example:
git checkout github-username/component/feature
will change the current branch to be github-username/component/feature.
NOTE:
Creating a new branch and swapping to it can be done in a single command via:
git checkout -b github-username/component/feature master
Committing changes
While working on a feature or a bug fix, you will be editing source code files. After editing a file, you might want to commit those changes to your local repository in order to checkpoint your work, or track the change you made. By default, git doesn’t track any files in the repository unless they are explicitly added to the repository. In order to add a file to the repository, you can use the following command:
git add path/to/file/in/repo
In addition to causing git to track the file, this will also stage all changes in the file. The next commit that is created will contain all changes in the stage by default. In order to commit only the changes in the staging area, the following command can be used:
git commit
NOTE: The git add command will only stage the changes in the file at the time of the git add. If the file is subsequently modified after a git add, and then a git commit is issued, the commit will not contain changes between the git add and the git commit.
This will open up an editor where a commit message can be written. The first line is the subject line and should be relatively short and concise. The second line should be blank, and every line below that can be the “body” of the commit message with as much information as needed.
An alternative way of committing all changes to any files that are currently tracked by the repository is:
git commit -a
This will again open an editor for a commit message to be entered.
Remote repositories
Git is a distributed content versioning system (DCVS). When you clone a repository, you make a local repository that is essentially a backup of whatever you cloned. When you commit, the commit only occurs in your local repository. In order to allow other people to make use of the commits you make, or to get your commits incorporated into master they need to be “pushed” to a remote.
A remote is a repository that you would like to communicate with, that is not in the local working directory. When you clone a repository, a default remote is created named “origin” that points to whatever you cloned. A remote is really just an alias to the location of another repository you either want to read or write with.
Before communicating with remotes, you might want to add or remove remotes. In order to add and remove remotes the git remote command can be used. The two uses are as follows:
git remote add remote-name protocol:address/to/repo # Creates a remote
git remote remove remote-name # Removes a remote
In order to communicate with remotes, there are three actions. push, pull, and fetch.
The push command can be used to write some commits from your local repository to a remote repository. For example:
git push <remote-name> <local-branch>:<remote-branch>
can be used to write a local branch to a remote branch on the remote pointed to by the name provided. An explicit example would be:
git push ACME-Climate/ACME feature:github-username/component/feature
In this case, we’re writing the branch feature to the remote origin and changing its name to core/feature.
git push is the command used when writing history to remotes. In order to read history from remotes, the git pull and git fetch commands can be used.
The git fetch command will update the local repository with the most recent history from a specific remote, without modifying any local branches in the repository. It can be used as follows:
git fetch <remote-name>
The git pull command is a combination of two operations. The first is a git fetch in order to update the history in the local repository of the branches that exist on the remote repository. The second is a git merge which merges the changes into the current working directory. It is only recommended to use this if you know what you’re doing. It can easily cause unexpected problems. It can be used as follows:
git pull <remote-name> <branch-name>
Submitting a pull request
Once you think development is finished on your project, you should push the project to the shared repository (a remote), and submit a pull request for the integration of your project into master. A pull request initiates a request to merge your branch into another branch. It is publicly documented in combination with the subsequent code review and discussion, and the outcome of pull request
Once the branch is pushed to the shared repository, you can navigate to the shared repository on github. On the shared repository’s webpage, there is a “Compare and pull request” button that can be used to initiate a pull request. You should verify the head and base of your pull request are correct. The pull request should also be provided with a very descriptive message and title. The reviewer will review the description as part of the pull request, because the description will be the commit message used when the merge is finished.
NOTE: The last commit on your branch should not be a merge commit merging master into your branch. The reviewer might run into unexpected merge conflicts when merging your branch into master. If that happens, the reviewer will instruct you on how to help resolve these conflicts.
This document can be used to help with pull request related issues.
Handling reviewer comments
During a code review there are several steps. The first step is for a reviewer to review your pull request description, along with the code and commits from the pull request. The reviewer is allowed to request the following from a developer:
Modify the pull request description
Modify commit history (Including removal, reordering, squashing, etc)
Modify code. This could be requested for stylistic reasons, or functional reasons.
Modifying the pull request description is as simple as editing the wording of the pull request description via github’s website.
Modifying the commit history requires rebasing. In this case, you want to figure out what the base of your current branch is. It is likely that you branched off of master, in which case you can use the merge-base command to determine your base. For example:
git merge-base github-username/component/my-feature ACME-Climate/ACME/master
Will tell you the commit that is the base of your branch. If you perform an interactive rebase onto the merge base, you can modify the history of your branch without introducing merge conflicts associated with changing your base. For example:
git rebase -i $(git merge-base HEAD ACME-Climate/ACME/master)
Finally, modifying code might require introducing new commits, or editing previous commits.
Testing a feature
After a reviewer is satisfied with all parts of a pull request, both the reviewer and the developer will perform testing on the pull request. The developer may be required to point a reviewer to a specific test case or namelist that can help the reviewer test the feature. In addition to testing the feature, it is the reviewers responsibility to verify the pull request does not introduce issues related to the previous functionality of master.
This can either be performed on the branch itself, or in an integration branch. The reviewer will inform the developer where testing should take place.
Fixing bugs
During the testing phase, bugs may be found related to the feature. Bug fixes can be committed inline to the feature branch. This allows a bug fix to show up as an individual commit but remain part of the development history for that feature.
Final merge and conflict resolution
Finally, after testing is complete the reviewer is ready to merge your feature into master. When they are doing this, they might run into unexpected merge conflicts they are unable to resolve.
If they do happen to run into this situation, you may be requested to help with the process. The easiest way to help is to create a new branch at the HEAD of the branch your pull request was submitted from. This branch should have the same name as the other branch with -resolved appended to the end. After the branch is created, you can merge ACME-Climate/ACME/master into it, and push the resolved version onto the shared repository. This gives the reviewer a version of the code that is merged and resolved, but allows the reviewer to ensure the history maintains the standards.
Never use the github automerge! DO NOT PRESS THE GREEN BUTTON!
Merges to master will be done locally by integrators. See Integrator Guide for more info.
Utilizing the repository history
Seeing as all of the ACME developers are stewards of the repository history, it would be useful for developers to understand how to make use of the history they are maintaining.
The most useful command for making use of the history is the git log command. This command has an abundance of optional arguments and this second can be used to get an idea of what you can do with git log.
The basic usage of this command gives the equivalent of an svn log. Where you get a list of all commits and their commit messages. By default, git log only shows commits that are reachable in the history of the HEAD. Optional arguments can be used to look at different (or all) branches.
One extremely useful version of git log gives a command line graph of the history.
git log --oneline --graph --decorate
git log can also only show local branches that match a certain naming convention. i.e.
git log --branches=*pattern*
The --branches option can be replaced with --remotes to only show remote tracking branches that match a pattern.
A specific commit has three pieces of information. The first is the commit’s tree (i.e. the files and directories contained in the commit), the second is the commit’s message (the commit message issued when creating the commit), and the third is a list of the commit’s parents. There can be multiple parents for a single commit, which would imply the commit was a merge commit. Git stores the order of the parents, with the first parent always being the commit the merge was initiated from. This is useful to note, because within our workflow first parent commits on master should always be merge commits bringing in features or bug fixes. git log can be used to narrow your view to only see first parent commits as well, via:
git log --first-parent
As you develop a new feature, you might migrate files from one place to another, or even rename the file. In order to have git show you all renames of a file, you can use the following command:
git log --name-only --follow -- path/to/file
Any of these options can be combined to give more flexibility to exploring the history of a git repository and make the history more useful.
Incorporating another feature
When developing a new feature, another developer might have created a feature you require for your work, or another developer might have made large changes to the interfaces you make use of. This section will describe how to incorporate those changes into your branch.
NOTE:
This action can have negative consequences and cause issues in the future, so it is important to know what you’re doing prior to doing any of these.
Cherry-pick method
The first option for incorporating changes from another development line into your development history is via a cherry-pick. A cherry-pick will copy individual commits (or a range of commits) from one point in history onto the HEAD of your current development line. It should be used when the number of commits your future development efforts depend on are small (order 1-10). It can be used as follows:
git cherry-pick <commit-ish>
This method is the easiest to use and one of the least likely to create issues in future development, and allows the most flexible review modifications. One downside to this method, however, is that the commits that are cherry-picked will now occur twice in the history.
Merge Method
The second option for incorporating changes into your development history is via a merge. A merge will attempt to merge an arbitrary number of other commits (or branches) into your current branch. It can be used as follows:
git merge [commit-ish1] [commit-is2] …
In this case, you can list how ever many commits you want on this line, and git will attempt to merge them all into the commit you are currently working on (into your working directory).
The merge will allow you to enter a commit message. The commit message should be very descriptive. It should explain what you’re merging, and why you’re merging it.
NOTE:
A merge creates a fixed point in history. The merge commit fixes all history before it, which limits possibilities for cleaning up history during a code review. A merge should be avoided if at all possible, but in some cases it is necessary. If it is necessary, try to clean up all history prior to the merge before beginning the merge.
Rebase Method
The third and final option for incorporating changes into your development history is via a rebase. A rebase allows you to modify commits. This includes operation such as squashing, re-ordering, deleting, etc. When you create a branch, the base of the branch is the commit you branched off of. A rebase allows you to modify what the base of your branch is. The following diagram can be used to visualize a rebase:
When performing a rebase, you specify the new base for your work. Rebase will then replay your work onto the new base. For example:
git rebase -i new-base
Will replay the history from the current branch on top of the new-base. The -i flag in this case allows interactive modification of the commits to replay. It should be used to verify what is going to happen after the rebase.
NOTE:
Commits that occur prior to a merge cannot be modified, or rebased. In general, if you previously merged within your branch (i.e. to incorporate an “external feature”) or if your branch was previously merged into another branch, you should not rebase anymore, because it will cause conflicts during future work.
Summary
Below is a summary of the above sections. It will provide an ordered list of commands for a typical workflow. This will not describe why you would use these commands, or even what they do, simply a typical order for a standard project. Some items will be left out, like incorporating changes from an external project that your project is dependent on.
git clone git@github.com:ACME-Climate/ACME.git
(only need to do this once per development platform (your laptop, Mira,Titan, etc.)cd ACME
git checkout -b github-username/component/feature acme-vXX
Repeat the following until development is complete:
vim file # make changes to files, or add new files
git add new-files
git commit -a # Edit commit message, based on template
Optional: git rebase -i <commit>
Optional: Incorporate externally developed feature into your feature
Option 1: git cherry-pick <commit>
Option 2: git merge [commit] [branch]
Option 3: git rebase -i <new-base>
Perform tests as needed throughout development process, but at a minimum before you submit a pull request.
git push ACME-Climate/ACME github-username/component/feature
(push to the shared repository on github)Visit http://github.com/ACME-Climate/ACME and submit a pull request to ACME/master