Some of the subdirectories in the ACME E3SM code contain code that was brought in from another repository.
There are two ways to bring in code from one git repo to another: subtree and submodule. ACME uses both and our procedures are described in Sharing External Code using Git Subtree and Sharing External Code using Git Submodules. Any source brought in as an external must be independently buildable and testable in that external.
Subtrees
...
Terms
- external any subdirectory in E3SM whose entire content is from another repository
- upstream master the repository everyone is sending their changes to. Only one so that everyone pulling from the same source is working with all the same changes. The source of an external.
Methods for including externals
- submodule a link that points to another git repository. See Sharing External Code using Git Submodules .
- recursive submodule: a submodule that is inside another submodule.
- subtree a set of files and commits from another git repo, brought in to your repo so that it looks like original content. See Sharing External Code using Git Subtree.
- monorepo the entire repo is added once (usally with subtree) and developed within E3SM. Might maintain a separate build/test system.
- buildtime the repo is checked out to the $CIME_OUTPUT_ROOT/$CASE/bld when compiling a case for the first time. Code does not exist in E3SM repo.
Rules for adding externals
- Group leads must approve adding a new external. The group needing the external can decide on the method of inclusion. A POC for the upstream master must be identified and the IG group leads notified.
- The upstream master for the external must be build-able and testable on its own.
- For submodules, the linked repo must be readable by all E3SM developers. If the upstream master can not be made readable, a fork must be maintained in E3SM-Project github and E3SM then links to that fork.
Development in externals
In general, there are several ways to develop with external code
- strict 1-way: all development is merged to the upstream master first and flows down to E3SM. No exceptions.
- mostly 1-way: prefer strict 1-way but some exceptions can be made for emergency fixes
- 2-way: developers can make changes in E3SM or the upstream master. POC keeps them in sync.
More specific instructions for externals are below.
Table of Externals currently in ACME
| Name | Subdirectory in ACME | external method | upstream master | development method | POC | Notes |
|---|---|---|---|---|---|---|
| CIME | cime | subtree | https://github.com/ |
| ESMCI/cime |
5cc4a90bd04d9273abb00f71762ebba63f2c3531
2-way | |||
| MCT | cime/externals/MCT | subtree | https://github.com/MCSclimate/MCT |
bfbe1c0a7f43d743c549679a5ad9d4ade1f700fe
MCT_2.9.0
mostly 1-way | usually brought in with CIME | ||
| PIO | cime/externals/ |
| pio* | subtree | https://github.com/NCAR/ParallelIO |
67c69713ee282f1b1c2eb7facdc3572a53490ad0
mostly 1-way | usually brought in with CIME | |||||
| SCORPIO | externals/scorpio externals/scorpio_classic | submodule | https://github.com/E3SM-Project/scorpio | strict 1-way | Jayesh Krishna | E3SM uses this and ignores cime/externals/pio |
| Kokkos | externals/kokkos | submodule | git@github.com:E3SM-Project/kokkos.git branch e3sm/kokkos | strict 1-way | James Foucar | |
| MPAS-Source | components/mpas-source | submodule | strict 1-way | MPAS-Ocean, Seaice and land ice all use code from here. | ||
| RRTMGP | components/cam/src/physics/rrtmgp/external | submodule | git@github.com:RobertPincus/rte-rrtmgp.git branch develop | strict 1-way | Ben Hillman | |
| COSP2 | components/cam/src/physics/cosp2/external | submodule | git@github.com:CFMIP/COSPv2.0.git branch CESM |
c2572f19d671c35a4cca26911a55ef78b3ba2829
CMake_Fortran_utils_150308
_v2.1.4 | strict 1-way | |||||
| MPP | components/clm/src/external_models/mpp | submodule | https://github.com/MPP-LSM/MPP branch alm/develop | strict 1-way | Gautam Bisht | |
| FATES | components/clm/src/external_models/fates | submodule | strict 1-way | |||
| SBETr | components/clm/src/external_models/sbetr | submodule | git@github.com:BeTR-biogeochemistry-modeling/sbetr.git | strict 1-way | Gautam Bisht | |
| CVMix | components/mpas-source/src/core_ocean/cvmix | rescursive submodule | https://github.com/CVMix/CVMix-src | strict 1-way | Jon Wolfe | used to be not in-source and fetched at build time. |
| ocean BGC | components/mpas-source/src/core_ocean/BGC | recursive submodule | https://github.com/E3SM-Project/Ocean-BGC | strict 1-way | Jon Wolfe | used to be not in-source and fetched at build time. |
Externals planned for E3SM
...
| Name | Subdirectory in ACME | external method | upstream master | development method | POC | Notes |
|---|---|---|---|---|---|---|
| CICE column physics | TBD | TBD | ? | ? | ? | |
| GCAM | TBD | submodule? | https://github.com/ |
| JGCRI/ |
491d28dda46fb5dae93bcd50f9b6511e24a46465
| gcam-core | strict 1-way | Katherine Calvin (Unlicensed) |
Specific external development instructions
CIME:
Code location: E3SM/cimeAny code development in E3SM/cime/config/acme can be committed directly to E3SM through the normal PR process.
Code development in other parts of CIME should be committed first with a PR in https://github.com/ESMCI/cime. It will be then brought to E3SM by a subtree merge as in Maintaining the CIME subtree in ACME.
Report issues about the Case Control System, driver or coupler directly to https://github.com/ESMCI/cime. config file issues can be reported to E3SM gitub.