Versions Compared

Old Version 29

changes.mady.by.user James Foucar

Saved on

compared with

New Version Current

changes.mady.by.user James Foucar

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • Squares = callstacks
  • Ovals = input/output files
  • Green = python
  • Blue = perl
  • Black = raw text
  • Purple = CMakePink =
  • XMLArrow out = Produces
  • Arrow in = Consumes/uses

Explaining a bit further, the build of an E3SM case is primarily influenced by the Setup (case.setup, represented by the left side of the diagram) and Build (case.build, represented by the right side of the diagram) case phases. Case.setup will copy cmake_macros from your repository to your case. These macros include a generic Macros.cmake file which uses case settings to include the various compiler and machine macros needed for the case. Macros.cmake will be copied directly into your case and the rest of the contents of cmake_macros will be recursively copied into $case/cmake_macros. These macro files contain things like which compilers to use, which compiler flags to use, and which link flags to add. The next key task for setup is building namelists for each active component. As best as I can tell, our builds are not influenced by namelist files, but the buildnml scripts are responsible for calling the build configure scripts ($component/bld/configure) which are still in Perl for all our core components. The build configure scripts are responsible for producing two key input files for the build phase, "Filepath" and "CCSM_cppdefs", which will be placed in $case/Buildconf/${component}conf (e.g. $case/Buildconf/camconf/Filepath). Filepath is list of directories that will be globbed for source files. An interesting feature of our build system is that no repeated file basenames are allowed. For example, if Filepath contains directories A and B and A/foo.f90 and B/foo.f90 both exist, then only A/foo.f90 will be compiled by our build system (assuming A came first in the Filepath list). I believe the original intent of this approach was to allow for custom per-case srcmods. CCSM_cppdefs is a list of c preprocessor directives that will be applied to every compilation of every source file in the component. Both Filepath and CCSM_cppdefs are tweakable by users (users can modify these files in-place and re-run case.build).

...

  • E3SM/cime_config/machines/config_machines.xml: Contains list of valid compilers (COMPILERS) and environment setup (modules, environment_variables).
  • E3SM/cime_config/machines/config_compilers.xml: DEPRECATED. Contains specifications (flags, link flags, exe names) for compilers (gcc, pgi) as well as machine-specific modifications for those compilers. This system can be reactivated by setting CIME_NO_CMAKE_MACRO=1 in your environment.
  • E3SM/cime_config/machines/cmake_macros/*: Contains specifications (flags, link flags, exe names) for compilers (gcc, pgi) as well as machine-specific modifications for those compilers. Replaces config_compilers.xml.
  • E3SM/cime_config/machines/Depends.$compiler.$machine.cmake: Contains machine/compiler-specific flag changes for specific E3SM component files
  • E3SM/cime_config/scripts/Toolscustomize/e3sm_compile_wrap.py: The default compile/link wrapper. It is used to produce timings of all operations
  • E3SM/cime_config/scripts/machines/scripts/*: Python scripts to assist in managing cmake macro files.
  • E3SM/components/CMakeLists.txt: The root CMakeLists.txt file for E3SM, mostly just iterates over components and calls functions
  • E3SM/components/cmake/cmake_util.cmake: Contains some generally useful functions for cmake
  • E3SM/components/cmake/build_mpas_model.cmake: Implements the handoff between E3SM's CMake system to MPAS's Cmake system

  • E3SM/components/cmake/common_setup.cmake: Cmake setup common to all core components, this is only invoked ones (AFTER mpas is configured)

  • E3SM/components/cmake/build_model.cmake: Implements how an individual core component gets configures (involves reading the Filepath and CCSM_cppdefs, globbing files, etc)
  • E3SM/components/cmake/modules/*.cmake: Cmake modules for finding packages we depend on
  • E3SM/components/cmake/*/CMakeLists.txt: Very thin CMakeLists.txt files for each component type, this is just to ensure each component get it's own build area
  • E3SM/components/*/bld/configure: Perl configure script for component, should produce Filepath and CCSM_cppdefs files
  • E3SM/components/*/cime_config/buildnml: The buildnml script for component, controls what flags get passed to $component/bld/configure
  • E3SM/components/*/cime_config/buildlib: The legacy buildlib driver for the classic deprecated Makefile-based build system
  • E3SM/components/*/cime_config/buildlib_cmake: A insertion point for last-minute, component-specific changes before cmake is called. For example, mosart uses this file, not bld/configure, to generate Filepath and CCSM_cppdefs
  • E3SM/components/mpas-source/src/CMakeLists.txt: The root CMakeLists.txt for MPAS
  • E3SM/components/mpas-source/src/cmake_utils.cmake: Contains some generally useful functions for configuring MPAS
  • E3SM/components/mpas-source/src/build_core.cmake: A generic implementation for building MPAS cores
  • E3SM/components/mpas-source/src/core_landice/landice.cmake: The CMake definition of the landice core
  • E3SM/components/mpas-source/src/core_ocean/ocean.cmake: The CMake definition of the ocean core
  • E3SM/components/mpas-source/src/core_seaice/seaice.cmake: The CMake definition of the seaice core
  • E3SM/share/buildlib: Python build wrapper scripts for our SHAREDLIB packages. CIME invokes these during the SHAREDLIB build phase.
  • $case/Buildconf/*conf: The various build conf dirs for all active components, the Filepath and CCSM_cppdefs files for each component live here
  • $case/Macros.cmake: The file that includes the relevant files in cmake_macros. You can modify this locally in your case.
  • $case/cmake_macros/*: The files that ultimately sets flags, link flags, etc. You can modify this locally in your case.
  • $case/Macros.make: A makefile macro derived from the cmake macros, needed for the sharedlibs that still use Makefile build systems. This is an auto-generated file that you should never edit.
  • $case/SourceMods/src.$component/*: Alternative source files to use instead of the ones from the component in your source dir
  • $EXEROOT/cmake-bld: The root cmake build/binary directory 
  • $case/Tools/e3sm_compile_wrap.py: The case-specific compiler wrapper, this will start out as a copy of the e3sm_compile_wrap.py from the repo. Users can customize this one.

...

As of recent (2020) PRs, the E3SM build system will always wrap compilations and links with the e3sm_compile_wrap.py script. The default version of this script lives in E3SM/cime_config/scripts/Toolscustomize/e3sm_compile_wrap.py and will be copied to all cases in $case/Tools. The one in the case is what actually gets used by the build system and is modifiable by the user at any time. The default behavior of this script is to time add timing info for all links and compilations to the e3sm.bldlog. A handy command for listing actions from cheapest to most expensive: `grep "built in" e3sm.bldlog.* | sort -n -k5,5` . This makes it very easy to find expensive actions.

...

The SourceMods mechanism is a way of bundling code changes within a case. We only recommend using SourceMods for small, short-term changes that just apply to one or two cases. For larger or longer-term changes, including gradual, incremental changes towards a final solution, we highly recommend making changes in the main source tree, leveraging version control (git). The SourceMods directory lives under the case and has subdirectories for each active component, example $case/SourceMods/src.eam. For any source file $name.$suffix in the source mod dir for a component, that file will take "precedence" over files of the same name in repo for that component. Our build system will automatically detect SourceMods changes and will know it needs to reconfigure (recreate cache files) when changes to SourceMods are made.

CSM_Share

This code lives in E3SM/share and is built during the sharedlibs build. It has a cmake build system very similar to one of our core components. Depends files for csm_share live in E3SM/share and need to be kept separate from the "main" Depends files.

Using custom TPLs/sharedlibs

Developers can force the build system to use custom installations of E3SM dependencies by making changes to ${PACKAGE_NAME}_ROOT in their shell environment. As an example, if you set Trilinos_ROOT, your build will build against that Trilinos.