The Design Document page provides a description of the algorithms, implementation and planned testing including unit, verification, validation and performance testing. Please read Step 1.3 Performance Expectations that explains feature documentation requirements from the performance group point of view.
Design Document
Expand | ||
---|---|---|
| ||
The first table in Design Document gives overview of this document, from this info the Design Documents Overview page is automatically created. In the overview table below 4.Equ means Equations and Algorithms, 5.Ver means Verification, 6.Perf - Performance, 7. Val - Validation
Use the symbols below (copy and paste) to indicate if the section is in progress or done or not started. |
...
Page Properties | ||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||||||||
|
Expand | |||
---|---|---|---|
| |||
|
Title: NH Dycore with SL Transport
Requirements and Design
E3SM Water Cycle Group
Date:
Summary
Contributors: Peter Bosler, Andrew Bradley, Oksana Guba , Mark Taylor, Andrew Steyer (Unlicensed), Paul Ullrich
Requirements
- Solve the nonhydrostatic equations with a HEVI approach
- Support a mass coordinate hydrostatic option to ease integration into E3SM
- Improve accuracy over E3SM v1 dycore ( E3SM v1 dycore = "preqx" + Eulerian transport)
- Improve performance over E3SMv1 dycore
Algorithmic Formulations
The formulation of the nonhydrostatic equations is documented in a journal paper to be submitted to JAMES: https://www.overleaf.com/read/bqycjpqjthnc. Details include:
- Nonhydrostatic equations in spherical geometry with a Lorenz staggered vertical discretization.
- Conserves a discrete energy when coupled with HOMME's existing mimetic spectral element horizontal discretization.
- Primitive variable formulation
- Supports terrain following mass or height coordinates and for both Eulerian or vertically Lagrangian discretizations.
- Energy conservation is obtained via a term-by-term balance in the kinetic, internal and potential energy budgets, ensuring an energy-consistent discretization with no spurious sources of energy.
The SL transport algorithm is documented in two recently accepted SISC papers:
- The property preservation methods are described and analyzed in this article (open access) and are implemented and unit tested in the CEDR subpackage of COMPOSE.
- For the advection step, a cell-integrated incremental remap SL method is described in a paper currently in the production stage; that will be linked when it's available. The geometry calculations are implemented and unit tested in the SIQK subpackage of COMPOSE.
Further details are as follows:
- SL methods minimize the number of communication rounds and volume for tracer transport per unit of simulated time, relative to other methods.
- Tracer transport must be shape preserving, mass conserving, and mass tracer consistent. CEDR provides these properties.
- For the advection step, we have opted to use an interpolation method rather than a cell-integrated method. This will be described in a paper we are currently drafting. Since a reference is not available yet, we will describe the key ideas here:
- CEDR handles all aspects of property preservation efficiently. It is needed for CISL as well as ISL.
- Interpolation SL (ISL) methods are the most efficient SL methods for four reasons:
- The remap formula is computationally parsimonious.
- An optimal implementation of communication is parsimonious. ISL requires ~4x data movement than CISL.
- It is easy to go to high order because edges do not have to be represented at high order, since there are no explicit edges.
- ISL accuracy can easily be boosted with p-refinement.
- Thus, ISL is a better choice than CISL: it is much faster, and it can also provide extremely high accuracy if desired.
- The key challenge is to find interpolants yielding a stabilized ISL on an SE mesh. We have done this, and these will be described in the paper. Roughly, interpolation formulas supported by a single spectral element are searched for the highest-order one that is unconditionally stable on the periodic transport problem.
Design and Implementation
Planned Verification and Unit Testing
Unit tests will be developed to test Plans for verification and unit testing was developed and approved under DOE Scidac transport and nonhydrostatic projects. They include unit tests for all new aspects of the algorithms and formulation. Verification will be done , verification using DCMIP test cases Functional tests will be added and the addition of functional tests to the E3SM test suite. The results are described W10 NH Dycore with SL Transport Verification Phase 1
Planned Validation Testing
Validation will be performed with E3SM v1 F compsets, using the standard E3SM diagnostics package. We run a 1 degree FC compset, FC5AV1C-L, simulation for 6 years and create 5-year (year 2 to 6) climatologies from it. We plot climatologies vs observations and vs default configuration. The default configuration consists of default hydrostatic dycore with default settings. Preliminary runs are described in /wiki/spaces/COM/pages/941096965 . There we produced runs with SL transport and theta-l dycore (hydrostatic only). The results are described W10 NH Dycore with SL transport Validation Phase 1
Planned Performance Testing
As this work is limited to the HOMME dynamical core, we will use the E3SM Performance Group's /wiki/spaces/EPG/pages/905969931. We expect the NH dycore with SL transport will be significantly faster at all processor counts and on all platforms. The results are described W10 NH Dycore with SL transport Performance Phase 1