The Design Document page provides a description of the algorithms, implementation and planned testing including unit, verification, validation and performance testing. Please read Step 1.3 Performance Expectations that explains feature documentation requirements from the performance group point of view.
Design Document
| Expand | ||
|---|---|---|
| ||
The first table in Design Document gives overview of this document, from this info the Design Documents Overview page is automatically created. In the overview table below 4.Equ means Equations and Algorithms, 5.Ver means Verification, 6.Perf - Performance, 7. Val - Validation
Use the symbols below (copy and paste) to indicate if the section is in progress or done or not started. |
...
| Page Properties | ||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||||||||
|
...
| Expand | |||
|---|---|---|---|
| |||
|
Title:
...
Physics Grid (Physgrid) With High-Order, Property-Preserving Remap
Requirements and Design
...
Date:
...
July-August 2019
Summary
The
purpose of this section is to summarize what capability is to be added to the ACME Ocean and Ice system through this design process. It should be clear what new code will do that the current code does not. Summarizing the primary challenges with respect to software design and implementation is also appropriate for this section. Finally, this statement should contain general statement with regard to what is “success.”Requirements
Requirement: name-of-requirement-here
Date last modified: // date
Contributors: @ name (add your name to this list if it does not appear)
Each requirement is to be listed under a ”section” heading, as there will be a one-to-one correspondence between requirements, design, proposed imple- mentation and testing. Requirements should not discuss technical software issues, but rather focus on model capability. To the extent possible, require- ments should be relatively independent of each other, thus allowing a clean design solution, implementation and testing plan.Algorithmic Formulations
Design solution: short-description-of-proposed-solution-here
Date last modified:// date
Contributors: (add your name to this list if it does not appear)
For each requirement, there is a design solution that is intended to meet that requirement. Design solutions can include detailed technical discussions of PDEs, algorithms, solvers and similar, as well as technical discussion of performance issues. In general, this section should steer away from a detailed discussion of low-level software issues such as variable declarations, interfaces and sequencing.
Design and Implementation
Implementation: short-desciption-of-implementation-here
Date last modified: // date
Contributors: (add your name to this list if it does not appear)
This section should detail the plan for implementing the design solution for requirement XXX. In general, this section is software-centric with a focus on software implementation. Pseudo code is appropriate in this section. Links to actual source code are appropriate. Project management items, such as svn branches, timelines and staffing are also appropriate. How do we typeset pseudo code?Planned Verification and Unit Testing
Verification and Unit Testing: short-desciption-of-testing-here
Date last modified:
Contributors: (add your name to this list if it does not appear)
How will XXX be tested? i.e. how will be we know when we have met requirement XXX. Will these unit tests be included in the ongoing going forward?
Planned Validation Testing
Validation Testing: short-desciption-of-testing-here
Date last modified:
Contributors: (add your name to this list if it does not appear)
How will XXX be tested? What observational or other dataset will be used? i.e. how will be we know when we have met requirement XXX. Will these unit tests be included in the ongoing going forward?
Planned Performance Testing
Performance Testing: short-desciption-of-testing-here
Date last modified:
Contributors: (add your name to this list if it does not appear)
How will XXX be tested? i.e. how will be we know when we have met requirement XXX. Will these unit tests be included in the ongoing going forward?high-level problem is to run the physics parameterizations on the same element grid as dynamics but a different nodal grid. For brevity, this capability is called physgrid. This feature requires modifying infrastructure to support two different grids, and write software to implement high-order, property-preserving remap algorithms.
There are two primary motivations for physgrid. First, on a quasi-uniform cubed sphere mesh, physics columns are given roughly uniform areas. Second, for nphys < 3, we can expect speedup of the physics parameterizations top-level calls of up to 9/nphys^2.
A narrative of development with extended results is available here: /wiki/spaces/COM/pages/1010171942
Requirements
Requirements for infrastructure include the following:
1. Generate domain and topography files for physgrid and put these on the input server.
2. Update configuration XML files to be able to express physgrid parameters and list default files.
Requirements for remap include the following:
1. In each element (and thus also globally), remap should conserve mass prior to the DSS. After the DSS, mass should be conserved globally.
2. In each element, remap of a tracer should not introduce new extrema prior to the DSS. After the DSS, there should be no new global extrema.
3. In each element, remap of a tracer should be tracer consistent. The DSS maintains tracer consistency.
4. Remap of horizontal velocity should avoid problems at the poles.
5. Algorithms should work seamlessly on quasi-uniform and RRM SE quad meshes. In particular, we strongly prefer to rely on element-neighbor data only through HOMME's standard halo exchange patterns.
6. For nphys > 2, order of accuracy (OOA) in the l2 norm should be at least 2 on a smooth test function.
Algorithmic Formulations
The following design document, which we shall prefer to as the "latex design doc," describes algorithms: Physgrid High-Order Remap
Design and Implementation
The code is organized as follows:
- homme/src/share/gllfvremap_mod.F90: Implementation of
algorithms. The gllfvremap_mod module exports the following primary
public API:- gfr_init: Initialize data structures and geometry data. Called in dyn_comp.F90.
- gfr_finish: Deallocate memory. Called in stepon.F90.
- gfr_fv_phys_to_dyn_topo: Record FV topography data and remap it to GLL. Later, FV topography data can be either remapped from GLL to FV, which provides a consistent average area within an FV cell independent of basis, or provided as the raw data read from the topography file. Called in inidat.F90.
- gfr_dyn_to_fv_phys: Remap T, u, v, omega_p, q, p_s, phi_s from GLL to FV bases. Called in dp_coupling.F90.
- gfr_fv_phys_to_dyn: Remap T, u, v, q tendencies or states from FV to GLL bases. Called in dp_coupling.F90.
- To support testing outside of the module, it provides a few additional public functions. Inside the module, the public function gfr_test iterates through a number of unit tests (test type 1) that test element-level area and mass conservation, shape preservation, reversibility of maps when expected, and order of accuracy (OOA); we summarize these quantities as properties and OOA.
- homme/src/share/gllfvremap_test_mod.F90: Tests (type 2) of global properties and OOA using the public API.
- homme/src/test_src/dcmip16_wrapper.F90: DCMIP 2016 moist baroclininc instability test (type 3) using physgrid. Implemented in dcmip2016_test1_pg and dcmip2016_test1_pg_forcing. Tests the public API.
This software is operational and fully testable in standalone HOMME. It is operational and type 1 tests can be called in principle in the E3SM build.
Type 2 tests can be run in a loop over n_e to produce convergence plots. The type-3 test permits us to study instantaneous snapshots of, e.g., q_v in a nontrivial setting but for which only a few compute nodes and approximately 10 minutes of computation are needed.
Planned Verification and Unit Testing
The previous section describes the types of tests. Results of the order of accuracy test are in the latex design doc, Figure 2.
Planned Validation Testing
We are running 6 years of the ne30 FC5AV1C-L compset with the default ne30np4 and new ne30pg2 grid configurations.
Planned Performance Testing
We will record and analyze timers in our F compset runs on Cori-KNL.