W9 CLUBB_v2 Design Doc
The Design Document page provides a description of the algorithms, implementation and planned testing including unit, verification, validation and performance testing. Please read Step 1.3 Performance Expectations that explains feature documentation requirements from the performance group point of view.
Design Document
In the table below 4.Equ means Equations and Algorithms, 5.Ver means Verification, 6.Perf - Performance, 7. Val - Validation, 
- competed, 
- in progress, 
- not done
Overview table for the owner and an approver of this feature
1.Description | Version 2 of CLUBB |
| 2.Owner | Vince Larson |
| 3.Created | 27 June 2019 |
| 4.Equ | - competed |
| 5.Ver | - competed |
| 6.Perf | - competed |
| 7.Val | - competed |
| 8.Approver | |
| 9.Approved Date |
Title: Version 2 of CLUBB
Requirements and Design
Atmosphere Group
Date: 20190627
Summary
The v1 version of CLUBB left pockets of supersaturation for the microphysics. To mitigate this, the v2 version of CLUBB does CLUBB's saturation adjustment immediately before microphysics when ipdf_call_placement = ipdf_pre_advance_fields.
CLUBB_v2 includes performance optimizations that may lead to speedups, but also may be vitiated by a performance degradation in the CLUBB interface that was introduced in order to change the position of the saturation adjustment.
CLUBB_v2 also adds new options that may be interest when tuning E3SMv2:
* CLUBB's gustiness can be enhanced if C_wp2_splat is set to non-zero values. This may enhance the benefits found with Po-Lun Ma connection of CLUBB's gustiness and surface processes.
* CLUBB_v2 can now prognose momentum fluxes if l_predict_upwp_vpwp = .true. This is designed to reduce ocean surface stress in cumulus regions.
The purpose of this section is to summarize what capability is to be added to the E3SM through this design process. It should be clear what new code will do that the current code does not. Summarizing the primary challenges with respect to software design and implementation is also appropriate for this section. Finally, this statement should contain general statement with regard to what is “success.”
Requirements
Requirement 1: Move saturation adjustment immediately before microphysics
Requirement 2: Add option to enhance gustiness
Requirement 3: Add option to prognose momentum fluxes
Date last modified: // 27 Jun 2019
Contributors: Vince Larson, Brian Griffin (Unlicensed)
Algorithmic Formulations
Design solution:
Date last modified:// 27 Jun 2019
Contributors: Vince Larson, Brian Griffin (Unlicensed)
The prognostic equation for momentum flux is shown in Eqn. (4.7) of CLUBB's tech doc, (Larson 2017). The saturation adjustment and enhanced gustiness are described in code comments.
Design and Implementation
Implementation:
Date last modified: // 27 Jun 2019
Contributors: Vince Larson, Brian Griffin (Unlicensed), Balwinder Singh
The new CLUBB code is available at CLUBB's release page, https://github.com/larson-group/clubb_release/, accessible after registration at https://carson.math.uwm.edu/larson-group/clubb_site/signup/.
CLUBB source code is free to be pulled into E3SMv2 and distributed as it was in E3SMv1.
Planned Verification and Unit Testing
Verification and Unit Testing: Unit tests for CLUBB's PDF closure and conservation of scalars are available.
Date last modified: 27 Jun 2019
Contributors: Vince Larson, Brian Griffin (Unlicensed), Andy Salinger
Perhaps these tests could be included in E3SM's standard unit test suite.
Planned Validation Testing
Validation Testing: Compset FC5AV1C-L was used with ne3_ne30 grid to validate CLUBB version 2 model
Date last modified: 27 Jun 2019
Contributors: Vince Larson, Balwinder Singh
Balwinder show that when CLUBB_v2 flags are reverted as in CLUBB_v1, then the global solutions are similar (within 1 W/m2 of SWCF). However, the new options available in CLUBB_v2 offer opportunities for improvements to global simulations during the tuning process.
Two six year simulations (control and test) were run on ne30_ne30 grid using compset FC5AV1C-L on Anvil using Intel compiler. Years 2 to 5 are analyzed using E3SM diags package:
Test simulation: SMS.ne30_ne30.FC5AV1C-L.anvil_intel.clbv2_rbsd_1_ed716943a4c
Control Siulation: SMS.ne30_ne30.FC5AV1C-L.anvil_intel.def_mstr_b5aaa6b7d228_clbv2_rbsd_1
E3SM diagnostics are available at the following web pages:
The history files are at (HPSS NERSC): A:/home/b/bsingh/SMS.ne30_ne30.FC5AV1C-L.anvil_intel.clbv2_rbsd_1_ed716943a4c.tar and A:/home/b/bsingh/SMS.ne30_ne30.FC5AV1C-L.anvil_intel.def_mstr_b5aaa6b7d228_clbv2_rbsd_1.tar
These files are archived as "tar" using HTAR.
Climo files: Climo files are inside the tar archive at: <test case name>/run/climo_fv/2-6yr/. For example: "SMS.ne30_ne30.FC5AV1C-L.anvil_intel.clbv2_rbsd_1_ed716943a4c/run/climo_fv/2-6yr/"
Planned Performance Testing
Performance Testing: Tests of E3SM on Anvil
Date last modified: 30 July 2019
Contributors: Balwinder Singh
To measure performance of CLUBBV2, two 5 days simulations (grid:ne30_ne30, Compset: FC5AV1C-L) were run on Anvil using Intel compiler. Following are the timings:
Control | Test | |
SYPD | 5.86 | 5.89 |
Advance CLUBB core (Wall clock time) | 1.915503e+04 | 1.641459e+04 |
CLUBB interface (Wall clock time) | 2.915716e+04 | 2.658124e+04 |
Advance CLUBB core is about 14% faster in CLUBBV2. These results are from a single run so we might need to evaluate the performance with more rigor.