The Design Document page provides a description of the algorithms, implementation and planned testing including unit, verification, validation and performance testing. Please read  Step 1.3 Performance Expectations that explains feature documentation requirements from the performance group point of view. 

Design Document

In the table below 4.Equ means Equations and Algorithms, 5.Ver means Verification, 6.Perf - Performance, 7. Val - Validation,    - competed,  - in progress,  - not done

Overview table for the owner and an approver of this feature

1.Description	Version 2 of CLUBB
2.Owner	Vince Larson
3.Created	27 June 2019
4.Equ	- competed
5.Ver	- competed
6.Perf	- competed
7.Val	- competed
8.Approver
9.Approved Date

Title: Version 2 of CLUBB

Requirements and Design

Atmosphere Group

Date: 20190627 

Summary

The v1 version of CLUBB left pockets of supersaturation for the microphysics.   To mitigate this, the v2 version of CLUBB does CLUBB's saturation adjustment immediately before microphysics when ipdf_call_placement = ipdf_pre_advance_fields.

CLUBB_v2 includes performance optimizations that may lead to speedups, but also may be vitiated by a performance degradation in the CLUBB interface that was introduced in order to change the position of the saturation adjustment.

CLUBB_v2 also adds new options that may be interest when tuning E3SMv2:

 * CLUBB's gustiness can be enhanced if C_wp2_splat is set to non-zero values.  This may enhance the benefits found with Po-Lun Ma connection of CLUBB's gustiness and surface processes.

 * CLUBB_v2 can now prognose momentum fluxes if l_predict_upwp_vpwp = .true.  This is designed to reduce ocean surface stress in cumulus regions.

The purpose of this section is to summarize what capability is to be added to the E3SM  through this design process. It should be clear what new code will do that the current code does not. Summarizing the primary challenges with respect to software design and implementation is also appropriate for this section. Finally, this statement should contain general statement with regard to what is “success.”

Requirements

Requirement 1: Move saturation adjustment immediately before microphysics

Requirement 2: Add option to enhance gustiness

Requirement 3: Add option to prognose momentum fluxes

Date last modified: // 27 Jun 2019 
Contributors: Vince Larson, Brian Griffin (Unlicensed)

Algorithmic Formulations

Design solution:

Date last modified:// 27 Jun 2019
Contributors: Vince Larson, Brian Griffin (Unlicensed)

The prognostic equation for momentum flux is shown in Eqn. (4.7) of CLUBB's tech doc, (Larson 2017).  The saturation adjustment and enhanced gustiness are described in code comments.

Design and Implementation

Implementation:

Date last modified: // 27 Jun 2019
Contributors: Vince Larson, Brian Griffin (Unlicensed), Balwinder Singh

The new CLUBB code is available at CLUBB's release page, https://github.com/larson-group/clubb_release/, accessible after registration at https://carson.math.uwm.edu/larson-group/clubb_site/signup/.

CLUBB source code is free to be pulled into E3SMv2 and distributed as it was in E3SMv1.

Planned Verification and Unit Testing 

Verification and Unit Testing: Unit tests for CLUBB's PDF closure and conservation of scalars are available.

Date last modified:  27 Jun 2019
Contributors: Vince Larson, Brian Griffin (Unlicensed), Andy Salinger

Perhaps these tests could be included in E3SM's standard unit test suite.

Planned Validation Testing 

Validation Testing: Compset FC5AV1C-L was used with ne3_ne30 grid to validate CLUBB version 2 model

Date last modified: 27 Jun 2019
Contributors: Vince Larson, Balwinder Singh

Balwinder show that when CLUBB_v2 flags are reverted as in CLUBB_v1, then the global solutions are similar (within 1 W/m2 of SWCF).  However, the new options available in CLUBB_v2 offer opportunities for improvements to global simulations during the tuning process.

Two six year simulations (control and test) were run on ne30_ne30 grid using compset FC5AV1C-L on Anvil using Intel compiler. Years 2 to 5 are analyzed using E3SM diags package:

Test simulation: SMS.ne30_ne30.FC5AV1C-L.anvil_intel.clbv2_rbsd_1_ed716943a4c

Control Siulation: SMS.ne30_ne30.FC5AV1C-L.anvil_intel.def_mstr_b5aaa6b7d228_clbv2_rbsd_1 

E3SM diagnostics are available at the following web pages:

Test Vs. Control

Test Vs. OBS

Control Vs. OBS

The history files are at (HPSS NERSC): A:/home/b/bsingh/SMS.ne30_ne30.FC5AV1C-L.anvil_intel.clbv2_rbsd_1_ed716943a4c.tar and  A:/home/b/bsingh/SMS.ne30_ne30.FC5AV1C-L.anvil_intel.def_mstr_b5aaa6b7d228_clbv2_rbsd_1.tar

These files are archived as "tar" using HTAR.

Climo files: Climo files are inside the tar archive at: <test case name>/run/climo_fv/2-6yr/. For example: "SMS.ne30_ne30.FC5AV1C-L.anvil_intel.clbv2_rbsd_1_ed716943a4c/run/climo_fv/2-6yr/"

Planned Performance Testing 

Performance Testing: Tests of E3SM on Anvil

Date last modified: 30 July 2019
Contributors: Balwinder Singh

To measure performance of CLUBBV2, two 5 days simulations (grid:ne30_ne30, Compset: FC5AV1C-L) were run on Anvil using Intel compiler. Following are the timings:

Advance CLUBB core is about 14% faster in CLUBBV2. These results are from a single run so we might need to evaluate the performance with more rigor.