W17_MPAS-Ocean 3D varying GM bolus kappa Design Document

Owned by Luke Van Roekel (Unlicensed)
Last updated: Aug 20, 2019 by Mark Petersen
4 min read

The Design Document page provides a description of the algorithms, implementation and planned testing including unit, verification, validation and performance testing. Please read  Step 1.3 Performance Expectations that explains feature documentation requirements from the performance group point of view. 

Design Document

 Click here for instructions to fill up the table below ......

The first table in Design Document gives overview of this document, from this info the Design Documents Overview page is automatically created.

In the overview table below 4.Equ means Equations and Algorithms, 5.Ver means Verification, 6.Perf - Performance, 7. Val - Validation

  • Equations: Document the equations that are being solved and describe algorithms
  • Verification Plans: Define tests that will be run to show that implementation is correct and robust. Involve unit tests to cover range of inputs as well as benchmarks.
  • Performance expectations: Explain the expected performance impact from this development
  • Validation Plans: Document what process-based, stand-alone component, and coupled model runs will be performed, and with what metrics will be used to assess validity

Use the symbols below (copy and paste) to indicate if the section is in progress or done or not started.

In the table below 4.Equ means Equations and Algorithms, 5.Ver means Verification, 6.Perf - Performance, 7. Val - Validation,   (tick) - competed, (warning) - in progress, (error) - not done

Overview table for the owner and an approver of this feature

1.Description

3D varying GM bolus kappa and phase speed
2.OwnerLuke Van Roekel (Unlicensed)
3.Created 
4.Equ (tick) 
5.Ver(tick) 
6.Perf(tick)
7.Val(tick)
8.ApproverLuke Van Roekel (Unlicensed), Mark Petersen
9.Approved Date19 Aug 2019
 Click here for Table of Contents ...

Table of Contents



Title: 3D varying GM bolus Kappa and phase speed

Requirements and Design

E3SM Water Cycle Group

Date:   

Summary

The Gent McWilliams (GM) parameterization of Gent and McWilliams (1995) is used to represent unresolved mesoscale eddies.  In MPAS, the bolus velocity is calculated following Ferrari et al. (2010), which cast the problem as a simple ODE, i.e.

In equation 16a, c is the phase speed of the first baroclinic mode, N2 is the Brunt Vaisala Frequency, \gamma is the overturning stream function, and kappa is the bolus diffusivity.  Presently MPAS uses constant values for both kappa and c, as suggested in the highlighted text.  This design document describes the relaxation of this constant in time and space assumption.


Requirements

Requirement: Improvements should fit with the variable resolution GM capability

Date last modified:  
Contributors: Luke Van Roekel (Unlicensed), Mark Petersen

The changes to equation 16a should fit with the current ramping of GM in variable resolution simulations

Requirement: Phase speed should be a function of space (2-D only) and time

Date last modified:  
Contributors: Luke Van Roekel (Unlicensed), Mark Petersen

Later portions of the Ferrari et al paper describe c as the first baroclinic mode phase speed, which is defined as NH/pi, where N is integrated over the column depth H.  This suggests c should vary in the horizontal and in time.

Requirement: Kappa should vary in 3D space and time

Date last modified:  
Contributors: Luke Van Roekel (Unlicensed), Mark Petersen

Conversations with Griffies and Adcroft suggest there is no reason to fix kappa in equation 16a above.  Danabasoglu and Marshall (2007) suggest kappa should vary like

Algorithmic Formulations

Design solution: new computations should not interfere with GM bolus ramp for variable resolution

Date last modified:   
Contributors: Luke Van Roekel (Unlicensed)

The ramped GM value is stored in gmBolusKappa, which is initialized first.  Our variable kappa computations use this array, preserving the ramped value.

Design solution: Add computation of c to the GM bolus routine

Date last modified:   
Contributors: Luke Van Roekel (Unlicensed)

The code has been modified to include a computation of depth averaged stratification to get c, e.g.

cGMphaseSpeed(:) = config_gravWaveSpeed_trunc
if (config_gm_lat_variable_c2) then
nEdges = nEdgesArray( 3 )
!$omp do schedule(runtime) private(cell1, cell2, k, ltSum, BruntVaisalaFreqTopEdge, sumN2, countN2, bottomAv)
do iEdge = 1, nEdges
cell1 = cellsOnEdge(1,iEdge)
cell2 = cellsOnEdge(2,iEdge)
sumN2 = 0.0
countN2 = 0
ltSum = 0.0

do k=2,maxLevelEdgeTop(iEdge)

BruntVaisalaFreqTopEdge = 0.5_RKIND * (BruntVaisalaFreqTop(k,cell1) + BruntVaisalaFreqTop(k,cell2))
BruntVaisalaFreqTopEdge = max(BruntVaisalaFreqTopEdge, 0.0_RKIND)

sumN2 = sumN2 + BruntVaisalaFreqTopEdge*layerThicknessEdge(k,iEdge)
ltSum = ltSum + layerThicknessEdge(k,iEdge)
countN2 = countN2 +1

enddo

if(countN2 > 0) cGMphaseSpeed(iEdge) = max(config_gm_min_phase_speed ,sqrt(sumN2/ltSum)*ltSum / 3.141592_RKIND)

enddo
!$omp end do

else
cGMphaseSpeed(:) = config_gravWaveSpeed_trunc
endif


Design solution: Allow bolus Kappa to vary in time and space via N2

Date last modified:   
Contributors: Luke Van Roekel (Unlicensed)

We have implemented the calculation of Danabasoglu and Marshall 2007, i.e.

!$omp do schedule(runtime) private(cell1, cell2, k, BruntVaisalaFreqTopEdge, maxN)
do iEdge = 1,nEdges
cell1 = cellsOnEdge(1,iEdge)
cell2 = cellsOnEdge(2,iEdge)

maxN = -1.0_RKIND
do k=2,maxLevelEdgeTop(iEdge)
BruntVaisalaFreqTopEdge = 0.5_RKIND * (BruntVaisalaFreqTop(k,cell1) + BruntVaisalaFreqTop(k,cell2))
BruntVaisalaFreqTopEdge = max(BruntVaisalaFreqTopEdge, 0.0_RKIND)

maxN = max(maxN,BruntVaisalaFreqTopEdge)

enddo

do k=2,maxLevelEdgeTop(iEdge)
BruntVaisalaFreqTopEdge = 0.5_RKIND * (BruntVaisalaFreqTop(k,cell1) + BruntVaisalaFreqTop(k,cell2))
BruntVaisalaFreqTopEdge = max(BruntVaisalaFreqTopEdge, 0.0_RKIND)

kappaGM3D(k,iEdge) = gmBolusKappa(iEdge)*max(config_gm_min_stratification_ratio, &
BruntVaisalaFreqTopEdge / (maxN + 1.0E-10_RKIND))
if(kappaGM3D(k,iEdge) <= 0) then
print *, 'val = ',kappaGM3D(k,iEdge),k,iEdge
stop
endif
enddo
enddo
!$omp end do


Design and Implementation

Implementation: Modifications to GM calculation

Date last modified:  
Contributors: Luke Van Roekel (Unlicensed)

We have added the following namelist options

  • config_gm_kappa_lat_depth_variable: Enables full space time varying bolus kappa. 
  • config_gm_kappa_lat_variable: as in first option, but integrates kappa value in vertical
  • config_gm_lat_variable_c2: Use the space time varying phase speed.

The following arrays are also added for analysis and debugging

  • kappaGM3D, cGMphaseSpeed, kappaGM2D

Planned Verification and Unit Testing 

Verification and Unit Testing: comparison against offline single column test

Date last modified:   

Contributors: Luke Van Roekel (Unlicensed), Mark Petersen


The new implementation has been tested against and analytic notebook based on equation 16a.  The verification passed.

Planned Validation Testing 

Validation Testing: G and B-cases

Date last modified:  


Contributors: Luke Van Roekel (Unlicensed)

a 50 year B-case and 62 year CORE2 case have been run confirming that the bolus kappa implementation is behaving as expected.  The changes also mitigate the lab sea bias seen in v1 LR.  Analysis is here

https://compy-dtn.pnl.gov/vanr302/20190723.testLuke2.A_WCYCL1850S.ne30_oECv3.compy_20_25/

Planned Performance Testing 

Performance Testing: short-desciption-of-testing-here

Date last modified:  
Contributors: Mark Petersen


Review the number of loops added, verify correct index order. Compare before/after in MPAS-Ocean stand alone with new feature on.

This was done with these timers.

Before:
3 gm bolus velocity 0.03464

After:
3 gm bolus velocity 0.04092

2 time integration 1.86810

Total time increase for this test, which is 6 timesteps in the low-resolution nightly regression suite, is 0.0044/1.868 = 0.0023, or 2.3% of the model integration time for the ocean.