L12 Design Document for Land Model Benchmarking

The Design Document page provides a description of the algorithms, implementation and planned testing including unit, verification, validation and performance testing. Please read  Step 1.3 Performance Expectations that explains feature documentation requirements from the performance group point of view. 

Design Document

 Click here for instructions to fill up the table below ......

The first table in Design Document gives overview of this document, from this info the Design Documents Overview page is automatically created.

In the overview table below 4.Equ means Equations and Algorithms, 5.Ver means Verification, 6.Perf - Performance, 7. Val - Validation

  • Equations: Document the equations that are being solved and describe algorithms
  • Verification Plans: Define tests that will be run to show that implementation is correct and robust. Involve unit tests to cover range of inputs as well as benchmarks.
  • Performance expectations: Explain the expected performance impact from this development
  • Validation Plans: Document what process-based, stand-alone component, and coupled model runs will be performed, and with what metrics will be used to assess validity

Use the symbols below (copy and paste) to indicate if the section is in progress or done or not started.

In the table below 4.Equ means Equations and Algorithms, 5.Ver means Verification, 6.Perf - Performance, 7. Val - Validation,   (tick) - competed, (warning) - in progress, (error) - not done

 

Overview table for the owner and an approver of this feature

1.Description

Create a land model benchmarking capability for ACME
2.OwnerForrest M. Hoffman (Unlicensed)
3.Created 
4.Equ(tick) 
5.Ver(tick) 
6.Perf(tick) 
7.Val(tick) 
8.ApproverWilliam Riley (Unlicensed), Peter Thornton
9.Approved Date 
V1.0Accepted
 Click here for Table of Contents ...

Table of Contents

 

Title: Create a Land Model Benchmarking Capability for ACME

Requirements and Design

ACME Land  Group

Date:  

Summary

A land model benchmarking package will be implemented, leveraging the ongoing ILAMB development in the RGCM-funded BGC Feedbacks SFA, for routine and systematic assessment of land model fidelity through comparison with observational data. Initially, a recent version of the ILAMB prototype will be installed and tested using ACME Land Model (ALM) output. New model–data comparisons will be added by the team of ACME benchmark developers for evaluating the impact of new feature implementation on the overall scientific performance of ALM in coupled and uncoupled configurations. The Workflow Team will be encouraged to adopt this evolving benchmarking package for operational use for evaluating relevant contemporary land and Earth system model simulations. A next-generation version of ILAMB, currently under development and offering parallel processing capabilities, will be used after v1 model activities are complete.

Requirements

Requirement: Install and test the ILAMB prototype code and processed observational data on rhea.ccs.ornl.gov

Date last modified:  
Contributors: Forrest M. Hoffman (Unlicensed)

A recent version of the ILAMB prototype code (nominally version 1.2.1) and the associated processed observational data will be installed on rhea.ccs.ornl.gov. This package will be tested using sample output from CLM4.0CN, CLM4.5BGC, and a corresponding offline ALM simulation.

Algorithmic Formulations

Design solution: Scoring metrics will be developed as needed for model–data comparison

Date last modified:  
Contributors: Forrest M. Hoffman (Unlicensed)

 

All new observational data will be accompanied by metrics for comparison with model output. These metrics will be clearly documented in the evolving Description of Scoring Metrics and Data Processing.

Design and Implementation

Implementation: Install and test the ILAMB prototype code and processed observational data on rhea.ccs.ornl.gov

Date last modified:  
Contributors: Forrest M. Hoffman (Unlicensed)

 

ILAMB code will be copied to the target system along with the observational data package and example model output. The ILAMB code will be tested and ported as necessary for correct operation on the target system.

Planned Verification and Unit Testing 

Verification and Unit Testing: Install and test the ILAMB prototype code and processed observational data on rhea.ccs.ornl.gov

Date last modified:   
Contributors: Forrest M. Hoffman (Unlicensed)

 

Resulting diagnostics and scores will be compared with the same model and code set at http://redwood.ess.uci.edu/mingquan/www/ILAMB/AllDiagnostics/.

Planned Validation Testing 

Validation Testing: Install and test the ILAMB prototype code and processed observational data on rhea.ccs.ornl.gov

Date last modified:  
Contributors: Forrest M. Hoffman (Unlicensed)

 

Model output from ALM will be run through the system and compared with that from CLM4.0CN and CLM4.5BGC. If all of the diagnostics and scores are reasonable for all three model outputs, the package will be considered to have passed validation.

Planned Performance Testing 

Performance Testing: Install and test the ILAMB prototype code and processed observational data on rhea.ccs.ornl.gov

Date last modified:  
Contributors: Forrest M. Hoffman (Unlicensed)

 

No performance testing is planned for the ILAMB prototype. Considerable performance testing is being performed for the next-generation package.