The Design Document page provides a description of the algorithms, implementation and planned testing including unit, verification, validation and performance testing. Please read Step 1.3 Performance Expectations that explains feature documentation requirements from the performance group point of view.
Design Document
In the table below 4.Equ means Equations and Algorithms, 5.Ver means Verification, 6.Perf - Performance, 7. Val - Validation, 
- competed, 
- in progress, 
- not done
Overview table for the owner and an approver of this feature
1.Description | Uncertainty quantification framework |
| 2.Owner | Khachik Sargsyan, Daniel Ricciuto |
| 3.Created | |
| 4.Equ | |
| 5.Ver | |
| 6.Perf | |
| 7.Val | |
| 8.Approver | Peter Thornton, William Riley (Unlicensed) |
| 9.Approved Date |
Title: Uncertainty Quantification Framework
Requirements and Design
ACME Land Group
Date:
Summary
A set of Python scripts is being included in the ACME Land Model v1 (ALMv1) to perform basic forward uncertainty quantification tasks, such as surrogate model construction and variance-based sensitivity analysis. Besides pre- and post-processing steps, the scripts heavily utilize functionalities of Uncertainty Quantification Toolkit (UQTk) being developed at SNL-CA as a lightweight C++/Python UQ library. Version 3.0 of UQTk will be available publicly in spring - any time before that the library is available upon request (see http://www.sandia.gov/uqtoolkit/). The general workflow consists of three steps (a) pre-processing: input parameter sampling, (b) ensemble simulation of ALM, (c) post-processing: for each output quantity of interest, build a polynomial surrogate and compute Sobol sensitivity indices.
Requirements
Requirement: Generate input parameter ensemble
Date last modified:
Contributors: Khachik Sargsyan, Daniel Ricciuto
One needs a selection of ALM parameters with associated value ranges for each parameter in order to generate ensemble input. This information is provided in the file Parameters_TitanUQ_ACME.csv. Note that some inputs are dependent making the sampling slightly less trivial.
Requirement: Perform ALM ensemble simulations
Date last modified:
Contributors: Daniel Ricciuto
Add text here...
Requirement: Polynomial Chaos surrogate model construction
Date last modified:
Contributors: Khachik Sargsyan
Given the ensemble of simulations corresponding to varied high-dimensional input, build a Polynomial Chaos (PC) – i.e. essentially a polynomial fit – surrogate for the input->output map.
Requirement: Variance-based uncertainty decomposition
Date last modified:
Contributors: Khachik Sargsyan
Given the polynomial fit (PC surrogate), one can extract global sensitivity indices, also known as Sobol sensitivity or variance-based decomposition. This is output variance fractional attribution to the input parameters used for surrogate construction.
Algorithmic Formulations
For each requirement, there is a design solution that is intended to meet that requirement. Design solutions can include detailed technical discussions of PDEs, algorithms, solvers and similar, as well as technical discussion of performance issues. In general, this section should steer away from a detailed discussion of low-level software issues such as variable declarations, interfaces and sequencing.
Design solution: Generate input parameter ensemble
Date last modified:
Contributors: Khachik Sargsyan, Daniel Ricciuto
The MaxEnt principle requires uniform sampling on the feasible input parameter space that respects the input constraints. Rosenblatt transformation (RT) is employed. For linear constraints such as a_1+a_2+a_3=1, one can avoid using RT by simply removing a_3 from the varying parameter list, and enforcing a_3=1-a_1-a_2 in order to prepare the input set for ALM.
Design solution: Perform ALM ensemble simulations
Date last modified:
Contributors: Daniel Ricciuto
Add text here...
Design solution: Polynomial Chaos surrogate model construction
Date last modified:
Contributors: Khachik Sargsyan
For high-dimensional input space, one necessarily operates in sparse-data regime. For example, 10000 simulations of varying 65 model parameters cover a small fraction of the 65-dimensional space only. Thus, it is necessary to a) estimate the surrogate uncertainty due to lack-of-data, and b) enforce adaptive polynomial selection in order to avoid overfitting. Bayesian compressive sensing (BCS) provides both capabilities. Details of the algorithm can be found in the paper Sargsyan_IJUQ.pdf
Design solution: Variance-based uncertainty decomposition
Date last modified:
Contributors: Khachik Sargsyan
The major advantage of PC-based surrogate is the access to Sobol sensitivity information without extra computational effort, via collecting the appropriate polynomial terms together.
Design and Implementation
Implementation: Generate input parameter ensemble
Date last modified:
Contributors: Khachik Sargsyan, Daniel Ricciuto
The samples are generated as a preprocessing step, via Python script prepr.py. The syntax is prepr.py <csv file> <number of samples> [parameters to exclude]. For example,
prepr.py Parameters_TitanUQ_ACME.csv 10000 lf_flig fr_flig generates the following output files:
- pdomain_x.dat - input ranges for the unconstrained parameters
- pnames_clm.dat, pnames_x.dat - parameter names, the full list, and the unconstrained list, respectively.
- input_clm.dat, input_x.dat - input ensembles, the full input for ALM, and the 'computational' unconstrained input in [-1,1] for surrogate construction.
Implementation: Perform ALM ensemble simulations
Date last modified:
Contributors: Daniel Ricciuto
Add text here per how to get from input file 10000x68 to a netCDF...
One can run postp.py 0 <observable> to read the NetCDF file for a selected observable (e.g. GPP, TLAI, TOTSOMC, TOTVEGC, EFLX_LH_TOT) and store it in Python's numpy format for further use.
Furthermore, postp.py 1 <observable> <site_id> generates several diagnostic plots for a selected observable and site.
Implementation: Polynomial Chaos surrogate model construction
Date last modified:
Contributors: Khachik Sargsyan
PC surrogate for specific observable and site can be obtained by running postp.py 2 <observable> <site_id> which generates....
One can use the multi-threading routine uqmulti.py in order to launch such construction in parallel for all observables and sites.
Implementation: Variance-based uncertainty decomposition
Date last modified:
Contributors: Khachik Sargsyan
Sensitivity information for specific observable and site can be obtained by running postp.py 3 <observable> <site_id> which generates....
One can use the multi-threading routine uqmulti.py in order to launch sensitivity computation in parallel for all observables and sites.
Planned Verification and Unit Testing
Verification and Unit Testing: short-desciption-of-testing-here
Date last modified:
Contributors: (add your name to this list if it does not appear)
How will XXX be tested? i.e. how will be we know when we have met requirement XXX. Will these unit tests be included in the ongoing going forward?
Planned Validation Testing
Validation Testing: short-desciption-of-testing-here
Date last modified:
Contributors: (add your name to this list if it does not appear)
How will XXX be tested? What observational or other dataset will be used? i.e. how will be we know when we have met requirement XXX. Will these unit tests be included in the ongoing going forward?
Planned Performance Testing
Performance Testing: short-desciption-of-testing-here
Date last modified:
Contributors: (add your name to this list if it does not appear)
How will XXX be tested? i.e. how will be we know when we have met requirement XXX. Will these unit tests be included in the ongoing going forward?