Document This page documents the E3SM recommended mapping files specifications for running the E3SM model and analysis of output. These should analyzing the output, which should be used for all V2 E3SMv2 configurations. This will also ease the transition to inline mapping in V2 E3SMv2 with the MOAB based coupler which has inline support for Tempest algorithms.
For each map (i.e. FV-FV mono, FV-SE monotr), see the sections below on naming conventions and specific command line options used to produce each map.
| Table of Contents |
|---|
Recommended
...
Settings
E3SMv2 with 2x2 physgrid (i.e. "pg2")
ATM FV grids (with physics on the "pg2" grid, used in V2) coupled to FV ocean and FV land. E3SMv2 will still use the spectral element dynamical core on the GLL grid, but the coupling between component always happens between the physics columns, which are now on a FV grid.
- fluxes: FV→FV mono
- state variables:
- ocn→atm: currently this map must be the same as used for fluxes (FV→FV mono)
- lnd↔atm and atm→ocn
- if source grid resolution >> target grid resolution: "FV→FV mono"
- if source grid resolution <= target grid resoluiton: use ESMF's bilin.
(replace by will be replaced by TR's FV→FV intbilin when it becomes available)
...
E3SMv1
ATM SE grids with physics on the "np4" grid (used in V1), coupled to FV ocean and FV land
...
. E3SMv1 uses the same spectral element GLL grid for both dynamics and physics in the atmosphere, so coupling between components happens on this grid.
- SE→FV for fluxes: use "SE→FV mono"
- SE → FV SE→FV for state variables: SE->FV intbilin
- FV → SE FV→SE for fluxes: FV→SE monotr
- The best map is the transpose of the SE->FV map computed above:
- This is the most important map, as it is used to compute the domain files and defines the ocean/land mask on the atm/land grid.
- FV → SE FV→SE for state variables
- ocn→atm: currently this map must be the same as used for fluxes (FV→SE monotr)
- lnd→atm: if land grid resolution >> atmosphere resolution: "FV→SE mono".
- lnd→atm: if land grid resolution <= atmosphere resolution: ESMF's bilin. (replace by FV→SE intbilin if it becomes available)
...
analysis and
...
input data
...
(lat-lon grids)
For ESMF maps, we previously recommend bilinear maps for upscaling and aave for downscaling. All the TempestRemap algorithms recommended here should work well for either upscaling or downscaling. The choice depends mostly on the tradeoff between monotonicity and accuracy:
...
- Lat/Lon → SE
- FV→SE "mono". Disabling the conservation constraint with --noconserve does not improve accuracy
- NOTE: For Lat/Lon grids, check the min/max of the weights - if the values are far outside the "[0,1]" interval, may need more coefficients for Vandermonde inverse near poles: edit LinearRemapFV.cpp: nRequiredFaceSetSize = 2*nCoefficients.
- SE → SE
- SE→SE "mono"
...
Motivation
...
for Moving to TempestRemap
Most mapping files used in the E3SM coupled system and used to remap to lat/lon grids for analysis are based on the ESMF_Regrid package. The ESMF remap algorithms and underlying tools are quite good, but they are targetted targeted at finite volume grids and assume cell centered data. The TR algorithms have native support for vertex centered finite element grids, such as used by the E3SM atmosphere dycore.
...
- Some compsets use ESMF bilinear maps for state variables The bilinear algorithm introduces aliasing artificats artifacts when mapping from a high-resolution to low-resolution region (downscaling). Some v1 compsets already use TR highorder high-order for atm2ocn maps, which avoid this issue.
- We currently use ESMF conservative maps for fluxes. This algorithm uses piecewise-constant finite volume reconstruction. These maps are good for downscaling, but will produce blocky artifacts when upscaling.
- ESMF conservative maps for spectral element grids requires us to first generate a "dual grid" that puts a finite volume around each spectral element node. Constructing this dual grid requires running a Matlab tool that uses Newton iteration to find the dual grid. For a high-resolution grid, this procedure can take several days. (For information on the dual grid, see CAM-SE Grid Overview). The dual grid is contained in the SCRIP format metadata file.
- TR algorithms natively support finite element grids, do not require the SCRIP dual grid, and give more accurate results when remapping to/from a spectral element atmosphere grid.
- TR algorithm recommended for state variables is automatically downscaling, so can always be used when mapping from high-res to low-res grids.
- Inline mapping: TempestRemap algorithms are part of the MOAB coupler, making it possible to eliminate mapping files and have them computed as needed.
- Land/Ocean Mask consistency: The flux ocn2atm map is used to define the land/ocean mask on the atmosphere grid. All other ocn2atm maps (state and vectors) must have the same range as the map uses for fluxes. That is, if a point on the atmosphere grid receives ocean data from the flux map, it must also receive ocean data from all other maps. The aave and bilinear maps do not have the same range, and thus if an aave map is used for fluxes, it must be used for all other ocn2atm maps. We speculate that TR maps all have the same range and thus we can use high order maps for ocn2atm state and vectors.
Notation
...
and Abbreviations
SE = Atmosphere spectral element dycore mesh ( CAM-SE Grid Overview ). Usually in "np4" configuration where each element has a 4x4 array of GLL nodes.
...
Note 2: For SE atmosphere grids, the Exodus file is what is used by the model, and it contains the corners of all the spectral elements. The dual grid (SCRIP format) metadata files are not needed by TempestRemap.
Map
...
File Naming Convention
See Mapping file algorithms and naming convention page.
The Overlap Mesh
IMPORTANT!!!: MAPS grids often have holes where there are no mesh points. When computing the TempestRemap overlap grid (step 1 below), the grid with less coverage must be the "a" grid, while the global grid should be the "b" grid. To generate the overlap grid:
./GenerateOverlapMesh --a $ocngrid --b $atmgrid --out overlap_mesh.g
Generic TR Commands to Generate Mapping Files
Update to TR options 2020/2/21: to fix precision issues, with row sums and conservation errors not close to machine precision at high resolution meshes, added the "–correct_areas" option to the recommendations. TR options to generate the various maps:
- SE→FV mono . (conservative, monotone, 1st order)
- ./GenerateOfflineMap --in_mesh $atmgrid --out_mesh $ocngrid --ov_mesh overlap_mesh.g --in_type cgll --in_np 4 --out_type fv --mono --correct_areas --out_map map_atm2ocn_mono.nc
- SE→FV intbilin (nonconservative, monotone, higher order)
- ./GenerateOfflineMap --in_mesh $atmgrid --out_mesh $ocngrid --ov_mesh overlap_mesh.g --in_type cgll --in_np 4 --out_type fv --mono3 --noconserve --correct_areas --out_map map_atm2ocn_intbilin.nc
- SE→FV highorder. (conservative, nonmonotone)
- ./GenerateOfflineMap --in_mesh $atmgrid --out_mesh $ocngrid --ov_mesh overlap_mesh.g --in_type cgll --in_np 4 --out_type fv --correct_areas --out_map map_atm2ocn_highorder.nc
- FV→SE monotr . (conservative, monotone, 1st order)
- The transpose of the SE→FV mono map:
- ./GenerateTransposeMap --in map_atm2ocn_mono.nc --out map_ocn2atm_monotr.nc
- FV→SE highorder . (conservative, non-monotone)
- ./GenerateOfflineMap --in_mesh $ocngrid --out_mesh $atmgrid --ov_mesh overlap_mesh.g --in_type fv --in_np 2 --out_type cgll --out_np 4 --out_map map_ocn2atm_highorder.nc
- Does these maps need "–correct_areas" ?
- FV→SE mono . (conservative, monotone, 1st order)
- ./GenerateOfflineMap --in_mesh $ocngrid --out_mesh $atmgrid --ov_mesh overlap_mesh.g --in_type fv --in_np 1 --out_type cgll --out_np 4 --out_map map_ocn2atm_mono.nc
- Does these maps need "–correct_areas" ?
- FV→SE intbilintr (nonconservative, nonmonotone, higher order)
- The transpose of the SE→FV intbilin map. No negative weights, but weights can be larger than 1.
- ./GenerateTransposeMap --in map_atm2ocn_intbilin.nc --out map_ocn2atm_intbilintr.nc
- FV→FV " aave " (conservative, monotone, 1st order)
- The classic cell integrated piecewise constant map. Equivalent to the ESMF "aave" map and NCO's built in map.
- ./GenerateOfflineMap --in_mesh $ocngrid --out_mesh $atmgrid --ov_mesh overlap_mesh.g --in_type fv --in_np 1 --out_type fv --out_np 1 -correct_areas --out_map map_ocn2atm_mono.nc
- FV→FV highorder . (conservative, non-monotone, high order)
- ./GenerateOfflineMap --in_mesh $atmgrid --out_mesh $ocngrid --ov_mesh overlap_mesh.g --in_np 2 --in_type fv --out_type fv --out_np 1 -correct_areas --out_map map_atm2ocn_highorder.nc
- --in_np 2 means all the nearest neighbor cells will be used for a higher order reconstruction of the source data, which will then be integrated over the target cell. Unlike the SE case, no mass matrix is used for an FV target grid, and thus --out_np is ignored (and will always be 1 internally).
Testing
NCO as of 4.9.0-beta01 can test maps for conservation and consistency. See: Assess and Address Regridding Weights in Map-Files with ncks.
...
We had some issues with TempestRemap, where the initial land mask was not very good. These were resolved here: /wiki/spaces/CMDV/pages/128827525
TR support in NCO and ncremap
The TR maps discussed in "Recommended Settings" above include many options and requirements (transposes, ocean grid first) that might slow-down casual TR users. NCO's ncremap provides a "Make Weight Files" (MWF) mode that automatically generates and correctly names all recommended TR (or ESMF) maps. Instructions and examples for MWF mode are summarized on the ncremap page here, with more in-depth discussion in the NCO manual here.