Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. fluxes:  
    1. if source grid resolution uniformly  >> target grid resolution:   FV→FV TR aave if source grid resolution <= target grid resolution:   V3 is
    2. considering adopting otherwise:    FV2+CAAS 
  2. state and vector maps:  
    1. ocn→atm: The state map's non-zero pattern must be a superset of the ocn→atm flux map.  
      1. This usually means we must use the same map as used for fluxes.    
      2. The FV2+CAAS  approach  (with aave basemap) is the best choice (not sure if this was tested in time for V3?).        
      lnd↔atm and atm→ocn
      1. if source grid resolution ocn resolution uniformly >> atm resolution:  TR aave 
      2. otherwise:   FV2+CAAS   
    2. lnd↔atm and atm→ocn
      1. if source grid resolution uniformly >> target grid resolution (fine-to-coarse):  "FV→FV aave"  TR  ave or intbilin
      2. if source grid resolution <= target grid resolution (nearly equivalent resolutions):   Use TR bilinif source grid resolution uniformly << target grid resolution (coarse-to-fine):  FV2+CAAS.   Needed for atm→lnd in trigrid.
      3. For these statemaps, intbilin should be better than aave or bilin, but not sufficiently tested for V3. 
      4. FV2+CAAS should be comparable to intbilin.   FV2+CAAS will have a small additional performance cost.    
      riv → ocn
      1. continue TR bilin or intbilin  (bilin slightly faster )
      2. otherwise:   TR intbilin or FV2+CAAS  (intbilin slightly faster)
    3. riv → ocn
      1. continue to use custom maps generated by E3SM/cime/tools/mapping/gen_mapping_files/runoff_to_ocn

"FV2+CASS" refers to the nonlinear approach from /wiki/spaces/COM/pages/3771301899 where an aave and 2nd order FV map are used with Compute-And-Assured-Sum (CAAS) to provide a monotone conservative map that avoids the blocky artifacts of aave maps and is more accurate than aave maps.  FV2+CAAS is comparable in accuracy to intbilin, adds conservation, but could be slightly more expensive.  

E3SMv3 with pg2 simplified instructions.  Accurate robust maps, but will increase computational cost of coupler:    

  1. fluxes:  FV2+CAAS
  2. state:  FV2+CASS
  3. riv → ocn
    1. continue to use custom maps generated by E3SM/cime/tools/mapping/gen_mapping_files/runoff_to_ocn


E3SMv2 with pg2

ATM FV grids with physics on the 2x2 physgrid (i.e. pg2) coupled to FV ocean and FV land. E3SMv2 will still use the spectral element dynamical core on the GLL grid, but the coupling between component always happens between the physics columns, which are now on a FV grid. 

fluxes:  FV→FV mono

state and vector maps:  

  1. ocn→atm: currently this map must be the same as used for fluxes (FV→FV mono)
    1. This restriction is because the range of the state and vector maps must match the range of the flux map.  
  2. lnd↔atm and atm→ocn
    1. if source grid resolution >> target grid resolution (fine-to-coarse):  "FV→FV mono"  
    2. if source grid resolution <= target grid resolution (coarse-to-fine):  use ESMF's bilin.
    3. New as of 2022/10:  TR's FV→FV intbilin should work well for all resolution combinations, replacing both maps above.  Not yet tested in E3SM.  

E3SMv1

ATM SE grids with physics on the "np4" grid (used in V1), coupled to FV ocean and FV land. E3SMv1 uses the same spectral element GLL grid for both dynamics and physics in the atmosphere, so coupling between components happens on this grid.

...

For ESMF maps, we previously recommend bilinear maps for coarse-to-fine and aave for fine-to-coarse.  All the TempestRemap algorithms recommended here should work well for either coarse-to-fine or fine-to-coarse.  

SE → Lat/Lon:   "intbilin"

  1. "intbilin" if exact conservation is not needed.
  2. "mono" if exact conservation is needed and 1st order accurate is ok (i.e. for fine-to-coarse maps).    

Lat/Lon → SE 

  1. FV→SE "mono".   Disabling the conservation constraint with --noconserve does not improve accuracy

SE → SE

  1. SE→SE "mono"


...

Background

Motivation for Moving to TempestRemap

...

Issues with the current algorithms supported by the ESMF tools: 

Some compsets use ESMF bilinear maps for state variables The bilinear algorithm introduces aliasing artifacts for fine-to-coarse maps.  Some v1 compsets already use TR high-order for atm2ocn maps, which avoid this issue.    

We currently use ESMF conservative maps for fluxes. This algorithm uses piecewise-constant finite volume reconstruction. These maps are good for fine-to-coarse mapping, but will produce blocky artifacts with coarse-to-fine maps.   

ESMF conservative maps for spectral element grids requires us to first generate a "dual grid" that puts a finite volume around each spectral element node.  Constructing this dual grid requires running a Matlab tool that uses Newton iteration to find the dual grid. For a high-resolution grid, this procedure can take several days.  (For information on the dual grid, see SE Atmosphere Grid Overview (EAM & CAM)).  The dual grid is contained in the SCRIP format metadata file.  

TR algorithms natively support finite element grids, do not require the SCRIP dual grid, and give more accurate results when remapping to/from a spectral element atmosphere grid.

TR algorithm recommended for state variables is cell integration based, so can always be used when mapping from high-res to low-res grids.

Inline mapping: TempestRemap algorithms are part of the MOAB coupler, making it possible to eliminate mapping files and have them computed as needed.

Land/Ocean Mask consistency:   The flux ocn2atm map is used to define the land/ocean mask on the atmosphere grid. All other ocn2atm maps (state and vectors) must have the same range as the map uses for fluxes.  That is, if a point on the atmosphere grid receives ocean data from the flux map, it must also receive ocean data from all other maps.  The aave and bilinear maps do not have the same range, and thus if an aave map is used for fluxes, it must be used for all other ocn2atm maps.   We speculate that TR maps all have the same range and thus we can use high order maps for ocn2atm state and vectors.   

Notation and Abbreviations

...

E3SM v2 (all maps are FV->FV)

Bilinear maps:

  • With E3SMv2's transition to PG2, we now often need bilinear maps for state variables.  These currently are not supported by TR and need to be computed with the ESGF utility
  • ESGF options for bilinear maps:  ADD HERE.  

FV→FV aave (conservative, monotone, 1st order)

  • The classic cell integrated piecewise constant map.  Equivalent to the ESMF "aave" map and NCO's built in map.  
  • ./GenerateOfflineMap --in_mesh $ocngrid --out_mesh $atmgrid --ov_mesh overlap_mesh.g --in_type fv --in_np 1 --out_type fv --out_np 1 --correct_areas  --out_map map_ocn2atm_mono.nc

FV→FV integrated bilinear

  • promising new capability in TR 2022/10, not yet tested in E3SM

E3SM v1 (maps with SE grids.  also sometimes used to produce E3SM v2 initial conditions which are on SE grids)

SE→SE mono (conservative, monotone, 1st order)

  • ./GenerateOfflineMap --in_mesh $atmgrid --out_mesh $atmgrid2 --ov_mesh overlap_mesh.g --in_type cgll --in_np 4  --out_type cgll --out_np 4 --mono --correct_areas --out_map map_atmgrid_to_atmgrid2_mono.nc

SE→FV mono (conservative, monotone, 1st order)

  • ./GenerateOfflineMap --in_mesh $atmgrid --out_mesh $ocngrid --ov_mesh overlap_mesh.g --in_type cgll --in_np 4  --out_type fv  --mono --correct_areas --out_map map_atm2ocn_mono.nc

SE→FV intbilin (nonconservative, monotone, higher order)

  • ./GenerateOfflineMap --in_mesh $atmgrid --out_mesh $ocngrid --ov_mesh overlap_mesh.g --in_type cgll --in_np 4  --out_type fv --mono3 --noconserve --correct_areas --out_map map_atm2ocn_intbilin.nc

SE→FV highorder. (conservative, nonmonotone)

  • ./GenerateOfflineMap --in_mesh $atmgrid --out_mesh $ocngrid --ov_mesh overlap_mesh.g --in_type cgll --in_np 4  --out_type fv --correct_areas --out_map map_atm2ocn_highorder.nc

FV→SE monotr (conservative, monotone, 1st order)

  • The transpose of the SE→FV mono map: 
  • ./GenerateTransposeMap --in map_atm2ocn_mono.nc --out map_ocn2atm_monotr.nc

FV→SE highorder (conservative, non-monotone)

  • ./GenerateOfflineMap --in_mesh $ocngrid --out_mesh $atmgrid --ov_mesh overlap_mesh.g --in_type fv --in_np 2 --out_type cgll --out_np 4  --out_map map_ocn2atm_highorder.nc
  • Do these maps need "–correct_areas" ?

FV→SE mono (conservative, monotone, 1st order)

  • ./GenerateOfflineMap --in_mesh $ocngrid --out_mesh $atmgrid --ov_mesh overlap_mesh.g --in_type fv --in_np 1 --out_type cgll --out_np 4 --out_map map_ocn2atm_mono.nc
  • Do these maps need "–correct_areas" ?

FV→SE intbilintr  (nonconservative, nonmonotone, higher order)

  • The transpose of the SE→FV intbilin map.  No negative weights, but weights can be larger than 1.   
  • ./GenerateTransposeMap --in map_atm2ocn_intbilin.nc --out map_ocn2atm_intbilintr.nc

The Overlap Mesh

IMPORTANT!!!: MAPS grids often have holes where there are no mesh points. When computing the TempestRemap overlap grid (step 1 below), the grid with less coverage must be the "a" grid, while the global grid should be the "b" grid.   To generate the overlap grid:

...

Check the quality of the mapping file with "ncks --chk_map map.nc"

sum of area weights is close to physical area 

mapping weights are between 0 and 1.  for higher order maps, it may be acceptable for small overshoots, but we need to test this. 

Consistency:  Row sum of weights = 1.

  1. Should be 1 to machine precision for all grids where a >= b.  (global → regional)
  2. For grids with a<b (such as ocn→atm) row sums should be 1.0 at all ocean points, 0 at all land points, and between 0 and 1 at coastal points.   

Conservation:  area_b weighted column sum of weights =  area_a (exact for conservation, close otherwise)  

  1. Should be equal to machine precision for all grids where a <= b. (regional → global)
  2. For grids with a>b (such as atm->ocn) should be equal to machine precision at all ocean points, 0 at all land points and between 0 and 1 at coastal points
  3. Non-conservatives maps are expected to have small errors.   The traditional ESMF bilinear map does not include areas in the map file and this metric might not be accurate.  

no missing target points (unless source grid has holes)


NOTE:   The land mask, as used by the land model is computed by remaping the ocean mask (on the ocean grid) to the atmosphere grid.  On the ocean grid, the ocean/land mask is all 0's or 1's  (that is, every point is either ocean or land).  Actually, on an MPAS grid, the land points are all removed, so the mask is all ocean.   This field is then mapped on to the land grid (when creating domain files).  This results in a land mask on the atmosphere/land grid with fractional land points ( 0< mask<1) near coastlines.  

...