...

The example below shows the commands needed to generate all mapping files for a tri-grid configuration with the atmosphere on the ne30pg2 grid. Note that this example of wrapping TempestRemap with ncremap assumes the mapping algorithm for TempestRemap. For more control over this, and to use the latest recommended maps for mapping between each component, see Transition to TempestRemap for Atmosphere gridsRecommended Mapping Procedures for E3SM Atmosphere Grids:

atm_grid_file=ne30pg2.g
ocn_grid_file=ocean.oEC60to30v3.scrip.181106.nc
lnd_grid_file=SCRIPgrid_0.5x0.5_nomask_c110308.nc
atm_name=ne30pg2
ocn_name=oEC60to30v3
lnd_name=r05
alg_name=mono

...

The resulting scrip file can be used for mapping with ESMF or plotting physgrid data on the native grid. 

Topography

The dycore requires smoothed topography on the GLL grid, and the physics parameterization for turbulent mountain stress operates on the physics (FV) grid. PRs #3267 and #3406 introduced a new file format for topography and related tools that provide consistent topography data on each grid. There are two new tools to support this new treatment of topography.

The tools address the following requirements.

1. The dycore needs geopotential phi_s at GLL points, and the physics needs phi_s at FV cell centers.
2. Physics parameterizations need SGH, SGH30, LANDFRAC, and LANDM_COSLAT computed on the FV grid.
3. We require that the map of the GLL phi_s data be equal to the FV phi_s data.
4. We want to run HOMME's smoother on GLL phi_s.
5. We do not want to use the GLL dual grid, which requires using an unscalable tool; we want to use only the FV dual grid, which can be created straightforwardly and quickly using, e.g., TempestRemap.

To meet these requirements, we introduce a new topography file format that has all data at FV points and, in addition, phi_s at the GLL points.

The first tool is a simple converter, for convenience. The input is an old GLL topography file. The output is a topography file in the new GLL-physgrid format. SGH, SGH30, LANDFRAC, and LANDM_COSLAT are not quite as good as with second tool, but this converter works in one step:

$ cat input.nl
&ctl_nl
ne = 30
/
&vert_nl
/
&analysis_nl
tool = 'topo_convert'
infilenames = 'USGS-gtopo30_ne30np4_16xdel2-PFC-consistentSGH.nc', 'USGS-gtopo30_ne30np4pg2_16xdel2-PFC-consistentSGH_converted'
/

mpirun -np 8 homme_tool < input.nl

homme_tool is an executable in ${homme_build}/src/tool/homme_tool produced in a standard configuration and build of standalone homme.

The second tool chain provides the best-quality GLL-physgrid file. It has five steps, illustrated for the case of ne30pg2:

TempestRemap: Create GLL, pg2, and pg4 grids. pg2 is our target. pg4 is a high-resolution intermediate grid.

# Generate the element mesh.
${tempest_root}/bin/GenerateCSMesh --alt --res 30 --file topo2/ne30.g
# Generate the target physgrid mesh.
${tempest_root}/bin/GenerateVolumetricMesh --in topo2/ne30.g --out topo2/ne30pg2.g --np 2 --uniform
# Generate a high-res target physgrid mesh for cube_to_target.
${tempest_root}/bin/GenerateVolumetricMesh --in topo2/ne30.g --out topo2/ne30pg4.g --np 4 --uniform
# Generate SCRIP files for cube_to_target.
${tempest_root}/bin/ConvertExodusToSCRIP --in topo2/ne30pg4.g --out topo2/ne30pg4_scrip.nc
${tempest_root}/bin/ConvertExodusToSCRIP --in topo2/ne30pg2.g --out topo2/ne30pg2_scrip.nc

cube_to_target, run 1: Compute phi_s on the pg4 grid.

${e3sm_root}/components/cam/tools/topo_tool/cube_to_target \
  --target-grid topo2/ne30pg4_scrip.nc \
  --input-topography topo2/USGS-topo-cube3000.nc \
  --output-topography topo2/ne30pg4_c2t_topo.nc

These warnings appear to be innocuous:

sum of weights is negative - negative area? -1.8651290104823655E-009 675 3000

...

homme_tool: Map phi_s on the pg4 grid to phi_s on the GLL grid. Smooth these GLL phi_s data. Finally, map these GLL phi_s data to pg2. An example namelist for an ne30 case is shown below:

$ cat input.nl
&ctl_nl
ne = 30
smooth_phis_numcycle = 16
smooth_phis_nudt = 28e7
hypervis_scaling = 0 
hypervis_order = 2
se_ftype = 2 ! actually output NPHYS; overloaded use of ftype
/
&vert_nl
/
&analysis_nl
tool = 'topo_pgn_to_smoothed'
infilenames = 'ne30pg4_c2t_topo.nc', 'ne30np4pg2_smoothed_phis'
/

mpirun -np 8 ./homme_tool < input.nl

Note that for RRM grids, mesh_file will need to be specified in the ctl_nl section as well, and ne should be set to 0. smooth_phis_numcycle is an adjustable parameter, and can be lowered for less smoothing. smooth_phis_nudt is resolution dependent. Generally, for uniform cubed-sphere grids, set resolution via ne = 30, smooth_phis_numcycle = 16, and smooth_phis_nudt = 28e7 * (30/NE)**2. For RRM, set ne = 0, smooth_phis_numcycle = 12, and smooth_phis_nudt = 4e-16.

cube_to_target, run 2: Compute SGH, SGH30, LANDFRAC, and LANDM_COSLAT on the pg2 grid, using the pg2 phi_s data.

${e3sm_root}/components/cam/tools/topo_tool/cube_to_target \
  --target-grid ne30pg2_scrip.nc \
  --input-topography USGS-topo-cube3000.nc \
  --smoothed-topography ne30np4pg2_smoothed_phis1.nc \
  --output-topography USGS-gtopo30_ne30np4pg2_16xdel2.nc

ncks: Append the GLL phi_s data to the output of step 4.

ncks -A ne30np4pg2_smoothed_phis1.nc USGS-gtopo30_ne30np4pg2_16xdel2.nc

USGS-gtopo30_ne30np4pg2_16xdel2.nc is the final GLL-physgrid topography file.

The second tool chain can be modified if you have a GLL PHIS field that you want to preserve exactly:

...

Run two ncks lines:

...

Several modifications to the topography generation are needed to support PG2 grids.  Complete details are given in the V2 (and later) instructions under

Atmospheric Topography Generation

Regional Refinement

Regionally refined grids require no special steps when using the physgrid because the regional refinement happens at the element level, and the physgrid only changes the physics column arrangement within the element. Note however that the input namelist for homme_tool will need to specify ne = 0 and mesh_file for RRM grids.

...