V3 Topography: GLL/PG2 grids

The tools address the following requirements.

1. The dycore needs geopotential phi_s at GLL points, and the physics needs phi_s at FV cell centers.

2. Physics parameterizations need SGH, SGH30, LANDFRAC computed on the FV grid.

3. We require that the map of the GLL phi_s data be equal to the FV phi_s data.

4. We want to run HOMME's smoother on GLL phi_s.

We illustrate the steps in the case of ne30pg2. Note that for RRM grids, in the homme_tool namelist files,  mesh_file will need to be specified in the ctl_nl section as well, and ne should be set to 0. In the V3 tool, we have switched to using tensor HV smoothing, and thus all other settings, including smooth_phis_nudt are now grid and resolution independent.

Note: the example below is using the “USGS-topo-cube3000.nc” source file which has a 3 km resolution. If the target E3SM grid is finer than 3km, once should consider using the 800m “USGS-topo-cube12000.nc” from 800m cubed topo generation from GMTED2010 15s DEM

1. TempestRemap: Create GLL and pg2 grid template files for the “USGS-topo-cube3000” high res data and the target EAM grid. The USGS-topo-cube3000 grid is computed by Lauritzen’s bin_to_cube tool and is in fact a “ne3000pg1” grid, with a resolution of approximately 3km. This is the high-resolution grid that is downsampled to the model grid and used to compute surface roughness. We are currently using this for generating SCREAM 3km grids, but this is not optimal and we should consider switching to something finer, such as USGS-topo-cube12000.

   # Generate the element mesh. ${tempest_root}/bin/GenerateCSMesh --alt --res 30 --file topo2/ne30.g # Generate the target physgrid mesh and SCRIP file: ${tempest_root}/bin/GenerateVolumetricMesh --in topo2/ne30.g --out topo2/ne30pg2.g --np 2 --uniform ${tempest_root}/bin/ConvertMeshToSCRIP --in topo2/ne30pg2.g --out topo2/ne30pg2_scrip.nc # Generate exodus & SCRIP file for the USGS-topo-cube3000.nc file: GenerateCSMesh --alt --res 3000 --file ne3000.g ConvertMeshToSCRIP --in ne3000.g --out ne3000pg1.scrip.nc # Generate GLL SCRIP file for target grid: for RRM grids, this SCRIP files are good enough # for topo downsampling, but not conservative enough for use in the coupled model: eval $( ${e3sm_root}/cime/CIME/Tools/get_case_env ) homme_tool_root=${e3sm_root}/components/homme/test/tool cd ${homme_tool_root} cmake \ -C ${homme_tool_root}/../../cmake/machineFiles/{machine}.cmake \ -DBUILD_HOMME_WITHOUT_PIOLIBRARY=OFF \ -DPREQX_PLEV=26 ${homme_tool_root}/../../ make -j4 homme_tool rm -f input.nl cat > input.nl <<EOF &ctl_nl ne = 30 mesh_file = "none" / &vert_nl / &analysis_nl tool = 'grid_template_tool' output_dir = "./" output_timeunits=1 output_frequency=1 output_varnames1='area','corners','cv_lat','cv_lon' output_type='netcdf' !output_type='netcdf4p' ! needed for ne1024 io_stride = 16 / EOF mpirun -np 8 ./src/tool/homme_tool < input.nl # make the 'scrip' file for target GLL grid ncks -O -v lat,lon,area,cv_lat,cv_lon ne30np4_tmp1.nc ne30np4_tmp.nc ncl E3SM/components/homme/test/tool/ncl/HOMME2SCRIP.ncl name=\"ne30np4\" ne=30 np=4

2. cube_to_target, run 1: Compute phi_s on the np4 grid. Any reasonable non-interpolatory map can be used for this step. As of 2021/5, we use “cube_to_target” for this step, which requires a GLL SCRIP file for the target grid. In the V2 tool chain, this SCRIP file was avoided by first mapping to a “pg4” grid (using an FV SCRIP file) as an intermediate step. But with the very little smoothing used in V3 topography files, this intermediate step was found to introduce too much smoothing itself.

   # option1: use TR # # create a TR monotone conservative map from ne3000pg1.scrip.nc to ne30np4 # map USGS-topo-cube3000.nc to ne30np4_topo.nc # # option 2: # follow V2 procedures. Map USGS-topo-cube3000.nc to ne30pg4_topo.nc # then combine smoothing (step 3 below) along with map to ne30np4_topo.nc # but using the proposed V3 smoothing settings # # option 3: use cube_to_target # ${e3sm_root}/components/cam/tools/topo_tool/cube_to_target \   --target-grid topo2/ne30np4_scrip.nc \   --input-topography topo2/USGS-topo-cube3000.nc \   --output-topography topo2/ne30np4_topo.nc

   These warnings appear to be innocuous:

   sum of weights is negative - negative area? -1.8651290104823655E-009 675 3000

3. homme_tool: Starting with the unsmoothed topo data on the GLL grid, apply dycore specific smoothing.

   $ cat input.nl &ctl_nl ne = 30 smooth_phis_p2filt = 0 smooth_phis_numcycle = 6 # increase for more smoothing smooth_phis_nudt = 4e-16 hypervis_scaling = 2 se_ftype = 2 ! actually output NPHYS; overloaded use of ftype / &vert_nl / &analysis_nl tool = 'topo_pgn_to_smoothed' infilenames = 'ne30np4_topo.nc', 'ne30np4pg2_smoothed_phis' /   mpirun -np 8 ./homme_tool < input.nl

   This uses the standard tensor laplace smoothing algorithm with 6 iterations. We use the naming convention “xNt”, where N is the value used for smooth_phis_numcycle and “t” denotes the use of the tensor laplace.

4. cube_to_target, run 2: Compute SGH, SGH30, LANDFRAC, and LANDM_COSLAT on the pg2 grid, using the pg2 phi_s data.

   ${e3sm_root}/components/cam/tools/topo_tool/cube_to_target \   --target-grid ne30pg2_scrip.nc \   --input-topography USGS-topo-cube3000.nc \   --smoothed-topography ne30np4pg2_smoothed_phis1.nc \   --output-topography USGS-gtopo30_ne30np4pg2_x6t.nc

5. ncks: Append the GLL phi_s data to the output of step 4.

   ncks -A ne30np4pg2_smoothed_phis1.nc USGS-gtopo30_ne30np4pg2_x6t.nc

USGS-gtopo30_ne30np4pg2_x6t.nc is the final GLL-physgrid topography file.