V1 Topography: GLL grids

For an overview of the Topography generation process, see the parent page: Running E3SM on New Atmosphere Grids

Requirements:

• High resolution USGS topography file named USGS-topo-cube3000.nc (located in the CESM inputdata server here; note this is on a 3 km cubed sphere grid)
• SCRIP-format atmosphere grid file
• topography tool (components/cam/tools/topo_tool/cube_to_target)
• homme_tool  

For "pg2" grids (E3SM v2):  dont follow this page and instead see the instructions on  Special Considerations for FV Physics Grids 

Step (a)

To begin Step (a) with a SCRIP file in hand,  interpolate the high resolution USGS data. The source topography file is the USGS high resolution topography on a 3 km cube sphere grid. This data exists on the CESM inputdata server here.  The data has also been downloaded to NERSC computing facilities and is located here:

/project/projectdirs/acme/inputdata/atm/cam/hrtopo/USGS-topo-cube3000.nc

Interpolation to the target grid is done using the cube_to_target tool. This tool has recently been modified to accept arguments via the command line, which prevents having to edit the source code and recompile to run the different steps of a) interpolating the USGS topography to the target grid and b) calculating the subgrid surface roughness based on the interpolated topography and one that has been smoothed by the dynamical core.  If successful, this script prints a clean exit with "Done running cube_to_target."

#!/bin/bash

grid_name=ne4
output_root=/gscratch/bhillma/e3sm/grids/${grid_name}
atm_scrip_file="${output_root}/ne4np4b_scrip.nc"

# Set paths
e3sm_root="${HOME}/codes/e3sm/branches/master"
datestring=`date +'%y%m%d'`
cube_to_target_root=${e3sm_root}/components/cam/tools/topo_tool/cube_to_target
input_topography_file=/project/projectdirs/acme/inputdata/atm/cam/hrtopo/USGS-topo-cube3000.nc
output_topography_file=${output_root}/topo/USGS_${grid_name}_unsmoothed_${datestring}.nc

# Get machine-specific modules
${e3sm_root}/cime/tools/configure && source .env_mach_specific.sh

# build the code
cd ${cube_to_target_root}
export FC=ifort INC_NETCDF=${NETCDF_DIR}/include LIB_NETCDF=${NETCDF_DIR}/lib
make clean && make
echo "Done building cube_to_target."

# run the code
echo "Run cube_to_target to interpolate topography..."
mkdir -p `dirname ${output_topography_file}`
${cube_to_target_root}/cube_to_target \
	--target-grid ${atm_scrip_file} \
	--input-topography ${input_topography_file} \
	--output-topography ${output_topography_file}
echo "Done running cube_to_target."

Step (b)

The unsmoothed topography then needs to be smoothed using the smoothing operator in HOMME. This is done using the homme_tool utility.  Follow the instructions in the Running E3SM on New Atmosphere Grids page, SCRIP section, to configure and build this tool.  A sample namelist for the topo smoothing option is given in components/homme/test/tool/toposmooth_gll.nl

1. For cubed sphere grids:
   1. Set resolution via set ne = 4  (for ne4 example here).
   2. Set  smooth_phis_numcycle=16.    (adjustable paramemter, lower for less smoothing)
   3. Set smooth_phis_nudt.  This is a resolution dependent parameter, recommend value for cubed sphere grids is 28e7 * ( 30/NE)**2
2. For RRM grids, we use the tensor laplace operator so that the amount of smoothing is proportional to the resolution
   1. ne=0, mesh_file=/path/to/exodus-rrm-grid-file.g
   2. Set  smooth_phis_numcycle=12.    (adjustable paramemter, lower for less smoothing)
   3. smooth_phis_nudt = 4e-16.  
3. Set infilenames=/path/to/unsmoothed/PHISdata

After running the tool, there should be output file 'phis-smoothed1.nc' with one of the fields named "PHIS".  We then extract just the variables needed for the EAM topo file:

• ncks -O -v PHIS,lat,lon phis-smoothed1.nc phis-smoothed2.nc

Step (c)

The topography tool then needs to be re-ran to compute the subgrid surface roughness. This is done by passing an extra argument to the topography tool with the smoothed topography output from the standalone HOMME run above:

# Get machine-specific modules
e3sm_root=${HOME}/codes/e3sm/branches/update-topo-tool
${e3sm_root}/cime/tools/configure && source .env_mach_specific.sh

# Set paths; datestring will probably be different if the workflow takes multiple days!
datestring=`date +'%y%m%d'`
atm_scrip_file="${output_root}/templates/ne4np4b_scrip_${datestring}.nc"
input_topography_file="${inputdata_root}/atm/cam/hrtopo/USGS-topo-cube3000.nc"
output_topography_file="${output_root}/topo/USGS_ne4np4_consistentSGH_${datestring}.nc"
smoothed_topography_file="phis-smoothed2.nc"   # output of step(b) above

# Run the code
${cube_to_target_root}/cube_to_target \
	--target-grid ${atm_scrip_file} \
	--input-topography ${input_topography_file} \
	--output-topography ${output_topography_file} \
	--smoothed-topography ${smoothed_topography_file}

This should output the final topography file with consistent subgrid surface roughness, ready to run in a full atmosphere configuration with the new resolution.