Page Comparison

15 Apr 2020  NERSC Directory Change Notice  

...

...

...

TempestRemap needs to be built from source from the Github repository. This is straightforward on Cori and Edison. Note that a parallel version of netCDF should be used. This can be accomplished on cori/edison be executing module swap cray-netcdf cray-netcdf-hdf5parallel or by sourcing an env_mach_specific.sh from a working case on the target machine before building. Then, 

...

• High resolution USGS topography file named USGS-topo-cube3000.nc (located in the CESM inputdata server here; note this is on a 3 km cubed sphere grid)
• SCRIP-format atmosphere grid file
• topography tool (components/cam/tools/topo_tool/cube_to_target)stand-alone HOMME (as of 2019/1/6
• homme_tool  (same tool used to create SCRIP files described above).

Topography needs to be interpolated from a high resolution USGS file, and then doctored up a bit to allow the model to run stably with the new topography. The tool chain used to compute topography is documented in the following paper:  

...

Step (b) requires running the smoothing tool included with E3SM: e3sm/components/homme/test/template/smoothtopo.job. This script should homme_tool's smoothing algorithm.  The namelist should be edited to specify the resolution (NE value) or grid file (RRM) grid and the amount of smoothing
(TODO: allow running this script with command line arguments instead of manual editing so that the task can be automated).   

...

The unsmoothed topography then needs to be smoothed using the smoothing operator in HOMME. This is done by performing a 1-timestep simulation using standalone HOMME with special options set. There is a script to do this in the HOMME component shipped with E3SM, located at using the homme_tool utility.  Follow the instructions above in the SCRIP section to configure and build this tool.  A sample namelist for the topo smoothing option is given in components/homme/test/templatetool/smoothtopo.job. The following changes need to be made in the script

1. Edit SBATCH directives at the top of the script;
   change #SBATCH --account=acme, #SBATCH --partition=regular, #SBATCH --constraint=knl.
2. Set MACH = $HOMME/cmake/machineFiles/cori-knl.cmake
   (appropriate for whatever architecture you are using; see directory for options)
3. Set resolution via set ne = 4
   (for ne4 example here).
4. Set smooth_phis_numcycle;
   (the default here is 16, but this parameter should probably be revisited for higher resolution grids)
5. Set input and output topography file names via the topo and topoout variables

The script can then be run via "sbatch smoothtopo.job". Note that the script needs to be run twice; the first pass will build HOMME, and the second pass will run HOMME (the first pass can be run directly on a login node, the second pass needs to be submitted to the queue). The smoothed topography will be saved to the filename selected via the topoout variable set in the script. Note that the smoothtopo.job script sets the working directory to ${HOME}/scratch1 by default, so the output (smoothed) topography file should be at ${HOME}/scratch1/preqx/template/${topoout}.toposmooth_gll.nl

1. For cubed sphere grids:
   1. Set resolution via set ne = 4  (for ne4 example here).
   2. Set  smooth_phis_numcycle=16.    (adjustable paramemter, lower for less smoothing)
   3. Set smooth_phis_nudt.  This is a resolution dependent parameter, recommend value for cubed sphere grids is 28e7 * ( 30/NE)**2
2. For RRM grids, we use the tensor laplace operator so that the amount of smoothing is proportional to the resolution
   1. ne=0, mesh_file=/path/to/exodus-rrm-grid-file.g
   2. Set  smooth_phis_numcycle=12.    (adjustable paramemter, lower for less smoothing)
   3. smooth_phis_nudt = 4e-16.  
3. Set infilenames=/path/to/unsmoothed/PHISdata

After running the tool, the output will be named "geos", in the file phis-smoothed1.nc, along with other unnecessary fields.  To convert this file into an EAM topo file:

• ncks -O -v geos,lat,lon phis-smoothed1.nc phis-smoothed2.nc
• ncrename -v geos,PHIS phis-smoothed2.nc

Step (c)

The topography tool then needs to be re-ran to compute the subgrid surface roughness. This is done by passing an extra argument to the topography tool with the smoothed topography output from the standalone HOMME run above:

...

• land grid descriptor file in SCRIP format 
• ESMF_RegridWeightGen
• geographic distribution for each land surface type along with grid descriptor files for each of those surface types
• mkmapdata.sh (found in components/clm/tools/shared/mkmapdata/)
• mksurfdata.pl (found in components/clm/tools/clm4_5/mksurfdata_map/)

...