This page is devoted to instruction in ncremap. It describes steps necessary to create grids, and to regrid datasets between different grids with ncremap. Some of the simpler regridding options supported by ncclimo are also described at Generate, Regrid, and Split Climatologies (climo files) with ncclimo. This page describes those features in more detail, and other, more boutique features often useful for custom regridding solutions.
The Zen of Regridding
Most modern climate/weather-related research requires a regridding step in its workflow. The plethora of geometric and spectral grids on which model and observational data are stored ensures that regridding is usually necessary to scientific insight, especially the focused and variable resolution studies that E3SM models conduct. Why does such a common procedure seem so complex? Because a mind-boggling number of options are required to support advanced regridding features that many users never need. To defer that complexity, this HOWTO begins with solutions to the prototypical regridding problem, without mentioning any other options. It demonstrates how to solve that problem simply, including the minimal software installation required. Once the basic regridding vocabulary has been introduced, we solve the prototype problem when one or more inputs are "missing", or need to be created. The HOWTO ends with descriptions of different regridding modes and workflows that use features customized to particular models, observational datasets, and formats. The overall organization, including TBD sections (suggest others, or vote for prioritizing, below), is:
...
Take a moment to compare the methods for EAM/ELM and for MPAS. They are nearly identical except for the variable names, experiment names and directories, map-files (so far nothing surprising or important) AND the additional MPAS options in ${mpas_opt}. We will return to discuss those soon. Each command-set begins with setting experiment-dependent I/O directories and a map-filefiles. Other experiments will require changing these to the appropriate I/O directories, yet the map-file remains the same unless the native or destination grid changes. The next three or four lines in each command-set configure the splitter and regridder with options that many ncclimo/ncremap users have never before tried before. Finally the list of input files and all the configuration options are sent to ncclimo. The entire procedure for the user boils down to creating then executing this one splitter command for each desired variable.
Regridding is performed only if the splitter (i.e., ncclimo) is invoke invoked with the --map option that supplies a suitable mapfile from the native to the desired destination grid. CMIP6 will only distribute data on 2D structured grids, yet E3SM will itself distribute the timeseries on native (unstructured grids). Hence the commands above construct both the native timeseries (stored in ${drc_out}) and the regridded timeseries for CMIP6 (stored in ${drc_rgr}). Internally, the splitter constructs the native grid timeseries for the same time segement for all requested variables in parallel, waits for completion, and then calls the regridder (ncremap) in parallel with all timeseries for that segment, waits, then continues to the next segment.
...
The main difference between generating the timeseries for the Historical ensemble and the DECK PI experiment is the need to loop over the ensemble. Here the splitter command is not backgrounded so that one member experiment is processed at a time (to avoide avoid overwhelming nodes with demands for I/O and RAM demands). Set the input directory in the ensemble loop, and ensure the globbing pattern for filenames matches the naming convention used for all five members. It is important to consider Consider whether to output to member-specific directories or to a single, ensemble-wide directory. If the former, then nothing special need be done. If the latter, use the --fml_nm (family-name) option as above to avoid members overwriting one another's timeseries by creating member-spcific identical timeseries names (that will overwrite one another) and to create instead member-specific timeseries names like
CLDLOW_H1_185001_201412.nc
CLDLOW_H2_185001_201412.nc
...