This page is devoted to instruction in ncremap. It describes steps necessary to create grids, and to regrid datasets between different grids with ncremap. Some of the simpler regridding options supported by ncclimo are also described at Generate, Regrid, and Split Climatologies (climo files) with ncclimo. This page describes those features in more detail, and other, more boutique features often useful for custom regridding solutions.
The Zen of Regridding
Most modern climate/weather-related research requires a regridding step in its workflow. The plethora of geometric and spectral grids on which model and observational data are stored ensures that regridding is usually necessary to scientific insight, especially the focused and variable resolution studies that E3SM models conduct. Why does such a common procedure seem so complex? Because a mind-boggling number of options are required to support advanced regridding features that many users never need. To defer that complexity, this HOWTO begins with solutions to the prototypical regridding problem, without mentioning any other options. It demonstrates how to solve that problem simply, including the minimal software installation required. Once the basic regridding vocabulary has been introduced, we solve the prototype problem when one or more inputs are "missing", or need to be created. The HOWTO ends with descriptions of different regridding modes and workflows that use features customized to particular models, observational datasets, and formats. The overall organization, including TBD sections (suggest others, or vote for prioritizing, below), is:
...
The main difference between generating the timeseries for the Historical ensemble and the DECK PI experiment is the need to loop over the ensemble. Here the splitter command is not backgrounded so that one member experiment is processed at a time (to avoid overwhelming nodes with I/O and RAM demands). Set the input directory in the ensemble loop, and ensure the globbing pattern for filenames matches the naming convention used for all five members. Consider whether to output to member-specific directories or to a single, ensemble-wide directory. If the former, then nothing special need be done. If the latter, use the --fml_nm (family-name) option as above to avoid identical timeseries names (that will overwrite one another) and to create instead member-specific timeseries names like
CLDLOW_H1_185001_201412.nc
CLDLOW_H2_185001_201412.nc
...
Advanced Regridding VI: Initial Condition Files
First, use the right tool. ncremap can regrid an initial conditions (IC) file, both vertically and horizontally. However, generating scientifically validated IC files for new model resolutions is best done with a lengthy workflow (https://acme-climate.atlassian.net/wiki/spaces/ED/pages/872579110/Adding+support+for+new+grids) in which regridding plays a relatively small role. That said, ncremap is a good tool to place an atmospheric state onto a new grid where it can then be nudged into a valid IC file, or to place the contents of an IC file on a rectangular grid (as shown below) where it is easier to plot. Regridding atmosphere IC files was straightforward until E3SM v2 when the atmosphere separated the dynamical and physical grids. This resulted in IC files containing two sets of grid variables, so now we must regrid v2 IC files with two successive invocations of ncremap:
ncremap --map=map_ne30pg2_to_cmip6_180x360_nco.20200901.nc foo.eam.i.2001-01-01-00000.nc foo.ncncremap --map=map_ne30np4_to_cmip6_180x360_nco.20200901.nc foo.nc foo.eam.i.2001-01-01-00000.rgr.nc
The first command remaps the variables on the PG2 physics grid to the desired output grid. This is straightforward since the physics grid variables (SICTHK, ICEFRAC, TS1...) use ncol (the ncremap default) as the horizontal coordinate. The second command remaps the remaining variables, stored on the NP4 dynamics grid with horizontal dimension ncol_d, to the same output grid. The second invocation automatcally regrids all the variables on the dynamics grid (with the ncol_d dimension) because the intermediate file foo.nc no longer has an ncol dimension.
To regrid only the variables with a non-default (e.g., not ncol) horizontal dimension in a file with multiple horizontal dimensions, one would explicitly select the non-default dimension using the ncremap -R option, whose argument is passed directly to the underlying regridder (ncks):
ncremap -R '--rgr col_nm=ncol_d' --map=map_ne30np4_to_cmip6_180x360_nco.20200901.nc foo.eam.i.2001-01-01-00000.rgr.nc foo.nc
Epilogue: User-Suggested Examples
...