This page is devoted to instruction in ncremap. It describes steps necessary to create grids, and to regrid datasets between different grids with ncremap. Some of the simpler regridding options supported by ncclimo are also described at Generate, Regrid, and Split Climatologies (climo files) with ncclimo. This page describes those features in more detail, and other, more boutique features often useful for custom regridding solutions.
The Zen of Regridding
Most modern climate/weather-related research requires a regridding step in its workflow. The plethora of geometric and spectral grids on which model and observational data are stored ensures that regridding is usually necessary to scientific insight, especially the focused and variable resolution studies that E3SM models conduct. Why does such a common procedure seem so complex? Because a mind-boggling number of options are required to support advanced regridding features that many users never need. To defer that complexity, this HOWTO begins with solutions to the prototypical regridding problem, without mentioning any other options. It demonstrates how to solve that problem simply, including the minimal software installation required. Once the basic regridding vocabulary has been introduced, we solve the prototype problem when one or more inputs are "missing", or need to be created. The HOWTO ends with descriptions of different regridding modes and workflows that use features customized to particular models, observational datasets, and formats. The overall organization, including TBD sections (suggest others, or vote for prioritizing, below), is:
...
The regridding problem most commonly faced is converting output from a standard resolution model simulation to equivalent data on a different grid for visualization or intercomparison with other data. The EAM v1 model low-resolution simulations are performed and output on the ne30np4 SE (spectral element) grid, aka the "source grid". Data on this source grid The corresponding EAM v2 simulations were conducted on the ne30pg2 FV (finite volume) grid. E3SM source grids like these are called “unstructured” because they have only one horizontal dimension (i.e., 1D) which makes them difficult to visualize. The recommended (fxm: link) 2D grid for analysis of these simulations is the 129x256 FV (finite-volume) grid, aka the "destination grid"2D (latitude-longitude) grids for analysis (aka the "destination grid") of v1 simulations was the 129x256 Cap grid (the gridcells at the poles look like yarmulke caps), and since v2 is the 180x360 equi-angular grid used by CMIP. The single most important capability of a regridder is the intelligent application of weights that transform data on the input grid to the desired output grid. These weights are stored in a "map-file", a single file that which contains all the necessary weights and grid information necessary. While most regridding problems revolve around creating the appropriate map-file, this prototype problem assumes that has already been done, so the appropriate map-file (map.nc) already exists and ncremap can immediately transform the input dataset (dat_src.nc) to the output (regridded) dataset (dat_rgr.nc):
| Code Block |
|---|
ncremap -m map.nc dat_src.nc dat_rgr.nc |
This solution is deceptively simple because it conceals the choices, paths, and options required to create the appropriate map.nc for all situations. We will discuss creating map.nc later after showing more powerful and parallel ways to solve the prototype problem. The solution above only works for users savvy enough to know how to find appropriate pre-built map-files. E3SM v1.0 map-files are available here. Many commonly used maps and grids can also be found in my (@czender's) directories as ~zender/data/[maps,grids] at most DOE High Performance Computing (HPC) centers. Take a minute now to look at these locations.
Pre-built map-files use the (nearly) standardized naming convention map_srcgrd_to_dstgrd_algtyp.YYYYMMDD.nc, where srcgrd and dstgrd are the source and destination grid names, algtyp is a shorthand for the numerical regridding algorithm, and YYYYMMDD is the date assigned to the map (i.e., the date the map was created). The source grid in the example above is called ne30np4, the destination is called fv129x256. The only A pre-built map for this the v1 combination is map.nc = map_ne30np4_to_fv129x256_aave.20150901.nc, and for v2 is map_ne30pg2_to_cmip6_180x360_aave.20200201.nc. What is "aave"? Weight generators can use about a dozen interpolation algorithms for regridding, and each has a shorthand name. For now, it is enough to know that the two most common algorithms are (first-order) conservative area-average regridding ("aave") and bilinear interpolation ("bilin" or "blin"). Hence this conservatively regrids dat_src.nc to dat_rgr.nc with first order accuracy:
...
To regrid multiple files with a single command, supply ncremap with the source and regridded directory names (drc_src, drc_rgr). It regrids every file in drc_src and places the output in drc_rgr:
| Code Block |
|---|
ncremap -m map.nc -I drc_src -O drc_rgr |
...
When an output directory is not specified, ncremap writes to the current working directory. When the output and input directories are the same, ncremap appends a string (based on the destination grid resolution) to each input filename to avoid name collisions. Finally, be aware that multiple-file invocations of ncremap execute in parallel by default. Power users will want to tune this as described in the section on "Intermediate Regridding".
...