Page Comparison

This page is devoted to instruction in ncremap. It describes steps necessary to create grids, and to regrid datasets between different grids with ncremap. Some of the simpler regridding options supported by ncclimo are also described at Generate, Regrid, and Split Climatologies (climo files) with ncclimo. This page describes those features in more detail, and other, more boutique features often useful for custom regridding solutions.

The Zen of Regridding

Most modern climate/weather-related research requires a regridding step in its workflow. The plethora of geometric and spectral grids on which model and observational data are stored ensures that regridding is usually necessary to scientific insight, especially the focused and variable resolution studies that E3SM models conduct. Why does such a common procedure seem so complex? Because a mind-boggling number of options are required to support advanced regridding features that many users never need. To defer that complexity, this HOWTO begins with solutions to the prototypical regridding problem, without mentioning any other options. It demonstrates how to solve that problem simply, including the minimal software installation required. Once the basic regridding vocabulary has been introduced, we solve the prototype problem when one or more inputs are "missing", or need to be created. The HOWTO ends with descriptions of different regridding modes and workflows that use features customized to particular models, observational datasets, and formats. The overall organization, including TBD sections (suggest others, or vote for prioritizing, below), is:

...

ncremap -m map_ne30np4_to_fv129x256_aave.20150901.nc dat_src.nc dat_rgr.nc # EAMv1
ncremap -m map_ne30pg2_to_cmip6_180x360_aave.20200201.nc dat_src.nc dat_rgr.nc # EAMv2

Before looking into map-file generation in the next section, try a few ncremap features. For speed's sake, regrid only selected variables:

...

You might find multiple grid-files that contain the string "129x256". Grid-file names are often ambiguous. The grid-file global metadata (ncks -M grid.nc) often displays the source of the grid. These metadata, and sometimes the actual data (fxm: link), are usually more complete and/or accurate in files with a YYYYMMDD-format date-stamp. For example, the metadata in file 129x256_SCRIP.20150901.nc clearly state it is an FV grid and not some other type of grid with 129x256 resolution. The metadata in 129x256_SCRIP.130510 tell the user nothing about the grid boundaries, and some of the data are flawed. When grids seem identical except for their date-stamp, use the grid with the later date-stamp. The curious can examine a grid-file (ncks -M -m grid.nc) and easily see it looks completely different from a typical model or observational data file. Grid-files and data-files are not interchangeable.

Multiple grid-files also contain the string "ne30". These are either slightly different grids, or the same grids store in different formats meant for different post-processing tools. The different SE (and FV) grid types are described with figures and described here (https://acme-climate.atlassian.net/wiki/spaces/Docs/pages/34113147/Atmosphere+Grids). As explained there, many people will want the "dual-grid" with pentagons. The correct grid-file for this is ne30np4_pentagons.091226.nc. Do not be tempted by SE grid-files named with "latlon".

All grid-files discussed so far are in SCRIP-format, named for the Spherical Coordinate Remapping and Interpolation Package (authored by @pjones). Other formats exist and are increasingly important, especially for SE grids. For now just know that these other formats are also usually stored as netCDF, and that some tools allow non-SCRIP formats to be used interchangeably with SCRIP.

...

ncremap -s ne30np4_pentagons.091226.nc -g 129x256_SCRIP.20150901.nc -m map_ne30np4_to_fv129x256_nco.YYYYMMDD.nc # EAMv1
ncremap -s ne30pg2.nc -g cmip6_180x360_scrip.20181001.nc -m map_ne30pg2_to_cmip6_180x360_nco.YYYYMMDD.nc # EAMv2

The map-file files above is are named with alg_typ="nco" because the ncremap default interpolation algorithm is the first-order conservative NCO algorithm (NB: before NCO 4.9.1 the default algorithm was ESMF "bilin"). To re-create the "aave" map in the first example, invoke ncremap with "-a conserve":

ncremap -a conserve -s ne30np4_pentagons.091226.nc -g 129x256_SCRIP.20150901.nc -m map_ne30np4_to_fv129x256_aave.YYYYMMDD.nc
ncremap -a conserve -s ne30pg2.nc -g cmip6_180x360_scrip.20181001.nc -m map_ne30pg2_to_cmip6_180x360_nco.YYYYMMDD.nc

This takes a few minutes, so save custom-generated map-files for future use. Computing weights to generate map-files is much more computationally expensive and time-consuming than regridding, i.e., than applying the weights in the map-file to the data. We will gloss over most options that weight-generators can take into consideration, because their default values often work well. One option worth knowing now is "-a". The invocation synonyms for "-a" are "--alg_typ", "--algorithm", and "--regrid_algorithm". These are listed in the square brackets in the self-help message that ncremap prints when it is invoked without argument, or with "--help":

zender@firn:~$ ncremap
...
-a alg_typ Algorithm for weight generation (default nco_con) [alg_typ, algorithm, regrid_algorithm]
ESMF algorithms: bilinear|conserve|conserve2nd|nearestdtos|neareststod|patch
NCO algorithms: nco_con (1st order conservative algorithm similar to "conserve")
Tempest algorithms: fv2fv|fv2fv_flx|fv2fv_stt|fv2se_flx|fv2se_stt|fv2se_alt|se2fv_flx|se2fv_stt|se2fv_alt|se2se|tempest
...

One valid option argument for each supported interpolation type is shown separated by vertical bars. The arguments shown have multiple synonyms that are equivalent. For example, "-a conserve" is equivalent to "-a aave" and to "--alg_typ=conservative". Use the longer option form for clarity and precision, and the shorter form for conciseness. The full list of synonyms, and the complete documentation, is at http://nco.sf.net/nco.html#alg_typ. The NCO algorithm "nco_conserve" is the default algorithm. Commonly-used algorithms that invoke ERWG are "bilinear" and "conservative". TR options are discussed below. Peruse the list of options now, though defer a thorough investigation until you reach the "Intermediate Regridding" section.

...