This page is devoted to instruction in ncremap. It describes steps necessary to create grids, and to regrid datasets between different grids with ncremap. Some of the simpler regridding options supported by ncclimo are also described at Generate, Regrid, and Split Climatologies (climo files) with ncclimo. This page describes those features in more detail, and other, more boutique features often useful for custom regridding solutions.
The Zen of Regridding
Most modern climate/weather-related research requires a regridding step in its workflow. The plethora of geometric and spectral grids on which model and observational data are stored ensures that regridding is usually necessary to scientific insight, especially the focused and variable resolution studies that E3SM models conduct. Why does such a common procedure seem so complex? Because a mind-boggling number of options are required to support advanced regridding features that many users never need. To defer that complexity, this HOWTO begins with solutions to the prototypical regridding problem, without mentioning any other options. It demonstrates how to solve that problem simply, including the minimal software installation required. Once the basic regridding vocabulary has been introduced, we solve the prototype problem when one or more inputs are "missing", or need to be created. The HOWTO ends with descriptions of different regridding modes and workflows that use features customized to particular models, observational datasets, and formats. The overall organization, including TBD sections (suggest others, or vote for prioritizing, below), is:
Software Requirements
Prototypical Regridding I: Use Existing Map-file
Prototypical Regridding II: Create Map-file from Known Grid-files
Prototypical Regridding III: Infer Grid-file from Data-file
Prototypical Regridding IV: Manual Grid-file Generation
Intermediate Regridding I: Treatment of Missing-Data (Done)
Intermediate Regridding II: MPAS-mode (TBD)TempestRemap
Intermediate Regridding III: TempestRemap Parallelism (DoneTBD)
Intermediate Advanced Regridding IV: Parallelism (TBD)I: MPAS-mode
Advanced Regridding III: Regional SE Output (RRG-mode) (Done)
Advanced Regridding IIIII: Sub-Gridscale Regridding (SGS-mode) (TBD)
Advanced Regridding IIIIV: Make All Weight Files (MWF-mode) (Done)
Software Requirements:
At a minimum, install a recent version NCO on your executable $PATH with the corresponding library on your $LD_LIBRARY_PATH. NCO installation instructions are here (fxm: link). We highly recommend installing NCO through the conda package. That will automatically install another important piece of the regridding toolchain, the ESMF_RegridWeightGen (aka ERWG) executable. Execute 'ncremap --config' to verify you have a working installation:
...
In practice, it may make sense to use the default conservative treatment with conservative regridding, and the renormalized treatment with other (non-conservative) regridding algorithms such as bilinear interpolation or nearest-neighbor. Another consideration is whether the fields being regridded are fluxes or state variables. For example, temperature (unlike heat) and concentrations (amount per unit volume) are not physically conserved quantities under areal-regridding so it often makes sense to interpolate them in a non-conservative fashion, to preserve their fine-scale structure. Few researchers can digest the unphysical values of temperature that the conservative treatment produces in regions rife with missing values. On the other hand are mass and energy fluxes, which should be physically conserved under areal-regridding. One should consider both the type of field and its conservation properties when choosing a missing value treatment.
Intermediate Regridding II:
...
TempestRemap
TempestRemap is the chief alternative to ERWG for regridding weight-generation, and is sleighted for increased adoption in E3SM. Tempest algorithms, written by @paulullrich, have many numerical advantages described in papers and at https://acme-climate.atlassian.net/wiki/spaces/Docs/pages/178848194/Transition+to+TempestRemap+for+Atmosphere+grids. Tempest lacks pre-built binary package support under most OSs and on most DOE machines so users must often download and install it by following the instructions at https://github.com/ClimateGlobalChange/tempestremap. Verify ncremap can access your Tempest installation as described in the above section on "Software Requirements". Once installed, Tempest can be as easy to use as ERWG with SCRIP grid-files, e.g.,
...
These maps, applied to appropriate flux and state variables, should exactly reproduce the online remapping in the E3SM v2/v3 coupler. However, explicitly generating all six standard maps this way is not recommended because ncremap includes an MWF-mode (for "Make All Weight Files") described below. MWF-mode generates and names, with one command and in a self-consistent manner, all combinations of E3SM global atmosphere<->ocean maps with both ERWG and Tempest.
Intermediate Regridding
...
III: Parallelism (TBD)
Advanced Regridding
...
:
...
Advanced procedures have in common that they activate non-standard processing modes for ncremap. These modes do something different, or in addition to, the standard weight-generation and application. Generally these modes were created in order to automate frequently recurring workflows that can leverage the ncremap infrastructure so long as various bells and whistles are introduced along the way. Please let us know if you have ideas for new or improved processing modes.
Advanced Regridding I: MPAS-mode
MPAS models produce output in their own format distinct from CESM-heritage models. MPAS-mode invokes three pre-processing steps to massage MPAS datasets until they are amenable to regridding. These steps include missing value annotation, missing value treatment, and dimension permutation. We will shortly describe these steps in order. First, though, MPAS-mode, like most other ncremap modes, is explicitly invoked with the -P md_nm option where md_nm is the mode name, i.e.,
ncremap -P mpas -m map.nc dat_src.nc dat_rgr.nc
Many model and observational datasets use missing values that are not annotated in the standard manner. For example, the MPAS ocean and ice models use -9.99999979021476795361e+33 as the missing value, yet (at least from 2015-2018) do not store this in a _FillValue attribute with any variables. To prevent arithmetic from treating these values as valid, MPAS-mode automatically puts this value in a _FillValue attribute for all double-precision variables via
ncatted -t -a _FillValue,,o,d,-9.99999979021476795361e+33 dat_src.nc
Oddly, the MPAS land-ice model uses -1.0e36 for missing values, so currently MPAS-LI users must explicitly supply this missing value to ncremap:
ncremap -P mpas --mss_val=-1.0e36 -m map.nc dat_src.nc dat_rgr.nc
Next, MPAS datasets usually have masked regions (e.g., non-ocean cells) yet MPAS users like to visualize regridded data with realistic (not blocky) boundaries along those cells and so they decided to, by default, treat missing values with the renormalization approach described above in the section on Treatment of Missing Values. Hence MPAS-mode automatically invokes ncremap with maximum renormalization, equivalent to
ncremap --rnr_thr=0.0 -m map.nc dat_src.nc dat_rgr.nc
Finally, ncremap requires the horizontal spatial dimension(s), whether latitude and longitude or some unstructured dimension, to be the final (most-rapidly-varying) dimension(s) of input datasets. MPAS datasets natively place their spatial dimension (typically nCells) closer to the least-rapidly-varying position. While this makes perfect sense from an I/O-efficiency point-of-view for unstructured models, it does not play well with regridders. Hence MPAS-mode permutes the input dimensions to a regridder-friendly order with
ncpdq -a Time,nVertLevels,nVertLevelsP1,maxEdges,MaxEdges2,nCategories,ONE,nEdges,nCells dat_src.nc dat_tmp.nc
The combination of these three data manipulations defines MPAS-mode.
Advanced Regridding II: Regional SE Output (RRG-mode)
EAM and CAM-SE will produce regional output if requested to with the finclNlonlat namelist parameter. Output for a single region can be higher temporal resolution than the host global simulation. This facilitates detailed yet economical regional process studies. Regional output files are in a special format that we call RRG (for "regional regridding"). An RRG file may contain any number of rectangular regions. The coordinates and variables for one region do not interfere with other (possibly overlapping) regions because all variables and dimensions are named with a per-region suffix string, e.g., lat_128e_to_134e_9s_to_16s. ncremap can easily regrid RRG output from an FV-dycore because ncremap can infer (as discussed above) the regional grid from any FV data file. Regridding regional SE data, however, is more complex because SE gridcells are essentially weights without vertices (as and SE weight-generators are not yet flexible enough to generate the regional weights. To summarize, regridding RRG data leads to three SE-specific difficulties (#1-3 below) and two difficulties (#4-5) shared with FV RRG files:
...
RRG-mode supports most normal ncremap options, including input and output methods and regridding algorithms.
Advanced Regridding
...
III: Sub-Gridscale Regridding (SGS-mode) (TBD)
Advanced Regridding
...
IV: Make All Weight Files (MWF-mode)
As mentioned above in the TempestRemap section, ncremap includes an MWF-mode (for "Make All Weight Files") that generates and names, with one command and in a self-consistent manner, all combinations of E3SM global atmosphere<->ocean maps with both ERWG and Tempest. MWF-mode automates the laborious and error-prone process of generating numerous map-files with various switches. Its chief use occurs when developing and testing new global grid-pairs for the E3SM atmosphere and ocean components. Invoke MWF-mode with a number of specialized options to control the naming of the output map-files:
...