Page Comparison

This page is devoted to instruction in ncremap. It describes steps necessary to create grids, and to regrid datasets between different grids with ncremap. Some of the simpler regridding options supported by ncclimo are also described at Generate, Regrid, and Split Climatologies (climo files) with ncclimo. This page describes those features in more detail, and other, more boutique features often useful for custom regridding solutions.

The Zen of Regridding

Most modern climate/weather-related research requires a regridding step in its workflow. The plethora of geometric and spectral grids on which model and observational data are stored ensures that regridding is usually necessary to scientific insight, especially the focused and variable resolution studies that E3SM models conduct. Why does such a common procedure seem so complex? Because a mind-boggling number of options are required to support advanced regridding features that many users never need. To defer that complexity, this HOWTO begins with solutions to the prototypical regridding problem, without mentioning any other options. It demonstrates how to solve that problem simply, including the minimal software installation required. Once the basic regridding vocabulary has been introduced, we solve the prototype problem when one or more inputs are "missing", or need to be created. The HOWTO ends with descriptions of different regridding modes and workflows that use features customized to particular models, observational datasets, and formats. The overall organization, including TBD sections (suggest others, or vote for prioritizing, below), is:

...

Advanced procedures have in common that they activate non-standard processing modes for ncremap. These modes do something different, or in addition to, the standard weight-generation and application. Generally these modes were created in order to automate frequently recurring workflows that can leverage the ncremap infrastructure so long as various bells and whistles are introduced along the way. Please let us know if you have ideas for new or improved processing modes.

...

MPAS models produce output in their own format distinct from CESM-heritage models. MPAS-mode invokes three pre-processing steps to massage MPAS datasets until they are amenable to regridding. These steps include missing value annotation, missing value treatment, and dimension permutation. We will shortly describe these steps in order. First, though, MPAS-mode, like most other ncremap modes, is explicitly invoked with the -P md_nm option where the md_nm is the mode name, i.e.,some variation of the MPAS component model name:

ncremap -P mpas -m map.nc dat_src.nc dat_rgr.nc # Generic MPAS mode, imperfect, prefer component-specific mode names
ncremap -P mpasocean -m map.nc dat_src.nc dat_rgr.nc # Important for correct vertical interpolation
ncremap -P mpasseaice -m map.nc dat_src.nc dat_rgr.nc # Automatically weights by timeMonthly_avg_iceAreaCell
ncremap -P mali -m map.nc dat_src.nc dat_rgr.nc # Automatically uses MALI _FillValue

Many model and observational datasets use missing values that are not annotated in the standard manner. For example, the MPAS ocean and ice models use -9.99999979021476795361e+33 as the missing value, yet (at least from 2015-2018) do not store this in a _FillValue attribute with any variables. To prevent arithmetic from treating these values as valid, MPAS-mode automatically puts this value in a _FillValue attribute for all doublefloating-precision point variables via

ncatted -t -a _FillValue,,o,d,-9.99999979021476795361e+33 -a _FillValue,,o,f,-9.99999979021476795361e+33 dat_src.nc

Oddly, the MPAS land-ice model uses -1.0e36 for missing values, so currently MPAS-LI users must explicitly supply this missing value to ncremap:, or invoke ncremap with the -P mali option

ncremap -P mpas --mss_val=-1.0e36 -m map.nc dat_src.nc dat_rgr.nc # Explicitly supply missing value
ncremap -P mali -m map.nc dat_src.nc dat_rgr.nc # Let ncremap know dataset is from MALI

Next, MPAS datasets usually have masked regions (e.g., non-ocean cells) yet MPAS users like to visualize regridded data with realistic (not blocky) boundaries along those cells and so they decided to, by default, treat missing values with the renormalization approach described above in the section on Treatment of Missing Values. Hence MPAS-mode automatically invokes ncremap with maximum renormalization, equivalent to

...

Finally, ncremap requires the horizontal spatial dimension(s), whether latitude and longitude or some unstructured dimension, to be the final (most-rapidly-varying) dimension(s) of input datasets. MPAS datasets natively place their spatial dimension (typically nCells) closer to the least-rapidly-varying position. While this makes perfect sense from an I/O-efficiency point-of-view for unstructured models, it does not play well with regridders. Hence all MPAS-mode permutes modes permute the input dimensions to a regridder-friendly order (i.e., ending with nCells) with a command of the form

ncpdq -a Time,nVertLevels,nVertLevelsP1,maxEdges,MaxEdges2,nCategories,ONE,nEdges,nCells dat_src.nc dat_tmp.nc

...