This page is devoted to instruction in ncremap. It describes steps necessary to create grids, and to regrid datasets between different grids with ncremap. Some of the simpler regridding options supported by ncclimo are also described at Generate, Regrid, and Split Climatologies (climo files) with ncclimo. This page describes those features in more detail, and other, more boutique features often useful for custom regridding solutions.
The Zen of Regridding
Most modern climate/weather-related research requires a regridding step in its workflow. The plethora of geometric and spectral grids on which model and observational data are stored ensures that regridding is usually necessary to scientific insight, especially the focused and variable resolution studies that E3SM models conduct. Why does such a common procedure seem so complex? Because a mind-boggling number of options are required to support advanced regridding features that many users never need. To defer that complexity, this HOWTO begins with solutions to the prototypical regridding problem, without mentioning any other options. It demonstrates how to solve that problem simply, including the minimal software installation required. Once the basic regridding vocabulary has been introduced, we solve the prototype problem when one or more inputs are "missing", or need to be created. The HOWTO ends with descriptions of different regridding modes and workflows that use features customized to particular models, observational datasets, and formats. The overall organization, including TBD sections (suggest others, or vote for prioritizing, below), is:
...
RRG-mode supports most normal ncremap options, including input and output methods and regridding algorithms.
Advanced Regridding III: Sub-Gridscale Regridding (SGS-mode)
ncremap has a sub-gridscale (SGS) mode that performs the special pre-processing and weighting necessary to conserve fields that represent fractional spatial portions of a gridcell, and/or fractional temporal periods of the analysis. Spatial fields output by most geophysical models are intensive, and so by default the regridder attempts to conserve the integral of the area times the field value such that the integral is equal on source and destination grids. However some models (like ELM, CLM, CICE, and MPAS-Seaice)
...
output gridcell values intended to apply to only a fraction sgs_frc (for "sub-gridscale fraction'') of the gridcell. The sub-gridscale fraction usually changes spatially with the distribution of land and ocean, and spatiotemporally with the distribution of sea ice and possibly vegetation. For concreteness consider a sub-grid field that represents the land fraction. Land fraction is less than one in gridcells that resolve coastlines or islands. ELM and CLM happily output temperature values valid only for a small (i.e., sgs_frc << 1) island within the larger gridcell. Model architecture dictates this behavior and savvy researchers expect it. The goal of the NCO weight-application algorithm is to treat SGS fields as seamlessly as possible so that those less familiar with sub-gridscale models can easily regrid them correctly.
Fortunately, models like ELM and CLM that run on the same horizontal grid as the overlying atmosphere can use the same mapping-file as the atmosphere, so long as the SGS weight-application procedure is invoked. Not invoking an SGS-aware weight application algorithm is equivalent to assuming sgs_frc = 1 everywhere. Regridding sub-grid values correctly versus incorrectly (e.g., with and without SGS-mode) alters global-mean answers for land-based quantities by about 1% for horizontal grid resolutions of about one degree. The resulting biases are in intricately shaped regions (coasts, lakes, sea-ice floes) and so are easy to overlook.
To invoke SGS mode and correctly regrid sub-gridscale data, specify the names of the fractional area sgs_frc and, if applicable, the mask variable sgs_msk (strictly, this is only necessary if these names differ from their respective defaults landfrac and landmask). Trouble will ensue if sgs_frc is a percentage or an absolute area rather than a fractional area (between zero and one). ncremap must know the normalization factor sgs_nrm by which sgs_frc must be divided (not multiplied) to obtain a true, normalized fraction. Datasets (such as those from CICE) that store sgs_frc in percent should specify the option --sgs_nrm=100 to instruct ncremap to normalize the sub-grid area appropriately before regridding. ncremap will re-derive sgs_msk based on the regridded values of sgs_frc: sgs_msk = 1 is assigned to destination gridcells with sgs_frc > 0.0, and all others sgs_msk = 0. As of NCO version 4.6.8 (released June, 2017), invoking any of the options --sgs_frc, --sgs_msk, or --sgs_nrm, automatically triggers SGS-mode, so that also invoking -P sgs is redundant though legal. As of NCO version 4.9.0 (released December, 2019), the values of the sgs_frc and sgs_msk variables should be explicitly specified. In previous versions they defaulted to landfrac and landmask, respectively, when -P sgs was selected. This behavior still exists but will likely be deprecated in a future version.
The area and sgs_frc fields in the regridded file will be in units of sterradians and fraction, respectively. However, ncremap offers custom options to reproduce the idiosyncratic data and metadata format of two particular models, ELM and CICE. When invoked with -P elm (or -P clm), a final step converts the output area from sterradians to square kilometers. When invoked with -P cice, the final step converts the output area from sterradians to square meters, and the output sgs_frc from a fraction to a percent.
# ELM/CLM: output "area" in [sr]
ncremap --sgs_frc=landfrac --sgs_msk=landmask in.nc out.nc
ncremap -P sgs in.nc out.nc # Deprecated in 4.9.0
# ELM/CLM pedantic format: output "area" in [km2]
ncremap -P elm in.nc out.nc # Same as -P clm, alm, ctsm
# CICE: output "area" in [sr]
ncremap --sgs_frc=aice --sgs_msk=tmask --sgs_nrm=100 in.nc out.nc
# CICE pedantic format: output "area" in [m2], "aice" in [%]
ncremap -P cice in.nc out.nc
# MPAS-Seaice: both commands are equivalent
ncremap -P mpasseaice in.nc out.nc
ncremap --sgs_frc=timeMonthly_avg_iceAreaCell in.nc out.nc
It is sometimes convenient to store the sgs_frc field in an external file from the field(s) to be regridded. For example, CMIP-style timeseries are often written with only one variable per file. NCO supports this organization by accepting sgs_frc arguments in the form of a filename followed by a slash and then a variable name:
ncremap --sgs_frc=sgs_landfrac_ne30.nc/landfrac -m map.nc in.nc out.nc
Files regridded using explicitly specified SGS options will differ slightly from those regridded using the -P elm or -P cice options. The former will have an area field in sterradians, the generic units used internally by the regridder. The latter produces model-specific area fields in square kilometers (for ELM) or square meters (for CICE), as expected in the raw output from these two models. To convert from angular to areal values, NCO assumes a spherical Earth with radius 6,371,220 m or 6,371,229 m, for ELM and CICE, respectively. The ouput sgs_frc field is expressed as a decimal fraction in all cases except for -P cice which stores the fraction in percent. Thus the generic SGS and model-specific convenience options produce equivalent results, and the latter is intended to be indistinguishable (in terms of metadata and units) to raw model output. This makes it more interoperable with many existing analysis scripts.
Advanced Regridding IV: Make All Weight Files (MWF-mode)
...