This page is devoted to instruction in ncremap. It describes steps necessary to create grids, and to regrid datasets between different grids with ncremap. Some of the simpler regridding options supported by ncclimo are also described at Generate, Regrid, and Split Climatologies (climo files) with ncclimo. This page describes those features in more detail, and other, more boutique features often useful for custom regridding solutions.
The Zen of Regridding
Most modern climate/weather-related research requires a regridding step in its workflow. The plethora of geometric and spectral grids on which model and observational data are stored ensures that regridding is usually necessary to scientific insight, especially the focused and variable resolution studies that E3SM models conduct. Why does such a common procedure seem so complex? Because a mind-boggling number of options are required to support advanced regridding features that many users never need. To defer that complexity, this HOWTO begins with solutions to the prototypical regridding problem, without mentioning any other options. It demonstrates how to solve that problem simply, including the minimal software installation required. Once the basic regridding vocabulary has been introduced, we solve the prototype problem when one or more inputs are "missing", or need to be created. The HOWTO ends with descriptions of different regridding modes and workflows that use features customized to particular models, observational datasets, and formats. The overall organization, including TBD sections (suggest others, or vote for prioritizing, below), is:
...
ncremap -m map.nc dat_src.nc dat_rgr.nc # Integral-preserving treatment of missing data is used by default
ncremap --preserve=integral -m map.nc dat_src.nc dat_rgr.nc # Explicitly specifiedspecify integral-preserving treatment
ncremap --preserve=mean -m map.nc dat_src.nc dat_rgr.nc # Explicitly specifiedspecify mean-preserving treatment
By default, ncremap implements the integral-preserving, conservative approach because it has useful properties, is simpler to understand, and requires no additional parameters. However, this approach will often produce unrealistic gridpoint values (e.g., a temperature of 10 Kocean temperatures < 100 K and that is not a typo) values along coastlines or near data gaps where state variables are regridded to/from small fractions of a gridcell. The mean-preserving approach solves this problem, yet incurs others, like non-conservation. The mean-preserving approach (aka re-normalization) ensures that the output values are physically consistent and do not stick-out like sore thumbs in a plot, although the integral of their value times area is not conserved.
To employ the renormalization algorithm instead, supply the threshold weight for valid destination values with The third option, the "sliding scale" approach, is slightly more complex and much more flexible than the all-or-nothing integral- or mean-preserving examples above. The sliding-scale refers to the fraction of the destination gridcell that must be overlapped by valid source gridcells. Users can specify the renormalization threshold weight rnr_thr which is the required valid fraction of destination gridcells with the renormalization threshold option "--rnr_thr=rnr_thr". Valid values must cover The weight-application algorithm then ensures that valid values overlap at least the fraction rnr_thr of the each destination gridcell for that gridcell to meet the threshold for a non-missing destination value. When rnr_thr is exceeded, the mean valid value is renormalized by in the valid area and is placed in the destination gridcell. If the valid area covers less than rnr_thr, then the destination gridcell is assigned the missing value. Valid values of rnr_thr range from zero to one. A threshold weight of rnr_thr = 0.0 indicates that any amount (no matter how small) of valid data should be represented by its mean value on the output grid. Keep in mind though, that this threshold is potentially a divisorthe actual valid area divides a sum (the area-weighted integral of the valid values), and values of zero or very near to zero in the divisor can lead to floating-point underflow and divide-by-zero errors. For convenience, ncremap permits users to specify a threshold weight of . A threshold weight of rnr_thr = 1.0 indicates that the entire destination gridcell (to within machine precision) must be overlapped by valid data in order for that destination gridcell to be assigned a valid (non-missing) value. Threshold weights 0.0 < rnr_thr < 1.0 invoke the real-power of the sliding scale. Remote sensing classification schemes applied to L2 data may require, for example, valid retrievals over at least 50% of source pixels before assigning a valid value to a destination gridcell. This is equivalent to rnr_thr = 0.5. And so,
ncremap --rnr_thr=0.
...
ncremap25 -m map.nc dat_src.nc dat_rgr.nc # Conservative treatment of missingSliding scale: Destination value preserves mean of input data, requires at least 25% valid input data
ncremap --rnr_thr=0.150 -m map.nc dat_src.nc dat_rgr.nc # Renormalized treatment of missing data
...
Sliding scale: Destination value preserves mean of input data, requires at least 50% valid input dataThese sliding scale examples specify that valid values must cover at least 10% 25% and 50% of the destination gridcell to meet the threshold for a non-missing destination value. With actual valid destination areas of , say 25% or 50%, the renormalized this approach algorithm would produce destination values greater than the conservative algorithm by factors of four or and two, respectively.
In practice, it may make sense to use the default conservative treatment with conservative regridding, and the renormalized treatment with other (non-conservative) regridding algorithms such as bilinear interpolation or nearest-neighbor. Another consideration is whether the fields being regridded are fluxes or state variables. For example, temperature (unlike heat) and concentrations (amount per unit volume) are not physically conserved quantities under areal-regridding so it often makes sense to interpolate them in a non-conservative fashion, to preserve their fine-scale structure. Few researchers can digest the unphysical values of temperature that the conservative treatment produces in regions rife with missing values. On the other hand, mass and energy fluxes should be physically conserved under areal-regridding. One should consider both the type of field and its conservation properties when choosing a missing value treatment.
...