...
As of NCO version 4.9.2 (March, 2020), adding the --frac_b_nrm flag changes the map-checker into a read-write algorithm that first diagnoses the map-file statistics described above and then re-writes the weights (and weight-derived statistics frac_a and frac_b) to compensate or "fix" issues that poor-quality input grids can cause. Input grids can and often do have regions that are not tiled by any portion of any input gridcell. For example, many FV ocean grids (such as MPAS) are empty (have no gridcells) in land regions beyond the coasts. Some FV ocean grids have gridcells everywhere and mask (i.e., screen-out) the non-ocean gridcells by setting the mask value to zero. Both these designs are perfectly legal. What is illegal, yet sometimes encountered in practice, is overlapping gridcells on the same input grid. Such an input grid is said to be self-overlapping.
Documentation above this line is generic to all map-files, and documentation below describes specifically applies only to map-files constructed with one particular grid, HYDRO1K. Would it be more logical and easier to digest if we separated the HYDRO1K documentation below into its own sub-page?
The surface topography dataset grid SCRIPgrid_1km-merge-10min_HYDRO1K-merge-nomask_c130402.nc (hereafter the HYDRO1K grid or hydro1k.nc for short) used by E3SM and CESM is self-overlapping. It contains numerous (about 17) quadrilateral gridcells that partially overlap one another. These are cells of differing orientations and sizes that seem to originate from different measurements. They are not repeated or duplicate gridcells. Weight-generators that receive the same input area twice might (if they do not take precautions to idenfity the issue, which no known weight-generators do) double-weight the self-overlapped region(s). In other words, self-overlapping input grids can lead weight-generators to produce values frac_b >> 1.0. Applying these weights would lead to exaggerated values on the destination grid.
Before describing the NCO method to address self-overlapping grids, we should mention that the HYDRO1k grid also contains one gridcell whose vertices are stored in clockwise (CW) not counter-clockwise (CCW) order. This gridcell violates the CCW ordering convention followed by all known weight generators and can lead to failture failure and carnage if not corrected. The practical solution is to re-order the vertices in that gridcell, thereby creating a modified input grid that is completely CCW (yet still suffers from self-overlapping gridcells):
ncap2 -O --no_tmp_fl -s 'grid_corner_lon(50085604,:)={100.024421691894531,100.024421691894531,100.012207031250000,100.012207031250000};grid_corner_lat(50085604,:)={42.598628997802734,42.589633941650391,42.589637756347656,42.598632812500000};' hydro1k.nc hydro1k_ccw.nc
NCO commands to produce maps that convert HYDRO1K elevation data (on the fixed, pure-CCW grid) to any desired output grid should include appropriate options for niceness, format, and speed. First, the HYDRO1K grid is memory-intensive. Map-files based For complex reasons, the map HYDRO1K to ne1024np4 requires the Git code-branch hmb-hydro2-fix (located at https://github.com/nco/nco/tree/hmb-hydro2-fix). The master NCO branch is fine for all other target grids, including ne512pg2 and ne1024pg2.
NCO commands to produce maps that convert HYDRO1K elevation data (on the fixed, pure-CCW grid) to any desired output grid should include appropriate options for niceness, format, and speed. First, the HYDRO1K grid is memory-intensive. Map-files based on it must be created on a node with plenty of RAM. For example, the HYDRO1K-to-ne1024np4 map requires about 200 120 GB RAM (if the SCRIP overlap mesh is NOT requested). Etiquette requires lowering the priority of memory and CPU-intensive jobs with the nice command on shared resources such as login-nodes. Second, the output weight and vertice arrays may exceed the capacities of the netCDF3 classic and 64-bit offset formats so we recommend setting the output format to CDF5 or to netCDF4-classic with the -5 or -7 switches, respectively. Third, the weight-generation code is threaded and our preliminary benchmarks show that it can scale well up to at least six threads. These options yield commands of the form
...
With six threads on our desktop benchmarking Linux box (e3sm.ess.uci.edu) the command above completes in just under three four hours. Such maps will suffer in correctness due to self-overlapping input gridcells as discussed above. The map-checker helps diagnose the extent of the self-overlapping issue:
...
SCRIPgrid_1km-merge-10min_HYDRO1K-merge-nomask_c130402.ncSCRIPgrid_1km-merge-10min_HYDRO1K-merge-nomask_c130402_ccw.ncmap_hydro1k_to_ne1024np4_nco.20200210.ncmap_hydro1k_to_ne1024np4_nco.20200210_fix.nc
20200423 Update: While writing this documentation we discovered a regression in the NCO weight-generation that produces more (~100+) frac_b WARNINGS than we found in February. Henry Butowsky (Unlicensed) is working to understand and fix the cause of the regression, and will post another update here once more progress is made20200710: Update
For complex reasons the map HYDRO1K to ne1024np4 requires the Git code-branch hmb-hydro2-fix (located at https://github.com/nco/nco/tree/hmb-hydro2-fix). The master NCO branch is fine for all other target grids, including ne512pg2 and ne1024pg2.
20200423: Update:
Ben Hillman this documentation may be useful in constructing high-resolutions ELM IC’s.