Overview

...

During the ACME era (c. 2015-16), we conducted extensive evaluation of AMWG, UV-CDAT, and NCO codes for generating and regridding climatology files (see /wiki/spaces/ATM/pages/29164477), we have and determined that NCO provides the most correct answers, has the best metadata, and is fastest. Until UV-CDAT bests NCO in these measures we advocate using NCO for creating climatologies(These conclusions did not consider any Xarray or cloud-based methodologies, and so perhaps should be revisited.) The NCO climatology and regridding software ncclimo and ncremap, respectively, have evolved, and maintained back-compatibility so they now support all component models of E3SM (and CESM), up to and including EAMxx and MALI.

In climatology generation mode, the NCO operator ncclimo ingests "raw" data consisting of interannual sets of files, each containing sub-daily (diurnal), daily, monthly, or yearly averages, and from these produces climatological daily, monthly, seasonal, and/or annual means. Alternatively, in timeseries reshaping (aka “splitter”) mode, ncclimo will subset and temporally split the input raw data timeseries into per-variable files spanning the entire period. ncclimo can will optionally regrid (by calling ncremap) all output files in either mode. The primary ncremap documentation is here. This presentation, given at the Albuquerque workshop on 20151104, conveys much of the information presented below, and some newer information, in a more graphical format. 

Prerequisites

...

Usencclimoif possible. It requires and comes with NCO version 4.6.0 and later.  Since about early 2018, the preferred way to obtain NCO for E3SM analysis is with the E3SM-Unified Conda package, which installs numerous analysis packages in a platform-independent manner and, as importantly, allows you to skip reading the rest of this paragraph. Those who need only NCO, or who wish to avoid Conda, should read-on. The newest versions of NCO are installed on rhea.ccs.ornl.gov at ORNL, blues/anvil.lcrc.anl.gov: at ANL, cori.nersc.gov (NERSC), aims4.llnl.gov (LLNL), cheyenne.ucar.edu (NCAR), and compy.pnl.gov (PNNL). The ncclimo and ncremap scripts should find the latest versions automatically, and not require any module or path changes. To use other (besides the ncclimo and ncremap scripts) NCO executables from the command-line or from your own scripts may require loading modulesall major DOE supercomputers in C. Zender’s home directory (usually ~zender/[bin,lib]), and semi-recent versions are sometimes available as machine modules (e.g., module load nco). This is site-specific and not under my (CZ's) control. You might check that the default NCO is recent enough (try module load nco, then ncks --version) or use developers' executables/libraries (in ~zender/[bin,lib] on all machines). Follow these directions on the NCO homepage to install on your own machines/directories. It can be as easy as apt-get install nco, dnf install nco, or conda install -c conda-forge nco, or you can build/install from scratch with configure;make;make install. 

...