...
| Code Block |
|---|
ncclimo -P mpasocean -s 1980 -e 1983 -i drc$drc_in -o drc$drc_out # MPAS-O ncclimo -P mpasseaice -s 1980 -e 1983 -i drc$drc_in -o drc$drc_out # MPAS-I |
Some data are best evaluated with custom-defined seasons, e.g., JFM instead of DJF, or two-month seasons such as FM or ON. ncclimo supports up to eleven (and counting) seasons, although by default it only computes MAM, JJA, SON, and DJF. As of NCO 4.6.8, use the --seasons (or --csn) option to specify additional or alternate seasons:
| Code Block |
|---|
ncclimo -mP mpascicempasseaice --seasons=jfm,jas,ann -s 1980 -e 1983 -i drc$drc_in -o drc$drc_out # MPAS-I |
The climatological annual mean, ANN, is also computed automatically when MAM, JJA, SON, and DJF are all requested (which is the default, so ANN is always computed by default). Use –-seasons=none to completely turn-off seasonal and annual-mean climatologies.
MPAS climos are unaware of missing values until/unless the input files are "fixed". We recommend that the person who produces the simulation annotate all floating point variables with the appropriate _FillValue prior to invoking ncclimo. Run something like this once in the history file directory:
...
If/when MPAS generates the _FillValue attributes itself, this step can and should be skipped (MPAS developers: please let Charlie Zender know when MPAS “fixes” this “feature”). All other ncclimo ncclimo features like regridding (below) are invoked identically for MPAS as for EAM/ELM users although under-the-hood ncclimo does do some special ncremap (if invoked) specially pre-processing processes (dimension permutation, metadata annotation) for MPAS . A five-year oEC60to30 MPAS-O climo with regridding to T62 takes < 10 minutes on rhea.data.
High Frequency climos (produce monthly, seasonal, and annual climatological means of diurnal cycles from diurnally resolved input data)
As of NCO 4.9.4 (September, 2020), ncclimowill can produce climatologies that retain the diurnal cycle resolution provided by the input data. These “high frequency climos” are useful for characterizing the diurnal cycle of processes typically retained in EAM/ELM h1-h4 history outputfiles, high-frequency observational analyses (e.g., MERRA2, ERA5), and similar data. In all respects except two, high frequency climo features are invoked and controlled by the same options as traditional climo generation from monthly mean input. The most significant difference is that the user must supply the filenames of high-frequency input data via any of the four methods outlined above for splitting. High-frequency climo input dataset names are too complex for ncclimoto automatically generate (as it does for monthly-mean input), so one must supply the names via standard input, positional arguments, or filename globbing, or directory location exactly as for splitter mode described above. The second difference is that the user must supply the --clm_md=hfc option to tell ncclimo to operate in climo-generation rather than splitter mode:
| Code Block |
|---|
ncclimo -P eam --clm_md=hfc --yr_srt=1 --yr_end=250 --var=FSNT,AODVIS --map=$map_fl --drc_out=$drc_out < input_list ls *cam.h0.0[012]??* | ncclimo --P eam --clm_md=hfc --yr_srt=1 --yr_end=250 --var=FSNT,AODVIS --map=$map_fl --drc_out=$drc_out ncclimo -P eam --clm_md=hfc --var=FSNT,AODVIS --yr_srt=1 --yr_end=250 --map=$map_fl --drc_out=$drc_out $drc_in/*cameam.h0h4.0[012]??*.nc ncclimo -P eam --clm_md=hfc --var=T,Q,RH --yr_srt=1 --yr_end=250 --drc_in=$drc_in --map=$map_fl --drc_out=$drc_out |
In high-frequency mode, ncclimoautomatically determines the number of timesteps per day (which must be an integer >= 1). In high-frequency mode the --caseid option is optional since the user provides all the input filenames. If provided, caseid is used to rename the output filenames (much similar to the same as the --fml_nm option).
Annual climos (produce climotological means from annual-mean input data)
Not all model or observed history files are created as monthly means. To create a climatological annual mean from a series of annual mean inputs (such as from a land ice model), select ncclimo's annual climatology mode with the -C ann (or --clm_md=ann) option:
...
ncclimo
...
-P cism --clm_md=ann -h h -c caseid -s 1851 -e 1900 -i $drc_in -o $drc_out
The options -m mdl_nm and -h hst_nm (that default to "cameam" and "h0", respectively) tell ncclimo how to construct the input filenames. The above formula names the files caseid.cism.h.1851-01-01-00000.nc, caseid.cism.h.1852-01-01-00000.nc, and so on. Annual climatology mode produces a single output file (or two if regridding is selected), and in all other respects behaves the same as monthly climatology mode.
Daily climos (interannual day-of-year statistics from multi-year daily-to-diurnally-resoluved input data)
High frequency timeseries are often available as daily means. To create a climatological daily mean from a series of daily mean inputs, select daily mode with the "--clm_md=dly" option. What is computed? In daily mode, ncclimo produces 365 output files, each consisting of the interannual average of the given day-of-year. If the input data are at sub-daily resolution (e.g., 8 timesteps per day for three hourly data), then by default all timesteps in a day are averaged into the daily output. An option to preserve the diurnal cycle for each day is in the works, contact Charlie Zender if interested. Also on the drawing board is a simple method to obtain min/max/avg etc. at the same time, or at least with the same command.Daily output timeseries are often divided amongst files with hard-to-predict names, e.g., because the number of days in a
Daily output timeseries are often divided amongst files with hard-to-predict names, e.g., because the number of days in a file, days in a month, and timesteps in a day may all vary. Thus ncclimo in daily mode requires the user to supply the input filenames. ncclimo will not construct input filenames itself in daily mode (unlike monthly or annual mode). Use the same filename input methods that the splitter mode (above) accepts. Putting it all together, produce a daily climatology with something like
...
cd
...
${DATA}/ne30/raw;ls
...
h1
...
.nc
...
|
...
ncclimo -P eam --clm_md=dly --job_nbr=8
...
--caseid=famipc5_ne30_v0.3_00007
...
--yr_srt=2001
...
--yr_end=2009
...
--var=PRECT,TREFHT
...
--drc_out=${DATA}/ne30/clm
The "--job_nbr" option tells how many days to independently compute at one time on a given node. See the NCO manual for more details.
Regridding (climos and other files)
...
Regridding is a standalone operation carried out by ncremap. See the full ncremap documentation for examples of standalone operation (including MPAS!). For convenience, ncclimo will (optionally) call ncremap to regrid When given the --map (or -r) option, ncclimo calls ncremap during climatology generation to produce climatology files on both the native and desired analysis grids. Only the ncremap features most relevant to ncclimo are described here. Regridding while producing climos is virtually free, because it is performed on idle nodes/cores after the monthly climatologies have been computed and while the seasonal climatologies are being computed. (This load“load-balancing balancing” can save half an hour on ne120 datasets.) To regrid, simply pass the desired mapfile name with "-r map.nc", e.g., "--r $map=${DATA}/maps/map_ne120np4_to_fv257x512_aave.20150901.nc".
Specifying '-O drc$drc_rgr' (NB: uppercase letter "O") causes ncclimo to place the regridded files in the directory ${drc_rgr}. These files have the same names as the native grid climos from which they were derived. There is no namespace conflict because they are in separate directories. These files also have Until ~2020, symbolic links to their AMWG filenames were also created by default. If 'This can be manually enabled with --amwg_lnk. If -O drc$drc_rgr' is not specified, ncclimo places all regridded files in the native grid climo output directory , ${drc_out}, specified by '-o drc$drc_out' (NB: lowercase letter "o") . To avoid namespace conflicts when both climos are stored in the same directory, the names of the regridded files are suffixed by affixed with the destination geometry string obtained derived from the mapfile, e.g., '*_climo_fv257x512_bilin.nc'. These files also have symbolic links to their AMWG filenames.
...
ncclimo
...
-P eam -c
...
famipc5_ne30_v0.3_00003
...
-s
...
1980
...
-e
...
1983
...
-i
...
$drc_in
...
-o
...
$drc_out
...
ncclimo
...
-P eam -c
...
famipc5_ne30_v0.3_00003
...
-s
...
1980
...
-e
...
1983
...
-i
...
$drc_in
...
-o $drc_out -r
...
$map_file
...
...
ncclimo
...
-P eam -c
...
famipc5_ne30_v0.3_00003
...
-s
...
1980
...
-e
...
1983
...
-i
...
$drc_in
...
-o $drc_out -r
...
$map_file
...
-O
...
$drc_rgr
The above commands perform a climatology without regridding, then with regridding (all climos stored in ${drc_out}), then with regridding and storing regridded files separatelystoring regridded files separately (in ${drc_rgr}). Paths specified by $drc_in, $drc_out, and $drc_rgr may be relative or absolute. An alternative to regridding during climatology generation is to manually regrid afterwards with ncremap, which has more specialized features built-in for regridding. To use ncremap to regrid a climatology in $drc_out and place the results in $drc_rgr, use something like
| Code Block |
|---|
ncremap --map=map.nc -I drc$drc_out -m map.nc -O drc$drc_rgr |
| Code Block |
|---|
ls drc_out/*climo* | ncremap --m map=map.nc -O drc$drc_rgr |
As of 20170526 and version 4.6.7, ncremap supports sub-gridscale (SGS) regridding. Though designed for ELM and MPAS-Seaice, this feature is configurable for other SGS datasets as well. In sub-grid mode, ncremap performs substantial pre- and post-processing so that regridding conserves fields that may represent only a fraction of the entire gridcell. The sub-gridscale fraction represented by each field is contained in a variable (set with the option --sgs_frc) whose default is landfrac. SGS mode eases regridding of datasets (e.g., from ELM, CLM, CICE, MPAS-Seaice) that output data normalized to a gridcell fraction rather than to its entire extent. SGS mode automatically derives new binary masks (--sgs_msk, defaults to landmask) and allows for additional normalization (--sgs_nrm). Specific flavors of SGS can be selected (with -P elm, or -P clm, -P cice, or mpas-seaiceP mpasseaice). These ensure regridded datasets recreate the idiosyncratic units (e.g., %, km2) employed by raw ELM, CLM, CICE, and MPAS-Seaice model output.
...
ncremap
...
-P
...
elm
...
--
...
map=map_ne30pg2_to_cmip6_180x360_nco.20200901.nc
...
elm_in.nc
...
output.nc
...
ncremap
...
-P
...
elm
...
-s
...
src_grd.nc
...
-d
...
1x1.nc
...
elm_data.nc
...
output.nc
...
ncremap
...
-
...
P mpasseaice
...
--map=map_oEC60to30v3_to_cmip6_180x360_aave.20181001.nc
...
mpasseaice_in.nc
...
output.nc
...
ncremap
...
-P
...
mpasseaice
...
-s
...
src_grd.nc
...
-d
...
1x1.nc
...
mpasseaice_data.nc
...
output.nc
Full documentation on SGS mode is here: http://nco.sf.net/nco.html#sgs. Note that ncclimo does not (yet anyway) call ncremap with the SGS option. One gets SGS features only by manually invoking ncremap as above. This is mainly because parallelization with SGS is much more efficient with standalone ncremap than through ncclimo. We solicit your feedback on SGS behavior and future features...
...
Coupled Simulations
ncclimo works on all E3SM component models, including the coupler. It can simultaneously generate climatologies for a coupled run, where climatologies mean both native and regridded monthly, seasonal, and annual averages as per the AG specification. Here are template commands for a recent to fully climatologize and regrid a coupled simulation:
| Code Block |
|---|
caseid=20160121.A_B2000ATMMOD.ne30_oEC.titan.a00
drc_in=/lustre/atlas1/cli112/proj-shared/golaz/ACME_simulations/20160121.A_B2000ATMMOD.ne30_oEC.titan.a00/run
map_atm=${DATA}/maps/map_ne30np4_to_fv129x256_aave.20150901.nc
map_lnd=$map_atm
map_ocn=${DATA}/maps/map_oEC60to30v3_to_cmip6_180x360_aave.20181001.nc
map_ice=$map_ocn |
| Code Block |
|---|
ncclimo -P eam -p mpi -c ${caseid} -m cam -s 2 -e 5 -i $drc_in -r $map_atm -o ${DATA}/acme/atm ncclimo -P elm -c ${caseid} -m clm2 -c ${caseid} -s 2 -e 5 -i $drc_in -r $map_lnd -o ${DATA}/acme/lnd ncclimo -pP mpimpasocean -p mpi -m mpas -s 2 -e 5 -i $drc_in -r $map_ocn -o ${DATA}/acme/ocn ncclimo -P mpasseaice -m mpasseaice -s 2 -e 5 -i $drc_in -r $map_ice -o ${DATA}/acme/ice |
The atmosphere and ocean model output is significantly larger than the land and ice model output. These commands recognize that by using different parallelization strategies that may (rhea standard queue) or may not (cooley or rhea bigmem queue) be required, depending on the RAM fatness of the analysis nodes, as explained below. As of late 2016 (and NCO v. 4.6.3-alpha03), the MPAS models do not utilize the $caseid option. They use their own , evolving MPAS naming convention. When fed the '-m' options shown aboveBy default, ncclimo processes the MPAS "hist.am.timeSeriesStatsMonthly" analysis members.
Extended climos:
...