Page Comparison

...

MPAS O/I considerations:

MPAS ocean and ice models currently have their own (non-CESM'ish) naming convention that guarantees output files have the same names for all simulations. By default ncclimo analyzes the "timeSeriesStatsMonthly" analysis member output (tell CZ if you want options for other AM output). ncclimo recognizes input files as being MPAS-style when invoked with "-m mpaso" or "-m mpascice" like this:

ncclimo -m mpaso    -s 1980 -e 1983 -i drc_in -o drc_out # MPAS-O

ncclimo -m mpascice -s 1980 -e 1983 -i drc_in -o drc_out # MPAS-I

MPAS climos are unaware of missing values until/unless the input files are "fixed". I recommend that the person who produces the simulation annotate all floating point variables with the appropriate _FillValue prior to invoking ncclimo. Run something like this once in the history file directory:

for fl in `ls hist.*` ; do
  ncatted -O -t -a _FillValue,,o,d,-9.99999979021476795361e+33 ${fl}
done

If/when MPAS O/I generates the _FillValue attributes itself, this step can and should be skipped. All other ncclimo features like regridding (below) are invoked identically for MPAS as for CAM/CLM users although under-the-hood ncclimo does do some special pre-processing (dimension permutation, metadata annotation) for MPAS. A five-year oEC60to30 MPAS-O climo with regridding to T62 takes < 10 minutes on rhea.

Annual climos (produce annual climos from annual input files):

Not all model or observed history files are created as monthly means. To create a climatological annual mean from a series of annual mean inputs, select ncclimo's annual climatology mode with the "-C ann" option:

...

Timeseries Reshaping mode, aka Splitting:

As of version 4.6.4, ncclimo will automatically switch to timeseries reshaping mode if it receives a list of files through a pipe to stdin, or, alternatively, placed as positional arguments (after the last command-line option), or if neither of these is done and no caseid is specified, in which case it assumes all *.nc files in drc_in constitute the input file list. These examples invoke reshaping mode in the three possible ways:

drc_in=/scratch2/scratchdirs/golaz/ACME_simulations/20161117.beta0.A_WCYCL1850S.ne30_oEC_ICG.edison/run
map_fl=${DATA}/maps/map_ne30np4_to_fv129x256_aave.20150901.nc
# Pipe list to stdin
cd $drc_in
ls *cam.h0.0[012]??* | ncclimo --dbg=0 --yr_srt=1 --yr_end=250 --var=FSNT,AODVIS --map=${map_fl} --drc_out=$drc_out
# List as positional arguments
ncclimo -v FSNT,AODVIS -s 1 -e 9 -o $drc_out $drc_in/*mdl*000[1-9]*.nc
# Read directory
ncclimo -v T,Q,RH -s 1 -e 9 -i $drc_in -o $drc_out

Assuming each input file is a monthly average comprising the variables T, Q, and RH, then the output will be FSNT_000101_000912.nc and AODVIS_000101_000912.nc. ncclimo reshapes the input so that the outputs are continuous timeseries of each variable taken from all input files.  When necessary, the output is split into segments each containing no more than ypf_max (default 50) years-per-file, i.e., FSNT_000101_005012.nc, FSNT_005101_009912.nc, FSNT_010001_014912.nc, etc. Change the maximum number of years-per-output-file with the --ypf_max=ypf_max option. The remainder of this documentation refers to climatology-generation mode.

MPAS O/I considerations:

MPAS ocean and ice models currently have their own (non-CESM'ish) naming convention that guarantees output files have the same names for all simulations. By default ncclimo analyzes the "timeSeriesStatsMonthly" analysis member output (tell CZ if you want options for other AM output). ncclimo recognizes input files as being MPAS-style when invoked with "-m mpaso" or "-m mpascice" like this:

ncclimo -m mpaso    -s 1980 -e 1983 -i drc_in -o drc_out # MPAS-O

ncclimo -m mpascice -s 1980 -e 1983 -i drc_in -o drc_out

...

Regridding (climos and other files):

Regridding is a standalone operation carried out by ncremap. See the full ncremap documentation for examples of standalone operation (including MPAS!). For convenience, ncclimo will (optionally) call ncremap to regrid during climatology generation to produce climatology files on both the native and desired analysis grids. Only the ncremap features most relevant to ncclimo are described here. Regridding while producing climos is virtually free, because it is performed on idle nodes/cores after the monthly climatologies have been computed and while the seasonal climatologies are being computed. This load-balancing can save half an hour on ne120 datasets. To regrid, simply pass the desired mapfile name with "-r map.nc", e.g., "-r ${DATA}/maps/map_ne120np4_to_fv257x512_aave.20150901.nc". Although this should not be necessary for normal use, you may pass any options specific to regridding with "-R opt1 opt2". 

Specifying '-O drc_rgr' (NB: uppercase "O") causes ncclimo to place the regridded files in the directory ${drc_rgr}. These files have the same names as the native grid climos from which they were derived. There is no namespace conflict because they are in separate directories. These files also have symbolic links to their AMWG filenames. If '-O drc_rgr' is not specified, ncclimo places all regridded files in the native grid climo output directory, ${drc_out}, specified by '-o drc_out' (NB: lowercase "o") . To avoid namespace conflicts when both climos are stored in the same directory, the names of the regridded files are suffixed by the destination geometry string obtained from the mapfile, e.g., '*_climo_fv257x512_bilin.nc'. These files also have symbolic links to their AMWG filenames.

ncclimo -c famipc5_ne30_v0.3_00003 -s 1980 -e 1983 -i drc_in -o drc_out

ncclimo -c famipc5_ne30_v0.3_00003 -s 1980 -e 1983 -i drc_in -r map_file -o drc_out

ncclimo -c famipc5_ne30_v0.3_00003 -s 1980 -e 1983 -i drc_in -r map_file -o drc_out -O drc_rgr

ncremap -I drc_out -m map.nc -O drc_rgr

ls drc_out/*climo* | ncremap -m map.nc -O drc_rgr

Timeseries Reshaping mode, aka Splitting:

As of version 4.6.4, ncclimo will automatically switch to timeseries reshaping mode if it receives a list of files through a pipe to stdin, or, alternatively, placed as positional arguments (after the last command-line option), or if neither of these is done and no caseid is specified, in which case it assumes all *.nc files in drc_in constitute the input file list. These examples invoke reshaping mode in the three possible ways:

drc_in=/scratch2/scratchdirs/golaz/ACME_simulations/20161117.beta0.A_WCYCL1850S.ne30_oEC_ICG.edison/run
map_fl=${DATA}/maps/map_ne30np4_to_fv129x256_aave.20150901.nc
# Pipe list to stdin
cd $drc_in
ls *cam.h0.0[012]??* | ncclimo --dbg=0 --yr_srt=1 --yr_end=250 --var=FSNT,AODVIS --map=${map_fl} --drc_out=$drc_out
# List as positional arguments
ncclimo -v FSNT,AODVIS -s 1 -e 9 -o $drc_out $drc_in/*mdl*000[1-9]*.nc
# Read directory
ncclimo -v T,Q,RH -s 1 -e 9 -i $drc_in -o $drc_out

...

 # MPAS-I

MPAS climos are unaware of missing values until/unless the input files are "fixed". I recommend that the person who produces the simulation annotate all floating point variables with the appropriate _FillValue prior to invoking ncclimo. Run something like this once in the history file directory:

for fl in `ls hist.*` ; do
  ncatted -O -t -a _FillValue,,o,d,-9.99999979021476795361e+33 ${fl}
done

If/when MPAS O/I generates the _FillValue attributes itself, this step can and should be skipped. All other ncclimo features like regridding (below) are invoked identically for MPAS as for CAM/CLM users although under-the-hood ncclimo does do some special pre-processing (dimension permutation, metadata annotation) for MPAS. A five-year oEC60to30 MPAS-O climo with regridding to T62 takes < 10 minutes on rhea.

Annual climos (produce annual climos from annual input files):

Not all model or observed history files are created as monthly means. To create a climatological annual mean from a series of annual mean inputs, select ncclimo's annual climatology mode with the "-C ann" option:

ncclimo -C ann -m cism -h h -c caseid -s 1851 -e 1900 -i drc_in -o drc_out

Regridding (climos and other files):

Regridding is a standalone operation carried out by ncremap. See the full ncremap documentation for examples of standalone operation (including MPAS!). For convenience, ncclimo will (optionally) call ncremap to regrid during climatology generation to produce climatology files on both the native and desired analysis grids. Only the ncremap features most relevant to ncclimo are described here. Regridding while producing climos is virtually free, because it is performed on idle nodes/cores after the monthly climatologies have been computed and while the seasonal climatologies are being computed. This load-balancing can save half an hour on ne120 datasets. To regrid, simply pass the desired mapfile name with "-r map.nc", e.g., "-r ${DATA}/maps/map_ne120np4_to_fv257x512_aave.20150901.nc". Although this should not be necessary for normal use, you may pass any options specific to regridding with "-R opt1 opt2". 

Specifying '-O drc_rgr' (NB: uppercase "O") causes ncclimo to place the regridded files in the directory ${drc_rgr}. These files have the same names as the native grid climos from which they were derived. There is no namespace conflict because they are in separate directories. These files also have symbolic links to their AMWG filenames. If '-O drc_rgr' is not specified, ncclimo places all regridded files in the native grid climo output directory, ${drc_out}, specified by '-o drc_out' (NB: lowercase "o") . To avoid namespace conflicts when both climos are stored in the same directory, the names of the regridded files are suffixed by the destination geometry string obtained from the mapfile, e.g., '*_climo_fv257x512_bilin.nc'. These files also have symbolic links to their AMWG filenames.

ncclimo -c famipc5_ne30_v0.3_00003 -s 1980 -e 1983 -i drc_in -o drc_out

ncclimo -c famipc5_ne30_v0.3_00003 -s 1980 -e 1983 -i drc_in -r map_file -o drc_out

ncclimo -c famipc5_ne30_v0.3_00003 -s 1980 -e 1983 -i drc_in -r map_file -o drc_out -O drc_rgr

ncremap -I drc_out -m map.nc -O drc_rgr

ls drc_out/*climo* | ncremap -m map.nc -O drc_rgr

Coupled Runs:

ncclimo works on all ACME models. It can simultaneously generate climatologies for a coupled run, where climatologies mean both native and regridded monthly, seasonal, and annual averages as per the AG specification. Here are template commands for a recent simulation:

...