IN PROGRESS

...

Post-Processing with zppy (needs update)

To post-process a model run, do the following steps. Note that to post-process up to year n, then you must have short-term archived up to year n.

...

Make a bulleted list of links, e.g., for https://web.lcrc.anl.gov/public/e3sm/diagnostic_output/<username>/E3SM/v2/beta/<case_name>/mpas_analysis/ts_0001-0050_climo_0021-0050/, create a bullet “1-50 (time series), 21-50 (climatology)”.

Long Term Archiving with zstash

Simulations that are deemed sufficiently valuable should be archived using zstash for long-term preservation.

Compy / anvil / chrysalis

Compy, anvil and chrysalis do not have local HPSS. We rely on NERSC HPSS for long-term archiving. Archiving requires a few separate steps:

1. Run 'zstash create' to archive to local disk.

2. Using Globus, transfer zstash archive files (everything under the zstash/ subdirectory) to NERSC HPSS. Select the option to preserve original files modification date.

3. Run 'zstash check' to verify integrity of zstash archive (and their transfer).

4. Update simulation Confluence page with path to HPSS.

Helper scripts

Below are some helper scripts to facilitate steps (1) and (3) above.

batch_zstash_create.bash to batch archive a number of simulations on compy. Run inside a ‘screen’ session to avoid any interruption:

#!/bin/bash

# Run on compy

# Load E3SM Unified
source /share/apps/E3SM/conda_envs/load_latest_e3sm_unified.sh

# List of experiments to archive with zstash
EXPS=(\
20200827.alpha4_v1GM.piControl.ne30pg2_r05_EC30to60E2r2-1900_ICG.compy \
20200905.alpha4_dtOcn.piControl.ne30pg2_r05_EC30to60E2r2-1900_ICG.compy \
)

# Loop over simulations
for EXP in "${EXPS[@]}"
do
    echo === Archiving ${EXP} ===
    cd /compyfs/gola749/E3SM_simulations/${EXP}
    mkdir -p zstash
    stamp=`date +%Y%m%d`
    time zstash create -v --hpss=none  --maxsize 128 . 2>&1 | tee zstash/zstash_create_${stamp}.log
done

batch_zstash_check.bash to batch check a number of simulations on NERSC dtn. Run inside a ‘screen’ session to avoid any interruption:

#!/bin/bash

# Run on NERSC dtn

# Load environment that includes zstash
source /global/cfs/cdirs/e3sm/software/anaconda_envs/load_latest_e3sm_unified.sh

# List of experiments to archive with zstash
EXPS=(\
20200827.alpha4_v1GM.piControl.ne30pg2_r05_EC30to60E2r2-1900_ICG.compy \
20200905.alpha4_dtOcn.piControl.ne30pg2_r05_EC30to60E2r2-1900_ICG.compy \
)

# Loop over simulations
for EXP in "${EXPS[@]}"
do
    echo === Checking ${EXP} ===
    #cd /global/cscratch1/sd/golaz/E3SM_simulations
    cd /global/cfs/cdirs/e3sm/golaz/E3SM_simulations
    mkdir -p ${EXP}/zstash
    cd ${EXP}
    stamp=`date +%Y%m%d`
    time zstash check --hpss=/home/g/golaz/2020/${EXP} --workers 2 2>&1 | tee zstash/zstash_check_${stamp}.log
done

More info

Refer to zstash's best practices for E3SM for details.