NERSC
DECK v1 simulation original model output has been archived on NERSC HPSS using zstash. See below for some examples on how to retrieve files using zstash. For a more information, refer to the zstash documentation. Please contact Zeshawn Shaheen (Unlicensed) for help with zstash. The table below lists output location for the various simulations. Note that some archives initially contained a few corrupted files. These problems have all been corrected.
Simulation | HPSS location | Approximate size |
---|---|---|
piControl |
| 37 TB |
abrupt-4xCO2 |
| 12 TB |
1pctCO2 |
| 11 TB |
Historical H1 |
| 12 TB |
Historical H2 |
| 12 TB |
Historical H3 |
| 12 TB |
Historical H4 |
| 12 TB |
Historical H5 | /home/g/golaz/2018/E3SM_simulations/20180307.DECKv1b_H5.ne30_oEC.edison /home/t/tang30/2018/E3SM_simulations/20180307.DECKv1b_H5.ne30_oEC.edison | 12 TB |
AMIP A1 |
| 4 TB |
AMIP A1 1850allF |
| 5 TB |
AMIP A1 1850aeroF |
| 5 TB |
AMIP A2 |
| 4 TB |
AMIP A2 1850allF |
| 4 TB |
AMIP A2 1850aeroF |
| 4 TB |
AMIP A3 | /home/t/tang30/2018/E3SM_simulations/20180716.DECKv1b_A3.ne30_oEC.edison | 5 TB |
AMIP A3 1850allF | /home/t/tang30/2018/E3SM_simulations/20180716.DECKv1b_A3_1850allF.ne30_oEC.edison | 5 TB |
AMIP A3 1850aeroF | /home/t/tang30/2018/E3SM_simulations/20180716.DECKv1b_A3_1850aeroF.ne30_oEC.edison | 5 TB |
DAMIP hist-GHG H1 | /home/t/tang30/2019/E3SM_simulations/20190404.DECKv1b_H1_hist-GHG.ne30_oEC.edison | 12 TB |
DAMIP hist-GHG H2 | /home/t/tang30/2019/E3SM_simulations/20190404.DECKv1b_H2_hist-GHG.ne30_oEC.edison | 12 TB |
DAMIP hist-GHG H3 | /home/t/tang30/2019/E3SM_simulations/20190404.DECKv1b_H3_hist-GHG.ne30_oEC.edison | 12 TB |
Total | 197 TB |
Retrieving data with zstash
Below are some basic examples on how to retrive DECK output data on NERSC. You can retrieve directly from NERSC Edison or Cori machines, but you may see better performance if you log in to one of the NERSC DTN nodes. The examples below assume that you have installed a conda environment with the latest version of zstash (see zstash quickstart guide).
Activate environment
source activate zstash
Examples
Retrieve selected cam.h0 files from piControl
Let's say you want to retrieve all cam.h0 (monthly atmosphere output files) between years 0030 and 0069 for the piControl simulation. The zstash command would be
zstash extract --hpss=/home/g/golaz/2018/E3SM_simulations/repaired/20180129.DECKv1b_piControl.ne30_oEC.edison "*.cam.h0.00[3-6]?-??.nc"
where
is the path on HPSS listed in the table above, and "/home/g/golaz/2018/E3SM_simulations/repaired/20180129.DECKv1b_piControl.ne30_oEC.edison
*.cam.h0.00[3-6]?-??.nc
" is a matching pattern to select all cam.h0 files between years 0030 and 0069, enclosing it in double quotes avoids shell level substitution.
The extracted files will be stored in a sub-directory structure that preserves the original structure:
archive/atm/hist/20180129.DECKv1b_piControl.ne30_oEC.edison.cam.h0.0030-01.nc archive/atm/hist/20180129.DECKv1b_piControl.ne30_oEC.edison.cam.h0.0030-02.nc archive/atm/hist/20180129.DECKv1b_piControl.ne30_oEC.edison.cam.h0.0030-03.nc archive/atm/hist/20180129.DECKv1b_piControl.ne30_oEC.edison.cam.h0.0030-04.nc [...] archive/atm/hist/20180129.DECKv1b_piControl.ne30_oEC.edison.cam.h0.0069-09.nc archive/atm/hist/20180129.DECKv1b_piControl.ne30_oEC.edison.cam.h0.0069-10.nc archive/atm/hist/20180129.DECKv1b_piControl.ne30_oEC.edison.cam.h0.0069-11.nc archive/atm/hist/20180129.DECKv1b_piControl.ne30_oEC.edison.cam.h0.0069-12.nc