...
Warning15 Apr 2020 NERSC Directory Change Notice
Due to project's name change at NERSC from 'ACME' to 'E3SM' and NERSC’s file system update , the directory ' /project/projectdirs/acme/ ' is now ' /cfs/cdirs/e3sm '
NERSC DECK v1 simulation original model output has been archived on NERSC HPSS using zstash . See below for some examples on how to retrieve files using zstash. For a more information, refer to the zstash documentation . Please contact Zeshawn Shaheen (Unlicensed) for help with zstash . The table below lists output location for the various simulations. Note that some archives initially contained a few corrupted files. These problems have all been corrected .
Simulation HPSS location Approximate size
Failed files and tarballs zstash check
output piControl /home/g/golaz/2018/E3SM_simulations /repaired /20180129.DECKv1b_piControl.ne30_oEC.edison
New of repaired archives g golaz tang30 /2018/E3SM_simulations/repaired/20180129.DECKv1b_piControl.ne30_oEC.edison
37 TB
piControl-failed.txt 00000a.tar 000074.tar 000029.tar 000081.tar Total of 147 tar files
t tang30 golaz /2018/E3SM_simulations/ repaired/ 20180215.DECKv1b_abrupt4xCO2.ne30_oEC.edison
New of repaired archives g golaz tang30 /2018/E3SM_simulations/repaired/20180215.DECKv1b_abrupt4xCO2.ne30_oEC.edison
12 TB abrupt-4xCO2
-failed.txt 00002a.tar 00001d.tar 00000f.tar 000028.tar 000021.tar Total of 47 tar files abrupt-4xCO2.txt
(extension to 300 years )
/home/g/golaz/2019/E3SM_simulations/20190722.DECKv1b_abrupt4xCO2.ne30_oEC3.compy Alternate location: /home/t/tang30/2019/E3SM_simulations/20190722.DECKv1b_abrupt4xCO2.ne30_oEC3.compy 13 TB 1pctCO2 /home/g/golaz/2018/E3SM_simulations /repaired /20180215.DECKv1b_1pctCO2.ne30_oEC.edison
11 TB 1pctCO2-failed.txt 000000.tar
Total of 45 tar files Alternate location: /home/t/tang30/2018/E3SM_simulations/repaired/20180215.DECKv1b_1pctCO2.ne30_oEC.edison
11 TB Historical H1 /home/
t tang30 golaz /2018/E3SM_simulations /repaired /20180215.DECKv1b_H1.ne30_oEC.edison
New of repaired archives g golaz tang30 /2018/E3SM_simulations/repaired/20180215.DECKv1b_H1.ne30_oEC.edison
12 TB Historical
_H1-failed.txt 00001e.tar 000002.tar Total of 49 tar files Historical H2 H2 /home/g/golaz/2018/E3SM_simulations/repaired/20180220.DECKv1b_H2.ne30_oEC.edison Alternate location:
/home/t/tang30/2018/E3SM_simulations /repaired /20180220.DECKv1b_H2.ne30_oEC.edison
12 TB Historical
_H2-failed.txt 000014.tar 00002d.tar Total of 49 tar files Historical_H2.txt Historical H3 H3 /home/z/zshaheen/2018/E3SM_simulations/repaired/20180302.DECKv1b_H3.ne30_oEC.edison
Alternate locations:
/home/g/golaz/2018/E3SM_simulations /repaired /20180302.DECKv1b_H3.ne30_oEC.edison
12 TB Historical
_H3-failed.txt 000000.tar 00002a.tar 00001f.tar 000025.tar Total of 49 tar files Historical H4 H4 /home/t/tang30/2018/E3SM_simulations/20180305.DECKv1b_H4.ne30_oEC.edison
Alternate location: /home/
t tang30 golaz /2018/E3SM_simulations/20180305.DECKv1b_H4.ne30_oEC.edison
12 TB
No failures out of 49 tar files.
_H4.txtHistorical Historical_H5.txt H5 /home/g/golaz/2018/E3SM_simulations/20180307.DECKv1b_H5.ne30_oEC.edison
12 TB No failures out of 50 tar files. AMIP_A1.txt Alternate location:
/home/t/tang30/2018/E3SM_simulations/20180307.DECKv1b_H5.ne30_oEC.edison
12 TB AMIP A1 /home/t/tang30/2018/E3SM_simulations/20180316.DECKv1b_A1.ne30_oEC.edison
4 TB No failures out of 15 tar files. AMIP Alternate location:
/home/g/golaz/2018/E3SM_simulations/20180316.DECKv1b_A1.ne30_oEC.edison
4 TB AMIP A1 1850allF /home/t/tang30/2018/E3SM_simulations/20180508.DECKv1b_A1_1850allF.ne30_oEC.edison
5 TB No failures out of 15 tar files. Alternate location:
/home/g/golaz/2018/E3SM_simulations/20180508.DECKv1b _A1_1850allF
.txtAMIP .ne30_oEC.edison
5 TB AMIP A1 1850aeroF /home/t/tang30/2018/E3SM_simulations/20180508.DECKv1b_A1_1850aeroF.ne30_oEC.edison
5 TB No failures out of 15 tar files. Alternate location:
/home/g/golaz/2018/E3SM_simulations/20180508.DECKv1b _A1_1850aeroF
.txt AMIP_A2.txt .ne30_oEC.edison
5 TB AMIP A2 /home/t/tang30/2018/E3SM_simulations/20180622.DECKv1b_A2.ne30_oEC.edison
4 TB No failures out of 16 tar files. AMIP Alternate location:
/home/g/golaz/2018/E3SM_simulations/20180622.DECKv1b_A2.ne30_oEC.edison
4 TB AMIP A2 1850allF /home/t/tang30/2018/E3SM_simulations/20180622.DECKv1b_A2_1850allF.ne30_oEC.edison
4 TB No failures out of 15 tar files. Alternate location:
/home/g/golaz/2018/E3SM_simulations/20180622.DECKv1b _A2_1850allF
.txtAMIP .ne30_oEC.edison
4 TB AMIP A2 1850aeroF /home/t/tang30/2018/E3SM_simulations/20180622.DECKv1b_A2_1850aeroF.ne30_oEC.edison
4 TB No failures out of 15 tar files. Alternate location:
/home/g/golaz/2018/E3SM_simulations/20180622.DECKv1b _A2_1850aeroF
.txt AMIP_A3.txt .ne30_oEC.edison
4 TB AMIP A3 /home/t/tang30/2018/E3SM_simulations/20180716.DECKv1b_A3.ne30_oEC.edison
5 TB No failures out of 20 tar files. AMIP Alternate location:
/home/g/golaz/2018/E3SM_simulations/20180716.DECKv1b_A3.ne30_oEC.edison
5 TB AMIP A3 1850allF /home/t/tang30/2018/E3SM_simulations/20180716.DECKv1b_A3_1850allF.ne30_oEC.edison
5 TB No failures out of 19 tar files.
Alternate location:
/home/g/golaz/2018/E3SM_simulations/20180716.DECKv1b _A3_1850allF.
txt ne30_oEC.edison
5 TB AMIP A3 1850aeroF /home/t/tang30/2018/E3SM_simulations/20180716.DECKv1b_A3_1850aeroF.ne30_oEC.edison
Alternate location:
/home/g/golaz/2018/E3SM_simulations/20180716.DECKv1b_A3_1850aeroF.ne30_oEC.edison
5 TB
No failures out of 19 tar files.
AMIP_A3_1850aeroF.txt Total 161 TB 18/634 = 2.839% tar files failed. Known bad data Some users have reported errors when trying to extract from HPSS archive. The cause of the bad data is currently unknown; possibilities include a bug in zstash or data file being corrupted to/on HPSS.
Simulation Tar archive Content Rescue status abrupt-4xCO2 00002a.tar abrupt4xCO2_index00002a.txt Could extract files; they look like netCDF files but md5 checksums do not match original ones. Historical H2 00002d.tar H2_index00002d.txt Could extract first file, but nothing else. Historical H3 000000.tar H3_index000000.txt Chris Golaz was able to successfully rescue contents of tar archive; md5 checksums matched original ones. Data available under: /global/cscratch1/sd/golaz/ACME_simulations/rescue/20180302.DECKv1b_H3.ne30_oEC.edison
DAMIP hist-GHG H1 /home/t/tang30/2019/E3SM_simulations/20190404.DECKv1b_H1_hist-GHG.ne30_oEC.edison
Alternate location:
/home/g/golaz/2019/E3SM_simulations/20190404.DECKv1b_H1_hist-GHG.ne30_oEC.edison
12 TB DAMIP hist-GHG H2 /home/t/tang30/2019/E3SM_simulations/20190404.DECKv1b_H2_hist-GHG.ne30_oEC.edison
Alternate location:
/home/g/golaz/2019/E3SM_simulations/20190404.DECKv1b_H2_hist-GHG.ne30_oEC.edison
12 TB DAMIP hist-GHG H3 /home/t/tang30/2019/E3SM_simulations/20190404.DECKv1b_H3_hist-GHG.ne30_oEC.edison
Alternate location:
/home/g/golaz/2018/E3SM_simulations/repaired/20180302.DECKv1b_H3.ne30_oEC.edison
12 TB Projection P1 /home/t/tang30/2019/E3SM_simulations/20190807.DECKv1b_P1_SSP5-8.5.ne30_oEC.cori-knl
Alternate location:
/home/g/golaz/2019/E3SM_simulations/20190807.DECKv1b_P1_SSP5-8.5.ne30_oEC.cori-knl
8 TB Projection P2 /home/t/tang30/2019/E3SM_simulations/20190821.DECKv1b_P2_SSP5-8.5.ne30_oEC.cori-knl
Alternate location:
/home/g/golaz/2019/E3SM_simulations/20190821.DECKv1b_P2_SSP5-8.5.ne30_oEC.cori-knl
8 TB Projection P3 /home/t/tang30/2019/E3SM_simulations/20190821.DECKv1b_P3_SSP5-8.5.ne30_oEC.cori-knl
Alternate location:
/home/g/golaz/2019/E3SM_simulations/20190821.DECKv1b_P3_SSP5-8.5.ne30_oEC.cori-knl
8 TB Projection P4 /home/t/tang30/2019/E3SM_simulations/20190821.DECKv1b_P4_SSP5-8.5.ne30_oEC.cori-knl
Alternate location:
/home/g/golaz/2019/E3SM_simulations/20190821.DECKv1b_P4_SSP5-8.5.ne30_oEC.cori-knl
8 TB Projection P5 /home/t/tang30/2019/E3SM_simulations/20190821.DECKv1b_P5_SSP5-8.5.ne30_oEC.cori-knl
Alternate location:
/home/g/golaz/2019/E3SM_simulations/20190821.DECKv1b_P5_SSP5-8.5.ne30_oEC.cori-knl
8 TB DAMIP SSP5-8.5-GHG P1 /home/t/tang30/2019/E3SM_simulations/20191019.DECKv1b_P1_SSP5-8.5-GHG.ne30_oEC.cori-knl
Alternate location:
/home/g/golaz/2019/E3SM_simulations/20191019.DECKv1b_P1_SSP5-8.5-GHG.ne30_oEC.cori-knl
8 TB DAMIP SSP5-8.5-GHG P2 /home/t/tang30/2019/E3SM_simulations/20191019.DECKv1b_P2_SSP5-8.5-GHG.ne30_oEC.cori-knl
Alternate location:
/home/g/golaz/2019/E3SM_simulations/20191019.DECKv1b_P2_SSP5-8.5-GHG.ne30_oEC.cori-knl
8 TB DAMIP SSP5-8.5-GHG P3 /home/t/tang30/2019/E3SM_simulations/20191019.DECKv1b_P3_SSP5-8.5-GHG.ne30_oEC.cori-knl
Alternate location:
/home/g/golaz/2019/E3SM_simulations/20191019.DECKv1b_P3_SSP5-8.5-GHG.ne30_oEC.cori-knl
8 TB Total 261 TB
Retrieving data with zstash Below are some basic examples on how to retrive retrieve DECK output data on NERSC. You can retrieve directly from NERSC Edison or Cori machines machine , but you may see for better performance if you log in we strongly recommend login to one of the NERSC DTN nodes DTNs (data transfer nodes: dtn<01…04>.nersc.gov) . The examples below assume that you have installed a conda environment with the latest version of zstash (see zstash quickstart guide example below relies on the E3SM unified environment. Alternatively, you can also install zstash in your own conda environment (see zstash documentation ).
Activate environment If you are using bash shell:
Code Blocksource activate zstash /global/cfs/cdirs/e3sm/software/anaconda_envs/load_latest_e3sm_unified.sh
If you are using csh shell: Code Blocksource /global/cfs/cdirs/e3sm/software/anaconda_envs/load_latest_e3sm_unified.csh
Examples Retrieve selected cam.h0 files from piControl ...
Code Blockzstash extract --hpss=/home/g/golaz/2018/E3SM_simulations/repaired/ 20180129.DECKv1b_piControl.ne30_oEC.edison "*.cam.h0.00[3-6]?-??.nc"
where /home/g/golaz/2018/E3SM_simulations /repaired /20180129.DECKv1b_piControl.ne30_oEC.edison
is the path on HPSS listed in the table above, and "*.cam.h0.00[3-6]?-??.nc
" is a matching pattern to select all cam.h0 files between years 0030 and 0069, enclosing it in double quotes avoids shell level substitution.
...