v1 DECK output HPSS archive

 

Apr 15, 2020  NERSC Directory Change Notice  

Due to project's name change at NERSC from 'ACME' to 'E3SM' and NERSC’s file system update, the directory  '/project/projectdirs/acme/' is now '/cfs/cdirs/e3sm'

NERSC

DECK v1 simulation original model output has been archived on NERSC HPSS using zstash. See below for some examples on how to retrieve files using zstash. For a more information, refer to the zstash documentation. The table below lists output location for the various simulations. Note that some archives initially contained a few corrupted files. These problems have all been corrected.

Simulation

HPSS location

Approximate size

Simulation

HPSS location

Approximate size

piControl

/home/g/golaz/2018/E3SM_simulations/repaired/20180129.DECKv1b_piControl.ne30_oEC.edison
Alternate location:
/home/t/tang30/2018/E3SM_simulations/repaired/20180129.DECKv1b_piControl.ne30_oEC.edison

37 TB

abrupt-4xCO2

/home/g/golaz/2018/E3SM_simulations/repaired/20180215.DECKv1b_abrupt4xCO2.ne30_oEC.edison
Alternate location:
/home/t/tang30/2018/E3SM_simulations/repaired/20180215.DECKv1b_abrupt4xCO2.ne30_oEC.edison

12 TB

abrupt-4xCO2

(extension to 300 years)

/home/g/golaz/2019/E3SM_simulations/20190722.DECKv1b_abrupt4xCO2.ne30_oEC3.compy
Alternate location:
/home/t/tang30/2019/E3SM_simulations/20190722.DECKv1b_abrupt4xCO2.ne30_oEC3.compy

13 TB

1pctCO2

/home/g/golaz/2018/E3SM_simulations/repaired/20180215.DECKv1b_1pctCO2.ne30_oEC.edison
Alternate location:
/home/t/tang30/2018/E3SM_simulations/repaired/20180215.DECKv1b_1pctCO2.ne30_oEC.edison

11 TB

Historical H1

/home/g/golaz/2018/E3SM_simulations/repaired/20180215.DECKv1b_H1.ne30_oEC.edison
Alternate location:
/home/t/tang30/2018/E3SM_simulations/repaired/20180215.DECKv1b_H1.ne30_oEC.edison

12 TB

Historical H2

/home/g/golaz/2018/E3SM_simulations/repaired/20180220.DECKv1b_H2.ne30_oEC.edison
Alternate location:
/home/t/tang30/2018/E3SM_simulations/repaired/20180220.DECKv1b_H2.ne30_oEC.edison

12 TB

Historical H3

/home/z/zshaheen/2018/E3SM_simulations/repaired/20180302.DECKv1b_H3.ne30_oEC.edison

Alternate locations:

/home/g/golaz/2018/E3SM_simulations/repaired/20180302.DECKv1b_H3.ne30_oEC.edison

12 TB

Historical H4

/home/t/tang30/2018/E3SM_simulations/20180305.DECKv1b_H4.ne30_oEC.edison
Alternate location:
/home/g/golaz/2018/E3SM_simulations/20180305.DECKv1b_H4.ne30_oEC.edison

12 TB

Historical H5

/home/g/golaz/2018/E3SM_simulations/20180307.DECKv1b_H5.ne30_oEC.edison
Alternate location:
/home/t/tang30/2018/E3SM_simulations/20180307.DECKv1b_H5.ne30_oEC.edison

12 TB

AMIP A1

/home/t/tang30/2018/E3SM_simulations/20180316.DECKv1b_A1.ne30_oEC.edison

Alternate location:

/home/g/golaz/2018/E3SM_simulations/20180316.DECKv1b_A1.ne30_oEC.edison

4 TB

AMIP A1 1850allF

/home/t/tang30/2018/E3SM_simulations/20180508.DECKv1b_A1_1850allF.ne30_oEC.edison

Alternate location:

/home/g/golaz/2018/E3SM_simulations/20180508.DECKv1b_A1_1850allF.ne30_oEC.edison

5 TB

AMIP A1 1850aeroF

/home/t/tang30/2018/E3SM_simulations/20180508.DECKv1b_A1_1850aeroF.ne30_oEC.edison

Alternate location:

/home/g/golaz/2018/E3SM_simulations/20180508.DECKv1b_A1_1850aeroF.ne30_oEC.edison

5 TB

AMIP A2

/home/t/tang30/2018/E3SM_simulations/20180622.DECKv1b_A2.ne30_oEC.edison

Alternate location:

/home/g/golaz/2018/E3SM_simulations/20180622.DECKv1b_A2.ne30_oEC.edison

4 TB

AMIP A2 1850allF

/home/t/tang30/2018/E3SM_simulations/20180622.DECKv1b_A2_1850allF.ne30_oEC.edison

Alternate location:

/home/g/golaz/2018/E3SM_simulations/20180622.DECKv1b_A2_1850allF.ne30_oEC.edison

4 TB

AMIP A2 1850aeroF

/home/t/tang30/2018/E3SM_simulations/20180622.DECKv1b_A2_1850aeroF.ne30_oEC.edison

Alternate location:

/home/g/golaz/2018/E3SM_simulations/20180622.DECKv1b_A2_1850aeroF.ne30_oEC.edison

4 TB

AMIP A3

/home/t/tang30/2018/E3SM_simulations/20180716.DECKv1b_A3.ne30_oEC.edison

Alternate location:

/home/g/golaz/2018/E3SM_simulations/20180716.DECKv1b_A3.ne30_oEC.edison

5 TB

AMIP A3 1850allF

/home/t/tang30/2018/E3SM_simulations/20180716.DECKv1b_A3_1850allF.ne30_oEC.edison

Alternate location:

/home/g/golaz/2018/E3SM_simulations/20180716.DECKv1b_A3_1850allF.ne30_oEC.edison

5 TB

AMIP A3 1850aeroF

/home/t/tang30/2018/E3SM_simulations/20180716.DECKv1b_A3_1850aeroF.ne30_oEC.edison

Alternate location:

/home/g/golaz/2018/E3SM_simulations/20180716.DECKv1b_A3_1850aeroF.ne30_oEC.edison

5 TB

DAMIP hist-GHG H1

/home/t/tang30/2019/E3SM_simulations/20190404.DECKv1b_H1_hist-GHG.ne30_oEC.edison

Alternate location:

/home/g/golaz/2019/E3SM_simulations/20190404.DECKv1b_H1_hist-GHG.ne30_oEC.edison

12 TB

DAMIP hist-GHG H2

/home/t/tang30/2019/E3SM_simulations/20190404.DECKv1b_H2_hist-GHG.ne30_oEC.edison

Alternate location:

/home/g/golaz/2019/E3SM_simulations/20190404.DECKv1b_H2_hist-GHG.ne30_oEC.edison

12 TB

DAMIP hist-GHG H3

/home/t/tang30/2019/E3SM_simulations/20190404.DECKv1b_H3_hist-GHG.ne30_oEC.edison

Alternate location:

/home/g/golaz/2018/E3SM_simulations/repaired/20180302.DECKv1b_H3.ne30_oEC.edison

12 TB

Projection P1

/home/t/tang30/2019/E3SM_simulations/20190807.DECKv1b_P1_SSP5-8.5.ne30_oEC.cori-knl

Alternate location:

/home/g/golaz/2019/E3SM_simulations/20190807.DECKv1b_P1_SSP5-8.5.ne30_oEC.cori-knl

8 TB

Projection P2

/home/t/tang30/2019/E3SM_simulations/20190821.DECKv1b_P2_SSP5-8.5.ne30_oEC.cori-knl

Alternate location:

/home/g/golaz/2019/E3SM_simulations/20190821.DECKv1b_P2_SSP5-8.5.ne30_oEC.cori-knl

8 TB

Projection P3

/home/t/tang30/2019/E3SM_simulations/20190821.DECKv1b_P3_SSP5-8.5.ne30_oEC.cori-knl

Alternate location:

/home/g/golaz/2019/E3SM_simulations/20190821.DECKv1b_P3_SSP5-8.5.ne30_oEC.cori-knl

8 TB

Projection P4

/home/t/tang30/2019/E3SM_simulations/20190821.DECKv1b_P4_SSP5-8.5.ne30_oEC.cori-knl

Alternate location:

/home/g/golaz/2019/E3SM_simulations/20190821.DECKv1b_P4_SSP5-8.5.ne30_oEC.cori-knl

8 TB

Projection P5

/home/t/tang30/2019/E3SM_simulations/20190821.DECKv1b_P5_SSP5-8.5.ne30_oEC.cori-knl

Alternate location:

/home/g/golaz/2019/E3SM_simulations/20190821.DECKv1b_P5_SSP5-8.5.ne30_oEC.cori-knl

8 TB

DAMIP SSP5-8.5-GHG P1

/home/t/tang30/2019/E3SM_simulations/20191019.DECKv1b_P1_SSP5-8.5-GHG.ne30_oEC.cori-knl

Alternate location:

/home/g/golaz/2019/E3SM_simulations/20191019.DECKv1b_P1_SSP5-8.5-GHG.ne30_oEC.cori-knl

8 TB

DAMIP SSP5-8.5-GHG P2

/home/t/tang30/2019/E3SM_simulations/20191019.DECKv1b_P2_SSP5-8.5-GHG.ne30_oEC.cori-knl

Alternate location:

/home/g/golaz/2019/E3SM_simulations/20191019.DECKv1b_P2_SSP5-8.5-GHG.ne30_oEC.cori-knl

8 TB

DAMIP SSP5-8.5-GHG P3

/home/t/tang30/2019/E3SM_simulations/20191019.DECKv1b_P3_SSP5-8.5-GHG.ne30_oEC.cori-knl

Alternate location:

/home/g/golaz/2019/E3SM_simulations/20191019.DECKv1b_P3_SSP5-8.5-GHG.ne30_oEC.cori-knl

8 TB

Total

 

261 TB

Retrieving data with zstash

Below are some basic examples on how to retrieve DECK output data on NERSC. You can retrieve directly from NERSC Cori machine, but for better performance we strongly recommend login to one of the NERSC DTNs (data transfer nodes: dtn<01…04>.nersc.gov). The example below relies on the E3SM unified environment. Alternatively, you can also install zstash in your own conda environment (see zstash documentation).

Activate environment

If you are using bash shell:

source /global/cfs/cdirs/e3sm/software/anaconda_envs/load_latest_e3sm_unified.sh

If you are using csh shell:

source /global/cfs/cdirs/e3sm/software/anaconda_envs/load_latest_e3sm_unified.csh

Examples

Retrieve selected cam.h0 files from piControl

Let's say you want to retrieve all cam.h0 (monthly atmosphere output files) between years 0030 and 0069 for the piControl simulation. The zstash command would be

zstash extract --hpss=/home/g/golaz/2018/E3SM_simulations/repaired/20180129.DECKv1b_piControl.ne30_oEC.edison "*.cam.h0.00[3-6]?-??.nc"

where /home/g/golaz/2018/E3SM_simulations/repaired/20180129.DECKv1b_piControl.ne30_oEC.edison is the path on HPSS listed in the table above, and "*.cam.h0.00[3-6]?-??.nc" is a matching pattern to select all cam.h0 files between years 0030 and 0069, enclosing it in double quotes avoids shell level substitution.

The extracted files will be stored in a sub-directory structure that preserves the original structure: