Warning

icon	false

15 Apr 2020 NERSC Directory Change Notice

Due to project's name change at NERSC from 'ACME' to 'E3SM' and NERSC’s file system update, the directory '~~/project/projectdirs/acme/~~' is now '/cfs/cdirs/e3sm'

NERSC

DECK v1 simulation original model output has been archived on NERSC HPSS using zstash. See below for some examples on how to retrieve files using zstash. For a more information, refer to the zstash documentation. Please contact Zeshawn Shaheen (Unlicensed) for help with zstash. The table below lists output location for the various simulations. Note that some archives initially contained a few corrupted files. These problems have all been corrected.

Simulation	HPSS location	Approximate size

Failed files and tarballszstash check output


piControl	`/home/g/golaz/2018/E3SM_simulations/repaired/20180129.DECKv1b_piControl.ne30_oEC.edison`

New

Alternate location

of repaired archives

:
/home/

g

t/

golaz

tang30/2018/E3SM_simulations/repaired/20180129.DECKv1b_piControl.ne30_oEC.edison

37 TB

~~piControl-failed.txt~~
~~00000a.tar~~
~~000074.tar~~
~~000029.tar~~
~~000081.tar~~

~~Total of 147 tar files~~

~~piControl.txt~~

abrupt-4xCO2

/home/

t

g/

tang30

golaz/2018/E3SM_simulations/repaired/20180215.DECKv1b_abrupt4xCO2.ne30_oEC.edison

New

Alternate location

of repaired archives

:
/home/

g

t/

golaz

`tang30/2018/E3SM_simulations/repaired/20180215.DECKv1b_abrupt4xCO2.ne30_oEC.edison`	12 TB
abrupt-4xCO2

-failed.txt
~~00002a.tar~~
~~00001d.tar~~
~~00000f.tar~~
~~000028.tar~~
~~000021.tarTotal of 47 tar files~~

~~abrupt-4xCO2.txt~~

(extension to 300 years)

/home/g/golaz/2019/E3SM_simulations/20190722.DECKv1b_abrupt4xCO2.ne30_oEC3.compy
Alternate location:
/home/t/tang30/2019/E3SM_simulations/20190722.DECKv1b_abrupt4xCO2.ne30_oEC3.compy

13 TB

1pctCO2

/home/g/golaz/2018/E3SM_simulations/repaired/20180215.DECKv1b_1pctCO2.ne30_oEC.edison

11 TB

1pctCO2-failed.txt
000000.tar

Total of 45 tar files

1pctCO2.txt

Alternate location:`/home/t/tang30/2018/E3SM_simulations/repaired/20180215.DECKv1b_1pctCO2.ne30_oEC.edison`	11 TB
Historical H1	`/home/`

t

g/

tang30

golaz/2018/E3SM_simulations/repaired/20180215.DECKv1b_H1.ne30_oEC.edison

New

Alternate location

of repaired archives

:
/home/

g

t/

golaz

tang30/2018/E3SM_simulations/repaired/20180215.DECKv1b_H1.ne30_oEC.edison

12 TB

~~Historical_H1-failed.txt~~
~~00001e.tar~~
~~000002.tar~~

~~Total of 49 tar filesHistorical_H1.txt~~


Historical H2	`/home/`

t

g/

tang30

golaz/2018/E3SM_simulations/repaired/20180220.DECKv1b_H2.ne30_oEC.edison

New

Alternate location

of repaired archives

:
/home/

g

t/

golaz

`tang30/2018/E3SM_simulations/repaired/20180220.DECKv1b_H2.ne30_oEC.edison`	12 TB
Historical

_H2-failed.txt
~~000014.tar~~
~~00002d.tarTotal of 49 tar filesHistorical_H2.txt~~Historical H3

H3

/home/z/zshaheen/2018/E3SM_simulations/repaired/20180302.DECKv1b_H3.ne30_oEC.edison

Alternate locations:

/home/g/golaz/2018/E3SM_simulations/repaired/20180302.DECKv1b_H3.ne30_oEC.edison

12 TB

Historical

_H3-failed.txt
000000.tar
00002a.tar
00001f.tar
000025.tarTotal of 49 tar files

Historical_H3.txt

Historical H4

H4	`/home/t/tang30/2018/E3SM_simulations/20180305.DECKv1b_H4.ne30_oEC.edison`Alternate location: `/home/`

t

g/

tang30

golaz/2018/E3SM_simulations/20180305.DECKv1b_H4.ne30_oEC.edison

12 TB

No failures out of 49 tar files.

Historical

_H4.txtHistorical Historical_H5.txt

H5	`/home/g/golaz/2018/E3SM_simulations/20180307.DECKv1b_H5.ne30_oEC.edison`

12 TBNo failures out of 50 tar files.AMIP_A1.txt

`Alternate location:` `/home/t/tang30/2018/E3SM_simulations/20180307.DECKv1b_H5.ne30_oEC.edison`	12 TB
AMIP A1	`/home/t/tang30/2018/E3SM_simulations/20180316.DECKv1b_A1.ne30_oEC.edison`

4 TBNo failures out of 15 tar files.AMIP

Alternate location:

/home/g/golaz/2018/E3SM_simulations/20180316.DECKv1b_A1.ne30_oEC.edison

4 TB

AMIP A1 1850allF

/home/t/tang30/2018/E3SM_simulations/20180508.DECKv1b_A1_1850allF.ne30_oEC.edison

5 TBNo failures out of 15 tar files.

Alternate location:

/home/g/golaz/2018/E3SM_simulations/20180508.DECKv1b_A1_1850allF

.txtAMIP

`.ne30_oEC.edison`	5 TB
AMIP A1 1850aeroF	`/home/t/tang30/2018/E3SM_simulations/20180508.DECKv1b_A1_1850aeroF.ne30_oEC.edison`

5 TBNo failures out of 15 tar files.

Alternate location:

/home/g/golaz/2018/E3SM_simulations/20180508.DECKv1b_A1_1850aeroF.

txtAMIP_A2.txt

`ne30_oEC.edison`	5 TB
AMIP A2	`/home/t/tang30/2018/E3SM_simulations/20180622.DECKv1b_A2.ne30_oEC.edison`

4 TBNo failures out of 16 tar files.AMIP

Alternate location:

/home/g/golaz/2018/E3SM_simulations/20180622.DECKv1b_A2.ne30_oEC.edison

4 TB

AMIP A2 1850allF

/home/t/tang30/2018/E3SM_simulations/20180622.DECKv1b_A2_1850allF.ne30_oEC.edison

4 TBNo failures out of 15 tar files.

Alternate location:

/home/g/golaz/2018/E3SM_simulations/20180622.DECKv1b_A2_1850allF.

txtAMIP

`ne30_oEC.edison`	4 TB
AMIP A2 1850aeroF	`/home/t/tang30/2018/E3SM_simulations/20180622.DECKv1b_A2_1850aeroF.ne30_oEC.edison`

4 TBNo failures out of 15 tar files.

Alternate location:

/home/g/golaz/2018/E3SM_simulations/20180622.DECKv1b_A2_1850aeroF

.txtAMIP_A3.txt

`.ne30_oEC.edison`	4 TB
AMIP A3	`/home/t/tang30/2018/E3SM_simulations/20180716.DECKv1b_A3.ne30_oEC.edison`

5 TBNo failures out of 20 tar files.AMIP

Alternate location:

/home/g/golaz/2018/E3SM_simulations/20180716.DECKv1b_A3.ne30_oEC.edison

5 TB

AMIP A3 1850allF

/home/t/tang30/2018/E3SM_simulations/20180716.DECKv1b_A3_1850allF.ne30_oEC.edison

5 TB

No failures out of 19 tar files.

Alternate location:

/home/g/golaz/2018/E3SM_simulations/20180716.DECKv1b_A3_1850allF

.txt

.ne30_oEC.edison

5 TB

AMIP A3 1850aeroF

/home/t/tang30/2018/E3SM_simulations/20180716.DECKv1b_A3_1850aeroF.ne30_oEC.edison

Alternate location:

/home/g/golaz/2018/E3SM_simulations/20180716.DECKv1b_A3_1850aeroF.ne30_oEC.edison

5 TB

No failures out of 19 tar files.

AMIP_A3_1850aeroF.txtTotal161 TB18/634 = 2.839% tar files failed.

Known bad data

Some users have reported errors when trying to extract from HPSS archive. The cause of the bad data is currently unknown; possibilities include a bug in zstash or data file being corrupted to/on HPSS.

SimulationTar archiveContentRescue statusabrupt-4xCO200002a.tarabrupt4xCO2_index00002a.txt

Could extract files; they look like netCDF files but md5 checksums do not match original ones.

Historical H200002d.tarH2_index00002d.txt

Could extract first file, but nothing else.

Historical H3000000.tarH3_index000000.txt

Chris Golaz was able to successfully rescue contents of tar archive; md5 checksums matched original ones.
Data available under:

/global/cscratch1/sd/golaz/ACME_simulations/rescue/20180302.DECKv1b_H3.ne30_oEC.edison


DAMIP hist-GHG H1	`/home/t/tang30/2019/E3SM_simulations/20190404.DECKv1b_H1_hist-GHG.ne30_oEC.edison` `Alternate location:` `/home/g/golaz/2019/E3SM_simulations/20190404.DECKv1b_H1_hist-GHG.ne30_oEC.edison`	12 TB
DAMIP hist-GHG H2	`/home/t/tang30/2019/E3SM_simulations/20190404.DECKv1b_H2_hist-GHG.ne30_oEC.edison` `Alternate location:` `/home/g/golaz/2019/E3SM_simulations/20190404.DECKv1b_H2_hist-GHG.ne30_oEC.edison`	12 TB
DAMIP hist-GHG H3	`/home/t/tang30/2019/E3SM_simulations/20190404.DECKv1b_H3_hist-GHG.ne30_oEC.edison` `Alternate location:` `/home/g/golaz/2018/E3SM_simulations/repaired/20180302.DECKv1b_H3.ne30_oEC.edison`	12 TB
Projection P1	`/home/t/tang30/2019/E3SM_simulations/20190807.DECKv1b_P1_SSP5-8.5.ne30_oEC.cori-knl` `Alternate location:` `/home/g/golaz/2019/E3SM_simulations/20190807.DECKv1b_P1_SSP5-8.5.ne30_oEC.cori-knl`	8 TB
Projection P2	`/home/t/tang30/2019/E3SM_simulations/20190821.DECKv1b_P2_SSP5-8.5.ne30_oEC.cori-knl` `Alternate location:` `/home/g/golaz/2019/E3SM_simulations/20190821.DECKv1b_P2_SSP5-8.5.ne30_oEC.cori-knl`	8 TB
Projection P3	`/home/t/tang30/2019/E3SM_simulations/20190821.DECKv1b_P3_SSP5-8.5.ne30_oEC.cori-knl` `Alternate location:` `/home/g/golaz/2019/E3SM_simulations/20190821.DECKv1b_P3_SSP5-8.5.ne30_oEC.cori-knl`	8 TB
Projection P4	`/home/t/tang30/2019/E3SM_simulations/20190821.DECKv1b_P4_SSP5-8.5.ne30_oEC.cori-knl` `Alternate location:` `/home/g/golaz/2019/E3SM_simulations/20190821.DECKv1b_P4_SSP5-8.5.ne30_oEC.cori-knl`	8 TB
Projection P5	`/home/t/tang30/2019/E3SM_simulations/20190821.DECKv1b_P5_SSP5-8.5.ne30_oEC.cori-knl` `Alternate location:` `/home/g/golaz/2019/E3SM_simulations/20190821.DECKv1b_P5_SSP5-8.5.ne30_oEC.cori-knl`	8 TB
DAMIP SSP5-8.5-GHG P1	`/home/t/tang30/2019/E3SM_simulations/20191019.DECKv1b_P1_SSP5-8.5-GHG.ne30_oEC.cori-knl` `Alternate location:` `/home/g/golaz/2019/E3SM_simulations/20191019.DECKv1b_P1_SSP5-8.5-GHG.ne30_oEC.cori-knl`	8 TB
DAMIP SSP5-8.5-GHG P2	`/home/t/tang30/2019/E3SM_simulations/20191019.DECKv1b_P2_SSP5-8.5-GHG.ne30_oEC.cori-knl` `Alternate location:` `/home/g/golaz/2019/E3SM_simulations/20191019.DECKv1b_P2_SSP5-8.5-GHG.ne30_oEC.cori-knl`	8 TB
DAMIP SSP5-8.5-GHG P3	`/home/t/tang30/2019/E3SM_simulations/20191019.DECKv1b_P3_SSP5-8.5-GHG.ne30_oEC.cori-knl` `Alternate location:` `/home/g/golaz/2019/E3SM_simulations/20191019.DECKv1b_P3_SSP5-8.5-GHG.ne30_oEC.cori-knl`	8 TB
Total		261 TB

Retrieving data with zstash

Below are some basic examples on how to retrive retrieve DECK output data on NERSC. You can retrieve directly from NERSC Edison or Cori machinesmachine, but you may see for better performance if you log in we strongly recommend login to one of the NERSC DTN nodesDTNs (data transfer nodes: dtn<01…04>.nersc.gov). The examples below assume that you have installed a conda environment with the latest version of zstash (see zstash quickstart guideexample below relies on the E3SM unified environment. Alternatively, you can also install zstash in your own conda environment (see zstash documentation).

Activate environment

If you are using bash shell:

Code Block

language	bash

source activate zstash/global/cfs/cdirs/e3sm/software/anaconda_envs/load_latest_e3sm_unified.sh

If you are using csh shell:

Code Block

language	bash

source /global/cfs/cdirs/e3sm/software/anaconda_envs/load_latest_e3sm_unified.csh

Examples

Retrieve selected cam.h0 files from piControl

...

Code Block

language	bash

zstash extract --hpss=/home/g/golaz/2018/E3SM_simulations/repaired/20180129.DECKv1b_piControl.ne30_oEC.edison "*.cam.h0.00[3-6]?-??.nc"

where /home/g/golaz/2018/E3SM_simulations/repaired/20180129.DECKv1b_piControl.ne30_oEC.edison is the path on HPSS listed in the table above, and "*.cam.h0.00[3-6]?-??.nc" is a matching pattern to select all cam.h0 files between years 0030 and 0069, enclosing it in double quotes avoids shell level substitution.

...

Versions Compared

Old Version 24

New Version Current

Key

NERSC

Known bad data

Retrieving data with zstash

Activate environment

If you are using csh shell:

Examples

Retrieve selected cam.h0 files from piControl

Page Comparison

Versions Compared

Old Version 24

New Version Current

Key

NERSC

Known bad data

Retrieving data with zstash

Activate environment

If you are using csh shell:

Examples

Retrieve selected cam.h0 files from piControl