Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.


Excerpt

All data from production simulation campaigns will be release released to the public by being published to distributed ESGF archives.

Once the simulation campaign is completed and the overview publication from those simulations is accepted, the Group Lead of the respected simulation campaign should send request to Exec and Infrastructure leadership to publish the data with the details of the data. The Infrastructure Group is responsible for publishing the data, the Scientists running the simulations are responsible for long-term archive of the data. The E3SM project requires everyone to use zstash to long-term archive their data. After this step is completed the Infrastructure Group will publish the data to ESGF.

Data Publication

There are different ways the data can be published. Most of the data will simply be publish as is (not CMORized). The data that up to 3 ways to retrieve the E3SM data.

Default Data Publication to E3SM Project on ESGF

The default publication of the data involves publication of model output in the native file format on native grid from all E3SM components. See Default Set of Model Output for ESGF publication for details on default data steams that will be published to the dedicated E3SM project space on ESGF (https://esgf-node.llnl.gov/projects/e3sm/). 

The Group Leads or the Simulation Campaign Lead can request additional data to be published, for example data regridded to standard lon-lat Cartesian grid, or derived data like climatologies. Those request will be generally taken as a lower priority for publication.

The status and documentation on data published on in publication can be found at Publication to ESGF to E3SM Project page on confluence.


CMORized Data Publication to CMIP Project on ESGF

A subset of data for which the simulation experiment belongs to some MIP (for example CMIP6, HiResMIP,...etc) can also be published as part of the intercomparison effort , in this case it and will be published to a MIP project space in ESGF (https://esgf-node.llnl.gov/search/cmip6/?institution_id=E3SM-Project). In this case the data has to adhere to the strict formatting rules set by the intercomparison a MIP project. For CMIP intercomparisons we We use CMOR to rewrite the data to the desired standard, we call it CMORized output.

Publishing data 'AS IS' (not CMORized)

As is data can also be published on the native grid or regridded. The low res DECK CMIP6 experiment published 'as is' was published both on the native grid and regridded. We also publish the regridded climatology files.

Publishing CMORized data

The CMIP6 CMORized data will be regridded and is  The CMIP CMORized data is regridded and split to have one variable per file, according to CMIP requirement.

Released E3SM Data

ESGF E3SM home page

Data on ESGF – LLNL ESGF node:

Data Available on HPSS

Pages on e3sm.org Describing Released E3SM Data

...

, see 

Long-Tems Archive on HPSS

 Full data from simulations is archive for long-term storage using zstash and either stored on HPSS system (if available) or will be zstashed and moved to LLNL to E3SM's petabyte storage system for the long-term archive. Some of this data is being made world-readable, and the users who have accounts on those system can access the data there. 

Released E3SM Data

See Publicly Available Data on ESGF or go to the E3SM public website https://e3sm.org/data/get-e3sm-data/.