...
A Point on Policy: We will strive to vet the structural integrity of datasets we are presented for publication. We have certain tools to test for time indexing issues, time gaps, and other problems occasionally discovered in a dataset, and take action to correct such issues, involving the data authors where necessary. Where we lack the tools to conduct such checks, we will make a best-effort to publish the data "as is", where specifically requested. In particular, when datasets are comprised of files across multiple archive or archive paths (due most often to major restarts such as runs interrupted by a change in available hardware), and there is overlap of identically-named files that differ in content, but address the same time-period, we will default to accept the "more recently creation-dated" files as comprising the intended dataset content.
...
A subset of data for which the simulation experiment belongs to some MIP (for example CMIP6, HiResMIP,...etc) can also be published as part of the intercomparison effort and will be published to a MIP project space in ESGF (https://esgf-node.llnl.gov/search/cmip6/?institution_id=E3SM-Project). In this case the data has to adhere to the strict formatting rules set by a MIP project. We use CMOR to rewrite the data to the desired standard, we call it CMORized output. The CMIP CMORized data is regridded and split to have one variable per file, according to CMIP requirement, see
3) Long-
...
Term Archive on HPSS
Full data from simulations is archive for long-term storage using zstash and either stored on HPSS system (if available) or will be zstashed and moved to LLNL to E3SM's petabyte storage system for the long-term archive. Some of this data is being made world-readable, and the users who have accounts on those system can access the data there.
...