There are two major parts to the proper assimilation of simulation run data to the LLNL E3SM local archives. The first part involves the physical (Globus) transfer of the zstash-archived data to its destination directory. The second part is the discovery and codification of the contained material, enabling subsequent automated review and extraction as needed.
Archive Acquisition
By a process often involving mystical elements, you may learn of the need to acquire a simulation archive. You may hear voices indicating a confluence page, managed by a simulation project/group, detailing the runs conducted in a cloud of arcane verbiage, from which you must locate and decipher the following key treasure items
Campaign | Presently one of { BCG-v1, CRYO-v1, DECK-v1, HR-v1 }, although new campaigns are expected to appear |
|---|---|
Model | Presently one of { 1_0, 1_1, 1_1_ECA, 1_2 }, although new models are expected to appear |
Experiment(s) | Consulting tea leaves, burnt entrails, or the principals involved in conducting the simulations is recommended here. Do not expect the experiment name to support human comprehension in favor of compactness. |
Ensemble | Usually “ens1”, although “ens2” through “ens5” have appeared in the night sky. These must be embedded in the experiment name, when ens[n] > ens1 |
Archive Path(s) | This should be the NERSC HPSS path(s) to the zstash-formatted archive(s) associated with the four elements above. Take note of the archive_path_leaf_directory_name(s). |
With the above 5 elements in hand, you can craft the receiving directory or directories, “/p/user_pub/e3sm/archive/model/campaign/archive_path_leaf_directory_name(s).
You may then log in to Globus, activate the credentials for access to NERSC HPSS and to (currently) acme1.llnl.gov, open these in the corresponding Globus “File_Manager” transfer panes, navigate to the given NERSC HPSS archive path in one pane, and to the newly-created receiving archive path in the other pane, and launch the transfer(s). If there are multiple archives to transfer, and fortune shines upon you (the source directories all have a common parent directory), you can simply highlight multiple source directories in the NERSC pane, indicate the receiving parent directory in the acme1 pane ( /p/user_pub/e3sm/archive/model/campaign/ ) and all of the archives can be transferred in one pass, each creating the proper archive leaf-path directory. Otherwise you will need to conduct each transfer individually.
Archive Path-Mapping
(placeholder)