...
If a simulation was run on systems with HPSS: run zstash to archive the data to the local HPSS tape. If the HPSS is not on NERSC, use Globus to transfer the zstash *.tar and index.db files to NERSC HPSS:
If data were archived on Theta HPSS, to transfer to NERSC HPSS, use the Globus endpoint alcf#dtn_hpss and NERSC HPSS. (Make sure to opt for “verify file integrity after transfer“ as a transfer and Sync option, which checks if source and destination file checksums match)
If a simulation was run on a system without HPSS (i.e., anvil and compy): run zstash to generate index.db and tar files locally [Note: this zstash feature is under development and will be available soon, ~within this month?available after version 0.4.1] and transfer these files to NERSC HPSS using Globus].
Make sure permissions are correct for files on HPSS.
If you keep seeing issues with a specific transfer between endpoints, you can email
support@globus.orgwith the transfer ID. They can look at the transfer logs and facilitate the conversation between the endpoint admins.
Notes on using zstash:
To archive output from an E3SM simulation located under $CSCRATCH/E3SM_simulations/20170731.F20TR.ne30_ne30.edison:
...