Page Comparison

...

• readonly COMPSET="WCYCL1850" : compset (configuration)

• readonly RESOLUTION="ne30pg2_EC30to60E2r2": resolution (low-resolution coupled simulation in this case)

  • ne30 is the number of spectral elements for the atmospheric dynamics grid, while pg2 refers to the physics grid option. This mesh grid spacing is approximately 110 km.

  • EC30to60E2r2 is the ocean and sea-ice resolution. The grid spacing varies between 30 and 60 km.

  • For simulations with regionally refined meshes such as the N American atmosphere grid coupled to the WC14 ocean and sea-ice, replace with northamericax4v1pg2_WC14to60E2r3.

• readonly CASE_NAME="v2.LR.piControl":

  • v2.LR is a short custom description to help identify the simulation.

  • piControl is the type of simulation. Other options here include , but are not limited to: amip, F2010.

• readonly CASE_GROUP="v2.LR":

  • This will let you mark multiple cases as part of the same group for later processing (e.g., with PACE).

  • Note: If this is part of a simulation campaign, ask your group lead about using a case_group label. Otherwise, please use a unique name to distinguish from existing case_group label names, i.g. “v2.LR“.

# Code and compilation

• readonly CHECKOUT="20210702": date the code was checked out on in the form {year}{month}{day}. The source code will be checked out in sub-directory named {year}{month}{day} under <code_source_dir>.

• readonly BRANCH="master": branch the code was checked out from. Valid options include “master”, a branch name, or a git hash. For provenance purposes, it is best to specify the git hash.

• readonly DEBUG_COMPILE=false : option to compile with DEBUG flag (leave set to false)

...

Before starting a long production run, it is highly recommended to perform a few short tests to verify:

1. The model starts without errors.

2. The model produces BFB (bit-for-bit) results after a restart.

3. The model produces BFB results when changing PE layout.

(1) can spare you from a considerable amount of frustration. Imagine submitting a large job on a Friday afternoon, only to discover Monday morning that the job started to run on Friday evening and died within seconds because of a typo in a namelist variable or input file.

...

To post-process a model run, do the following steps. Note that to post-process up to year n, then you must have short-term archived up to year n.

Install zppy

If you haven't already, check out the zppy repo in <code_source_dir>. Go to You can ask questions about zppy on https://github.com/E3SM-Project/zppy . Get path to clone by clicking the green "Code" button. Run git clone <path>./discussions/categories/general

Install zppy

Load the E3SM unified environment. For Chrysalis, this is: source /lcrc/soft/climate/e3sm-unified/load_latest_e3sm_unified_chrysalis.sh. Commands for other machines, and installation guide for development version of zppy can be found at https://e3sm-project.github.io/zppy/_build/html/main/getting_started.html

...

Simulations that are deemed sufficiently valuable should be archived using zstash for long-term preservation.

You can ask questions about zstash on https://github.com/E3SM-Project/zstash/discussions/categories/general

Compy, Anvil and Chrysalis do not have local HPSS. We rely on NERSC HPSS for long-term archiving.

...

Log into Globus, using your NERSC credentials: https://www.globus.org/

1. (Left hand side): Transfer from <the machine's DTN> <simulations_dir>/<case_name>/zstash

2. Click enter on path and "select all" on left-hand side

3. Transfer to NERSC DTN /global/cfs/cdirs/e3sm/<username>/E3SMv2/<case_name>

   1. Notice we're using cfs rather than scratch on Cori

4. Click enter on the path

5. Click “Transfer & Sync Options” in the center.

6. Choose:

   1. “sync - only transfer new or changed files” (choose “modification time is newer” in the dropdown box)

   2. “preserve source file modification times”

   3. “verify file integrity after transfer”

7. For “Label This Transfer”: “zstash <case_name> <machine name> to NERSC”

8. Click "Start" on the left hand side.

You should get an email from Globus when the transfer is completed. (On Chrysalis, for 165 years of data, this transfer takes ~13 hours).

...

Update simulation Confluence page with information regarding this simulation (For Water Cycle’s v2 work, that page is V2 Simulation Planning /wiki/spaces/ED/pages/2766340117 ) . In the zstash archive column, specify:

...