Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Code Block
breakoutModewide
export CODE_ROOT=~/gitwork/scream/ #or wherever you cloned the scream repo
export COMPSET=F2010-SCREAMv1      #or whatever compset you want
export RES=ne4_ne4                 #or whatever resolution you want
export PECOUNT=16x1                # Number of MPIs by number of threads. Should be divisible by node size
export CASE_NAME=${COMPSET}.${RES}.${PECOUNT}.test1  #name for your simulation.
export QUEUE=pdebug                #whatever the name of your debug or batch queue is
export WALLTIME=00:30              #HH:MM on Summit, HH:MM:SS on other machines
export COMPILER=intel              #which compiler to use (can be omitted on some machines)

Resolution optionsGrid options (just showing the best, most supported versions here):

Resolution

Grid name (aka $RES)

Notes

ne4

ne4_ne4

not sure about pg2?

ne30ne30

ne30pg2_ne30ne30pg2

ne120ne120

ne120pg2_r0125_oRRS18to6v3ne120pg2

ne256

ne256np4ne256pg2_oRRS18to6v3ne256pg2

ne512

ne512_r0125_oRRS18to6v3

we don’t have a pg2 grid yet. This one is sketchy

ne1024

ne1024pg2_ne1024pg2

Suggested PECOUNTs (not necessarily performant, just something to get started). Note that EAMv1 currently uses something like 0.04 to 0.07GB/element, so make sure you don’t add more elements/node than you have memory for.

...

Step 6: Change SCREAM settings

As of , this This is done by modifying namelist_scream.xml either by hand or by using the atmchange function which now comes bundled when you create a case. Explore namelist_scream.xml for variables you might want to change (but you shouldn’t have to change anything to run).

...

You can check it’s progress via squeue -u <username> on LLNL and NERSC systems. Use jobstat -u <username> or bjobs on Summit. Kill jobs with bkill <jobid> on Summit. Model output will be in the run subdirectory.

Rules for changing output mid-run

In CAM/EAMf90, changing your output at all mid-run requires doing a “branch run”. In EAMxx, users are welcome to add new output streams or stop existing streams any time they restart. This is enabled by EAMxx including the output frequency and averaging type in the name of all output files, so if you modify either of these things, you will create a new and unique output stream.

You may need to add Perform Restart: false to any new output stream you create mid-run (there’s still some debate about this).

** Bonus Content: How to run at an unsupported resolution! **

...

  1. Change Vertical__Coordinate__Filename to use the initial condition file for your new resolution

  2. Change Filename under Initial__Conditions → Physics__GLL subsection to also use that new initial condition file

  3. Change SPA__Remap__File to use one appropriate to map ne30 to your new resolution

  4. Change se_ne as appropriate

  5. change se_tstep and nu_top (recommended defaults for these and dtime are given in the awesome table on the EAM's HOMME Dycore Recommended Settings (THETA) page

** v1 ne256 notes (06/09/2022) **

Currently, getting ne256 v1 through initialization requires:

  1. Setting:

    Code Block
    ./xmlchange EPS_AGRID=1.e-9
  2. Setting:

    Code Block
    ./atmchange cubed_sphere_map=2
  3. For SPA remap file point to (has been uploaded to E3SM input data server as of 06/09/2022) atm/scream/init/map_ne30np4_to_ne256np4_mono.20220609.nc

  4. Make sure you are compiling with 128 vertical levels (currently, SCREAM compiles with 72 layers by default) .

    Code Block./xmlchange SCREAM_CMAKE_OPTIONS="SCREAM_NP 4 SCREAM_NUM_VERTICAL_LEV 128 SCREAM_NUM_TRACERS 10"