...
Code Block | ||
---|---|---|
| ||
export CODE_ROOT=~/gitwork/scream/ #or wherever you cloned the scream repo export COMPSET=F2010-SCREAMv1 #or whatever compset you want export RES=ne4_ne4 #or whatever resolution you want export PECOUNT=16x1 # Number of MPIs by number of threads. Should be divisible by node size export CASE_NAME=${COMPSET}.${RES}.${PECOUNT}.test1 #name for your simulation. export QUEUE=pdebug #whatever the name of your debug or batch queue is export WALLTIME=00:30 #HH:MM on Summit, HH:MM:SS on other machines export COMPILER=intel #which compiler to use (can be omitted on some machines) |
Resolution optionsGrid options (just showing the best, most supported versions here):
Resolution | Grid name (aka $RES) | Notes |
---|---|---|
ne4 | ne4_ne4 | not sure about pg2? |
ne30ne30 | ne30pg2_ne30ne30pg2 | |
ne120ne120 | ne120pg2_r0125_oRRS18to6v3ne120pg2 | |
ne256 | ne256np4ne256pg2_oRRS18to6v3ne256pg2 | |
ne512 | ne512_r0125_oRRS18to6v3 | we don’t have a pg2 grid yet. This one is sketchy |
ne1024 | ne1024pg2_ne1024pg2 |
Suggested PECOUNTs (not necessarily performant, just something to get started). Note that EAMv1 currently uses something like 0.04 to 0.07GB/element, so make sure you don’t add more elements/node than you have memory for.
...
Step 6: Change SCREAM settings
As of , this This is done by modifying namelist_scream.xml either by hand or by using the atmchange
function which now comes bundled when you create a case. Explore namelist_scream.xml for variables you might want to change (but you shouldn’t have to change anything to run).
...
You can check it’s progress via squeue -u <username>
on LLNL and NERSC systems. Use jobstat -u <username>
or bjobs
on Summit. Kill jobs with bkill <jobid>
on Summit. Model output will be in the run subdirectory.
Rules for changing output mid-run
In CAM/EAMf90, changing your output at all mid-run requires doing a “branch run”. In EAMxx, users are welcome to add new output streams or stop existing streams any time they restart. This is enabled by EAMxx including the output frequency and averaging type in the name of all output files, so if you modify either of these things, you will create a new and unique output stream.
You may need to add Perform Restart: false
to any new output stream you create mid-run (there’s still some debate about this).
** Bonus Content: How to run at an unsupported resolution! **
...
Change Vertical__Coordinate__Filename to use the initial condition file for your new resolution
Change Filename under Initial__Conditions → Physics__GLL subsection to also use that new initial condition file
Change SPA__Remap__File to use one appropriate to map ne30 to your new resolution
Change se_ne as appropriate
change se_tstep and nu_top (recommended defaults for these and dtime are given in the awesome table on the EAM's HOMME Dycore Recommended Settings (THETA) page
** v1 ne256 notes (06/09/2022) **
Currently, getting ne256 v1 through initialization requires:
Setting:
Code Block ./xmlchange EPS_AGRID=1.e-9
Setting:
Code Block ./atmchange cubed_sphere_map=2
For SPA remap file point to (has been uploaded to E3SM input data server as of 06/09/2022) atm/scream/init/map_ne30np4_to_ne256np4_mono.20220609.nc
Make sure you are compiling with 128 vertical levels (currently, SCREAM compiles with 72 layers by default) .
./xmlchange SCREAM_CMAKE_OPTIONS="SCREAM_NP 4 SCREAM_NUM_VERTICAL_LEV 128 SCREAM_NUM_TRACERS 10"Code Block