...
./xmlchange MAX_TASKS_PER_NODE=42
./xmlchange MAX_MPITASKS_PER_NODE=42 # i.e. NTHRDS is 1
./xmlchange NTASKS=42*100 # all comps stacked on 100 nodes
./xmlchange PSTRID_ATM=7 # i.e. ATM procs stride 7 (=$NCPU/$NGPU = 42/6) with 6 ATM procs per node
./xmlchange NTASKS_ATM=6*100 # ATM is now at 6 procs per node
Process striding helps to
...
Exclusive process stride – EXCL_STRIDE
In addition to steps 1--4 5 above,
./xmlchange EXCL_STRIDE_ATM=7 # i.e. only ATM runs on stride-7 procs, other components on the remaining procs
Exclusive striding mutually excludes ATM and non-ATM components from executing on cores assigned to ATM. This minimizes any core-sharing side-effects.
How to verify
...
component-to-
...
task placement
./xmlchange INFO_MPROF=2 # to log memory usage profiling from every MPI process
zgrep "comps" run/e3sm.log*.gz # and look for "(pe= $task_id comps= cpl ICE GLC WAV IAC ESP)"
...