...
- Choose a total number of tasks that is evenly divisible by the number of cores/node for your machine (e.g. asking for 14 total cores on a machine with 12 cores/node is dumb because you will be charged for 24 cores and 10 of them will sit idle).
Choose NTASKS_ATM so it evenly divides the number of spectral elements in your atmos grid. The number of elements is related to the number of atm grid points by ??? Having uneven numbers of elements per core is possible, but makes the model run much more slowly. The table below shows the # elements and acceptable core counts for ACME atm resolutions:
atm res # elements
acceptable core counts ne30 5400 5400,2700,1800,1350,1080,900,675,600,540,450,350,300,270, ...
ne120 ??? ??? - The MPAS components
work well at any core count but require mapping files of the form mpas-cice.graph.info.<ICE_CORE_COUNT>.nc and mpas-o.graph.info.<OCN_CORE_COUNT>.nc. These files are automatically downloaded from https://acme-svn2.ornl.gov/acme-repo/acme/inputdata/ocn/mpas-o/ and https://acme-svn2.ornl.gov/acme-repo/acme/inputdata/ice/mpas-cice/ by the ACME model, but may not exist yet if nobody has used these core counts yet. It is trivial to generate these files though. On edison, you can typemodule load metis/5.1.0
gpmetis <graph_file> <# pes>
where graph_file is something like https://acme-svn2.ornl.gov/acme-repo/acme/inputdata/ocn/mpas-o/oRRS15to5/mpas-o.graph.info.151209 and # pes is the number of cores you want to use for that component. For single component (e.g CORE-forced ocean, AMIP atmos) compsets, you might as well use a serial configuration because only the active component will consume appreciable time.
NOTE: this page is not done yet.
Testing PE Layout Timing: