Page Comparison

When you need to make a new mpas*.part.N file because you changed the number of nodes an MPAS component runs on.  These instructions including adding the file to the repo for others to use.

Step-by-step guide for MPSMPAS-Ocean

1. See if "metis" or "gpmetis" is available on your platform and add it to your path.
   1. On Cray systems, gpmetis is available after `module load metis`.
   2. Do "module spider metis" to find the module to load on your system.
   3. Install it from conda
2. Find the base mpas graph file called something like "mpas-o.graph.info.151031".  These files should be in inputdata/ocn/mpas-o respectively.
3. Run the metis command with the last argument being the number of nodes for the partition.
   1. Example:  "gpmetis mpas-cice.graph.info.151010 120"   will generate a *.part.120 file for mpas-cice.
4. Place the new file in the correct inputdata subdirectory for the model to find it.

...

Please do not do this for MPAS-Seaice, as the resulting graph files are are not well load balanced (many cores are wasted on equatorial regions without sea ice).

Making better load balanced graph files for MPAS-Seaice

Maybe ask for help from the MPAS team.  But here are detailed instructions for those who want to give it a go:

...

1. Create a conda environment with mpas_tools and the no-MPI verison version of ESMF:

   Code Block
   mamba create -y -n mpas_tools python=3.11 "mpas_toolstools>=0.20.0" "esmf=*8.2.0=nompi*" mamba activate mpas_tools

   Download

   
   

2. Run the sea ice graph generation tools

   Code Block
   cd /lcrc/group/e3sm/public_html/inputdata/ice/mpas-seaice/<mesh_name> simple_seaice_partitions -m <mesh_file> -p partitions/mpas-seaice.graph.info.<creation_date> -n <core_count> <core_count> <core_count>

   If you are not on LCRC, you need to download the required input files somewhere:
   https://web.lcrc.anl.gov/public/e3sm/mpas_standalonedata/mpas-seaice/partition/seaice_QU60km_polar.nc
   https://web.lcrc.anl.gov/public/e3sm/mpas_standalonedata/mpas-seaice/partition/icePresent_QU60km_polar.nc
   

   Run the sea ice graph generation tools

   Code Blockprepare_seaice_partitions -i seaice_QU60km_polar.nc -p icePresent_QU60km_polar.nc -m <mesh_file> -o . create_seaice_partitions -m <mesh_file> -o . -p mpas-seaice.graph.info.<creation_date> -n <core_count> <core_count> <core_count>

   and then point to them with the -d.
   
   Group read/write and world read permission are added automatically, so you should not need to chmod the files.

The mesh file is a ifle file with the mesh you are trying to partition, typically seaice.*.nc or mpassi.*.nc in inputfiles/ice/mpas-seaice/<mesh_name>.  The creation date needs to match the creation date of other sea ice partition files in the partitions directory (or it won't be found).  You can provide multiple core task counts in one call to createsimple_seaice_partitions, which will be much faster than calling that tool multiple times.  Put the resulting mpas-seaice.graph.info.<cration_date>.part.<core_count> files in /lcrc/group/e3sm/public_html/inputfiles/ice/mpas-seaice/<mesh_name>The tool should take care of setting permissions as well.