Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 15 Next »

When you need to make a new mpas*.part.N file because you changed the number of nodes an MPAS component runs on.  These instructions including adding the file to the repo for others to use.


Step-by-step guide for MPAS-Ocean


  1. See if "metis" or "gpmetis" is available on your platform and add it to your path.
    1. On Cray systems, gpmetis is available after `module load metis`.
    2. Do "module spider metis" to find the module to load on your system.
    3. Install it from conda
  2. Find the base mpas graph file called something like "mpas-o.graph.info.151031".  These files should be in inputdata/ocn/mpas-o respectively.
  3. Run the metis command with the last argument being the number of nodes for the partition.
    1. Example:  "gpmetis mpas-cice.graph.info.151010 120"   will generate a *.part.120 file for mpas-cice.
  4. Place the new file in the correct inputdata subdirectory for the model to find it.


Please do not do this for MPAS-Seaice, as the resulting graph files are 


Making better load balanced graph files for MPAS-Seaice

Maybe ask for help from the MPAS team.  But here are detailed instructions for those who want to give it a go:

http://mpas-dev.github.io/MPAS-Tools/stable/seaice/partition.html

MPAS-Seaice is rather poorly load balanced with standard MPAS graph partitions. This is because sea ice only exists in a very limited part of the domain. A script has been written that can generate somewhat better partitions. Each block is split into 3 non-contiguous sections: One for each polar region and a larger one for the equatorial region. For highly desired changes to this script contact Adrian Turner or Xylar Asay-Davis.

  1. Create a conda environment with mpas_tools and the no-MPI verison of ESMF:

    mamba create -y -n mpas_tools python=3.11 mpas_tools "esmf=*=nompi*"
mamba activate mpas_tools

  2. Download the required input files:
    https://web.lcrc.anl.gov/public/e3sm/mpas_standalonedata/mpas-seaice/partition/seaice_QU60km_polar.nc
    https://web.lcrc.anl.gov/public/e3sm/mpas_standalonedata/mpas-seaice/partition/icePresent_QU60km_polar.nc

  3. Run the sea ice graph generation tools

    prepare_seaice_partitions -i seaice_QU60km_polar.nc -p icePresent_QU60km_polar.nc -m <mesh_file> -o .
create_seaice_partitions -m <mesh_file> -o . -p mpas-seaice.graph.info.<creation_date> -n <core_count> <core_count> <core_count>

The mesh file is a ifle with the mesh you are trying to partition, typically seaice.*.nc or mpassi.*.nc in inputfiles/ice/mpas-seaice/<mesh_name>.  The creation date needs to match the creation date of other sea ice partition files in the directory (or it won't be found).  You can provide multiple core counts in one call to create_seaice_partitions, which will be much faster than calling that tool multiple times. 

Put the resulting mpas-seaice.graph.info.<cration_date>.part.<core_count> files in /lcrc/group/e3sm/public_html/inputfiles/ice/mpas-seaice/<mesh_name>.

  • No labels