When you need to make a new mpas*.part.N file because you changed the number of nodes an MPAS component runs on. These instructions including adding the file to the repo for others to use.
Step-by-step guide
Download script inputTreeDownload.sh
Run inputTreeDownload.sh to check out just the directory structure of the repository. This takes about 15 minutes. This is to avoid having to download the entire repository.
- See if "metis" or "gpmetis" is available on your platform and add it to your path.
- On Cray systems, gpmetis is available after `module load metis`.
- On Blues and Anvil, gpmetis is loaded with `soft add +metis-5.0.3`.
- On Mira, gpmetis can be loaded with `export PATH=/soft/libraries/alcf/current/xl/METIS/bin/:$PATH`
- You'll need to download the files for the grid you want to add a partition for:
- svn co --depth=files https://acme-svn2.ornl.gov/acme-repo/acme/inputdata/<path-to-dir> acme/inputdata/<path-to-dir>
- Find the base mpas graph file called "mpas-cice.graph.info.151020" or "mpas-o.graph.info.151031". These files should be in inputdata/ice/mpas-cice and inputdata/ocn/mpas-ocn respectively.
- Run the metis command with the last argument being the number of nodes for the partition.
- Example: "gpmetis mpas-cice.graph.info.151010 120" will generate a *.part.120 file for mpas-cice.
- Place the new file in the correct inputdata subdirectory for the model to find it.
- svn add and svn commit
Bonus round: Making better load balanced graph files for MPAS-Seaice
MPAS-Seaice is rather poorly load balanced with standard MPAS graph partitions. This is because sea ice only exists in a very limited part of the domain. A script has been written that can generate somewhat better partitions. Each block is split into 3 non-contiguous sections: One for each polar region and a larger one for the equatorial region. For highly desired changes to this script contact Adrian Turner.
- Download the MPAS-Tools repo.
- Install the mesh conversion tools (MPAS-Tools/grid_gen/mesh_conversion_tools)
- Ensure that your python build includes the netCDF4 library (http://unidata.github.io/netcdf4-python/)
- Run the sea ice graph generation script (MPAS-Tools/grid_gen/seaice_grid_tools)
usage: gen_seaice_mesh_partition.py [-h] -m MESHFILENAME -n NPROCS
[-c MPASCULLERLOCATION] [-o OUTPUTPREFIX]
[-p] [-g METIS]
Create sea ice grid partition
optional arguments:
-h, --help show this help message and exit
-m MESHFILENAME, --mesh MESHFILENAME
MPAS mesh file
-n NPROCS, --nprocs NPROCS
number of processors
-c MPASCULLERLOCATION, --culler MPASCULLERLOCATION
location of cell culler
-o OUTPUTPREFIX, --outprefix OUTPUTPREFIX
output graph file prefic
-p, --plotting create diagnostic plotting file of partitions
-g METIS, --metis METIS
name of metis utility