Page Comparison

...

These instructions assume you have already ported E3SM to your machine (you have it defined in config_machines.xml and can build and run cases.)

Almost all dependencies are in general optional for MOAB but are required for MOAB-in-E3SM. Features To build the MOAB library specifically for E3SM, the following dependencies are required and mandatory for scalable coupler workflows.MOAB has to be configured with these TPLs to work consistently with E3SM and to support both online and offline workflows. The appropriate features and tools needed within MOAB will get automatically built and enabled based on the dependencies when configuring the during MOAB libraryconfiguration.

• MPI: MOAB supports both MPICH- and OpenMPI-based libraries configured externally to enable scalable mesh manipulation algorithms. Required dependency for the MOAB-coupler. (Without MPI, only a serial version of MOAB will be built.) Use the same MPI library/wrappers as E3SM.

• HDF5: MOAB uses a native HDF5-based file format to represent the entire MOAB data model and this is the recommended interface and format for scalable I/O of MOAB meshes and associated data. If MPI is enabled, HDF5 should be built with parallel support. HDF5 support allows you to output any E3SM state maintained by MOAB in h5m files that are tremendously useful in debugging as they can be directly visualized with VisIt using the MOAB plugin (on a sphere and with the exact mesh). If you built HDF5 for E3SM (with parallel support), use that to enable HDF5 support in MOAB configuration. For MOAB-E3SM workflows, this is an optional dependency unless you expect to develop and debug MOAB driver.

• Parallel-NetCDF: MOAB library uses parallel-NetCDF (pnetcdf) library directly for reading NetCDF (v3) meshes in parallel and linear map weight files (for instance the r2o map). All other component maps are generally calculated online by the MOAB coupler. Use the same PNetCDF library as E3SM. For MOAB-E3SM workflows, this is a required dependency.

• Zoltan: Support for online or offline partitioning through Zoltan (optionally configured with dependency on ParMetis/Metis) can be utilized with the MOAB partitioner tool (mbpart). The runtime partitioning can be invoked with appropriate file read options (PARTITION_METHOD=RCBZOLTAN;). This dynamic rebalancing is important to achieve good scalability when reading nc files in parallel. For MOAB-E3SM workflows, this is a recommended required dependency.

• TempestRemap: Provides support for both offline and online remapping of Climate field data on unstructured spherical meshes. The offline remapping weights can be generated using the mbtempest tool. Refer to Offline remapping workflow with mbtempest for more details. For MOAB-E3SM workflows, this is a required dependency.

• Eigen3: A substitute for BLAS/LAPACK interfaces (if not available locally). This is a required dependency for MOAB when calculating weights with TempestRemap as we store the SparseMatrix linear map internally on every task and perform SpMV using the Eigen3 interfaces. For MOAB-E3SM workflows, this is a required dependency.

...

• Metis/ParMetis: MOAB can optionally use the Metis or ParMetis library for partitioning mesh files in an offline form through MOAB’s mbpart tool. This may be useful to pre-partition the mesh apriori in offline remapping workflows, especially when using Metis KWAY graph partitioning algorithm (MPAS).

• HDF5: MOAB uses a native HDF5-based file format to represent the entire MOAB data model and this is the recommended interface and format for scalable I/O of MOAB meshes and associated data. If MPI is enabled, HDF5 should be built with parallel support. HDF5 support allows you to output any E3SM state maintained by MOAB in h5m files, which are tremendously useful in debugging as they can be directly visualized with VisIt using the MOAB plugin (on a sphere and with the exact mesh). If you built HDF5 for E3SM (with parallel support), use that to enable HDF5 support in MOAB configuration.

• NetCDF: MOAB library optionally depends on the NetCDF(-C) libraries to compile the ExodusII reader/writer. If MPI is enabled, then the underlying HDF5 dependency for NetCDF should have parallel support i.e., serial HDF5 dependency will not pass internal configuration checks. If you built the NetCDF library for E3SM, consistently use that installation to enable NetCDF support in MOAB.

...

(Below use the same compiler, MPI and I/O libraries you have set up for E3SM. If you have multiple compiler-MPI options, you will need to build MOAB for each compiler family with corresponding TPL dependencies.)

• Override default compilers with environment variables or configure-time options with the compilers you use for E3SM: CC, CXX, FC, F77

• For the MPI installed for E3SM, use --with-mpi=$MPI_DIR

• For the PNetCDF installed, use --with-pnetcdf=$PNETCDF_DIR

• To simplify the process of building MOAB along with its dependenciesTPLs that are not used directly by E3SM, we currently offer a workflow to automatically pull the TPL sources and build it on the machine. These packages will be installed in the same location as the specified MOAB prefix (INSTALL_DIR).

...

• If you built NetCDF4 and HDF5, add --with-hdf5=$HDF5_DIR --with-netcdf=$NETCDF_DIR

• Execute the configure script with desired configuration options out-of-source (recommended) build directory. A typical MOAB configuration command with all TPLs enabled for E3SM pre-built for your machine might look like the one shown below, where $PACKAGE_DIR indicates the location of the installation prefix for the particular TPL package.

  Code Block

  $MOAB_SRC/configure CC=$E3SM_MPI_DIR/bin/mpicc \ CXX=$E3SM_MPI_DIR/bin/mpicxx \ FC=$E3SM_MPI_DIR/bin/mpif90 \ F77=$E3SM_MPI_DIR/bin/mpif77 \ --prefix=$MOAB_INSTALL_DIR \ --enable-debug --enable-optimize \ --with-mpi=$E3SM_MPI_DIR \ --with-hdf5=$E3SM_HDF5_DIR \ --with-netcdf=$E3SM_NETCDF_DIR \ --with-pnetcdf=$E3SM_PNETCDF_DIR \ --with-eigen3=$EIGEN3_INCLUDE_DIR \ --with-tempestremap=$TEMPESTREMAP_DIR \ --with-zoltan=$ZOLTAN_DIR

However, if external libraries like Eigen3, TempestRemap, and Zoltan are to be auto-downloaded and configured dynamically, then one can use the following command.

...

• Compile MOAB and supported tools: make -j4

• Verify configuration and build setup: make check (Some checks won’t may not pass without HDF5. Ignore those).

...

• Note: If running on machines that require a job submission, request an interactive node before you run make check

• To install the compiled libraries, headers, and tools in the prefix directory, run: make install

If you experience any issues during the configuration or linking phases of MOAB, please email the mailing list: moab-dev@mcs.anl.gov , or contact Vijay M , Iulian Grindeanu or #coupler on Slack for additional help.

Once you have verified MOAB has been built correctly, you need to update the CIME machine configuration to specify the MOAB_ROOT (point to the installation prefix).

Next, edit Edit the entry for your machine in config_machines.xml and make changes like below (example from pm-cpu configuration)

...

You can add “Vmoab” to any B watercycle or F case test.

Pre-built recipes (Conda)

...