As part of the efforts in the CMDV project, interfaces to integrate the MOAB unstructured mesh library with the TempestRemap remapping tool have been undertaken. Detailed information on the algorithmic and implementation aspects of this effort have been written in a manuscript submitted to Geoscientific Model Development [1]. This work has led to the development of a new offline remapping tool called mbtempest, which exposes the functionality to compute the supermesh or intersection mesh between two unstructured source and target component grids, in addition to using this supermesh for computing the remapping weights to project solutions between the grids. This functionality is part of the critical worflow with E3SM, where the generated remapping weights in the offline step are consumed by MCT at runtime to seamlessly transfer solution data between components (atm↔ocn, atm↔lnd, etc).
...
Note on DoF IDs for SE meshes
Currently, there is no consistent way to generate Consistently generating the global DoF ids for a spectral mesh in parallel, through MOAB is non-trivial since it requires information about all shared edges and a fixed traversal path that is independent of the parallel partitioning schema. While the actual ID assignments for DoFs are easy in serial, the assignment in parallel could be dependent depend on how the elements are ordered on the partitions, and may not even match the numbering in HOMME. Hence, the generated mapping weights may need a different permutation to be applied before passing it back to HOMME within E3SM. So, the recommended workflow for such a case is to run HOMME to get the
However, for standard Cubed-Sphere SE meshes, users can rely on an implementation in MOAB that uses TempestRemap to generate a CS mesh of a given resolution and use a numbering scheme that is consistent with HOMME to get the global DoF IDs. This temporary CS mesh is generated redundantly on every process only for the sake of computing the global IDs, out of which the relevant DoF numbering for SE GLL points are then stored as a user-specified MOAB tag. Once this is done, the rest of the workflow works seamlessly. This invalidates the need for the user to do any additional steps to compute remapping weights between CS-SE meshes and other lower-order discretizations.
For RRM grids or non-standard SE grids, the recommended workflow is to run HOMME to get the resolution of the grid required to generate the mapping file, and then store the DoF IDs as a GLOBAL_DOFS tag in MOAB, before being written to disk. This eliminates the need to get a consistent match when generating the weights in the MOAB mbtempest offline workflow since the tag in the file will be loaded in parallel along with the mesh during the setup phase of the tool.
Converting parallel h5m file to SCRIP file for E3SM
...
If steps (a)-(g) pass successfully, the MOAB libraries and tools, along with interfaces for TempestRemap will be installed in $INSTALL_PREFIX/moab directory. The offline remapping weight computation tool, mbtempest, will also be installed during this process and can then be used standalone to generate the weight files as needed.
References
1 Mahadevan, V. S., Grindeanu, I., Jacob, R., and Sarich, J.: Improving climate model coupling through a complete mesh representation: a case study with E3SM (v1) and MOAB (v5.x), Geosci. Model Dev. Discuss., https://doi.org/10.5194/gmd-2018-280, in review, 2018.