Abstract
As part of the ongoing CMDV effort, we are starting to rewrite and rethink many parts of the coupler to better meet ACME's near-term science and technical goals. The changes are multifold and are primarily motivated to simplify and improve the efficiency in the coupler workflow that is currently used in ACME.
We are working on integrating MOAB [1] in to the ACME suite in order to provide scalable communication infrastructure and to serve as a uniform representation for each of the model's (MPAS, HOMME) mesh and its specific decomposition. This would also allow for a unified checkpointing capability with parallel HDF5/NetCDF based I/O, for restarting runs.
The current offline model to compute the intersection of two grids on a sphere, followed by interpolation weights generation imposes a serious bottleneck in the computational workflow. In order to overcome this, and to allow dynamic decomposition in the physics models, we utilize the scalable intersection algorithms implemented in MOAB in combination with the high-order conservative remapping schemes exposed in TempestRemap [2] library. Leveraging this combination of libraries then provides a robust and accurate infrastructure for spatial remapping in the ACME coupler.
Additionally, modifications that expose runtime load-balancing in the current driver and algorithms to improve performance of the coupler by using a master-slave concept for partitioning are also being investigated on large-scale machines.
[1] MOAB, http://sigma.mcs.anl.gov
[2] Paul Ulrich, https://github.com/ClimateGlobalChange/tempestremap