2020-07-23 All-Hands Presentation Meeting Notes

Owned by Luke Van Roekel
Last updated: Sept 09, 2021 by Renata McCoy

Presenter: Peter Thornton

Title:

Progress toward a 1km offline land simulation capability for North America

Abstract:

With E3SM focusing part of its Phase 2 coupled simulation effort on simulations using a regionally-refined grid over North America, there is value in having an offline land science focus for the same region. One benefit of that approach would be the delivery of a “gold standard” set of results describing the surface weather and climate, as well as the energy, water, carbon, and biogeochemistry processes over the continent. These results could serve as a point of reference for coupled simulations, providing evaluation metrics for improvements in climate prediction skill when switching from standard model resolution to the regionally-refined North American nested grid. We know from previous offline land simulations that the quality of driving data has a major impact on the fidelity of process simulation outputs. Observational data resources over North America make it possible to derive very high-quality forcing information, at finer resolution than the proposed regional refined mesh. This allows aggregation of offline results to the coupled simulation grid, facilitating the construction of uncertainty ranges for the evaluation of the regionally-refined coupled results.

Over the past year a team at ORNL has developed the capability to execute these high resolution simulations, including development of a new sub-daily 1km surface weather forcing dataset for North America, and modification of ELM to make efficient use of GPU resources on Summit to carry out the large calculation and manage the IO streams. In this presentation we summarize the main efforts and provide a few illustrative examples of how the tasks have been accomplished. We use the OpenACC framework to add new capability for data movement and task parallelism on the Nvidia GPU devices on Summit, and we use the native MPI parallelism in E3SM to operate over multiple Summit nodes and multiple GPU devices per node. We highlight the use of a sophisticated functional unit testing framework to expedite the analysis, testing, and parallelization of ELM modules and sub-modules for GPU implementation. We provide current performance statistics and indicate ongoing work to optimize performance. A major conclusion is that the deep copy capability of OpenACC allows us to use the nested subgrid hierarchical data structure of ELM on GPUs with few modifications.

Date

 

Time

  • PT: 8:30 am
  • ET: 11:30 am

Call Info

  • web session:   https://global.gotomeeting.com/join/570361173                   
  • call number:    (571) 317-3122 Access Code: 570-361-173,            If busy, use alternate number: (773) 945-1029

    Joining from a video-conferencing room or system?       Dial: 67.217.95.2##570361173 ,  Cisco devices: 570361173@67.217.95.2 

Attendees


Presentation

Time
Title
Presenter
Presentation
Recording
Notes

30 min


Progress toward a 1km offline land simulation capability for North AmericaPeter Thorntonthornton high-res land july 2020.pptx

4 Movie (on the E3SM YouTube Channel)