E3SM-Unified 1.9.0 release notes

Owned by Xylar Asay-Davis
Last updated: Sept 13, 2023 by Jill Chengzhu Zhang
4 min read

Dear E3SM Team,

A new version of E3SM-Unified is available: 1.9.0.  E3SM-Unified is a combination conda and spack environment that includes a large amount of analysis and pre- and post-processing software for E3SM users.  The new version includes updates to ChemDyg, E3SM-Diags, e3sm_to_cmip, MPAS-Analysis, MOAB, NCO, ncvis, xcdat, zppy, zstash and many more packages.

New and improved in 1.9.0

ChemDyg 0.1.4: This new package provides diagnostics for the interactive gas-phase chemistry, newly available in E3SM, to support routine testing and evaluation of the chemistry performance.

E3SM Diags 2.9.0: This new release adds one new set: mixed phase partition diagnostics (mp_partition), also known as T5050 diagnostics. The ARM diagnostics (arm_diags) set is enhanced with new aerosol-cloud-interaction and aerosol activation metrics developed by ARM infrastructure team. The aerosol budget diagnostics (aerosol_budget) is updated by including global mean burdens, source/sink budgets and lifetimes for aerosol species contributed by Aerosol Working Group. GPCP v3.2 precipitation data is added to replace an older version (v2.2). Surface relative humidity and wind speed data set derived from ERA5 are added as standard evaluation sets. Release note page.

e3sm_to_cmip 1.10.0: This minor release includes a major revamp in the implementation of land and atm variable handlers. Handlers are now defined a single yaml file rather than as individual Python modules. More information can be found in the Variable Handlers documentation page.

The __main__.py module for running e3sm_to_cmip has also been refactored with a new E3SMtoCMIP class, which gets instantiated as objects e3sm_to_cmip runs. This makes executing, debugging, and testing e3sm_to_cmip significantly easier compared to running from the CLI and hoping things work. More information can be found in the Development Guide documentation page.

In general, the documentation was updated to make it more readable and informative. A new bash script was added to serve as an example for how to execute e3sm_to_cmip for end-to-end processing and cmorization of variables.

MPAS-Analysis 1.9.0: This release adds new waves analysis and moves to Adusumilli et al. (2020) for Antarctic melt-rate observations as well as numerous minor enhancements and bug fixes listed in the MPAS-Analysis v1.9.0 release notes.

MOAB 5.5.0: Significant improvements in both robustness and performance for parallel, conservative map generation, and extensions to support bilinear map projections through TempestRemap v2.2.0.

NCO 5.1.7: Simplified invocation of quantization and codecs. CF-compliant longitude boundaries. Fixes for Intel compilers. NCZarr features. Vertical interpolation improvements. EAMxx support.

ncvis: A newly developed graphical tool that supports visualizing NetCDF data on unstructured grid.

xcdat 0.5.0: This long-awaited minor release includes feature updates to support an optional user-specified climatology reference period when calculating climatologies and departures, and improved support for using custom time coordinates in temporal APIs.

This release also includes a bug fix for singleton coordinates breaking the swap_lon_axis() function. Additionally, Jupyter Notebooks for presentations and demos have been added to the documentation.

zppy 2.3.0: This release introduces plugin functionality – users can now create plugins to add custom tasks to zppy. Additionally, users can now 1) choose to generate all variables in climo or ts by setting vars="", 2) directly choose the value for the ncclimo --parallel option, and 3) generate hemispheric time-series plots. For a full explanation of changes, see https://github.com/E3SM-Project/zppy/releases.

zstash 1.4.0: This release adds a couple new options. The --include option allows users to specify which files to include. The --follow-symlinks option allows users to copy symlinks, which is helpful in preventing broken links. For a full explanation of changes, see https://github.com/E3SM-Project/zstash/releases.

Activation

As in previous versions, you can access the environment as usual by sourcing an activation script:

Acme1:

source /p/user_pub/e3sm_unified/envs/load_latest_e3sm_unified_acme1.sh

Andes:

source /gpfs/alpine/proj-shared/cli115/e3sm-unified/load_latest_e3sm_unified_andes.sh

Anvil:

source /lcrc/soft/climate/e3sm-unified/load_latest_e3sm_unified_anvil.sh

Chicoma:

source /usr/projects/e3sm/e3sm-unified/load_latest_e3sm_unified_chicoma-cpu.sh

Chrysalis:

source /lcrc/soft/climate/e3sm-unified/load_latest_e3sm_unified_chrysalis.sh

Compy:

source /share/apps/E3SM/conda_envs/load_latest_e3sm_unified_compy.sh

Frontier:

source /ccs/proj/cli115/software/e3sm-unified/load_latest_e3sm_unified_frontier.sh

Perlmutter (CPU nodes):

source /global/common/software/e3sm/anaconda_envs/load_latest_e3sm_unified_pm-cpu.sh

Details

After nearly 3 months of testing, the new version has been deployed on our supported machines: Acme1, Andes, Anvil, Chicoma, Chrysalis, Compy, Frontier and Perlmutter. We have dropped support for Cori-Haswell (decommissioned). Note: We encourage users at OLCF to use Andes, rather than Frontier, for processing and analysis.

On 6 machines (Anvil, Chicoma, Chrysalis, Compy, Frontier and Perlmutter) there are 6 packages of interest -- ESMF, ILAMB, MOAB, NCO, TempestExtremes and TempestRemap -- that have been built with Spack using system compilers and MPI libraries.  When you load E3SM-Unified on a compute node, you will have access to these versions, which can be run in parallel and which will typically run more efficiently than their counterparts in conda packages.

Next version

Testing of the next version (1.10.0) is planned to begin in early February 2024 and deployment is expected in early March.  To request packages and versions to include, make a comment on Next Version.

As always, please email me if there are any questions, or post an issue at https://github.com/E3SM-Project/e3sm-unified/issues.

Cheers,

Xylar and the Infrastructure Team