EAM and SCREAM/EAMxx uses the HOMME dycore package. HOMME has both Fortran and C++ versions with different options for the prognostic thermodynamic variable (preqx use temperature while theta-l uses potential temperature) and those in turn have some additional capabilities.
...
Summary of HOMME capabilities available for different versions.
SL-transport (C++ only) | phys grid | non-hydrostatic option | GPU option | Allows RRM grids | E3SM version used | |
HOMME preqx Fortran | no | yes | no | yes (OpenACC)no | yes | v1 |
HOMME preqx C++ | no | yes | no | yes (Kokkos) | no | only available in standalone HOMME |
HOMME theta-l Fortran | yes (Fortran calling C++/Kokkos) | yes | yes | yes (SL only via Kokkos, in a branch)1 | yes | v2,v3 |
HOMME theta-l C++ | yes (in a branch)not yet | yes | yes | yes (Kokkos) | not yet | v3v4, SCREAM |
When building Kokkos for theta-l Fortran, the "cpu" target is used by default because running only the SL on the GPU would not give good overall performance.
New features
...
in EAMxx to be considered for EAM V3:
Physics mass adjustment due to phase change: apply locally to dp3d instead of to surface pressure. (reduces magnitude of artificial pressure work term)Adopt tensorHV for NE30 grids. (it is the default for all other grids)
Turn on topographic improvements (pgrad_correction=1, hv_ref_profiles=6) and switch to rougher “x6t” topography
Using the THETA dycore in E3SM:
As of 2020/12 in E3SM master (efb184d525), Theta is the default. Default settings will use SL transport and Hydrostatic mode. dycore running in hydrostatic mode with SL transport is the default.
To switch to the older Eulerian transport add to user_nl_eam
...
tstep_type=5 is the 3rd order accurate KGU53 explicit method used for hydrostatic simulations. It can be used with NH simulations with very small timestepes, ~0.5s
tstep_type=9 is an IMEX method, combining an explicit Butcher table (KGU53, used for most equations) and an implicit Butcher table (used for terms responsible for vertically propagating acoustic waves not present in hydrostatic models).
tstep_type=10 is another IMEX method which uses KG52 for the explicit table. It has a slightly larger stability region.
For reference: original page documenting XML changes to integrate theta-l: /wiki/spaces/NGDNA/pages/1010794497
If you have an older version that still uses preqx, you can enable the THETA dynamical core withOlder preqx dycore is still available, but wont support newer namelist options:
./xmlchange CAM_TARGET=theta-lpreqx
Detailed description of timesteps and viscosity parameters
...
With RRM grids, the timesteps will be controlled by the highest resolution region. So with an RRM grid with refinement down to NE120, the timesteps should be close to what we run on a uniform cubed-sphere NE120 grid. The timesteps may need to be slightly smaller because of the deformed elements in the transition region. With a hiqh quality RRM mesh ( Max Dinv-based element distortion metric <= 4, see Generate the Grid Mesh (Exodus) File for a new Regionally-Refined Grid) we can usually run with the expected dt_dyn and dt_tracer values, and only the viscosity timesteps need to be slightly reduced.
Spreadsheet for looking at scaling with resolution of constant and tensor coefficient HV:
https://docs.google.com/spreadsheets/d/1LHTl2_A065pfdWC69OHmXvNL_v1484cXg7ZowhyEbPU/edit?usp=sharing
...
hypervis_scaling=3.0 ( turn on tensorHV option, required for RRM grids and default for must cubed-sphere grids( default for grids as of 2023/4 )
nu=3.4e-8
WARNING on dtime: in the table below we give "dtime", the physics timestep. This is also a namelist option in EAM, but it will be ignored. The only way to set dtime is to set ATM_NCPL = ( 24*60*60 / dtime) in env_run.XML
...
Resolution | Timesteps | Namelist settings | Notes | Tested? |
|---|---|---|---|---|
7.5 degree | dtime=7200 dt_tracer=3600 | nu_top = 2.5e5 se_tstep=600 | Ultra low res for regression testing only. | |
1 degree (NE30) | dtime=1800 dt_tracer=1800 | nu_top = 2.5e5 se_tstep=300 If backward compatability is needed for V2 water cycle, change HV defaults toE3SM v2 uses constant coefficient HV, enabled with: | E3SMv2 NE30 does not use tensorHV to avoid having to retune. Should change in v3. With With larger dt_remap( dt_remap=1800), we see occasional (every 2-3 years) dp3d limiter activation, meaning that the model levels are approaching zero. This appears to be due to strong divergence above tropical cyclones created by one of the parameterizations. | HS+topo(72L): H and NH (H can run at t 360s but not 400s with either tstep_type=4 or 5). dt_remap=600 runs with no limiter warnings, while dt_remap=900 crashes with dp3d<0 at surface. F-EAMv1-AQP1: H and NH FAV1C-L: H and NH |
NE45 | dtime=1200. dt_tracer=1200 dt_vis_tom=200 | nu_top=2.5e5 se_tstep=200 | ||
1/4 degree (NE120) | dtime=900 dt_remap=150 | nu_top=1e5 se_tstep=75 | CFL estimates suggest: dt_vis_tom*nu_top <= 31*2.5e5 nu_top=2.5e5 would need | HS+topo(72L): H and NH (with dt_remap=75 and theta limiter to handle unphysical boundary layer) F-EAMv1-AQP1: H and NH, both 72 and 128 levels (1+ years) FC5AV1C-H01B: NH 72L runs several years. |
12km (NE256) | dtime=600 dt_tracer=200 NOTE: these defaults were updated 2021/9 based on SCREAM v0.1 3km runs. But NE256 is known to run stably at slightly larger timsteps: dt_tracer=300 | nu_top=4e4 | nu_tom=4e4 is running at the code's estimate of the CFL limit with S=1.9 | F-EAMv1-AQP1:
FC5AV1C-H01B: NH 128L run for several months dt=37.5/75/300/600. Occasional problems near coastlines - considering ( ) reducing dtime, increasing HV, tunning CLUBB dtime=600 used in: https://agupubs.onlinelibrary.wiley.com/doi/10.1029/2021MS002805 |
6km (NE512) | dtime=200 dt_tracer=100 | nu_top=2e4 | CFL estimates suggest: dt_vis_tom*nu_top <= 1.7*2.5e5, nu_top=2.5e5 needs hypervis_subcycle_tom=13 | F-EAMv1-AQP1:
FC5AV1C-H01B: NH 128L run for 1 day with dt=18.75/37.5/150/300, then NaNs in microphysics (not yet debugged) |
3km (NE1024) | dtime=100 dt_tracer=50 | nu_top=1e4 | CFL estimates suggest: dt_vis_tom*nu_top <= 0.43*2.5e5 with nu_top=2.5e5, | F-EAMv1-AQP1:
FC5AV1C-H01B: SCREAM v0: run for 40 days with constantHV, dt=9.375/18.75/75/75. SCREAM v0.1: switch to tensorHV (slightly less diffusion), needs dt_dyn<=9s, dt_tracer<60 |
RRM | dtime=??? dycore timesteps should be set based on the finest region in the RRM. | nu_top=Uncertain - needs more research. Should probably switch to tensor laplacian. For NE30→NE120 grids, start with NE120 constant coefficient value, 1e5. | RRM uses a tensor HV formulation which scales with resolution dx^3.0 (For preqx, we used a dx^3.2 scaling. ) To determine the effective HV coefficient at a given resolution "dx", use: nu_tensor = nu_const *( 2*rearth /((np-1)*dx))^{hv_scaling} * rearth^{-4.0}. i.e. tensor nu=3.4e-8 when used at 1 degree resolution (dx=110,000m, np=4, rearth=6.376e6) is equivalent to 1e15 m^4/s. |
...
old page documenting XML changes to integrate theta-l: E3SM NH integration: namelist defaults