Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

EAM and SCREAM uses the HOMME dycore package.   HOMME has both Fortran and C++ versions with different options for the prognostic thermodynamic variable (preqx use temperature while theta-l uses potential temperature) and those in turn have some additional capabilities.

...

New namelist variables:  EAMv2 now allows the user to directly set the main dycore timesteps with new namelist options replacing "nsplit, rsplit and qsplit".   When using the new namelist options, the old variables a are set to:   nsplit=-1, rsplit=-1 and qsplit=-1.    For  The older namelist variables should be avoided, and they will removed when E3SM drops support for V1 compsets and the preqx dycore.   For information on the older namelist variables, see the PREQX section below.  

...

With RRM grids, the timesteps will be controlled by the highest resolution region.  So with an RRM grid with refinement down to NE120, the timesteps should be close to what we run on a uniform cubed-sphere NE120 grid.   The timesteps may need to be slightly smaller because of the deformed elements in the transition region.   With a hiqh quality RRM mesh ( Max Dinv-based element distortion metric <= 4, see Generate the Grid Mesh (Exodus) File for a new Regionally-Refined Grid) we can usually run with the expected dt_dyn and dt_tracer values, and only the viscosity timesteps need to be slightly reduced.  

Spreadsheet for looking at scaling with resolution of constant and tensor coefficient HV:

https://docs.google.com/spreadsheets/d/1LHTl2_A065pfdWC69OHmXvNL_v1484cXg7ZowhyEbPU/edit?usp=sharing

...

ResolutionTimestepsNamelist settingsNotesTested?
1 degree (NE30)

dtime=1800 

dt_tracer=1800
dt_remap=600
dt_dyn=dt_vis=dt_vis_q=300
dt_vis_tom=300



nu_top = 2.5e5    

se_tstep=300

dt_tracer_factor = 6
hypervis_subcycle_q=6

hypervis_scaling=0
nu=1e15

E3SMv2 NE30 does not use tensorHV to avoid having to retune. Should change in v3.

With dt_remap=1800, we see occasional (every 2-3 years) dp3d limiter activation, meaning that the model levels are approaching zero.  This appears to be due to strong divergence above tropical cyclones created by one of the parameterizations.

HS+topo(72L):  H and NH (H can run at t 360s but not 400s with either tstep_type=4 or 5).  dt_remap=600 runs with no limiter warnings, while dt_remap=900 crashes with dp3d<0 at surface.  

F-EAMv1-AQP1:  H and NH 

FAV1C-L:  H and NH


NE45

dtime=1200. 

dt_tracer=1200
dt_remap=400
dt_dyn=dt_vis=dt_vis_q=200

dt_vis_tom=200

nu_top=2.5e5

se_tstep=200
dt_tracer_factor=6
hypervis_subcycle_q=6




1/4 degree (NE120)

dtime=900  

dt_remap=150
dt_tracer=450  (could be as large as 650, but it has to divide 900)
dt_dyn=dt_vis=dt_vis_q=75
dt_vis_tom=75

nu_top=1e5

se_tstep=75

dt_tracer_factor = 6
hypervis_subcycle_q=6

CFL estimates suggest:

dt_vis_tom*nu_top <= 31*2.5e5

nu_top=2.5e5 would need
hypervis_subcycle_tom=3


HS+topo(72L):  H and NH (with dt_remap=75 and theta limiter to handle unphysical boundary layer)

F-EAMv1-AQP1:  H  and NH, both 72 and 128 levels (1+ years)

FC5AV1C-H01B:  NH 72L runs several years.   

12km

(NE256)

dtime=600  ???

dt_tracer=200
dt_remap=200/3.
dt_dyn=dt_vis=dt_vis_q=200/6.
dt_vis_tom = 200/6.


NOTE: these defaults were updated 2021/9 based on SCREAM v0.1 3km runs.  But NE256 is known to run stably at slightly larger timsteps:

dt_tracer=300
dt_remap=75
dt_dyn=dt_vis=dt_vis_q=37.5
dt_vis_tom = 37.5


nu_top=4e4
se_tstep=33.33333333333

dt_tracer_factor=6
hypervis_subcycle_q=6


nu_tom=4e4 is running at the code's estimate of the CFL limit with S=1.9


F-EAMv1-AQP1:

  • H-128L running with default timesteps. (1 month) 
  • NH-128L:
    • tstep_type=9:  Runs with dt=37.5/75/300/600.  crashes with dt=40/40/240/480 after 6d, dp3d<0 around layer 30.   
    • with tstep_type=10: dt=40/40/240/480  good (for 1 month).  

FC5AV1C-H01B:  NH 128L run for several months dt=37.5/75/300/600.  Occasional problems near coastlines - considering (  ) reducing dtime, increasing HV, tunning CLUBB

6km (NE512)

dtime=300 ???

dt_tracer=100
dt_remap=100/3.
dt_dyn=dt_vis=dt_vis_q=100/6.
dt_vis_tom=100/6.



nu_top=2e4
se_tstep=16.6666666666666

dt_tracer_factor = 6
hypervis_subcycle_q=6



CFL estimates suggest:

dt_vis_tom*nu_top <= 1.7*2.5e5,

nu_top=2.5e5 needs hypervis_subcycle_tom=13 

F-EAMv1-AQP1:

  • H-128L:  dt=20/40/120/240 good (ran 15d, but then died with "cloud cover" errors
  • NH-128L: 
    • tstep_type=9:  dt=20/40/120/240: crash bad EOS 2.4days.  dt=18.75/37.5/150/300, ran 20days.
    • tstep_type=10: dt=20/40/120/240 crash 3.1days.  bad dp3d layer 75

FC5AV1C-H01B:  NH 128L run for 1 day with dt=18.75/37.5/150/300, then NaNs in microphysics (not yet debugged)

3km (NE1024)

dtime=100

dt_tracer=50
dt_remap=16.6666
dt_dyn=dt_vis=dt_vis_q=8.3333
dt_vis_tom=8.3333

nu_top=1e4
se_tstep=8.3333333333333

dt_tracer_factor = 6
hypervis_subcycle_q=6


CFL estimates suggest:

dt_vis_tom*nu_top <= 0.43*2.5e5

with nu_top=2.5e5, 
hypervis_subcycle_tom=24


F-EAMv1-AQP1:

  • H-128L:  dt=10/20/60/120.  crashed with cloud_cover errors < 1d
  • NH-128L:  dt=9.375/18.75/75/150 ran 1d

FC5AV1C-H01B: 

SCREAM v0:  run for 40 days with constantHV, dt=9.375/18.75/75/75. 

SCREAM v0.1: switch to tensorHV (slightly less diffusion), needs dt_dyn<=9s, dt_tracer<60

RRM

dtime=???

dycore timesteps should be set based on the finest region in the RRM.  

nu_top=Uncertain - needs more research.  Should probably switch to tensor laplacian.  For NE30→NE120 grids, start with NE120 constant coefficient value, 1e5.  


RRM uses a tensor HV formulation which scales with resolution dx^3.0  (For preqx, we used a dx^3.2 scaling. )

To determine the effective HV coefficient at a given resolution "dx", use:

nu_tensor = nu_const *( 2*rearth /((np-1)*dx))^{hv_scaling} * rearth^{-4.0}.

i.e. tensor nu=3.4e-8 when used at 1 degree resolution (dx=110,000m, np=4, rearth=6.376e6) is equivalent to  1e15 m^4/s.  



...