During the V2 development process, we transitioned from CLUBB V1 to CLUBB V2. In CLUBB V2, a process controlled by c_k10 was modified to add finer control with c_k10 and c_k10h. The old behavior was recovered if c_k10h = c_k10 = 0.35. In V2 development, since c_k10h was a new parameter it was not set in the namelist and inheriting a default value of c_k10h = 1.0. This resulted in an unexpected degradation in the climate, which was not detected for several months. For more detailed background, see V2 Case Studies /wiki/spaces/CNCL/pages/3130818586
Can E3SM’s NBFB tests detect this difference? And how sensitive are they to changes in c_k10h?
...
RESULTS: All tests pass with roundoff level changes to c_k10h, and fail (detect statistically different results) with small changes in c_k10h. With the default thresholds, PGN is the most sensitive (detecting a statistical difference with a 1e-8 change, followed by TSC at 1e-32, and MVK at 2e-2). These results are correlated with the timescale of each test, with PGN looking at physics columns after 1 timstep, TSC time step convergence with 300 timesteps, and MVK examining 1 year climatologies.
...
c_10kh | Test Result (possible bug in scripts? fails for all values except when results are bfb) | Alternative test resultHui Wan 's criterion: PASS = , unless all values points in Pp_min plot > PASS thresholdin [5min,10min] range are FAIL | TestStatus.log Metrics region by region results Global, Land, Ocean |
|---|---|---|---|
0.35 (default) | PASS | PASS | PASS, PASS, PASS |
0.350001 | FAIL | PASS | PASS, PASS, PASS |
0.35001 | FAIL | PASS | FAIL, PASS, PASS pmin plot |
0.3501 | FAIL | PASS | PASS, PASS, PASS pmin plot |
0.351 | FAILFAIL | PASS | PASS, FAIL, PASS pmin plot |
0.36 | FAIL | FAIL | FAIL, FAIL, FAIL pmin.36.png |
0.37 | FAIL | FAIL | FAIL, FAIL, FAIL |
...