...
RESULTS: All tests pass with roundoff level changes to c_k10h, and fail (detect statistically different results) with small changes in c_k10h. With the default thresholds, PGN is the most sensitive (detecting a statistical difference with a 1e-8 change, followed by TSC at 1e-2, and MVK at 2e-2). These results are correlated with the timescale of each test, with PGN looking at physics columns after 1 timstep, TSC time step convergence with 300 timesteps, and MVK examining 1 year climatologies.
MVK
...
30 member ensemble of ~ 1 year simulations. Takes about 1.3 hours on 18 nodes.
Hack to reuse same “-c” case to run multiple experiments:
rm -f run/*.nc (otherwise we get PIO run time errors)
add “clubb_c_k10h=0.40” to user_nl_eam_???? files
./case.submit
MVK_P24x1.ne4_oQU240.F2010-CICE (Compy, 2011/11)
c_10kh | Test Result | TestStatus.log Metrics threshold=13 |
|---|---|---|
0.35 (default) | PASS | |
0.36 | PASS | reject 7/121 |
0.38 | FAIL | reject 21/121 |
0.40 | FAIL | reject 50/121 |
MVK_P36x1.ne4_oQU240.F2010 (Anvil 2022/6)
c_10kh | Test Result | TestStatus.log Metrics threshold=13 |
|---|---|---|
0.35 (default) | PASS | |
0.36 | PASS | reject 9/121 |
0.38 | FAIL | reject 34/121 |
PGN
20 member ensemble of ~ 1 timestep simulations. Takes about 1 min on 16 nodes.
Hack to reuse same “-c” case to run multiple experiments:
rm -f run/*.nc (otherwise we get PIO run time errors)
add “clubb_c_k10h=0.40” to user_nl_eam_???? files
./case.submit
PGN_P32x1.ne4_oQU240.F2010
...
20 member ensemble of ~ 1 timestep simulations. Takes about 1 min on 16 nodes.
(Compy, 2011/11)
c_10kh | Test Result | TestStatus.log Metrics T test (t,p) |
|---|---|---|
0.35 (default) | PASS | (0.000, 1.000) |
0.350000001d0 | PASS | (-1.424, 0.169) |
0.35000001 | FAIL | (-2.542, 0.019); |
0.36 | FAIL | (-12.564, 0.000); |
Hack to reuse same “-c” case to run multiple experiments:
...
rm -f run/*.nc (otherwise we get PIO run time errors)
...
add “clubb_c_k10h=0.40” to user_nl_eam_???? files
...
PGN_P32x1.ne4_oQU240.F2010 (Anvil, 2022/6)
c_10kh | Test Result | TestStatus.log Metrics T test (t,p) |
|---|---|---|
0.350000001d0 | PASS | (-1.424, 0.169) |
0.350000005d0 | PASS | (-1.549, 0.136) |
0.35000001 | FAIL | (-2.542, 0.019) |
0.35000002 | FAIL | (-2.619, 0.016) |
TSC_P36x1.ne4_ne4.F2010-CICE
...