...
Tests were run on Compy. All tests are first run with “-g” to generate baselines with E3SM master as of 2021/11/8. Then they are rerun with “-c” (compare to baseline), with various values of c_k10h
RESULTS: All tests pass with roundoff level changes to c_k10h, and fail (detect statistically different results) with small changes in c_k10h. With the default thresholds, PGN is the most sensitive (detecting a statistical difference with a 1e-8 change, followed by TSC at 1e-5, and MVK at 2e-2). These results are correlated with the timescale of each test, with PGN looking at physics columns after 1 timstep, TSC time step convergence with 300 timesteps, and MVK examining 1 year climatologies.
...