Versions Compared

Old Version 59

changes.mady.by.user Qi Tang

Saved on

compared with

New Version Current

changes.mady.by.user Qi Tang

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • : configuration and test of future projection run P1 (first ensemble member)
  •  : completed tests and submitted all 5 ensemble members
  •  : on hold due to quota issues
  •  : quota issue solved
  • : P2 simulation completed
  •  : P1 simulation completed
  •  : P3 simulation completed
  •  : P4 simulation completed
  • : P5 simulation completed

20190807.DECKv1b_P1_SSP5-8.5.ne30_oEC.cori-knl

...

#Simulation periodYearsNodesBFBFLAGTime stampDuration (hr)Raw core hoursCost per yearEffective SYPDNotes

1

2015-01-01 - 2021-05-01


125

true

190822-131413






22021-05-01 - 2021-08-21
125true190824-224204




32021-07-01 - 2023-08-02
125true190825-013247




42023-07-01 - 2023-10-08
125true190825-201045




52023-09-01 - 2023-11-22
125true190825-231312




62023-11-01 - 2024-01-01
125true190827-0047021.21
57,6993.33
72024-01-01 - 2030-01-016125true190827-10411446.34
61,7903.11
82030-01-01 - 2036-01-016125true190829-11133744.17
58,8983.26
92036-01-01 - 2038-09-172125true190906-005305



  • slurmstepd: error: *** STEP 24372975.0 ON nid06667 CANCELLED AT 2019-09-08T00:53:47 DUE TO TIME LIMIT ***

102038-01-01 - 2044-01-016125true190914-11502442.97
57,2953.35
112044-01-01 - 2050-01-016125true190916-13235142.78
57,0353.37
122050-01-01 - 2056-01-016125true190920-07140945.09
60,1133.19
132056-01-01 -
125true190922-162447



  • forrtl: error (65): floating invalid
142056-01-01 - 2057-01-061125true190925-133248



  • srun: error: Node failure on nid03307
152057-01-01 - 2063-01-016125true190928-22011143.75
58,3373.29
162063-01-01 - 2069-01- 016125true191002-215103

20190821.DECKv1b_P2_SSP5-8.5.ne30_oEC.cori-knl

Cori-knl jobs

2015 2015112319082213285120172022-09-01 - 20232029
#Simulation periodYearsNodesBFBFLAGTime stampDuration (hr)Raw core hoursCost per yearEffective SYPDNotes

1

46.00
61,3383.13
172069-01-01 - 2075-01-016125true191006-22015-11-01 - 2016-10-11125true190823-05505632016-09-01 - 2016-12-2202290343.11
57,4803.34
182075-01-01 - 2077-11-052125true190823-14515542016-11191008-005323



  • power outage
192077-01-01 - 20172083-0201-22016125true190823191013-191053200343544.44
59,2493.24
202083-01-01 - 20172089-0401-22016125true190824191016-01291507472162017-03-01 - 2022-10-21125true190824-112956742.93
57,2463.35
212089-01-01 - 2095-01-016125true190827191018-004702090935242.4858
5956,3507683.2438
82220232095-01-01 - 20292101-01-016125true190827191020-1030131026334544.99
6159,3219883.1392029-01-01 - 2029-01-25125true190829-092132
  • node failure
1020

20190821.DECKv1b_P2_SSP5-8.5.ne30_oEC.cori-knl

Cori-knl jobs

2035010161908291338126666
#Simulation periodYearsNodesBFBFLAGTime stampDuration (hr)Raw core hoursCost per yearEffective SYPDNotes

1

2015-01-01 - 2015-11-23


125

true

190822-

132851

42.5956,7823.38




11203522015-11-01 - 01 2016- 204010-11-145
125true190906190823-012450

  • slurmstepd: error: *** STEP 24373034.0 ON nid05169 CANCELLED AT 2019-09-08T01:25:10 DUE TO TIME LIMIT ***

122040055056




32016-09-01 - 01 2016- 2046-01-0112-22
125true190914190823-11502414515543.4657,9443.3113




204642016-11-01 - 2017-02-22
125true190823-191053




52017-01-01 - 20522017-0104-0122
125true190916190824-19505201291542.5256,6953.3914




2052-0162017-03-01 - 20582022-0110-0121
125true190920190824-07140911295643.5258,0253.31




152058-0172022-09-01 - 20642023-01-01
125true190922190827-162447004702432.6448
5859,1903503.3024
16820642023-01-01 - 20702029-01-016125true190928190827-2116561030134445.5099
5961,3283213.2413
17920702029-01-01 - 2029-01- 25
125true191003-142531

20190821.DECKv1b_P3_SSP5-8.5.ne30_oEC.cori-knl

Cori-knl jobs

2015 201504192015-03-01 - 2016-10-092029-032036-032044-
#Simulation periodYearsNodesBFBFLAGTime stampDuration (hr)Raw core hoursCost per yearEffective SYPDNotes

1

190829-092132



  • node failure
102029-01-01 - 2035-01-016125true190829-13381242.59
56,7823.38
112035-01-01 - 2040-11-145125true

190822-163651

2190906-012450



  • slurmstepd: error: *** STEP 24373034.0 ON nid05169 CANCELLED AT 2019-09-08T01:25:10 DUE TO TIME LIMIT ***

122040-01-01 - 2046-01-016125true190823-08142332016-09190914-11502443.46
57,9443.31
132046-01-01 - 20182052-0401-09016125true190823-20203842018-03190916-19505242.52
56,6953.39
142052-01-01 - 20222058-0501-28016125true190824-11431852022-05190920-07140943.52
58,0253.31
152058-01-01 - 20232064-0401-17016125true190825-22254962023-03190922-16244743.64
58,1903.30
162064-01-01 - 20292070-0401-14016125true190827190928-030655211656744.50
59,3283.24
172070-01-01 - 20292076-1201-07016125true190830-07215282029-11191003-14253144.65
59,5333.23
182076-01-01 - 20362082-0401-11016125true190830191006-170340022910944.11
58,8083.26
192082-01-01 - 20362084-0908-15162125true190905-204900102036-09191008-023343



  • power outage
202084-01-01 - 20372090-0501-17016125true190906-115943112037-05191013-21200343.59
58,1203.30
212090-01-01 - 20432096-1101-06016125true190906191016-191152083433122043-11-01 - 2044-01-18125true190908-1934321344.53
59,3683.23
222096-01-01 - 20442100-0601-17014125true190909-063024142044-05-01 - 2044-12-10125true190909-102354152044-11-01 - 2046-08-29191018-09093530.22
60,4503.18

20190821.DECKv1b_P3_SSP5-8.5.ne30_oEC.cori-knl

Cori-knl jobs

#Simulation periodYearsNodesBFBFLAGTime stampDuration (hr)Raw core hoursCost per yearEffective SYPDNotes

1

2015-01-01 - 2015-04-19


125

true

190822-163651






22015-03-01 - 2016-10-09
125true190823-081423




32016-09-01 - 2018-04-09
125true190823-202038




42018-03-01 - 2022-05-28
125true190824-114318




52022-05-01 - 2023-04-17
125true190825-222549




62023-03-01 - 2029-04-14
125true190827-030655




72029-03-01 - 2029-12-07
125true190830-072152




82029-11-01 - 2036-04-11
125true190830-170340




92036-03-01 - 2036-09-15
125true190905-204900




102036-09-01 - 2037-05-17
125true190906-115943




112037-05-01 - 2043-11-06
125true190906-191152




122043-11-01 - 2044-01-18
125true190908-193432




132044-01-01 - 2044-06-17
125true190909-063024




142044-05-01 - 2044-12-10
125true190909-102354




152044-11-01 - 2046-08-29
125true190909-160140




162046-07-01 - 2048-05-01
125true190913-150410




172048-03-01 - 2048-08-16
125true190915-200640




182048-07-01 - 2050-02-28
125true190915-234117




192050-01-01 - 2054-05-18
125true190917-080701




202054-05-01 - 2055-05-21
125true190918-170706




212055-05-01 - 2055-12-16
125true190920-021029




222055-11-01 - 2058-01-26
125true190921-154950




232058-01-01 - 2059-0827
125true190924-034141




242059-07-01 - 2060-01-09
125true190925-143757




252060-01-01 - 2060-08-09
125true190926-012525




262060-07-01 - 2060-11-22
125true190926-075728




272060-11-01 - 2061-09-16
125true190928-064331




282061-09-01 - 2062-11-18
125true190929-122208




292062-11-01 - 2064-01-09
125true191001-234801




302064-01-01 - 2064-06-24
125true191005-220429




312064-05-01 - 2064-07-31
125true191007-020816




322064-07-01 - 2065-01-01
125true191007-095553




332065-01-01 - 2065-07-05
125true191013-184928




342065-07-01 - 2066-01-03
125true191013-234708




352066-01-01 - 2072-01-016125true191020-01362843.42
57,8953.32
362072-01-01 - 2078-01-016125true191022-04474245.25
60,3293.18
372078-01-01 - 2084-01-016125true191024-05343044.71
59,6103.22
382084-01-01 - 2090-01-016125true191029-19502944.69
59,5893.22
392090-01-01 - 2095-12-275125true191031-203215



  • slurmstepd: error: *** STEP 25464109.0 ON nid02751 CANCELLED AT 2019-11-02T20:32:32 DUE TO TIME LIMIT
402095-01-01 - 2100-01-015125true191102-21235035.03
56,0463.43

20190821.DECKv1b_P4_SSP5-8.5.ne30_oEC.cori-knl

Cori-knl jobs

#Simulation periodYearsNodesBFBFLAGTime stampDuration (hr)Raw core hoursCost per yearEffective SYPDNotes

1

2015-01-01 - 2015-04-14


125

true

190822-210322






22015-03-01 - 2016-01-29
125true190823-102928




32016-01-01 - 2016-11-29
125true190823-190051




42016-11-01 - 2022-05-10
125true190824-114309




52022-05-01 - 2028-06-20
125true190827-030702




62028-05-01 - 2034-09-12
125true190830-131533




72034-09-01 - 2035-03-07
125true190905-204900




82035-03-01 - 2035-10-30
125true190906-120744




92035-09-01 - 2036-09-19
125true190907-040452




102036-09-01 - 2040-08-30
125true190907-161648




112040-07-01 - 2040-12-14
125true190909-160140063058




161220462040-0711-01 - 20482041-0506-0106
125true190913190909-150410102354




171320482041-0305-01 - 20482043-0801-1631
125true190915190909-200640160135




181420482043-0701-01 - 20502044-0210-2808
125true190915190913-234117150400




191520502044-0109-01 - 20542045-0508-1820
125true190917190914-080701044011




201620542045-0507-01 - 20552051-0511-2116
125true190918190914-170706130126




211720552051-0511-01 - 20552056-1201-1622
125true190920190917-021029080706




221820552056-1101-01 - 20582057-01-2621
125true190921190918-154950170706




231920582057-01-01 - 20592057-08-082709
125true190924190920-034141021028




242020592057-07-01 - 20602059-0108-0930
125true190925190921-143757155954




252120602059-0107-01 - 20602061-0803-0904
125true190926190924-012525033938




262220602061-0703-01 - 20602061-1109-2214
125true190926190925-075728143757




272320602061-1109-01 - 20612062-0904-1602
125true190928190926-064331010555




282420612062-0903-01 - 2062-1107-1801
125true190929190926-12220829080008




252062-07-01 - 2062-1107-01 - 
125true

20190821.DECKv1b_P4_SSP5-8.5.ne30_oEC.cori-knl

Cori-knl jobs

2015-01-01 - 2015-04-14190822210322
#Simulation periodYearsNodesBFBFLAGTime stampDuration (hr)Raw core hoursCost per yearEffective SYPDNotes

1

190928-064333



  • rof_run_mct :: MOSART clock is not in sync with Master Sync

262062-05-01 - 2063-03-17
125true190929-144003




272063-03-01 - 2066-11-15
125true191001-043933




22820152066-0311-01 - 20162067-0104-2913
125true190823191005-102928220559




32920162067-0103-01 - 20162067-1109-2920
125true190823191007-190051093820




43020162067-1109-01 - 20222068-0503-1002
125true190824191013-114309185844




53120222068-0503-01 - 20282068-06-2008
125true190827191014-030702003023




63220282068-05-01 - 20342068-0908-1205
125true190830191019-131533191622




73320342068-0907-01 - 20352069-0302-0701
125true190905191026-204900063501




83420352069-0301-01 - 20352075-1001-30016125true190906-12074492035-09191030-12585847.17
62,8953.05
352075-01-01 - 20362081-0901-19016125true190907-040452102036-09191101-14301845.87
61,1583.14
362081-01-01 - 20402087-0801-30016125true190907-161648112040-07191103-12424743.09
57,4473.34
372087-01-01 - 20402093-1201-14016125true190909-063058122040-11-01 - 2041-06-06191106-07184446.07
61,4213.13
382093-01-01 - 2099-01-016125true190909-102354132041-05191108-22340446.00
61,3393.13
392099-01-01 - 20432100-01-31011125true190909-160135142043-01-01 - 2044-10-08125true190913-150400152044-09-01 - 2045-08-20125true190914-044011162045-07-01 - 2051-11-16191111-1214327.49
59,8913.21

20190821.DECKv1b_P5_SSP5-8.5.ne30_oEC.cori-knl

Cori-knl jobs

2051-112057
#Simulation periodYearsNodesBFBFLAGTime stampDuration (hr)Raw core hoursCost per yearEffective SYPDNotes

1

2015-01-01 - 2021-01-01

6

125

true

190914-13012617

190821-163236

43.90
58,5363.28
22021-01-01 - 20562027-01-22016125true190917-080706182056190823-12452543.71
58,2863.29
32027-01-01 - 20572033-01-21016125true190918-17070619true190826-23064745.69
60,9263.15
42033-01-01 - 20572036-0803-09033125true190920190829-021028202057-07-01 - 2059-08-30125true190921-155954212059-07-01 - 2061-03-04050543



  • slurmstepd: error: *** STEP 24201048.0 ON nid02756 CANCELLED AT 2019-08-31T05:05:54 DUE TO TIME LIMIT ***
52036-01-01 - 2042-01-016125true190924-033938222061-03190906-16342843.83
58,4423.29
62042-01-01 - 20612048-0901-14016125true190925-143757232061-09190916-13231642.30
56,4003.40
72048-01-01 - 20622048-0410-0230
125true190926190918-010555182423




24820622048-0309-01 - 20622049-0705-0110
125true190926190920-080008014000




25920622049-0705-01 - 20622051-0708-0102
125true190928190921-064333

  • rof_run_mct :: MOSART clock is not in sync with Master Sync

262062-05154950




102051-07-01 - 20632052-0312-1710
125true190929190924-144003052724




271120632052-0311-01 - 20662055-1105-1527
125true191001190925-043933000334




281220662055-1105-01 -  2056-09-14
125true

20190821.DECKv1b_P5_SSP5-8.5.ne30_oEC.cori-knl

Cori-knl jobs

619082116323663662051-07true
#Simulation periodYearsNodesBFBFLAGTime stampDuration (hr)Raw core hoursCost per yearEffective SYPDNotes

1

2015-01-01 - 2021-01-01190926-010605




132056-09-01 - 2057-12-21
125true190928-052549




142057-11-01 - 2059-01-01
125true190929-122203




152058-11-01 - 2062-09-06
125true191001-04530343.9058,5363.282




2021-01162062-09-01 - 20272062-0112-01296
125true190823191005-12452523090643.7158,2863.293




2027-01172062-11-01 - 20332063-0103-0111
125true190826191007-23064701413945.6960,9263.15




42033-01182063-03-01 - 20362063-0308-0307
125true190829-050543
  • slurmstepd: error: *** STEP 24201048.0 ON nid02756 CANCELLED AT 2019-08-31T05:05:54 DUE TO TIME LIMIT ***
52036-01-01 - 2042-01-01191007-101710




192063-07-01 - 2064-02-05
125true190906191013-16342818141543.8358,4423.29




62042202064-01-01 - 20482064-0108-01
125true190916-13231642.3056,4003.4072048-01-01 - 2048-10-30191013-233743




212064-07-01 - 2070-12-04
125true191024-111729




222070-11-01 - 2077-05-11
125true190918191102-182423140804




82320482077-0901-01 - 20492083-0501-10016125true190920-01400092049-05191104-16385547.03
62,7073.06
242083-01-01 - 20512089-0801-02016125true190921191106-15495018523147.10
62,4263.08
252089-01-01 - 20522094-12-10065125true190924191109-052724112052-11-01 - 2055-05-27125true190925-000334122055-05-01 - 2056-09-14034314



  • slurmstepd: error: *** STEP 25643536.0 ON nid02543 CANCELLED AT 2019-11-11T03:43:22 DUE TO TIME LIMIT ***
262094-01-01 - 2099-11-245125true190926191111-010605132056-09-01 - 2057-12-21125true190928-052549142057-11-01 - 2059-01-01125true190929-122203152058-11-01 - 2062-09-06121432



  • slurmstepd: error: *** STEP 25700921.0 ON nid02554 CANCELLED AT 2019-11-13T12:14:36 DUE TO TIME LIMIT ***
272099-01-01 - 2100-01-011125true191001191113-045303162062-09-01 - 1251739418.09
64,7192.97


Output files on Cori scratch:

/global/cscratch1/sd/tang30/E3SM_simulations/20190807.DECKv1b_P1_SSP5-8.5.ne30_oEC.cori-knl
/global/cscratch1/sd/tang30/E3SM_simulations/20190821.DECKv1b_P2_SSP5-8.5.ne30_oEC.cori-knl
/global/cscratch1/sd/tang30/E3SM_simulations/20190821.DECKv1b_P3_SSP5-8.5.ne30_oEC.cori-knl
/global/cscratch1/sd/tang30/E3SM_simulations/20190821.DECKv1b_P4_SSP5-8.5.ne30_oEC.cori-knl
/global/cscratch1/sd/tang30/E3SM_simulations/20190821.DECKv1b_P5_SSP5-8.5.ne30_oEC.cori-knl

Files on

...

NERSC HPSS:

v1 DECK output HPSS archive

...

Net TOA radiation:

Surface air temperature:

Image Modified

Net TOA SW and LW:

Image Modified