Versions Compared

Old Version 39

changes.mady.by.user Qi Tang

Saved on

compared with

New Version Current

changes.mady.by.user Qi Tang

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • : configuration and test of future projection run P1 (first ensemble member)
  •  : completed tests and submitted all 5 ensemble members
  •  : on hold due to quota issues
  •  : quota issue solved
  • : P2 simulation completed
  •  : P1 simulation completed
  •  : P3 simulation completed
  •  : P4 simulation completed
  • : P5 simulation completed

20190807.DECKv1b_P1_SSP5-8.5.ne30_oEC.cori-knl

...

#Simulation periodYearsNodesBFBFLAGTime stampDuration (hr)Raw core hoursCost per yearEffective SYPDNotes

1

2015-01-01 - 2021-05-01


125

true

190822-131413






22021-05-01 - 2021-08-21
125true190824-224204




32021-07-01 - 2023-08-02
125true190825-013247




42023-07-01 - 2023-10-08
125true190825-201045




52023-09-01 - 2023-11-22
125true190825-231312




62023-11-01 - 2024-01-01
125true190827-0047021.21
57,6993.33
72024-01-01 - 2030-01-016125true190827-10411446.34
61,7903.11
82030-01-01 - 2036-01-016125true190829-11133744.17
58,8983.26
92036-01-01 - 2038-09-172125true190906-005305



  • slurmstepd: error: *** STEP 24372975.0 ON nid06667 CANCELLED AT 2019-09-08T00:53:47 DUE TO TIME LIMIT ***

102038-01-01 - 2044-01-016125true190914-11502442.97
57,2953.35
112044-01-01 - 2050-01-016125true190916-13235142.78
57,0353.37
122050-01-01 - 2056-01-016125true190920-07140945.09
60,1133.19
132056-01-01 -
125true190922-162447



  • forrtl: error (65): floating invalid
142056-01-01 - 2057-01-061125true

...

190925-133248



  • srun: error: Node failure on nid03307
152057-01-01 - 2063-01-016125true190928-22011143.75
58,3373.29
162063-01-01 - 2069-01-016125true191002-21510346.00
61,3383.13
172069-01-01 - 2075-01-016125true191006-02290343.11
57,4803.34
182075-01-01 - 2077-11-052125true191008-005323



  • power outage
192077-01-01 - 2083-01-016125true191013-20034344.44
59,2493.24
202083-01-01 - 2089-01-016125true191016-07472142.93
57,2463.35
212089-01-01 - 2095-01-016125true191018-09093542.58
56,7683.38
222095-01-01 - 2101-01-016125true191020-10263344.99
59,9883.20

20190821.DECKv1b_P2_SSP5-8.5.ne30_oEC.cori-knl

Cori-knl jobs

#Simulation periodYearsNodesBFBFLAGTime stampDuration (hr)Raw core hoursCost per yearEffective SYPDNotes

1

2015-01-01 - 2015-11-23


125

true

190822-132851






22015-11-01 - 2016-10-11
125true190823-055056




32016-09-01 - 2016-12-22
125true190823-145155




42016-11-01 - 2017-02-22
125true190823-191053




52017-01-01 - 2017-04-22
125true190824-012915




62017-03-01 - 2022-10-21
125true190824-112956




72022-09-01 - 2023-01-01
125true190827-0047022.48
59,3503.24
82023-01-01 - 2029-01-016125true190827-10301345.99
61,3213.13
92029-01-01 - 2029-01-25
125true190829-092132



  • node failure
102029-01-01 - 2035-01-016125true190829-13381242.59
56,7823.38
112035-01-01 - 2040-11-145125true190906-012450



  • slurmstepd: error: *** STEP 24373034.0 ON nid05169 CANCELLED AT 2019-09-08T01:25:10 DUE TO TIME LIMIT ***

122040-01-01 - 2046-01-016125true190914-11502443.46
57,9443.31
132046-01-01 - 2052-01-016125true190916-19505242.52
56,6953.39
142052-01-01 - 2058-01-016125true190920-07140943.52
58,0253.31
152058-01-01 - 2064-01-016125true190922-16244743.64
58,1903.30
162064-01-01 - 2070-01-016125true190928-21165644.50
59,3283.24
172070-01-01 - 2076-01-016125true191003-14253144.65
59,5333.23
182076-01-01 - 2082-01-016125true191006-02291044.11
58,8083.26
192082-01-01 - 2084-08-162125true191008-023343



  • power outage
202084-01-01 - 2090-01-016125true191013-21200343.59
58,1203.30
212090-01-01 - 2096-01-016125true191016-08343344.53
59,3683.23
222096-01-01 - 2100-01-014125true191018-09093530.22
60,4503.18

20190821.DECKv1b_P3_SSP5-8.5.ne30_oEC.cori-knl

Cori-knl jobs

#Simulation periodYearsNodesBFBFLAGTime stampDuration (hr)Raw core hoursCost per yearEffective SYPDNotes

1

2015-01-01 - 2015-04-19


125

true

190822-163651






22015-03-01 - 2016-10-09
125true190823-081423




32016-09-01 - 2018-04-09
125true190823-202038




42018-03-01 - 2022-05-28
125true190824-114318




52022-05-01 - 2023-04-17
125true190825-222549




62023-03-01 - 2029-04-14
125true190827-030655




72029-03-01 - 2029-12-07
125true190830-072152




82029-11-01 - 2036-04-11
125true190830-170340




92036-03-01 - 2036-09-15
125true190905-204900




102036-09-01 - 2037-05-17
125true190906-115943




112037-05-01 - 2043-11-06
125true190906-191152




122043-11-01 - 2044-01-18
125true190908-193432




132044-01-01 - 2044-06-17
125true190909-063024




142044-05-01 - 2044-12-10
125true190909-102354




152044-11-01 - 2046-08-29
125true190909-160140




162046-07-01 - 2048-05-01
125true190913-150410




172048-03-01 - 2048-08-16
125true190915-200640




182048-07-01 - 2050-02-28
125true190915-234117




192050-01-01 - 2054-05-18
125true190917-080701




202054-05-01 - 2055-05-21
125true190918-170706




212055-05-01 - 2055-12-16
125true190920-021029




222055-11-01 - 2058-01-26
125true190921-154950




232058-01-01 - 2059-0827
125true190924-034141




242059-07-01 - 2060-01-09
125true190925-143757




252060-01-01 - 2060-08-09
125true190926-012525




262060-07-01 - 2060-11-22
125true190926-075728




272060-11-01 - 2061-09-16
125true190928-064331




282061-09-01 - 2062-11-18
125true190929-122208




292062-11-01 - 2064-01-09
125true191001-234801




302064-01-01 - 2064-06-24
125true191005-220429




312064-05-01 - 2064-07-31
125true191007-020816




322064-07-01 - 2065-01-01
125true191007-095553




332065-01-01 - 2065-07-05
125true191013-184928




342065-07-01 - 2066-01-03
125true191013-234708




352066-01-01 - 2072-01-016125true191020-01362843.42
57,8953.32
362072-01-01 - 2078-01-016125true191022-04474245.25
60,3293.18
372078-01-01 - 2084-01-016125true191024-05343044.71
59,6103.22
382084-01-01 - 2090-01-016125true191029-19502944.69
59,5893.22
392090-01-01 - 2095-12-275125true191031-203215



  • slurmstepd: error: *** STEP 25464109.0 ON nid02751 CANCELLED AT 2019-11-02T20:32:32 DUE TO TIME LIMIT
402095-01-01 - 2100-01-015125true191102-21235035.03
56,0463.43

20190821.DECKv1b_P4_SSP5-8.5.ne30_oEC.cori-knl

Cori-knl jobs

3.2462029-01-01 - 2029-01-256662052-01-01 - 
#Simulation periodYearsNodesBFBFLAGTime stampDuration (hr)Raw core hoursCost per yearEffective SYPDNotes

1

2015-01-01 - 2015-1104-2314


125

true

190822-132851210322






22015-1103-01 - 2016-1001-1129
125true190823-055056102928




32016-0901-01 - 2016-1211-2229
125true190823-145155190051




42016-11-01 - 20172022-0205-2210
125true190823190824-191053114309




520172022-0105-01 - 20172028-0406-2220
125true190824190827-012915030702




620172028-0305-01 - 20222034-1009-2112
125true190824190830-112956131533




720222034-09-01 - 20232035-0103-0107
125true190827-0047022.4859,350190905-204900




820232035-0103-01 - 20292035-0110-0130
125true190827-10301345.9961,3213.139190906-120744




92035-09-01 - 2036-09-19
125true190907-040452




102036-09-01 - 2040-08-30
125true190829190907-092132161648
  • node failure
10




2029-01112040-07-01 - 20352040-0112-0114
125true190829-13381242.5956,7823.38112035-01-01 - 2040-11-145190909-063058




122040-11-01 - 2041-06-06
125true190909-102354




132041-05-01 - 2043-01-31
125true190906-012450

  • slurmstepd: error: *** STEP 24373034.0 ON nid05169 CANCELLED AT 2019-09-08T01:25:10 DUE TO TIME LIMIT ***

122040190909-160135




142043-01-01 - 20462044-0110-0108
125true190914190913-11502415040043.4657,9443.3113




2046-01152044-09-01 - 20522045-0108-0120
125true190916-19505242.5256,6953.3914190914-044011




162045-07-01 - 2051-11-16
125true190914-130126




172051-11-01 - 2056-01-22
125true190920190917-071409

20190821.DECKv1b_P3_SSP5-8.5.ne30_oEC.cori-knl

Cori-knl jobs

2015-01-01 - 2015-04-191908221636512050
#Simulation periodYearsNodesBFBFLAGTime stampDuration (hr)Raw core hoursCost per yearEffective SYPDNotes

1

080706




182056-01-01 - 2057-01-21
125true190918-170706




192057-01-01 - 2057-08-09
125true190920-021028




202057-07-01 - 2059-08-30
125true190921-155954




22120152059-0307-01 - 20162061-1003-0904
125true190823190924-081423033938




32220162061-0903-01 - 20182061-0409-0914
125true190823190925-202038143757




42320182061-0309-01 - 20222062-0504-2802
125true190824190926-114318010555




52420222062-0503-01 - 20232062-0407-1701
125true190825190926-222549080008




62520232062-0307-01 - 20292062-0407-1401
125true190827-03065572029-03190928-064333



  • rof_run_mct :: MOSART clock is not in sync with Master Sync

262062-05-01 - 20292063-1203-0717
125true190830190929-072152144003




82720292063-1103-01 - 20362066-0411-1115
125true190830191001-170340043933




92820362066-0311-01 - 20362067-0904-1513
125true190905191005-204900220559




102920362067-0903-01 - 20372067-0509-1720
125true190906191007-115943093820




113020372067-0509-01 - 20432068-1103-0602
125true190906191013-191152185844




123120432068-1103-01 - 20442068-0106-1808
125true190908191014-193432003023




133220442068-0105-01 - 20442068-0608-1705
125true190909191019-063024191622




143320442068-0507-01 - 20442069-1202-1001
125true190909191026-102354063501




153420442069-1101-01 - 20462075-0801-29016125true190909-160140162046-07191030-12585847.17
62,8953.05
352075-01-01 - 20482081-0501-016125true190913-150410172048-03191101-14301845.87
61,1583.14
362081-01-01 - 20482087-0801-16016125true190915191103-200640124247182048-07-01 - 2050-02-28125true190915-2341171943.09
57,4473.34
372087-01-01 - 20542093-0501-18016125true190917-080701202054-05191106-07184446.07
61,4213.13
382093-01-01 - 20552099-0501-21016125true190918-170706212055-05191108-22340446.00
61,3393.13
392099-01-01 - 20552100-1201-16011125true190920191111-021029121432222055-11-01 - 125true7.49
59,8913.21

20190821.DECKv1b_

...

P5_SSP5-8.5.ne30_oEC.cori-knl

Cori-knl jobs

190824-114309
#Simulation periodYearsNodesBFBFLAGTime stampDuration (hr)Raw core hoursCost per yearEffective SYPDNotes

1

2015-01-01 - 20152021-0401-1401

6

125

true

190822-210322

true

190821-163236

43.90
58,5363.28
220152021-0301-01 - 20162027-01-29016125true190823-10292812452543.71
58,2863.29
320162027-01-01 - 20162033-1101-29016125true190823-190051190826-23064745.69
60,9263.15
420162033-1101-01 - 20222036-0503-10033125true190829-050543



  • slurmstepd: error: *** STEP 24201048.0 ON nid02756 CANCELLED AT 2019-08-31T05:05:54 DUE TO TIME LIMIT ***
520222036-0501-01 - 2028-06-20125true190827-03070262028-05-01 - 2034-09-122042-01-016125true190906-16342843.83
58,4423.29
62042-01-01 - 2048-01-016125true190830-131533190916-13231642.30
56,4003.40
720342048-0901-01 - 20352048-0310-0730
125true190905190918-204900182423




820352048-09-0301 - 01 2049- 203505-10-30
125true190906190920-120744014000




920352049-0905-01 - 20362051-0908-1902
125true190907190921-040452154950




1020362051-0907-01 - 20402052-0812-3010
125true190907190924-161648052724




1120402052-0711-01 - 20402055-1205-1427
125true190909190925-063058000334




1220402055-1105-01 - 20412056-0609-0614
125true190909190926-102354010605




1320412056-0509-01 - 20432057-0112-3121
125true190909190928-160135052549




1420432057-0111-01 - 20442059-1001-0801
125true190913190929-150400122203




1520442058-0911-01 - 20452062-0809-2006
125true190914191001-044011045303




1620452062-0709-01 - 20512062-1112-1629
125true190914191005-130126230906




1720512062-11-01 - 20562063-0103-2211
125true190917191007-080706014139




1820562063-0103-01 - 20572063-0108-2107
125true190918191007-170706101710




1920572063-0107-01 - 20572064-0802-0905
125true190920191013-021028181415




2020572064-07-01 - 125true

20190821.DECKv1b_P5_SSP5-8.5.ne30_oEC.cori-knl

Cori-knl jobs

6190821163236
#Simulation periodYearsNodesBFBFLAGTime stampDuration (hr)Raw core hoursCost per yearEffective SYPDNotes

1

2015-01-01 - 2021-01-0101-01 - 2064-08-01
125true191013-233743




212064-07-01 - 2070-12-04
125true191024-111729




222070-11-01 - 2077-05-11
125true191102-140804




43.9058,5363.2822021232077-01-01 - 20272083-01-016125true190823191104-1245251638554347.7103
5862,2867073.2906
32420272083-01-01 - 20332089-01-016125true190826191106-2306471852314547.6910
6062,9264263.1508
42520332089-01-01 - 20362094-0312-030635125true190829191109-050543034314



  • slurmstepd: error: *** STEP 2420104825643536.0 ON nid02756 nid02543 CANCELLED AT 2019-0811-31T0511T03:05:54 DUE TO TIME LIMIT ***
52036-01-01 - 2042-01-016125true190906-16342843.8358,4423.2962042
  • 43:22 DUE TO TIME LIMIT ***
262094-01-01 - 20482099-0111-012465125true190916191111-13231642.3056,4003.4072048121432



  • slurmstepd: error: *** STEP 25700921.0 ON nid02554 CANCELLED AT 2019-11-13T12:14:36 DUE TO TIME LIMIT ***
272099-01-01 - 20482100-1001-30011125true190918191113-18242317394182048-09-01 - 2049-05-10125true190920-01400092049-05-01 - 125true.09
64,7192.97


Output files on Cori scratch:

/global/cscratch1/sd/tang30/E3SM_simulations/20190807.DECKv1b_P1_SSP5-8.5.ne30_oEC.cori-knl
/global/cscratch1/sd/tang30/E3SM_simulations/20190821.DECKv1b_P2_SSP5-8.5.ne30_oEC.cori-knl
/global/cscratch1/sd/tang30/E3SM_simulations/20190821.DECKv1b_P3_SSP5-8.5.ne30_oEC.cori-knl
/global/cscratch1/sd/tang30/E3SM_simulations/20190821.DECKv1b_P4_SSP5-8.5.ne30_oEC.cori-knl
/global/cscratch1/sd/tang30/E3SM_simulations/20190821.DECKv1b_P5_SSP5-8.5.ne30_oEC.cori-knl

Files on

...

NERSC HPSS:

v1 DECK output HPSS archive

...

Net TOA radiation:

Surface air temperature:

Image Modified

Net TOA SW and LW:

Image Modified