CMIP6 data conversion status

This is a working document showing the conversion status for publishing E3SM data to CMIP6. The variables listed are unordered, and pulled from the priority 1 request list as well as scientist input. If you think that an important variable is missing please add it.

The conversion formulas are for the most part compiled by a NON DOMAIN EXPERT (me) using the following resources. There is the very real possibility they may be incorrect, please verify the conversions in your area of expertise and mark the "verified" column with your name.

Much of this information is pulled from the previous discussion page E3SM data conversion for CMIP6, but I wanted to distill the information down to the bare necessities. 

Resources:

• CMIP6 data request https://www.earthsystemcog.org/projects/wip/CMIP6DataRequest
• CESM Amon conversion table for CMIP5: https://github.com/NCAR/CCP_Processing_Suite/blob/master/CMOR2/Xwalks
• http://clipc-services.ceda.ac.uk/dreq/data/tabs02/cmvme_CMIP_piControl_1_1.xlsx
• CMIP6 tables : http://clipc-services.ceda.ac.uk/dreq/index/miptable.html
• JSON tables on GitHub https://github.com/PCMDI/cmip6-cmor-tables

ATM

Amon (monthly atmospheric data)

Notes:

• Full request: http://clipc-services.ceda.ac.uk/dreq/u/MIPtable::Amon.html
• CESM Amon conversion table for CMIP5 (https://github.com/NCAR/CCP_Processing_Suite/blob/master/CMOR2/Xwalks/xwalk_CMIP5_Amon, thanks to Peter Caldwell for finding this).
• All fields must be interpolated in the horizontal to the new CMIP6 grids for which Charlie Zender prepared mapping files (see CMIP6 Grid-files and Map-files).
• Some 3d variables must be reported on model levels and others interpolated to the standard 19 pressure levels.

Additional tables to consider

• 3hr, 6hrLev, 6hrPlev, 6hrPlevPt
• CF3hr, CFday, CFmon (suggest consulting with Yuying Zhang on most useful variables and variables not impacted by COSP bug).
• AERmon

First set of atmospheric data has been cmorized provided by Sterling:

Data access:

• acme1 -> /p/user_pub/work/E3SM/1_0/cmip6_variables/piControl/atm/CMIP6
• Edison -> /global/cscratch1/sd/sbaldwin/CMIP6

{

    dimensions:

       time = UNLIMITED ; // (6000 currently)

       lat = 180 ;

       lon = 360 ;

       bnds = 2 ;

    variables:

       double time(time) ;

             time:bounds = "time_bnds" ;

             time:units = "days since 0001-01-01 00:00:00" ;

             time:calendar = "noleap" ;

             time:axis = "T" ;

             time:long_name = "time" ;

             time:standard_name = "time" ;

       double time_bnds(time, bnds) ;

       double lat(lat) ;

             lat:bounds = "lat_bnds" ;

             lat:units = "degrees_north" ;

             lat:axis = "Y" ;

             lat:long_name = "latitude" ;

             lat:standard_name = "latitude" ;

       double lat_bnds(lat, bnds) ;

       double lon(lon) ;

             lon:bounds = "lon_bnds" ;

             lon:units = "degrees_east" ;

             lon:axis = "X" ;

             lon:long_name = "longitude" ;

             lon:standard_name = "longitude" ;

       double lon_bnds(lon, bnds) ;

       double height ;

             height:units = "m" ;

             height:axis = "Z" ;

             height:positive = "up" ;

             height:long_name = "height" ;

             height:standard_name = "height" ;

       float tas(time, lat, lon) ;

             tas:standard_name = "air_temperature" ;

             tas:long_name = "Near-Surface Air Temperature" ;

             tas:comment = "near-surface (usually, 2 meter) air temperature" ;

             tas:units = "K" ;

             tas:cell_methods = "area: time: mean" ;

             tas:cell_measures = "area: areacella" ;

             tas:history = "2019-01-26T00:52:57Z altered by CMOR: Treated scalar dimension: \'height\'." ;

             tas:coordinates = "height" ;

             tas:missing_value = 1.e+20f ;

             tas:_FillValue = 1.e+20f ;

// global attributes:
:Conventions = "CF-1.7 CMIP-6.2" ;
:activity_id = "CMIP" ;
:branch_method = "Spin-up documentation" ;
:branch_time_in_child = 0. ;
:branch_time_in_parent = 0. ;
:contact = "Dave Bader (bader2@llnl.gov)" ;
:creation_date = "2019-01-26T00:52:57Z" ;
:data_specs_version = "01.00.23" ;
:experiment = "pre-industrial control" ;
:experiment_id = "piControl" ;
:external_variables = "areacella" ;
:forcing_index = 1 ;
:frequency = "mon" ;
:further_info_url = "https://furtherinfo.es-doc.org/CMIP6.E3SM-Project.E3SM-1-0.piControl.none.r1i1p1f1" ;
:grid = "gs1x1" ;
:grid_label = "gr" ;
:history = "2019-01-26T00:52:57Z ;rewrote data to be consistent with CMIP for variable tas found in table Amon.;\n",
"Output from 20180129.DECKv1b_piControl.ne30_oEC.edison" ;
:initialization_index = 1 ;
:institution = "LLNL (Lawrence Livermore National Laboratory, Livermore, CA 94550, USA); ANL (Argonne National Laboratory, Argonne, IL 60439, USA); BNL (Brookhaven National Laboratory, Upton, NY 11973, USA); LANL (Los Alamos National Laboratory, Los Alamos, NM 87545, USA); LBNL (Lawrence Berkeley National Laboratory, Berkeley, CA 94720, USA); ORNL (Oak Ridge National Laboratory, Oak Ridge, TN 37831, USA); PNNL (Pacific Northwest National Laboratory, Richland, WA 99352, USA); SNL (Sandia National Laboratories, Albuquerque, NM 87185, USA). Mailing address: LLNL Climate Program, c/o David C. Bader, Principal Investigator, L-103, 7000 East Avenue, Livermore, CA 94550, USA" ;
:institution_id = "E3SM-Project" ;
:mip_era = "CMIP6" ;
:nominal_resolution = "100 km" ;
:parent_activity_id = "CMIP" ;
:parent_experiment_id = "piControl-spinup" ;
:parent_mip_era = "CMIP6" ;
:parent_source_id = "E3SM-1-0" ;
:parent_time_units = "days since 0001-01-01" ;
:parent_variant_label = "r1i1p1f1" ;
:physics_index = 1 ;
:product = "model-output" ;
:realization_index = 1 ;
:realm = "atmos" ;
:references = "Golaz, J.-C., P. M. Caldwell, L. P. Van Roekel and co-authors, 2019: The DOE E3SM coupled model version 1: Overviewand evaluation at standard resolution. JAMES, submitted; http://e3sm.org" ;
:source = "E3SM 1.0 (2018): \n",
       "aerosol: MAM4 with resuspension, marine organics, and secondary organics (same grid as atmos)\n",
       "atmos: EAM (v1.0, cubed sphere spectral-element grid; 5400 elements with p=3; 1 deg average grid spacing; 90 x 90 x 6 longitude/latitude/cubeface; 72 levels; top level 0.1 hPa)\n",
       "atmosChem: Troposphere specified oxidants for aerosols. Stratosphere linearized interactive ozone (LINOZ v2) (same grid as atmos)\n",
       "land: ELM (v1.0, cubed sphere spectral-element grid; 5400 elements with p=3; 1 deg average grid spacing; 90 x 90 x 6 longitude/latitude/cubeface; satellite phenology mode), MOSART (v1.0, 0.5 degree latitude/longitude grid)\n",
       "landIce: none\n",
       "ocean: MPAS-Ocean (v6.0, oEC60to30 unstructured SVTs mesh with 235160 cells and 714274 edges, variable resolution 60 km to 30 km; 60 levels; top grid cell 0-10 m)\n",
       "ocnBgchem: none\n",
       "seaIce: MPAS-Seaice (v6.0, same grid as ocean)" ;
:source_id = "E3SM-1-0" ;
:source_type = "AOGCM AER" ;
:sub_experiment = "none" ;
:sub_experiment_id = "none" ;
:table_id = "Amon" ;
:table_info = "Creation Date:(21 March 2018) MD5:528cdc7b6fa35bf79fa0b69273987051" ;
:title = "E3SM-1-0 output prepared for CMIP6" ;
:variable_id = "tas" ;
:variant_label = "r1i1p1f1" ;
:license = "CMIP6 model data produced by E3SM is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License (https://creativecommons.org/licenses/). Consult https://pcmdi.llnl.gov/CMIP6/TermsOfUse for terms of use governing CMIP6 output, including citation requirements and proper acknowledgment. Further information about this data, including some limitations, can be found via the further_info_url (recorded as a global attribute in this file). The data producers and data providers make no warranty, either express or implied, including, but not limited to, warranties of merchantability and fitness for a particular purpose. All liabilities arising from the supply of the information (including any liability arising in negligence) are excluded to the fullest extent permitted by law." ;
:cmor_version = "3.4.0" ;
:tracking_id = "hdl:21.14100/c1e2e21a-5d05-4a26-8b86-21f947ab44a1" ;
:e3sm_source_code_doi = "10.11578/E3SM/dc.20180418.36" ;
:e3sm_source_code_reference = "https://github.com/E3SM-Project/E3SM/releases/tag/v1.0.0" ;
:doe_acknowledgement = "This research was supported as part of the Energy Exascale Earth System Model (E3SM) project, funded by the U.S. Department of Energy, Office of Science, Office of Biological and Environmental Research." ;
:computational_acknowledgement = "The data were produced using resources of the National Energy Research Scientific Computing Center, a DOE Office of Science User Facility supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC02-05CH11231." ;
:ncclimo_generation_command = "drc_in=\'/p/user_pub/work/E3SM/1_0/piControl/1deg_atm_60-30km_ocean/atmos/native/model-output/mon/ens1/v1\' # Input directory\n",
       "drc_out=\"/p/user_pub/work/E3SM/1_0/cmip6_variables/piControl/atm/ne30/climo\" # Native grid output directory\n",
       "drc_rgr=\"/p/user_pub/work/E3SM/1_0/cmip6_variables/piControl/atm/ne30/rgr\" # Regridded output directory\n",
       "drc_tmp=\'/p/cscratch/acme/baldwin32/tmp\' # Temporary/intermediate-file directory\n",
       "map=\"/export/zender1/data/maps/map_ne30np4_to_cmip6_180x360_aave.20181001.nc\" # Regridding map-file\n",
       "cmip6_opt=\'-7 --dfl_lvl=1 --no_cll_msr --no_frm_trm --no_stg_grd\' # CMIP6-specific options\n",
       "spl_opt=\'--yr_srt=1 --yr_end=500 --ypf=500\' # Splitter options\n",
       "vars=(\'TREFHT\' \'TS\' \'PSL\' \'PS\' \'QREFHT\' \'PRECC\' \'PRECL\' \'PRECSC\' \'PRECSL\' \'QFLX\' \'TAUX\' \'TAUY\' \'LHFLX\' \'CLDTOT\' \'FLDS\' \'FLNS\' \'FSDS\' \'SHFLX\' \'CLOUD\' \'CLDICE\' \'TGCLDIWP\' \'CLDLIQ\' \'RELHUM\' \'Q\' \'O3\' \'P0\' \'hyam\' \'hybi\' \'TMQ\' \'FLDSC\' \'FLUT\' \'FSDSC\' \'SOLIN\' \'FSNSC\' \'FSUTOA\' \'FSUTOAC\' \'FSNT\' \'FLNT\' \'T\' \'U\' \'V\' \'OMEGA\' \'Z3\')\n",
       "export TMPDIR=${drc_tmp}\n",
       "cd ${drc_in}\n",
       "\n",
       "for i in \"${vars[@]}\"\n",
       "do\n",
             " ls 20180129.DECKv1b_piControl.ne30_oEC.edison.cam.h0.0???-*.nc | ncclimo --var=${i} ${cmip6_opt} ${spl_opt} --map=${map} --drc_out=${drc_out} --drc_rgr=${drc_rgr} &> ${drc_out}/ncclimo_${i}.out &\n",
       "done" ;
:ncclimo_version = "4.7.9-alpha04" ;
}

LND

OCN

ICE. – Notz et al. 2016 – https://www.geosci-model-dev.net/9/3427/2016/gmd-9-3427-2016.pdf