...
Step 4: Indexing and publication
- Next log into the datanode data node you're going to be publishing to. Make sure you have an openid account for this node, and that your account has the "publisher" attribute. Note that this server needs to have access to both the mapfiles as well as the data directories. If you staged the data on another server, you'll need to copy over in the correct structure to the esgf node you're publishing too.
- Ensure that /esg/config/esgcet/esg.e3sm.ini is correct and there is an entry in /esg/config/esgcet/esg.ini for e3sm in the projects table
- source the conda environment
- Store your myproxy credentials locally (this will store your credentials at ~/.globus/certificate-file and activate them for the next 72 hours)
| Code Block |
|---|
myproxy-logon -s <your_esgf_identity_node_hostname> -l <your_myproxy_username> -o ~/.globus/certificate-file -t 72 |
- If you are publishing an experiment for the first time run "esginitialize -c"
- Run the following commands in the given order
- esgpublish --project e3sm --map /path/to/where/you/want/your/mapfiles/<your_first_mapfile>.map --service fileservice
- esgpublish --project e3sm --map /path/to/where/you/want/your/mapfiles/<your_first_mapfile>.map --service fileservice --noscan --thredds
- esgpublish --project e3sm --map /path/to/where/you/want/your/mapfiles/<your_first_mapfile>.map --service fileservice --noscan --publish
Step 5:
...
Verification
Your data should now be available on the given node. You can verify by opening a browser window and going to https://<your_esgf_node>/esg-search/search?project=e3sm
...
| Code Block |
|---|
<dataset_id> | optional_key1=value1 | optional_key2=value2 | ect | ect |
For example the first round of publication used the following:
| Code Block |
|---|
E3SM.1_0.piControl.1deg_atm_60-30km_ocean.atmos.129x256.climo.monClim.ens1 | science_driver="Water Cycle" | land_grid_resolution="1deg" | seaice_grid_resolution="60-30km"
E3SM.1_0.piControl.1deg_atm_60-30km_ocean.atmos.129x256.time-series.mon.ens1 | science_driver="Water Cycle" | land_grid_resolution="1deg" | seaice_grid_resolution="60-30km" | period="Perpetual 1850"
E3SM.1_0.piControl.1deg_atm_60-30km_ocean.land.native.model-output.mon.ens1 | science_driver="Water Cycle" | land_grid_resolution="1deg" | seaice_grid_resolution="60-30km" | period="Perpetual 1850" |
After the new mapfile has been created, envoke the esgadd_facetvalues command
| Code Block |
|---|
esgadd_facetvalues --project e3sm --map /path/to/your/additional/facets/map --noscan --thredds --service fileservice |
And finally, publish the newly updated dataset
| Code Block |
|---|
esgpublish --project e3sm --map /path/to/your/map/files/ --noscan --publish --service fileservice |