Checking files for CF compliance

Notes on CF compliance

The CF ("Climate and Forecast") convention for including metadata in NetCDF files (NetCDF API) that makes them "self describing", facilitating the processing and sharing of files by agreeing upon common metadata that should be included in the netCDF files (e.g. standard names, units, etc) so that files created by different authors can be easily shared.

Files that comply with the CF conventions are much more easily processed by analysis tools (including those used by ACME), and reduces the needs for "special case" codes for processing particular datasets.

The CF home page is found at http://cfconventions.org/

Web-based CF compliance checker

A web-based CF compliance checker is described on the CF Home page, and it is repeated here http://puma.nerc.ac.uk/cgi-bin/cf-checker.pl. You can use it to check any files you create for observational or model datasets. The check will identify non-compliant aspects of your netCDF files.

The web-based checker is very easy to use, and very informative. But it is painful to check many files, or to check very large netCDF files remotely. The code for that CF checker is written in python and it is in principle available for use on any platform, but it is very difficult to install, requiring old versions of python and numpy. BenjaminM (Unlicensed) investigated its use in our first assessment of CF compliance for the files in the NCAR NCL observational data directory.

ACME "quick-and-dirty" CF compliance checker

Phil Rasch (pnl.gov) wrote a lightweight shell script that uses curl to send a small version of the file to the checker, and accesses the NERC website from a linux/unix command line so you don't need to install the checker, and you can check many files quickly. The script is stored in the github /wiki/spaces/ATM/pages/31130398 at https://github.com/ACME-Climate/PreAndPostProcessingScripts/blob/master/utils/quick_and_dirty_cf_compliance_checker.

To obtain a copy of the script, it is best to check out the repository from github (making it easier to obtain future updates):

git clone git@github.com:ACME-Climate/PreAndPostProcessingScripts.git

the script is located in the repository at:

utils/quick_and_dirty_cf_compliance_checker

Usage

quick_and_dirty_cf_compliance_checker <filename.nc>

Dependencies

nco, ncdump, curl

(requires a relatively modern version of curl)

Alternate checker (yet to be evaluated)

An alternate compliance checker is available here https://bitbucket.org/mde_/cfchecker. It appears to be much more portable. The two checkers sometimes provide identify different issues. We will try it and update this page after evaluating it.

I am copying Susannah Burrows and BenjaminM (Unlicensed) on this so I know they see this page. Ben, would you consider trying to install the potentially more portable CF checker on the main ACME platforms where UV-CDAT is installed? If so, can you open a JIRA issue? Thanks! Phil