The VIIRS Climate Raw Data Record: An Easy-to-use Raw Data Set (Level 1b) for the Suomi-NPP Visible Infrared Imaging Radiometer Suite

The Visible Infrared Imaging Radiometer Suite (VIIRS) instrument is a key element of the Suomi-National Polar-Orbiting Partnership (NPP) satellite that was launched in October of 2011. VIIRS collects data in 22 spectral channels, from visible to long-wave infrared, at two different spatial resolutions (375 m and 750 m at nadir). The Climate Raw Data Record (C-RDR) Project at the NOAA National Climatic Data Center (NCDC) is producing a NOAA Level 1b data set for the VIIRS instrument. A Level 1b data set contains “unprocessed data at full resolution, time-referenced, and annotated with ancillary information including data quality indicators, calibration coefficients and georeferencing parameters.” [FGDC-STD-012-2002] This data set is being produced on an operational basis, and is being archived and made available for public use.

The NOAA Interface Data Processing Segment (IDPS) produces three primary levels of data sets for VIIRS. There are Raw Data Record (RDR), Sensor Data Record (SDR), and Environmental Data Record (EDR) data sets being produced on an operational basis for dissemination and archive. The RDR data set has been processed to NOAA Level 1a, and contains time-sequenced and indexed Consultative Committee for Space Data Systems (CCSDS) Space Packets filled with down-linked VIIRS engineering and science data. The contents of the CCSDS Space Packets have not been unpacked at this level.

The VIIRS C-RDR data set is being produced to fill a gap in the types of data sets being produced by the IDPS. The SDRs, while highly useful, contain data that has undergone calibration transformations, some of which are complex, and the engineering data that was used in the operations have not been preserved within the data sets. As a result, third parties that wish to use different calibration values or algorithms are forced to use the VIIRS Science RDR data set, which is difficult to use.

The VIIRS C-RDR data set addresses these issues by unpacking all of the engineering and science data from each VIIRS RDR data set file, as well as accompanying satellite position, velocity, attitude, and operation state data from the associated Spacecraft Diary RDR data set files, and stores the uncalibrated values as time-series variables in Network Common Data Form 4 (netCDF-4) files. The netCDF-4 format is platform-independent, binary, hierarchical, and self-describing. Each variable within a VIIRS C-RDR file is annotated with a description of the measurement it contains, information about the source, and specifications of valid limits and fill values.

The image data, which were differentially encoded and compressed using the RICE algorithm before being placed into CCSDS Space Packets by the VIIRS on-board processing, are decompressed and decoded, and stored as images in the VIIRS C-RDR files. Each VIIRS C-RDR file also contains file-level metadata conforming to the netCDF Climate and Forecast (CF) Conventions, the netCDF Attribute Convention for Dataset Discovery (ACDD), and the ISO 19115-2 Standard. Metadata elements, such as granule IDs, which are found in Suomi-NPP data product HDF5 format files are also present in C-RDR files, as an aid to understanding the provenance and processing history of the VIIRS C-RDR files.

To fully satisfy the NOAA Level 1b definition, the calibration coefficients must also be present. The VIIRS C-RDR data set satisfies this requirement by producing a VIIRS C-RDR Support Data file, which is a netCDF-4 file containing all of the VIIRS-specific auxiliary files (initialization parameters, processing coefficients, and lookup tables) needed to produce a VIIRS SDR file from a VIIRS RDR file. The values within each support data file are stored as individual variables within the netCDF file, following the descriptions found in the Suomi-NPP Common Data Format Control Book Volume 8. Each variable is annotated with a description of the measurement it contains, information about the source, and specifications of valid limits and fill values. The variables belonging to a given support data file are grouped together, and the group is annotated with all available file-level metadata, including original creation and effectivity dates. File-level metadata identify the contents by original Suomi-NPP file names, and provide envelope effectivity dates. The file-level metadata also contains elements conforming to the CF, ACDD and ISO 19115-2 conventions.

The variables contained within VIIRS C-RDR files are easily read by a number of existing software applications, and easily accessed in FORTRAN, C, C++, Java, or Python applications using netCDF APIs that are available for each language.

The body of this presentation provides details of the VIIRS C-RDR contents and structure, explains the C-RDR production methods and process, and explains how to search for and obtain VIIRS C-RDR files.