Reprocessing with the S-NPP Common Matchup Tool

Many parameters, look up tables, and heuristics govern the processing of sensor data records (SDRs) into environmental data records (EDRs). EDRs themselves are typically processed in a chain where upstream products such as a cloud mask profoundly influence downstream EDRs such as aerosol retrieval, surface reflectance, and vegetation index. The interplay between the dozens of look up tables, and hundreds of coefficients, parameters, and heuristics is very complex. Consequently, efforts to improve the performance of one EDR through algorithm, look up table, or coefficient changes carry the risk of degrading downstream products.

The Common Matchup and Reprocessing Tools, developed for calibration and validation of the VIIRS sensor on Suomi NPP, allows direct assessments of the impacts of parameter, look up table, and algorithmic changes on several VIIRS EDRs or Intermediate Products (IPs) simultaneously. The Common Matchup Tool currently runs as a Product Generation Executive (PGE) on NOAA's GRAVITE system in Suitland, MD, continuously storing small cut-outs of VIIRS data and spatially and temporally coregistered in situ data from Aeronet and SURFRAD sites. All input to the VIIRS algorithms such as land surface temperature, aerosol optical thickness, surface reflectance, and vegetation index, as well as corresponding in situ data and ancillary data is stored in daily HDF 5 files for each of these EDRs. These matchup results files contain everything needed to reprocess pixels around the matched in situ data.

The Reprocessing Tool ingests the matchup results files. From them, it creates new input files that can be fed back into the ground processing software, ADL or ADA, to reproduce the EDRs. The effect of changes to algorithms, look up tables, parameters, et cetera, in the ground processing software can be then be evaluated by measuring performance changes of the modified EDRs against the original in situ data. Computational performance is several orders of magnitude faster than the original EDR generation because only small clusters of pixels around the in situ sites need to be processed. Reprocessed results using the VIIRS aerosol optical thickness algorithm are shown as examples.