Aqua-MODIS and CALIPSO Subset Product Generation for OCO-2 Validation Using Python

Subsets are a vital part of many data management, processing, and validation plans and are especially important for satellite data due to storage and memory constraints. The CIRA Data Processing Center is using a Python scheme to routinely produce 5 Aqua-MODIS and 3 CALIPSO subset products in support of OCO-2 validation efforts. NASA's OCO-2 satellite is the first dedicated to the remote sensing of atmospheric carbon dioxide (CO₂). It leads the A-Train satellite constellation, approximately 7 minutes ahead of the Aqua satellite, flying in a roughly 700 km sun-synchronous orbit (i=98.2º) with a 1:36pm ascending node and a 16 day/233 orbit repeat cycle. The OCO-2 footprint is relatively small, approximately 2 km cross-track and 3 km along track, which maximizes the number of available cloud-free pixels for long-term analysis. NASA's Aqua satellite carries six instruments including MODIS, a 36-band spectroradiometer that measures visible and infrared radiation in a spectral range between 0.4-14.4 µm from which a wide range of data products are derived. MODIS collects data at three spatial resolutions, 250 m, 500 m, and 1 km, and has a viewing swath width of 2330 km that provides global coverage every 1-2 days. The data products that CIRA subsets to the OCO-2 ground track are the Level 1B half-kilometer resolution calibrated radiances and geolocation; the Level 1A 1-kilometer resolution ground elevation, scene information, solar and satellite geometry, and geolocation; the 1- and 5-kilometer resolution cloud product, and the 10-kilometer aerosol product. NASA's CALIPSO satellite is also a member of the A-Train satellite constellation, flying approximately 2 minutes behind Aqua. The CALIOP lidar on-board retrieves high-resolution vertical profiles of aerosols and clouds. The 70 m diameter footprint has an along-track resolution of 333 m and, like OCO-2, has a 16 day repeat cycle. The data products that CIRA subsets to the OCO-2 ground track are the 1-kilometer cloud layer product and the 5-kilometer cloud and aerosol layered products. The Python processing scheme makes use of the PyNIO, NumPy, SciPy, JSON, MySQL, and H5Py libraries. PyNIO replaced the PyHDF library for reading HDF-EOS input files due to its added NetCDF and GRIB capabilities, which will support easy integration of additional input datasets should the need arise. The other libraries allow for fast and efficient array processing (NumPy and SciPy), the creation and utilization of both static and dynamically-created header files for automated processing (JSON), easy connection to existing databases for quick population of JSON header files (MySQL), and reading and writing of HDF-5 files (H5Py). The processing structure employs a system of Python scripts and static and dynamically-created JSON configuration files to select HDF-EOS MODIS and CALIPSO input files, perform product-specific pre-processing (including interpolation, granule aggregation, and intermediate product creation), and match on a pixel-by-pixel basis to the OCO-2 geolocation using a great-circle nearest neighbor scheme. The resulting subset products are written to HDF-5 files and distributed to the OCO-2 science team via JPL. In addition to a discussion of the processing path, examples of analysis and visualization with resulting products will be shown. Although we focus on OCO-2, MODIS, and CALIPSO here, the long-term goal of this tool is a generalized collocation mechanism for comparing satellite data. The methodology will next be applied to NASA's CloudSat satellite, which should further improve and generalize the tool while simultaneously allowing for new integrated scientific study.