Using Machine Learning to Assess Solar Radiation Data Quality

This research aims to assess the integrity of incoming shortwave solar radiation sensor data from the Kansas Mesonet, which uses both Apogee CS 300 and CS 301 Silicon Pyranometers. These measurements are crucial for estimating evapotranspiration and supporting agricultural water management, especially as the Ogallala aquifer, one of Kansas’s main water supplies, continues to deplete. They also play a critical role in animal comfort measurements and fire danger calculations. Recently, the Mesonet has installed cameras at specific stations that capture still images focused on the horizon with both surface cover and the western sky shown. The project used these 15 minute images to evaluate historical pyranometer performance. A machine learning model was trained to classify station images as sunny or cloudy, and then applied to identify sunny days – defined by clear skies at 9AM, 12PM, and 3PM. Solar radiation data from these predicted sunny days were compared against clear sky solar radiation estimates using statistical methods such as root mean square error, absolute percent error, Pearson’s correlation coefficient, and the Wald test. Most stations exhibited minimal discrepancies, but this approach successfully detected erroneous solar radiation readings at the Norton 4SW weather station, where incorrect zero values were recorded under sunny conditions after a pyranometer replacement. A Flask-based web application was developed to visualize this analysis, aiding in the assessment of solar radiation data. While this tool effectively identified major issues, smaller complications such as calibration or minor sensor defects were harder to detect. Other limitations included manual image sorting, variability in model accuracy across stations, and a lack of clear, simple results.