Creating a Refined Quality-Controlled Truth Dataset to Improve Confidence in Lightning Detection

In order to properly train a machine learning model, the quality of the truth dataset is of utmost importance. For global lightning data, determining if a lightning detection is real or false can be a difficult task. This research aims to create a better quality-controlled truth dataset for global lightning data. To achieve this, we use cloud boundaries derived from longwave IR merged global satellite data to determine if lightning is in fact inside a cloud, thereby separating false lightning from real lightning. The satellites involved include the Geostationary Operational Environmental Satellites (GOES) 16 & 17, the Himawari-9 satellite, and the Meteosat 9 & 10 satellites. Additionally, detections by the legacy Earth Networks Total Lightning Network (ENTLN) dataset and the Geostationary Lightning Mapper (GLM; where available) provide further confidence in detected lightning's legitimacy. We employ this improved lightning dataset to recalculate confidence values used in the ENTLN algorithm to help determine the likelihood a detection is real lightning. Overall, the new dataset increases logistic regression model performance, resulting in higher POD and lower FAR across most regions. This improved performance will ultimately help increase the future accuracy of Earth Networks lightning data and the quality-controlled dataset can be used in future lightning validation efforts.