Forecast verification of the Current Icing Potential (CIP) product to predict observed lightning in the vicinity of U.S. Spaceports

Government spaceports employ extensive lightning detection networks that may not be available at commercial spaceports. As the number of commercial space operations increases, the Federal Aviation Administration identified the need for a method to diagnose the threat of triggered lightning at commercial spaceports without in-situ measurements. Charge separation that produces lightning is generated by the existence of water in solid and liquid states interacting. This mixed phase environment is also conducive to structural aircraft icing. Anecdotal observations of the Aviation Weather Center's Current Icing Potential (CIP) numerical weather prediction model indicated a potentially high correlation between lightning activity and forecast icing potential. Analysis of three years of USPLN lightning data at spaceports across the United States provided a measure of the lightning frequency at these locations. Relatively high statistical correlations between the CIP and lightning activity in both space and time were discovered, but so were negative correlations. Since it is not possible to define a correlation when one field is constant, such as CIP values greater than zero but with no observed lightning events, a forecast verification was conducted using a traditional contingency table of forecasted (CIP) versus observed lightning events. Forecast verification studies using CIP to predict lightning, and previous lightning to predict future lightning (a persistence forecast), were performed. Case studies were also conducted to determine the CIP's ability to diagnose lightning hazards, particularly lightning initiation, in a hypothetical operational setting. The forecast verification study, covering two years of lightning activity, determined the CIP's ability to diagnose lightning hazards was quite limited due to extensive overprediction. Critical Success Index (CSI) scores for CIP as a lightning predictor were less than 15% in all cases. In comparison, lightning persistence forecasts achieved CSI scores closer to 40%. During the case study analyses the CIP demonstrated a number of weaknesses. The CIP: 1) failed to capture lightning initiation, 2) overpredicted the extent of lightning, 3) missed whole regions of lightning, and 4) failed to capture lightning cessation. Queries of an aviation hazard database developed by ERAU and NASA showed CIP over predicted convection in general, not just lightning. It is difficult to draw any conclusions on the CIP's ability to diagnose lightning triggered by a launch vehicle due to insufficient documented cases of this hazard.