Forecasts for these specific conditions are issued for each landing opportunity. Verification of the forecasts presents several challenges. First, the exact location of the landing location is not known in advance. Rather, the landing location is a desired center point with an estimated uncertainty surrounding this location. The forecast and the verification must therefore attempt to account for the location uncertainty. In addition, the availability of observed weather data for verification depends greatly on the landing location. The wealth of observed data (surface observations, satellite, weather radar, and National Lightning Detection Network cloud-to-ground flash locations, for example) in the United States makes verification for these landing sites relatively easy. Even in Europe, SMG has access to a relatively large amount of surface observation data as well as lightning location data from the United Kingdom Meteorological Office and a Spanish lightning detection network. However, the verification of both winds and thunderstorms over Russia, Asia, and the Sea of Japan is hampered by the limited data available. Non-traditional verification data sources such as long-range lightning detection, satellite derived low level winds, and satellite imagery are used to overcome this limitation. However, the use of this data introduces a measure of subjectivity into the verification process.
Overall, the forecasts for both winds and thunderstorms are correct for greater than 97% of the landing opportunities and have shown a high degree of skill (Heidke skill score of 0.70 for winds and 0.46 for thunderstorms). However, these weather events have been a relatively rare condition occurring on less than 3% of the landing opportunities during this initial fall and winter evaluation period. Forecasters have not shown too great a tendency to over-forecast the occurrence of significant weather (bias is 1.6). Forecasters have had an 85% Probability of Detection (POD) of significant wind events, but the POD for thunderstorms has been lower at about 62%. The False Alarm Ratio for significant winds and thunderstorms are 39% and 62%, respectively. It is anticipated that the forecast skill will decrease somewhat during the spring and summer due to an increase in the amount of convective weather.