Challenges of incorporating the event-based perspective into verification techniques
A number of questions arise in considering "event-based" verification. For example, if the null events are indicated only implicitly, then the number of null events is indeterminate. Without a proper count of null events (particularly, the 'correct no' quarter of the contingency table) many verification scores such as Peirce Skill Score, ROC curves, and Extreme Dependency Score cannot be calculated. Another question is how forecast and observed events are paired in order to calculate verification scores. Events could be matched based on the time of onset of the event, the time of the midpoint of the event, or on a completely different characteristic (e.g., maximum strength, event duration, etc.). Furthermore, the forecast and observed events may exist at different scales, so that several distinct observed events may occur within the time frame of a single forecast event. The choice made in how to account for this or any of the questions listed here will impact the resulting skill measures.
Questions such as these will be considered in two contexts: convective weather impacting the airspace in the vicinity of an airport, and turbulence along a flight path. These two contexts involve very different spatial scales, which will allow for an examination of the sensitivity different verification approaches have to the nature of the weather event being examined.