Event-based turbulence verification

Recent turbulence evaluations by the FAA's Quality Assessment Product Development Team (QAPDT) have included some forms of flight-based verification approaches, specifically comparisons of the duration of turbulence events and time windowing (i.e., comparing forecast and observed turbulence over periods of 10, 20, 30 minutes). With the reduction of grid spacing between versions 3 and 4 of the Graphical Turbulence Guidance (GTG) from 13 km to 3 km, a closer correlation between forecasted turbulence fields and aircraft-observed turbulence is expected. Therefore, we believe it is now appropriate to expand evaluation methodology from the flight-based to event-based. By this is meant that scoring metrics can be applied to the twin forecast and observed turbulence time series; the scoring metrics will capture both the timing and duration of events. Two primary motivations for this approach are noted. First, the forecast is evaluated in the context of pilot experience, answering the question, “I see turbulence forecast 53 minutes into my flight. How likely is that to occur? What are the typical timing errors in these forecasts?” Second, a challenge in applying confidence intervals on performance measures for turbulence is the difficulty in determining the functional sample size (after accounting for autocorrelation--the lack of independence in neighboring data). Switching to an event-based approach sidesteps the effect of autocorrelation. This presentation will demonstrate the suitability of an event-based approach to turbulence forecast evaluation.