Use of EDR observations for verification of atmospheric turbulence forecasts

As part of the ongoing efforts within the Forecast Verification Section in the Global Systems Division at the Earth Systems Research Laboratory, the use of in-situ eddy dissipation rate (EDR) observations are being tested as an independent observation source for verification of turbulence forecasts. Until recently, the primary source of atmospheric turbulence observations for verifications has been voice pilot reports (PIREPs). It has been established that PIREPs maintain a degree of subjectivity based on human uncertainty in reporting turbulence in the atmosphere and were not intended to provide research quality observations. More recently, automated turbulence measurements of EDR have become available on select commercial aircraft providing an automated measure of turbulence every minute in specific geographical regions, leaving a gap in spatial coverage through regions of low air traffic density. Due to this frequent measurement, a high density of “no turbulence” is reported creating an oversampling of turbulence reports. To make the EDR dataset useful for verifying turbulence forecasts, oversampling of EDR observations must be addressed. To investigate the oversampling, a technique was developed to appropriately filter the “no” observations of turbulence while preserving the moderate-or-greater (MOG) observations. This filtering was performed by capturing the maximum turbulence observation per flight per given time window for a given study period. Time windows were varied from 5-60 minutes to determine the degree to which operationally significant turbulence reports were smoothed. Results indicate that a 5-minute sampling window is ideal. Beyond this rate, sampling appears to physically alias possible turbulence events. A short case study reveals ideal sampling rates at 5 and 10-minutes provide observations every 40-60 km, which according to previous research, fits within the horizontal dimensions of a pocket of clear air turbulence (< 60 km 85% of the time). Furthermore, when verification skill scores are computed for turbulence forecasts, a reduction in performance is greatest between the full EDR dataset and the 5-minute EDR dataset. The greatest improvement in forecast skill is shown in the Bias score. At the forecast MOG severity threshold, the Bias score reduces from 25 (significantly overforecasting) using the full set of EDR observations to 5 (slightly overforecasting) using a 5-minute EDR sampling. Also, relative operating characteristic (ROC) diagrams show area under the curve calculations to be within the 95% confidence interval between the full set of EDR observations and the 5-, 10-, and 20-minute EDR sampling for the entire evaluation period. Interestingly, while these diagrams show slight reduction in skill with each sub-sampling, skill scores using EDR observations for evaluating turbulence forecast accuracy (filtered and unfiltered) show greater skill than when PIREPs are used to evaluate turbulence forecast accuracy.