A Method to Verify Convective Weather in TAFs Specific to Aviation Applications

The FAA and ICAO mandate the use of TAFs for domestic and international aviation operations for flight planning and flight following. However, the TAF is also relevant and valuable to other actions and decisions in aviation operations. There have long been challenges developing a meaningful verification methodology for TAFs, especially associated with convective weather. The Weather Company (TWC) has collaborated with American Airlines (AA) to develop a new, automated verification methodology to assess the accuracy and applicability of TAFs. This verification will be beneficial for (1) forecaster review and forecast content improvement, (2) integration into AA’s operational metrics to measure airline performance, and (3) assessing the overall quality of services performed by TWC.

Convective forecasts are commonly provided in the following TAF groups: PROB30 and TEMPO which are conditional groups for thunderstorm potential; VCTS for thunderstorms within 25 miles; and main body TSTMs for high confidence, long period storms. METARs have commonly been used to assess TAF skill, but inconsistencies (1) in reporting thunderstorms and their distances from the airport between the automated and human observations and (2) among different airports results in a considerable observation error that biases verification. Therefore, both radar reflectivity and lightning density data are used to augment METAR observations of convection. In addition to providing a consistent and reliable observation record, these data also naturally contain information about the distance of convection, as identified by reflectivity pixels and/or non-zero lightning density, from an airport. These location data allow for the grouping of radar and lightning observations into the following categorization scheme that provides for a more flexible verification framework: TSTM over Field (< 5 mi), VCTS (5-10 mi), VCTS (10-25 mi), TSTMs within Gates (25 mi – furthest gate), and No TSTM. This scheme is further motivated by the inconsistent use and interpretation of the VCTS group in TAFs. Penalties ranging from 0–1 were identified for each combination of TAF forecast and observation category and are weighted by proximity to the TAF issuance time. This verification product generates both human and machine-readable spreadsheets that can be consumed by aviation leadership and data analytics teams. We plan to discuss additional features of this verification methodology and its potential applications and impacts during the talk.