A Performance Assessment of the National Ceiling and Visibility Analysis Product

In this presentation we summarize results from a formal quality assessment of the National Ceiling and Visibility Analysis product (NCVA), a gridded analysis that evaluates reported ceiling and visibility information for the purpose of improving the flight planning process. On behalf of the Federal Aviation Administration's Aviation Weather Research Program, and in support of an Aviation Weather Technology Transfer (AWTT) D4 (operational) decision point, this study was carried out to examine the following:

• The quality of the NCVA product with respect to a baseline analysis.

• The effect on the NCVA of utilizing a satellite-based cloud mask.

• The potential value of NCVA's frequent update-cycle to the flight planning process.

• NCVA's performance compared to the operational Weather Depiction Analysis, a product specifically referenced in the NCVA Concept of Use.

Constrained to the use of METARs for verification, this assessment employs a cross-validation technique to create independence between the observational set and the input set that is utilized by the NCVA algorithm. Cross-validation statistics produced from METAR data that are withheld from the NCVA should not be interpreted as measures of the algorithm's skill at METAR sites, but rather as a measure of performance of the NCVA grid at points away from METAR sites. The NCVA algorithm retains information from all operationally available METARs used as input, never altering data at grid points associated with available METAR reports.

In the absence of a secondary gridded product that represents current operational planning information, the analysis team created a proxy baseline product for comparison. This product, referred to in this report as the Nearest Neighbor Analysis (NN-A), simply uses the METAR closest to each evaluation point in determining ceiling, visibility, and flight category values. The study period consists of the summer months of 2008 and the winter months of 2008-09, chosen to provide significant data for stratification of results by season and time of day.

Results to be presented will highlight: (1) NCVA's performance compared with a baseline analysis; (2) the effect on quality from a satellite-based cloud mask; (3) the potential value to the flight planning process of frequent updates to this product; (4) a comparison against the Weather Depiction Analysis; and (5) an assessment of performance by sub-region and weather regime.

This research is in response to requirements and funding provided by the Federal Aviation Administration. The views expressed are those of the authors and do not necessarily represent the official policy and position of the U.S. Government.