Observation sets are critical to verification as they serve as the truth for comparison. However, incorporating observation sets, particularly for verification of aviation weather products, often comes with challenges. While flight-based observations such as pilot reports and automated in-situ observations are one of the few direct observations of aviation weather hazards such as icing and turbulence, they provide an biased sample of the atmosphere due to the fact that they often avoid the hazard being observed. Additionally, in the case of diagnostic or analysis products, these observation sets are often incorporated into the product being assessed.
To support the assessment process, the QA PDT performs investigations of new observation sets and develops verification techniques to provide performance information in an operational context. This involves developing an understanding of new datasets as compared to known ones, and identifying the characteristics and fields relevant to verification for the aviation hazard of interest. Examples of recent observations sets investigated and used for purposes of verification include GPM, AMDAR, CloudSAT, and CALIPSO. As part of the investigation and technique development, these datasets are compared to other observation sets such as METARs, soundings, and ground-based radar products to form a more comprehensive sampling of a specific hazard. This presentation will provide an overview of such observation sets, findings, and how they were incorporated into verification techniques for our assessments.