Representing radar validity for Canadian weather radars

Applications for weather radar data are becoming more diverse and with more sophisticated requirements, so a better representation of data quality is needed within Environment Canada's radar data handling system. In principle one can discuss four properties of radar data at each pixel: the measurement itself, its quality, its validity and the history of its processing. This presentation will focus on the validity issue, while touching on the other properties for context. Radar data differs from simpler weather observations, such as surface temperature, in the numerous ways that data can become either invalid or partially valid. At least five categories of validity can occur with a gridded radar dataset: 1) Valid non-zero measurement, 2) Below minimum detectable signal, 3) Not sampled, 4) Censored, 5) Corrupted and 6) Substituted. The “Not sampled” category can in turn be subdivided into two subcategories 3A) beyond the sampled range and 3B) in area known a priori to be impossible to sample. Censoring can also be either “hard” or “soft” depending on whether it is irreversible. Unfortunately, many existing data management systems blur these categories, which often need to be handled differently in subsequent processing. The existing Environment Canada system uses a scheme of “hard” censoring where a single value is used for all “bad” data. The talk will present the validity categories in more detail, and give examples of how different applications might handle them.