Are the data you rely on valid?

High quality data are essential to the validity of results in any study relating to Applied Climatology or Statistical Probabilities. One can not assume that all data collected and archived are truly of sufficiently high quality to assure the validity of the research conclusions or the predictions made from the processed data.

While great technological advancements have been made in meteorological instrumentation, automation, and communication, errors in climatological data bases are introduced in many ways and often remain undetected until after use in studies or predictions, despite the efforts of archival agencies to quality control data received.

Instruments may be miscalibrated, misaligned, or mistakenly read; digits in recorded values may be interchanged or erroneously transcribed; and transmission errors can “creep in” in todays electronic transmission of huge quantities of data.

The data are often assumed to be representative of the values intended to be measured, despite the fact even small scale instrument relocations on an airport, or subtle and unexpected environmental changes in the vicinity of the instruments can have significant impacts on the representativeness of the measured parameters.

“Quality Control” by deletion of suspect data can alter distributions, leading to invalid analyses of parameter distributions.

An assumption that all - or even the majority - of data errors or non representative measurements are detected and corrected in quality control process is unwarranted, as quality control procedures are neither designed to detect subtle calibration drifts, small errors in calibration, the effects of “minor instrument relocations,” or the true validity of extreme values.

These errors may vitally affect analyses of trends, critical decision values, or the determination of extreme values.

Examples of each of these types of invalid data from unquestioned data bases are discussed.