Creating a Globally-Consistent, Quality-Controlled, Point-Location, Historical Weather Database Utilizing Several Standardized Validation and Gap-Filling Methodologies

At MeteoGroup a long, quality controlled, gap-filled, station database has been created. The database contains long time series of various atmospheric parameters for about 20,000 stations worldwide. Currently, most climate analyses are carried out with an atmospheric model (e.g. ERA-40, CFSR etc.) in order to generate a spatially and temporally homogeneous gridded data set; however, many times information needed by end-users are for point-locations. In our approach we focus on a single point. For each synoptic hourly station a data series has been assimilated in three steps:

1) The data is quality controlled. Every parameter has its own set of checks, consisting of time consistency checks, gross checks and statistical checks. 2) Then the various data sources are compared and the best data point is chosen (and flagged). The flagging allows the user to be able to determine the original source of a particular value. 3) Any remaining missing data are filled using five different sophisticated statistical MOS techniques; a regular MOS system, a special MOS for hourly gaps, a special MOS for two hourly gaps, a special MOS for three hourly gaps and a spline methodology.

After the above steps are completed, the data in the database is then controlled over longer time periods. This is an improvement over the short term quality control, which only uses several weeks of data to check the quality. Two main checks are done:

1) Check for jumps in the data. Are there any jumps in the observed variables? Example: Average temperature for station A from 1980-1990 is 10 degrees and over 1990-2000 it is 25 degrees. 2) Check for extremes or outliers. Some outlying values are only noticeable when looking at all data. Example: Daily precipitation observation of 300mm is not uncommon, but it is suspicious if the station never reported higher than 80mm over a 40 year time period.

The Historical Weather Database (HDB) contains about 20,000 stations worldwide, with hourly data going back as far as 1931. Data is gap filled from 1957 using one of the MOS techniques. All data is accessible through a user-friendly web interface so it takes only a few seconds for a user to obtain a quality checked, gap-filled dataset. The HDB is an extremely useful source of information for climatological studies for both non-commercial and commercial parties such as wind and solar farm developers (climate surveys) and commodity traders. Additionally, this standardized methodology can be applied to proprietary weather networks for specialized studies. The main focus of the presentation will be the results of the long-term quality control done especially for the HDB.