1) They only compare data sets at pixels that are co-located with weather stations, limiting the area of study.
2) They resample and reproject data sets onto a common grid, distorting the original interpolations and introducing uncertainty into the comparisons.
To work around these limitations, we developed a new framework for comparing GMDs. We used this framework to compare maximum temperature (tmax), minimum temperature (tmin), and precipitation (ppt) interpolations from five GMDs across the state of Montana and then used unbiased Montana Mesonet weather station data to compare each GMD to measured ground data. Our findings show that at large temporal (30 years) and spatial (state of Montana) scales, all five data sets produce similar results for tmax, tmin, and ppt. At smaller temporal and spatial scales and at higher elevations, however, the predictions of data sets begin to diverge. Most notably, we found that interpolations of ppt between data sets begin to diverge at 1,500 meters and that daily interpolations of tmin and tmax can be biased by as much as 5°C relative to Montana Mesonet measurements in agricultural settings. Our results show that failing to acknowledge the differences between GMDs and physical weather measurements could significantly impact the final decisions and planning efforts of GMD users.