Verification of DWD MOSMIX and the meteoblue learning multimodel at more than 500 WMO stations worldwide

Accurate weather forecasting is essential for a wide range of applications, from daily decision-making to disaster preparedness. This study presents the various validation processes undertaken at the Swiss company meteoblue AG as we continuously assess the performance of various weather models to ensure the highest accuracy. We compare the performance of different forecasting models, specifically focusing on the Model Output Statistics (MOS) from Deutscher Wetterdienst (DWD) and meteoblue's machine-learning model (mLM). The validation involves a comprehensive analysis of key meteorological variables, including temperature, precipitation, wind speed, dew point temperature, and relative humidity. For each variable, appropriate validation metrics such as Mean Bias Error (MBE), Mean Absolute Error (MAE), and Root Mean Square Error (RMSE) are computed. These metrics provide a quantitative assessment of the models' performance across different lead times. The study specifically compares the performance of DWD's MOSMIX with meteoblue's mLM at over 500 stations, with detailed analysis over a three-month period (March-June 2024). Results indicate that both models show increasing MAE for temperature predictions with longer lead times, with DWD MOSMIX and mLM at the same accuracy level. However, both models struggled with precipitation forecasts, failing to achieve significant skill scores across various metrics. The mLM model tends to overpredict, while DWD MOSMIX often underpredicts precipitation. Further analysis extends to the validation of mLM, ICON, and ERA5 datasets against METAR station observations. The mLM model, when validated against METAR locations, outperforms both ICON and ERA5, demonstrating the model's superior ability to adapt to localized conditions. However, the analysis highlights variability across different stations, suggesting areas for future improvement, particularly in the handling of outliers and the refinement of precipitation predictions.