Can Machine Learning Features Identify Fitness of Meteorology Simulations for Application to Air Quality in Bogotá, Colombia?

Standard meteorological model performance evaluation (MPE) can be insufficient in determining “fitness” for application to air quality modeling.  Typical MPE compares predictions of temperature, wind, and humidity to community-based thresholds. Conceptually, these thresholds are used to measure the model’s capability to represent mesoscale features that cause variability in pollution from emissions. Thus, a method that instead examines features through machine learning could provide a better estimate of fitness. This work compares measures of fitness from standard MPE analysis to those derived from machine learning. Both threshold MPE and feature analysis are then evaluated as predictors of “acceptable” air quality model performance. We expect feature-based meteorological fitness to provide a more accurate measure of performance.

Meteorology and air pollution simulations for Bogotá, Colombia provide an ideal case study. This case is particularly interesting because the complex local topography presents challenges for the Weather Research and Forecasting (WRF) model. A k-means cluster analysis identified 4 dominant meteorological features associated with wind speed and direction. The model predictions are able to pass several MPE thresholds, but as expected show poor performance for wind direction error. By comparison, the model is more likely to reproduce the cluster analysis features. The four observationally-derived features have clear relationships with particulate matter concentrations, which suggests that reproducing the features will indicate better air quality model performance.

Meteorological model performance is fundamentally important to air quality modelling, but threshold MPE analysis may insufficiently describe fitness. We demonstrate the relationship between meteorological MPE, thresholds, and feature prediction. Then, we’ll discuss further the relationship between the evaluation techniques and model fidelity. Feature-based analysis may be particularly useful for high resolution modelling (1km or less). In these cases, small-scale variability could be spatially offset and cause poor performance with standard threshold analysis even when the general pattern is well reproduced.