Meteorological Interpretation of XAI Output Applied to a 3D Convolutional Neural Network Fog Prediction Model

A novel 3D convolutional neural network deterministic model (FogNet) was developed to predict fog at Automatic Weather Observing System (AWOS) station KRAS (Mustang Island Airport in Port Aransas) on the Middle Texas coast. The model performed superior to the High-Resolution Ensemble Forecast (HREF) system (available to National Weather Service operational meteorologists) when using skill-based performance metrics. (In order to compare FogNet and HREF performances, the probabilistic HREF output was transformed to a deterministic HREF by determining thresholds that maximize HREF performance.) High FogNet prediction performance was achieved with respect to advection fog events, yet weak performance was noted for radiation fog events. XAI techniques were applied to the model to determine the subset of feature and feature groups that strongly influenced model predictions without regard to prediction accuracy/skill (feature effect), and features that contributed to model performance (feature importance).

This study examines the most salient features and feature groups arising from various feature effect and importance methods. The features and feature groups that possess high feature importance provide useful information for operational meteorologists. Features with the greatest feature effect and importance captured several meteorological processes and/or ambient environmental conditions associated with fog, enhancing model trustworthiness. For example, spatial analysis of composite XAI feature effect results reveal that the greatest contribution of the features to FogNet predictions occurred in the vicinity of the target location, which demonstrates the critical importance of local forcing to fog development. Feature effect output provided insight as to why the model performed poorly with respect to radiation fog cases, which allowed for the identification of future changes to the feature set and feature group configuration necessary to improve performance.

This study is a collaborative effort between the National Weather Service Weather Forecast Office in Corpus Christi, Texas and the Conrad Blucher Institute for Surveying and Science at Texas A&M University–Corpus Christi.