Quantifying Inequity in Data-Driven and Physics-Based Weather Models

Skillful weather prediction is crucial for decision-making and the protection of life and property. This is especially true in preparation for extreme weather events such as heat waves, which studies have shown disproportionately impact historically underserved communities. These impacts can potentially be mitigated if decisions are guided by skillful weather forecasts. Such forecasts are traditionally produced using physics-based numerical models, which rely on fundamental physical knowledge and an initial state derived from a global network of observations. In contrast, data-driven models have recently been developed, which rely solely on historical data to produce forecasts that can meet or exceed the accuracy of physics-based models. However, human decision-making drives the placement of observation systems, prioritizing practicality (e.g., accessibility for maintenance) and mission objectives (e.g., support of aviation) over other considerations. Therefore, the data collected likely contain systemic biases owing to the geographic distribution of sensors. Since both paradigms of weather forecasting rely on observations, it is reasonable to believe that the predictions produced by these models may inherit some systemic biases from those contained within observations. Yet, little work has investigated how the spatial and temporal distribution of prediction skill coincides with different populations and regions, leading to a crucial knowledge gap: do different populations and regions benefit equally from the forecasts produced by these paradigms? In this work, I develop a machine-learning-based framework to quantify this potential inequity at high resolution across the country and apply it to forecasts of 2m temperature at a range of lead times. Using a variational auto-encoder, I reconstruct historical 2m temperature forecast errors, extracting the latent space as a new quantitative index of forecast skill with associated uncertainty. After investigating the interpretability of this index, I then integrate it with demographic data (e.g., income, poverty, race) from the American Community Survey by aggregating to the block-group level to examine the distribution of forecast skill among different populations. Comparing the results when this index is generated with physics-based and data-driven model outputs allows me to assess the influence of model paradigm on forecast equitability. As machine learning continues proliferating within the atmospheric and climate sciences, we must challenge ourselves to ensure our work is serving everyone, especially those who are the most vulnerable.