"Am I Confused or Is This Confusing?": Deep Uncertainty Quantification of North Pacific SST Variability

We will never have absolute confidence when projecting future climate states from past states because climate is chaotic. Therefore, when studying the climate system, we build probabilistic models (e.g., probabilistic neural networks) to approximate the distributions of possible future climate states conditioned on past data. When quantifying this uncertainty inherent in our system (which, in the machine learning (ML) world, is often called “aleatoric uncertainty” or “risk”), we can also attempt to quantify the range of possible distributions (models) which might describe our system (ML researchers call this “epistemic uncertainty,” “model uncertainty/ignorance”, or, simply, “uncertainty”). If we have a good measure of epistemic uncertainty, then we have a good idea when we can trust our probabilistic neural network–-because it is operating in a regime with which it is familiar–and we also know when our model is confused and making a shot in the dark. Ultimately, this allows us to gauge trust in our estimates of state-dependent (i.e., input-dependent) climate predictability, as well as identify when an instance of the climate system is fundamentally different from what our model has seen in training.

Neural network ensembles and second-order parametric neural networks are among the most effective and intuitive frameworks for epistemic uncertainty quantification in machine learning. Here, we evaluate the ability and practicality of these techniques for multi-annual North-Pacific SST anomaly prediction in the CESM2-LE, particularly with respect to gauging trust in estimates of state-dependent climate predictability and for the detection of out-of-training-distribution Pacific climate regimes.