Creating Synthetic Passive Microwave Observations Using Deep Learning to Aid Tropical Cyclone Forecasting

Deep learning has revolutionized our ability to perform image-to-image translation tasks. Diffusion models are gaining attention due to their ability to create sharper, more accurate images that capture the target's data distribution. In addition to utilizing traditional fully-connected and convolutional neural networks, we apply diffusion models to the task of generating synthetic passive microwave imagery. Using geostationary satellite data as inputs, the resulting synthetic imagery provides 10-minute, 4-km resolution images that fill in the gaps from polar-orbiting passive microwave data. Here we will discuss how applying diffusion models has improved our synthetic imagery, focusing on the strengths and weaknesses of diffusion with respect to its ability to provide useful information for tropical cyclone forecasting. One essential factor in providing helpful synthetic imagery is having appropriate evaluation methods, and we will discuss which evaluation tools have been most helpful in improving our product.

Learned aleatoric uncertainty estimates from our deep-learning neural networks, along with epistemic uncertainties from our multi-model consensus, provide well-calibrated uncertainty estimates that indicate how accurate the scene is without knowing the truth. Some deep learning models provide learned aleatoric uncertainty estimates, that when combined with epistemic uncertainty estimates from a model consensus, indicate how accurate the scene is without knowing the truth. Utilizing this information, we provide a scene-based uncertainty estimate for every image, with the goal that this 1) will allow forecasters to screen images based on an established threshold if desired, increasing user confidence; and 2) help illuminate biases in the synthetic imagery that will help users learn how to interpret the imagery and adjust to its shortcomings appropriately. Here we will assess the ability of our deep learning models to provide not only accurate and timely synthetic imagery, but also to provide well-calibrated scene-based uncertainty estimates that can also be utilized in real time to aid the overall value of this product.

Disclaimer: The scientific results and conclusions, as well as any views or opinions expressed herein, are those of the author(s) and do not necessarily reflect those of NOAA or the Department of Commerce.