'Real-Time Short-Term Quantitative Precipitation Forecasting Model Using VQGAN and Transformer for Operational Deployment'

In this study, we developed a real-time high resolution(1km) AI-based model for short-term(~6hrs) quantitative precipitation forecasting over the Korean Peninsula using generative model, VQGAN and transformer-based technologies. The models have been co-developed by the Korea Meteorological Administration (KMA) and KAIST. Currently, the KMA employs the Korea Local Analysis and Prediction System (KLAPS) and the McGill Algorithm for Precipitation Nowcasting by Lagrangian Extrapolation (MAPLE) for its nowcasting services. The model developed in this research is intended to replace these existing systems in operational forecasting. Our model utilizes a generative model, 3D-VQGAN, to compress high-resolution precipitation data into tokens, extracting precipitation patterns that are then predicted using a transformer model. The input data consists of 500m resolution radar reflectivity data over the Korean Peninsula(HSR_KMA) at 10-minute intervals, and up to 2 hours prior. The target data are also radar reflecitity. but it extends up to 6 hours into the future. It was trained on nine years of data (2014–2023, excluding 2020 for validation). The model predicts quantitative precipitation at a 1km resolution, with forecasts extending up to 6 hours at 10-minute intervals. We have been developing the model since 2021, and have ultimately selected three models as candidates for the final operational model. These models are based on the Memory-Efficient Bidirectional Transformer (MeBT) framework and are temporarily named MeBT24, MeBT12, and MeBT24_curr, respectively. Initially, memory constraints necessitated the development of models with relatively fewer layers and parameters, such as MeBT12, which could potentially impact learning performance. To address this, we enhanced the transformer’s memory efficiency with Flash-Attention and expanded the model structure. This allowed for an increase in parameters and the removal of bottlenecks, leading to the development of MeBT24. The model was trained to predict up to 3 hours, with recursive predictions extending up to 6 hours. We implemented a curriculum learning approach, where the model first learned short-term forecasts with strong causality, gradually extending the prediction length, thereby improving early precipitation flow accuracy. The model being presented in this study, MeBT24_curr, has completed curriculum learning for predictions up to 90 minutes, and training continues to extend this duration. All models have been trained using input data from the past 2 hours at 10-minute intervals, and experiments are currently underway to extend context length using past data up to 6 hours prior to improve contextual understanding. Performance analysis was conducted for all models, with MeBT24 outperforming MeBT12 overall. Although MeBT24_curr exhibited superior performance in the early stages of prediction (up to 110 minutes), its performance declined towards the end due to the short curriculum training period. The evaluation results for a subset of the validation dataset from 2020 are as follows. In the 1-hour forecast for precipitation above 0.1mm, both MeBT24 and MeBT24_curr achieved a CSI of 0.8, while MeBT12 slightly lagged with a CSI of 0.76. However, all models demonstrated a tendency to underestimate, with Bias scores of 0.6 for MeBT24 and 0.7 for both MeBT12 and MeBT24_curr. In the 6-hour forecast for precipitation above 0.1mm, the CSI was 0.65 for both MeBT24 and MeBT24_curr, while MeBT12 recorded a CSI of 0.59. The Bias scores were 0.69 for MeBT24, 0.57 for MeBT12, and 0.41 for MeBT24_curr. The analysis involved classifying and comparing cases across five types of precipitation: cyclonic, convective, orographic, frontal, and others. The results indicated that the model performed best in cyclonic, followed by frontal, and then orographic precipitation types (cyclonic > frontal > orographic). However, the prediction performance was relatively lower for convective and other types. These models, including real-time trial operations with web visualization, began on June 25, 2023 (Fig), and completed their trial operations by November.