Artificial Intelligence-Based Investigation of Severe Thunderstorms Producing Downbursts in Canada.

Introduction
Windstorms are the most severe weather hazard in Canada. Our recent analysis of insured losses (Hadavi et al., 2022) showed that ~75% of all weather-related catastrophes in Ontario and Quebec during 2008–2021 were wind disasters. We also demonstrated that convective storms are the most disastrous wind type, imposing ∼67% of the total wind damage in Canada. It is therefore crucial to be able to distinguish between environments that support severe thunderstorm winds and those that do not.

Methodology
This study develops seven machine learning models to identify environments prone to downburst-producing thunderstorms: logistic regression, Gaussian naive Bayes, decision trees, random forests, gradient boosted trees, support vector machines, and k-nearest neighbors. Figure 1 shows the overall workflow of the present research.
Two (binary) classes of downburst and non-downburst were created based on the downburst data provided by the Northern Tornadoes Project (NTP) and lightning data supplied by Environment Canada's Canadian Lightning Detection Network (CLDN). An ERA5 vertical profile was obtained for each grid cell in two categories, and after doing quality control, 38 convective parameters (selected from a total of 201) were calculated using the thundeR package as input features for machine learning models. Data were split into two independent subsets of training (80%) and testing (20%) to avoid overfitting, which happens when a model performs well on the training data by memorizing patterns but performs poorly on unseen data. A stratified split method was used to ensure that the ratio between two classes is maintained in each subset. For the time period 2019–2023, the sample size of the non-downburst class (19,132 samples) was found to be ~62 times that of the downburst class (306 samples), resulting in an imbalanced dataset. Machine learning methods often struggle to learn the concepts and patterns related to the minority class from an imbalanced dataset. Therefore, 19 different resampling methods were implemented to adjust data imbalance, and the results were compared in detail. Resampling adjustment was conducted only on the training data (not test data; Fig. 1f) to prevent synthetic overfitting; thus, the model evaluation (on test data) fully reflects the difficulty of predicting a rare event. Moreover, a hyperparameter search was performed using 5-fold cross validation to tune models. Ultimately, the final model configuration was trained on the entire training dataset and evaluated on the held-out testing dataset using several performance measures, such as Matthews Correlation Coefficient (MCC), Critical Success Index (CSI), Heidke Skill Score (HSS), precision, recall, and F1 scores. The considered metrics take into account not only the number of prediction errors but also the type of errors that are made (i.e., false positive vs. false negative), so they can deal with the class imbalance effectively.

Fig. 1. Overall workflow of the adopted methodology in the present study for developing and evaluating machine learning models.

Results
Results indicate that the random forest model significantly outperforms other models for the particular dataset of this study (Fig. 2 and Fig. 3).

Fig. 2. Model performance metrics of (a) MCC, (b) CSI, (c) HSS, and (d) the area under the ROC curve (AUC) for considered machine learning models. The boxplot distributions rely on 1000 bootstrapped samples generated from the held-out test dataset.

Fig. 3. (a) Receiver Operating Characteristic (ROC) curves for each ML model; the AUC value for each is shown in the legend. (b) Performance diagram for all ML models.

Finally, the techniques such as permutation importance and accumulated local effects are used to alleviate the opaque nature of ML methods. Permutation importance determine top ten most important variables (Fig. 4) and accumulated local effects describe how individual input features affect the prediction of a machine learning model and outputs (Fig. 5).
It is observed that all the selected features make meteorological sense and are in line with previous research studies. Also, the impact of each variable on downburst probability is meaningful. For example, in Fig. 5a, the ALE (i.e., the probability of a downburst) increases by ∼8% as cold pool strength increases from ∼6 to ∼14 °C, which is consistent with Romanic et al. (2022).

Fig. 4. Multipass backward results are in the top row, while multi-pass forward results are for the bottom row. Each column corresponds to a different error function used to assess loss of skill. (a), (d) AUC; (b), (e) area under the precision-recall curve; and (c), (f) area under the performance diagram curve. Bars are colored based on parameters’ categories: parcel (grey), thermodynamic (orange), kinematic (blue), and composite (dark red).

Fig. 5. Accumulated local effects (ALE) for the top nine predictors of the permutation test: (a) Cold Pool Strength (CPS), (b) Downburst Environment Index (DEI), (c) mean equivalent potential temperature between surface and 1 km, (d) moisture flux between surface and 2 km, (e) mean wind speed between surface and 1 km, (f) precipitable water, (g) mixing ratio, (h) bulk wind shear between surface and 6 km, and (i) lapse rate between surface and 1 km. Red line corresponds to the random forest model. Gray histograms in the background are the counts of points in each bin (corresponding to the right y axis).

To conclude, our findings indicate that relationships learned by the random forest model is physically reasonable, highlighting the potential utility of artificial intelligence–based tools for severe thunderstorm and downburst diagnostics.

References

Hadavi, M., Sun, L., & Romanic, D. (2022). Normalized insured losses caused by windstorms in Quebec and Ontario, Canada, in the period 2008–2021. International Journal of Disaster Risk Reduction, 80, 103222. https://doi.org/10.1016/j.ijdrr.2022.103222
Romanic, D., Taszarek, M., & Brooks, H. (2022). Convective environments leading to microburst, macroburst and downburst events across the United States. Weather and Climate Extremes, 37, 100474. https://doi.org/10.1016/j.wace.2022.100474