Using Python and Machine Learning Techniques for High-Impact Weather Nowcasting in an Operational Center in Brazil

In the last 10 years, weather radars and lightning detection network coverage have increased in Brazil, especially in southeastern region, an area prone to high impact weather, mostly related to Mesoscale Convective Systems. The major economic activity of this region is agriculture and energy production, directly dependent on precipitation distribution, water availability and severe storms impacts. Usually these events are also accompanied by atmospheric discharges, which can also cause energy distribution problems and shutdowns in the electric sector. Therefore, improving the monitoring and forecasting capabilities of these High Impact Weather (HIW) events, a type of event that can affect economy and endanger human life, help in the decision-making process and operational measures of energy companies, as well as help mitigate and even anticipate damage, allowing those decisions to be made. In this paper, we present the approach developed in an operational center of HIW nowcasting, which uses the information of S-Band weather radars and lightning detection networks. Severe storms monitoring and forecasting usually comprise the identification of the main processes: identification of active storm cells, tracking of the storms and forecast direction and velocity of the storm movement around an area. The focus of this work is tracking and very short range and nowcasting of these storms, aiming to study machine learning methods for short-term storm forecast on cells identified and tracked with TITAN (Thunderstorm Identification, Tracking, Analysis, and Nowcasting) system in different stages as well as include lightning information to find storm intensity and life cycle. Due to the nature of the phenomena represented in this work, machine learning methods were chosen because they are able to learn from the features presented and their relationships. Moreover, once the model is learned by the chosen method, the processing of the new entries occurs much faster allowing for nowcasting applications. The evaluation of the results were done according to meteorological concepts of the phenomena and also by comparing them with the forecast provided by TITAN for each cell, since it is a well-established tool used in meteorology. We applied supervised machine learning regression algorithms, such as generalized linear models (Bayesian ridge, Huber and Theil-Sen) and ensemble models (bagging, extra trees, and random forest). Some of the features of the thunderstorm used as input were: position, orientation, displacement historic, area, volume and maximum reflectivity value for the radar data, and also positive and negative cloud-to-ground lightning data. Lightning data, when comparing the same models with and without this information improved only 10% or less for the centroid location and ellipses size, but these results need to be reviewed or replaced by total lightning information. The best performance was achieved with the Random Forest and Linear Models algorithms, and its results proved to be satisfactory for the forecast of these events. We tested generalized linear models and also ensemble models, but random forest model showed better results for the storm centroid location and storm sizes. Since the study has already shown promising results, we included in this paper the results of classification and regression methods to identify storm cycle and dissipation using total lightning data from the Brazilian lightning detection network and the results are presented in terms of storm identification and location performance which is available in an operational nowcasting service. With this work we expect to improve our knowledge about these storms that impact the region and also our ability to forecast them in an operational environment.