Object-Based Storm Tracking Method for Convection Allowing Model Verification

John David Goode¹,², Sean W. Freeman¹

¹Department of Atmospheric and Earth Science, The University of Alabama in Huntsville, Huntsville, AL

²Earth System Science Center, The University of Alabama in Huntsville, Huntsville, AL

Short-range Convection Allowing Models (CAMs) are critical contributors to storm prediction and can be difficult to validate. End-users of these models, particularly forecasters, rely on these models to predict the proper storm types, in the correct location, at the right time. Historically, verification of these models has been performed on a grid-point basis, which can provide some insights on trends, but fails to identify individual storm performance. Newer object-based methods are an improvement in thunderstorm verification with CAMs, but many of these methods rely on pairing individual storms found in both the model and observations, and do not include a time component. Here, we introduce a tracked object-based verification methodology that allows for rapidly examining trends in CAM thunderstorm simulation performance that does not rely on paired storms and includes a time-based component. The Python package tobac (Tracking and Object-Based Analysis of Clouds) was utilized in creating an efficient, repeatable, and flexible method for comparing simulated data from a CAM to observational data. This method introduces a novel verification tool: a “fingerprint”. The track of each storm cell within the selected time period is plotted on a map—the model’s cells in one color, and the observed cells in another. By viewing these “fingerprints” overlaid, the output of the model is easy to evaluate intuitively.

As a demonstration of our methodology, subhourly reflectivity data from the High-Resolution Rapid Refresh (HRRR) model was compared to Multi-Radar Multi-Sensor (MRMS) reflectivity. MRMS data corresponding to the study period was downloaded, regridded to the HRRR’s grid, and both the model and the MRMS data were masked to the contiguous United States (CONUS). Data for the entire CONUS was processed across the entire year of 2023. During processing, we defined a cell as an area with 4+ adjacent pixels with a value greater than 25 dBZ; tobac is also able to distinguish between separate features within a large area of 25+ dBZ, such as individual cells within a mesoscale convective system. Initial results include that the HRRR underproduced diurnal storms in the NW CONUS. Because the output data is time-based, individual subsets across space or time can be analyzed. For example, the average cell count in HRRR at the end of its 18-hour forecasts was 38% higher than MRMS for the month of June; however, the average cell area produced by HRRR was substantially less than that produced by MRMS for the same time period. Total cell count by time, maximum reflectivity by cell, cell lifetime, and cell area are examples of other data readily output by this method.

This analysis method is promising for verifying new CAMs, including RRFS, RAPS, and FV3kmm, and new machine learning based CAM-like forecasts. Further, due to the speed at which it can be run, near-real-time verification is possible with this method, enabling rapid and ongoing evaluation of CAMs as they are run. This CAM verification method produces a variety of statistics, and “fingerprints” provide a detailed summary of a model run’s performance at a single glance—a powerful tool for forecast verification.