Evaluation of ARPS and GDST 1-6 Hour Forecasts

MIT/Lincoln Labs (MIT/LL) has developed the Growth and Decay Storm Tracker (GDST), which is designed to track the storm envelope of existing convection using radar data. The GDST produces forecasts up to 1 hour at resolutions as high as 1 km using a filter technique to generate a 2D motion field. The Advanced Regional Prediction System (ARPS) has been developed by the Center for Analysis and Prediction of Storms (CAPS) and is used to create real-time forecasts of convective events at resolutions of hundreds of meters and over forecasts periods of 6 hours or more. The purpose of this study is to provide insight into the accuracy of ARPS and GDST six-hour forecasts of position, coverage, and strength for five strongly-forced convective events. The ARPS and the GDST are initialized with WSR-88D data. A six-hour forecast period requires that storm system dynamical processes are considered. Hence, it is important to evaluate the accuracy of the GDST simulations, which has no dependence on dynamical process, beyond the first forecast hour.

Forecasts are compared using statistical indices such as Probability of Detection (POD), False Alarm Rate (FAR), and Critical Success Index (CSI). However, subjective parameters are also used to evaluate forecasts (e.g., storm intensity, initiation location, movement, growth and decay) including a synopsis of the storm-scale/mesoscale environment. Test cases are selected from the Spring Operations Period 1999 (SOP99) -- a real-time forecast testing period that demonstrates the value of initializing the model with Level-III (NIDS) WSR-88D radar data. Each case is selected based on the appearance of strongly-forced convective events that contain significant radar echoes observed no later than the first hour of the forecast period.

Results show it is paramount to use WSR-88D data to initialize the ARPS model. ARPS forecasts nearly always contain the proper mode and orientation of convective systems providing potential for accurate forecasts. However, ARPS did suffer from some consistent forecast problems associated primarily with spurious convection, areal coverage of higher reflectivities, and intensity of convection. In general, the GDST is able to discern the magnitude of propagation for convective systems as a whole. In addition to forecasting the movement of line-storms fairly well, the GDST also showed the ability to accurately forecast the movement of mature supercells in one event.

Generally, the GDST is able to produce a more accurate forecast over the initial 2-hour forecast period while ARPS forecasts tend to produce more skillful forecasts in the 2-6 hour forecast period. Although the size of this statistical sample is extremely small, POD and CSI values for the GDST are very near zero by the second hour of the forecast. However, they are higher than the averaged ARPS values through the first forecast hour. Beyond the second hour, the ARPS’s POD and CSI values rise with POD averaged values near 0.35 and CSI averaged values are approximately 0.13. The GDST produces lower FAR values throughout the first two hours, but both models generate high FAR values from 3 to 6 hours into the forecast period.