Excessive Rainfall Outlook Verification Product for WPC Forecasters

Flash flood and quantitative precipitation forecasts (QPF) of heavy rain are a
primary concern of the Weather Prediction Center (WPC). One critical WPC product,
called the Excessive Rainfall Outlook (ERO), forecasts the probability of rainfall
exceeding one, three, and six hour flash flood guidance (FFG). Unfortunately, ERO
forecasts have not been extensively verified against FFG or any flood observations in the
past few years. Furthermore, WPC forecasters need methods to verify their daily forecast
performance in addition to longer-term bulk statistics. This talk will detail the verification
and in-house website development of the ERO ranging from daily assessment plots to
bulk 3-year performance.

The ERO consists of four probabilistic risk categories of precipitation exceeding
FFG: marginal (5-10% probability of precipitation exceeding FFG), slight (10-20%),
moderate (20-50%) and high (50-100%). WPC uses dynamical and statistical models,
situational awareness, and other rules of thumb to generate EROs valid for days one
(current day), two (next day) and three (two days into the future). The ERO is verified
using the Model Evaluation Tools (MET) software against FFG within 40 km of a point
from 2015 to present using stage IV precipitation data. The ERO is also compared to
WPC’s Unified Flood Verification system, which consists of local storm reports (LSRs),
United States Geological Survey (USGS) river gauge observations, and Average
Recurrence Intervals (ARIs).

ERO average fractional coverage shows the seasonality of flooding events;
specifically the spring Mesoscale Convective Systems, summer monsoon and tropical
activity and winter lull. Longer-term verification confirms that the ERO probabilistic
forecasts are well calibrated, although observed probabilities are towards the higher end
of the probabilistic definition in 2018. Employing skill metrics such as Brier Score (BS)
or Area Under the Relative Operating Characteristic (AUC) shows that newer forecasts
are more skillful, and cool season forecasts are more skillful compared to the warm
season. However, AUC and BS are not appropriate for assessing skill for single events,
and WPC is exploring other methods to evaluate daily skill. Most of this verification is
available to WPC forecasters on an internal website.