National Weather Service warnings: comparing expectations with reality

The National Weather Service (NWS) is responsible for issuing severe thunderstorm and tornado warnings for the United States. The success of these warnings is primarily measured using two parameters: Probability of Detection (POD) and False Alarm Ratio (FAR). Although the NWS continuously seeks to provide the best possible warning services, little quantitative data is available on what NWS customers deem as "acceptable" POD and FAR scores. To gain insight into customer expectations of NWS warning skills, the Tulsa NWS Weather Forecast Office (WFO) conducted a survey of emergency managers, media, and NWS employees. Over 300 respondents were asked what they considered "acceptable" POD and FAR scores for severe thunderstorm and tornado warnings.

The POD and FAR scores deemed acceptable by NWS employees and some key NWS customers, namely emergency managers and the media, are compared against each other. These results suggest subtle but important differences between the expectations of the warning provider (NWS) and warning recipients (NWS customers). These expectations were further compared against actual POD and FAR scores of NWS offices. The differences between expectations and "reality" are dramatic, suggesting the need for continued warning operations improvement, severe weather research, NWS/customer dialogue, and public education.

Finally, correlations are made between actual POD, FAR, and warning lead times. This analysis seeks to demonstrate interdependencies between these important severe weather verification parameters. For example, if the NWS could alter its practices to meet customer expectations for one parameter (e.g., lowering its FAR), the analysis attempts to show what the resulting "cost" would be in another parameter (potentially reduced lead times). Such a cost/benefit analysis presumably would be valuable in NWS policy-making and goal-setting activities.