Developing Societal Outcome Indicators for NWS' Weather Ready Nation Program

This presentation will describe the work being done to design societal outcome indicators for the National Weather Service’s (NWS’s) Weather Ready Nation (WRN) program. NWS routinely tracks a set of internal performance metrics related to efficiency and accuracy such as “false alarm rates” and forecast accuracy. NWS, however, has become more interested in measuring its impact on societal outcomes. In this presentation, I will discuss the steps we took to design societal outcome measures for the WRN program, including the design and implementation of data collection efforts. We developed four sets of societal outcome metrics under the project: measures of public knowledge, measures of public preparedness, measures of the actions taken by the public once an event occurs, and measures of the reduced impact on weather-related injuries. A key aspect of the design was to leverage existing data sources within NWS for cost-effective long-term implementation. Although the initial focus of the work was on WRN, the results of the project can be seen as having a broader application to many NWS activities that are centered around providing information with the goal of influencing individual behavior. First, NWS provides information to the public with the goal of influencing individual behavior. This study was designed to measure societal outcomes at the individual level. Second, government agencies often struggle with collecting recurring data to measure individual-level outcomes due to budgetary constraints. We designed performance metrics for WRN taking this into account and we looked for data streams that could be leveraged within NWS to collect these data. Thus, our approach represented a cost-effective approach to designing societal outcome measures. Finally, performance metrics can provide useful data sources for evaluating programs over the longer-term. In our design of societal outcome measures for WRN, we explicitly took into account future evaluation efforts. Those who attend this session will learn about the process we used to measure individual-level outcomes associated with WRN.