Fact, Fiction, or Somewhere in Between: Assessing the Quality of Severe Weather Information in the Twitterverse

Social media is growing in popularity, especially during severe weather events. Among other things, people and organizations are using social media to send and receive information about severe weather before, during, and after events. Unfortunately, we know relatively little about the quality of this information—is it accurate and/or timely or inaccurate and/or outdated? We begin to answer these questions by systematically analyzing a random sample of 5,000 messages (“tweets”) that contain the word “tornado” that were published on Twitter before, during, and after the May 20, 2013 Oklahoma City area tornado. Our analysis proceeds in five steps. First, we identify the subset of tweets that contain verifiable information, such as the location of a tornado. Second, we verify (or “fact check”) the information contained in those tweets according to the best available information to date, like information from the NWS and SPC event archive sites. Third, we identify the subset of tweets that contain “actionable” information—i.e., information that can be used to make a decision about how to respond to a hazard, such as watch, warning, or location information. Fourth, we assess the timeliness of the actionable information contained in those tweets—was the information published in time for people to act upon it? Was it too late? Finally, we perform a logistic regression to explore some of the factors that affect the quality of the information contained in these tweets—i.e., user type/affiliation, social media experience, and reputation. This information will be used to answer questions about the reliability of severe weather information on social media, and how to enhance that reliability. A better understanding of these issues potentially could help agencies such as the National Weather Service and emergency management officials.