Understanding the socio-economic impacts of extreme weather provides a basis for understanding the value of improvements in weather forecasts. Data on damage from weather events is often used as the easiest indicator of the value of research to improve forecasts. Specifically such damage data is used to help identify “identifying weather phenomena that have limited forecast skill but important societal, economic, and environmental consequences” During an extreme weather event though, the task of estimating damages is often over-shadowed by higher priority objectives, such as preventing further damages and reducing deaths and injuries. Damage estimation is often limited to hindsight, with no consistent or reliable application of methods for damage estimation. Irregularities in estimating and reporting damages result in damage estimates of the same event varying by 40% or more among different data sources, leading to inadequate and even inaccurate comparisons over longer time periods. In our work to update and revise damage data in the Extreme Weather Sourcebook (http://www.sip.ucar.edu/sourcebook), we have confronted issues concerning the depth, accuracy and consistency of storm damage data collection. This type of data has been used in many peer reviewed articles to explore changes in impacts of weather over time but there has been very little open discussion of the quality of the damage data. This paper will present several issues with weather induced damage data quality – focusing on hurricane damage data – to prompt a dialogue about reliability of scattered and inconsistent data from multiple sources. We hope this will lead to efforts to reduce the error in reported damages and to better reporting and organization of storm damage data in the future. Fundamentally we will advocate a longer term effort to standardize collection, reporting, and archiving of data on weather related damage to provide reliable information for future decision making. Doing so is critical to providing a stronger basis for integrating efforts to improve forecasts of high impact weather events.