Rescuing data from a black hole: Consolidation of tropical cyclone data sets into an information repository for improved data access

With increasing data observations comes the challenge of managing larger quantities of data -- and how to effectively utilize that data. Aggregating existing discrete tropical cyclone text-based data sets into a single relational database repository provides a more refined access to specific data than previously possible. There are many potential benefits for the research and operations communities: large quantities of data are easily handled; flexible and exhaustive comparisons between data elements are possible; derived data can be computed and added to the repository; data elements that are normalized into a relational model are well-defined, and benefit from a consistent, logical representation of information; obvious data errors are caught through quality control checks inherent in metadata definitions.

Possible data sets for inclusion into a first generation repository include ATCF best track, objective aids, fix and SHIPS data, recon high-density, vortex, and dropsonde observations, HURDAT, HRD aircraft "pass data" (Rahn), RSMC best track data, modified best track (DeMaria), and synthesized best track (Kossin). A preliminary proof of concept on forty years of JTWC and NHC best track data was encouraging. Queries included calculating intensity changes between best track points, identifying the range of Vmax sorted by MSLP, and identifying periods of time in the historical record with no tropical cyclone activity. The repository will be significantly expanded by the inclusion of additional data sets, and then turned loose on willing members of the tropical cyclone community. Results will be presented at the conference.