Inside The National Massive Realtime Meteorological Data Processing System

Meteorological data is a typical non-structure data, which reaches dozens of TBs per day. Parsing,storage and access mode based on RDBMS and file system become the bottleneck of weather forecast data processing system. To fulfill the China national weather forecast platform MICAPS users' fast, in time query of realtime data, we depict a stable 7*24 distributed data parsing system, supporting dozens of TBs per day. According to the multi-dimension model and the user behavior of meteorological data, using non-relational key-value DDBMS, we design and implement a high performance massive data storage system. Practices have proven that realtime data parsing system and the massive data storage system based on non-relational key-value DDBMS are capable for storage, query and applications requirements of massive meteorological data. This system is the core system of China weather forecast data flow, shows excellent functions and performance.