How Elastic Cloud Computing Enables Efficient, Very High Resolution, Real-Time Radar Processing

Cloud computing offers many advantages over and represents a natural progression from the use of data centers. Weather Decision Technologies (WDT) has utilized data centers for many years to securely house all compute resources with high uptime availability. A portion of those resources have been dedicated over the past 12 years to real-time production of 3D radar mosaics and derived 2D products both domestically and internationally. Since 2013, WDT has been exploring how best to transition compute resources to the new cloud environment including, for the past year-and-a-half, radar processing over the CONUS domain.

Part of the advantage of using cloud computing is the elastic nature of its compute instance types. CPU and memory utilization for radar data processing is strongly correlated to the spatial coverage of precipitation systems.  Moreover, the type and scale of compute resources required depend greatly on the resolution and geographic extent of the domain used in construction of radar mosaics. Traditionally compute resources had to be dedicated for worst case scenarios (i.e. when a majority of radars operate in rapidly updating (AVSET, SAILS) severe weather modes (VCPs 11, 12, 212, etc.)) Thus, in more quiescent weather conditions, a vast majority of computing power would remain idle. Elastic computing offers the opportunity to use resources only when needed creating a much superior, efficient system.

To this end, WDT has developed new radar processing infrastructure to take advantage of the strengths of cloud computing.  WDT licenses the Warning Decision Support System – Integrated Information (WDSSII) from the National Severe Storms Laboratory to meet all operational radar processing needs.  In order to make efficient use of the cloud platform, a new head-node concept has been developed on top of WDSSII that automatically manages all resources in the construction of 3D mosaics and derived products. A description of this new system will be given including how other cloud computing resources (i.e. Amazon Web Services DynamoDB databases, Simple Notification Service (SNS), and Lambda (executing code without provisioning or managing servers)) help to create a total self-monitoring, self-regulating system. Furthermore, efficient use of compute resources has created the opportunity to drive the grid-spacing of products to very-high extents both spatially (250 meters) and temporally (2.5 minute updates). Lessons learned throughout this process including how to remain efficient, yet economical at the same time will be given in addition to how this new radar processing infrastructure is meeting demands of clients today.