Crowdsourcing Computations in the Cloud for Disaster Risk Management

Managing disaster risks related to storms, floods, drought, wildfires, and other natural hazards relies increasingly on interpreting earth observations and running numerical models. Peak computing demand during event detection, hazard assessment, or incident response may exceed agency computing budgets; however some of it may be met by distributing subtasks among hundreds or thousands of participating computers operated by members of the public. This “volunteered computing” approach has enabled large projects in mathematics, basic science, and climate research to harness the slack computing capacity of thousands of desktop computers. This capacity is likely to diminish in the coming years as desktops give way to battery-powered mobile devices (laptops, smartphones, tablets) in the consumer market; but as cloud computing becomes commonplace, it may restore most (or more) of this slack capacity -- if cloud users are offered an easy, trustworthy mechanism for participating. Such a "volunteered cloud computing" capability would not only crowdsource computations among cloud users; it also promises several unique advantages over its desktop-based counterpart: tasks distributed within a single cloud can rely on large, fast data transfers; granular billing mechanisms allow small slices of “interstitial” computing at zero marginal cost; and virtual storage volumes allow in-depth, fully reversible machine reconfigurations.

Volunteered cloud computing is especially suitable for tasks that are “embarrassingly parallel” (i.e., that can be broken into independent subtasks requiring no intercommunication) -- including tasks that require or produce large data volumes. Examples in disaster risk management include near-real-time satellite image processing and interpretation; pattern and trend detection across large observation series; and large model ensembles to account for many combinations and perturbations of key inputs. In the context of a major disaster, we estimate that cloud users might volunteer several thousand processor core-hours across a large provider such as Amazon Web Services. To explore this potential, we are building a volunteered cloud computing platform and targeting it to a disaster risk management context. Using a lightweight, fault-tolerant network protocol, this platform helps cloud users join parallel computing projects; provides accountability for donated computing; automates reconfiguration of users' virtual machines; and optimizes the use of “interstitial” computing. Initial applications include near-real-time fire detection from multispectral satellite imagery and flood risk mapping based on hydrological simulation trials.