Elements of a Scalable Infrastructure for Weather Forecaster Access to Joint Polar Satellite System (JPSS) Data

Each satellite in the Joint Polar Satellite System (JPSS) produces global observations and retrievals amounting to 2 Terabytes per day. Access to these data products by operational weather forecasters presents several challenges related to the size, structure, and complexity of the data; variable bandwidths and data rates; and near-real-time user needs.

The U.S. National Oceanic and Atmospheric Administration (NOAA)’s Product Distribution and Access (PDA) facility disseminates data products from JPSS to known subscribers, and fulfills requests from users of the Advanced Weather Interactive Processing System (AWIPS) in particular. Thanks to several mitigation measures (including the production of thinned, reformatted, and compressed products; scheduling / prioritization of data file transfers and requests; and spatial selection for smaller regions), a few basic service capabilities are adequate to provide forecasters with near-real-time access to the limited JPSS data they currently use.

But over the coming years, growing data volumes and data usage will likely require a more scalable data infrastructure, with additional capabilities to reduce unnecessary data movement. These may include tiered services for on-demand data subsetting and map rendering, with variable spatial resolutions and tiling; replication / predistribution of high-demand products; sending or fetching data only when triggered by events of interest (fixed or user-specified); or server-side execution of user-supplied queries or algorithms. To ensure a scalable storage and computing environment, this infrastructure may also rely on virtual products, computed on demand and cached, in place of some pre-computed products. This infrastructure may also take advantage of multiple communications channels (e.g., OneNWSnet and the Satellite Broadcast Network) and alternate data sources (e.g., Direct Readout). Finally, industry-standard protocols such the OGC Web services suite would let this infrastructure employ off-the-shelf (commercial or open-source) software, and thus more easily adapt to evolving technologies and computing environments, and serve dynamic user needs.