Streamlining On-Demand Access to Joint Polar Satellite System (JPSS) Data Products for Weather Forecasting

Observations from the Joint Polar Satellite System (JPSS) support National Weather Service (NWS) forecasters, whose Advanced Weather Interactive Processing System (AWIPS) Data Delivery (DD) will access JPSS data products on demand from the National Environmental Satellite, Data, and Information Service (NESDIS) Product Distribution and Access (PDA) service. Based on the Open Geospatial Consortium (OGC) Web Coverage Service, this on-demand service promises broad interoperability and frugal use of data networks by serving only the data that a user needs. But the volume, velocity, and variety of JPSS data products impose several challenges to such a service. It must be efficient to handle large volumes of complex, frequently updated data, and to fulfill many concurrent requests. It must offer flexible data handling and delivery, to work with a diverse and changing collection of data, and to tailor its outputs into products that users need, with minimal coordination between provider and user communities. It must support 24x7 operation, with no pauses in incoming data or user demand; and it must scale to rapid changes in data volume, variety, and demand as new satellites launch, more products come online, and users rely increasingly on the service.

We are addressing these challenges in order to build an efficient and effective on-demand JPSS data service. For example, on-demand subsetting by many users at once may overload a server’s processing capacity or its disk bandwidth – unless alleviated by spatial indexing, geolocation transforms, or pre-tiling and caching. Filtering by variable (/ band / layer) may also alleviate network loads, and provide fine-grained variable selection; to that end we are investigating how best to provide random access into the variety of spatiotemporal JPSS data products. Finally, producing tailored products (derivatives, aggregations) can boost flexibility for end users; but some tailoring operations may impose significant server loads. Operating this service in a cloud computing environment allows cost-effective scaling during the development and early deployment phases – and perhaps beyond. We will discuss how NESDIS and NWS are assessing and addressing these challenges to provide timely and effective access to JPSS data products for weather forecasters throughout the country.