ESPDS Product Distribution and Access—Making the Most of the ESPDS PDA User Portal

The Product Distribution and Access (PDA) application within the Environmental Satellite Product Distribution System (ESPDS) is a data-driven decision engine that provides the National Oceanic and Atmospheric Administration’s (NOAA’s) National Environmental Satellite, Data, and Information Service (NESDIS) with a flexible mechanism for distributing a wide range of satellite and ancillary data products from a variety of sources to a broad spectrum of consumers based on an operator-configurable rule set. PDA allows for the definition of a set of data products with a collection of associated file and metadata properties, which are then used to drive system behaviors for data acquisition, data characterization, persistence to short-term storage, matching to user-defined subscription and search criteria, and routing to consumers. The PDA User Portal provides approved data consumers with a “self-service” ability to search for and subscribe to data products received at PDA, to tailor the products according to their specific needs, and to specify the distribution mechanism to be used to obtain the data from PDA.

The data consumer-accessible Search and Subscription panels on the PDA User Portal offer a broad range of options that allow for significant flexibility when defining Search and Subscription criteria, but this flexibility comes as a cost of complexity -which facilitates an improved data delivery experience for seasoned users, but can be the cause of frustration for less experienced users. This presentation aims to cut through the complexity, providing users with information and techniques that will allow them to better leverage the available features of the PDA User Portal to obtain meteorological data necessary to their job function.

PDA provides “near real time” access to satellite meteorological and ancillary data products, routing files to consumers immediately upon receipt at PDA. PDA functions as an intermediate archive, storing data for a brief, configurable time period, which is nominally 7 days for GOES-R series data, 4 days for JPSS and S-NPP data, and 3 days for all other meteorological products. Ancillary data products are retained by version, with an Operator-specified number of versions of each ancillary data file maintained within the PDA Inventory at any given time.

Subscription definition through the PDA User Portal is accomplished by specifying a set of metadata characteristics associated with an individual product that are matched for delivery either upon file arrival at PDA or on a user-defined schedule. Search is accomplished in a fashion similar to subscription definition, but matches metadata characteristics to products already in the PDA Inventory. Search criteria can be saved via the portal, allowing users to re-run their saved searches at a later time without having to re-specify the search parameters.

Both Search and Subscription allow the user to specify a set of tailoring characteristics defining parameters to alter their selected data using a set of pre-defined data manipulation processes. Available tailoring functions include: geographic subsetting (a.k.a. “Sectorization”); projection remapping; data layer extraction; bit depth and resolution scale modification; conversion to other imagery formats; and, various forms of compression.

The PDA User Portal also allows data consumers to specify data delivery options for files matching Search and Subscription criteria, including push and pull delivery via SFTP and FTPS, and data delivery by HTTPS pull. Users may also request that an e-mail notification be sent indicating data availability.

In general, “Less is More” when specifying parameters on the PDA User Portal. The fewer data characteristics specified that are necessary to define the desired data, the better. For example, the GOES-R ABI Cloud and Moisture Imagery (CMI) products are always produced using the ABI instrument. There’s no need to additionally specify the ABI instrument as a differentiator when you’ve selected the CMI product. The GOES-R CONUS products always cover the same geographic region, covering the continental United States. There’s no need to specify a geographic bounding box to match when subscribing to these products. The fewer criteria specified, the simpler the database query match, and the faster the results are obtained.

Some criteria, particularly within the Tailoring portion of the Search and Subscription panels are applicable to only certain types of data products. For the most part, the Portal is designed to limit presented options to only those applicable to the selected product, but there are some cases where this isn’t possible, and a knowledgeable consumer can avoid them. For example, criteria related to Orbit Number, whether the data was collected during day or night, and whether the spacecraft was ascending or descending are applicable only to Products derived from polar-orbiting spacecraft. In keeping with the less-is-more philosophy, it’s better to omit these criteria (i.e., not specify them) when attempting to receive non-polar products.

In addition to the PDA User Portal GUI, ESPDS provides a machine-to-machine interface that allows for data discovery (i.e., a Search capability) and data transfer. The MMI includes a web API that allows external systems to discover data within the PDA catalog using metadata criteria as described above for Search and Subscription, and to request the transfer of data based on pre-established delivery criteria.

The ESPDS PDA User Portal offers a great deal of flexibility in obtaining meteorological and ancillary data products, but this flexibility makes it such that a more knowledgeable consumer is better able to make the most of its features. By increasing awareness of the capabilities of the interface, a better data delivery outcome can be achieved.