NOAA Observing System Integrated Analysis (NOSIA) Methodology

NOAA spends approximately ($2.7B) annually to develop, acquire, and operate operational and research-oriented earth observing systems. Due to budgetary pressures and the increasing size, cost and complexity of, and cost of earth observations, the question of which systems or combination of systems provide the greatest value has become paramount. To address this question, NOAA has developed a best-in-breed observing system portfolio analysis capability. The methodology NOAA uses to build and maintain the observing system portfolio analysis (NOSIA) capability has the following major elements:

● Value Modeling: NOAA's articulated its mission to operations relationships in terms of its mission-goals, mission service areas, key products and services, and data sources (includes direct observation, models, products and databases)--discussed in more detail in a companion presentation; ● Evaluation Scale: NOAA employed a single evaluation scale across all elements of NOSIA that provided a consistent mapping of qualitative inputs (swing weights, performance and satisfaction levels) into numerical 1-100 scale; ● Swing Weighting: NOAA used swing weighting to assess the impact of each data source on each Surveyed Product; swing weights were converted into math functions that represent relationships between nodes in the model ● NOAA used the PALMA (c) tool to build the portfolio analysis model; PALMA contains all the nodes, relationships, roll-up rules, and weights (over 20,000 connections); ○ NOAA has reached Initial Operating Capability for a single-period model ○ NOAA is developing a multi-period version that will take future capabilities and impacts into account along with budget profiles and schedule information; ● PALMA provides the capability to: ○ Perform optimization; using the model to identify observing portfolios that maximize NOAA performance under different budget scenarios; ○ Prune low-impact inputs and compress excessively high swing weights in order to level inputs amongst subject matter experts; and ○ Query the model using externally composed filters and tables in order to effectively extract results from the large number of connections and impacts--discussed in more detail in a companion briefing.

The NOSIA methodology was adopted by the White House Office of Science and Technology Policy for the 2012 Earth Observation Assessment which informed the 2014 National Plan for Civil Earth Observations. Key Words: NOAA, policy analysis, portfolio analysis, value model, earth observation, environmental observation, investment analysis, repeatable process, multi-criteria decision analysis, optimization, swing-weighting, value-model, mission-objective decomposition, impact analysis, cost-impact analysis, efficient frontier