Constructing NOAA's Value-based Tree: Representing Relationships between NOAA's Organization, Services, and Observing Systems

NOAA spends approximately ($2.7B) annually to develop, acquire, and operate operational and research-oriented earth observing systems. Due to budgetary pressures and the increasing cost and complexity of earth observations, the question of which systems or combination of systems provide the greatest value has become paramount. To address this question, NOAA has developed a best-in-breed observing system portfolio analysis capability. A Value-based Tree is a structure that graphically depicts the capability/mission/goal or objective to be achieved via a hierarchical decomposition of mission activities and the value achieved. Constructing a value-based tree in NOAA, has provided the agency a fundamental pillar for system assessment and analysis and a significant leap forward in the way analysts can view data. Providing a definitive and authoritative foundation for this effort within NOAA has been possible by consolidating and stabilizing data bases containing all of NOAA's programs' documented Priority 1 observing requirements and their associated products. Also critical has been the information obtained from NOAA's Next Generation Strategic Plan, Long-term Goals' Implementation Plans; and Line Office Strategic Plans; existing Government Performance and Results Act (GPRA) documentation and the invaluable inputs of Subject Matter Experts (SME) who as the actual practitioners and ultimate authorities in their corresponding fields provided expert testimony representing thousands of years of collective experience and knowledge. It is important to highlight that SMEs represented a cross-section of sources in the wide range of disciplines across NOAA including Weather Forecast Offices, National Science Centers, National Laboratories, Data Centers and Senior Headquarters personnel. As mandated by Dr. Kathryn Sullivan, NOAA's Administrator, The Technology, Planning and Integration for Observation Division in NESDIS (TPIO) in coordination with senior leadership, was responsible for engaging with Line Office representatives and conducting a series of interviews to address and identify the various levels of the value tree. While SME inputs have been considered subjective, expert testimony was used to assign a value or score to the direct use of Observing Systems or Data Sources and Tools to generate a product. The development of the parameter-specific value model was done using the PALMA© software tool, a capability developed by The MITRE Corporation with Federal funds. PALMA© captures the NOAA value chains and represents the linkages from top-level NOAA Long-term Goals down to key products, and from there, to supporting models and observing requirements. To score and assign value, a performance scale was created and used for both inputs and outputs, i.e., for the performance of Key Products produced, and the level of satisfaction for data sources and tools used to generate the products. The analysis team identified hierarchies based in the agency's organization and structure and quickly established the top “root” node of the tree representing the overall benefit to NOAA. Nodes on the first level below the top node (NOAA) are the four Long-term Goals stipulated in NOAA's Next Generation Strategic Plan (NGSP), Weather Ready Nation (WRN), Climate Adaptation and Mitigation (CLI), Healthy Oceans (HO) and Resilient Costs (RC). The analysis team identified candidate Mission Service Areas (MSAs) and coordinated the rationale behind each MSA with LO and Mission Goal liaisons. The next level in the tree would normally be Performance Measures (PMs). However, after the team conducted the research analysis, a final determination was made not to include PMs within the value tree, because most of the products identified in the analysis had no corresponding PM associated. Key Service Delivery Products (KPs) were identified by reviewing relevant directives and documents of the parameters in question. MSA-specific SMEs identified additional KPs and again, the analysis team worked with the SMEs and OSC LO representatives to define a subset of one or two KPs that reasonably represent the KPs for each MSA. In cases of multiple KPs, the team and the SMEs developed roll-up rules to reflect the relative importance among the KPs. Observing systems appear at the bottom level of the PALMA© value tree. To characterize these systems, the analysis team collected performance, schedule of operation, lifecycle costs and system dependencies. Observing requirements are folded into the tree via the respective observing systems that satisfy the documented requirements. Observing requirements documented in the Consolidated Observation Requirements List (CORL) were used to scope and define the set of relationships between observing and input / intermediate products for the subset of input products for which applicable numerical studies/analyses are not available. Data sources and tools (intermediate/input products needed to produce KPs were also classified into a set of categories, such as global models, regional or mesoscale models, satellite data or products, other remotely sensed data, external databases, historical and in situ data. NOAA Senior Leadership reviewed and approved these categories of intermediate/input products. The roll-up rules relating KPs to input products play a leading role in defining how the PALMA© model captures/addresses integrated observing capabilities, and the analysis team relied heavily on the expertise of SMEs at NOAA test beds, modeling and operational centers, and offices to develop these roll-up rules. The NOAA value tree is a flexible construct that can be extended to take additional value-oriented inputs into consideration such as GPRA measures, performance measures, Primary Mission Essential Functions and Activities, or societal or economic impacts. Future directions include working with civil agencies' social scientists to estimate the social and economic impacts from services flowing from National Civil Earth Observations.