NOAA, through the NOAA Observing Systems Committee (OSC), is working to develop a repeatable process to assess the most cost-effective observing system architecture to best meet NOAA's requirements for Upper Air observations. This analysis process is a value-based process that uses focused data collection efforts using mission specific Subject Matter Experts (SMEs) to populate a decision-assistance model tool (PALMA). In March 2011 a workshop was conducted with key scientists, operational practitioners and technical managers to identify relevant Mission Service Areas (MSAs). Eleven MSAs supporting Weather and 3 supporting Climate Adaptation and Mitigation were selected to best characterize the end-to-end product chain for upper air observations within NOAA. Within each MSA, Performance Measures (PM) and Key Service Delivery Products (KP) were collected to define a mission-oriented ‘value chain' describing the impact of upper air systems on NOAA's most important end-user products. A total of 144 PM/KP combinations were collected and then refined to 15-30 PM/KPs. Identification of appropriate performance measures is critical to the success of this study and is the first major effort in completing this analysis. Some objectives falling under NOAA Goals are not directly attached to an easily quantified PM, and the requirements for profile observations for these Goals will be added to this process through Subject Matter Expert (SME) input. These refinements will be completed through a series of SME meetings designed to collect and weigh the value of each element to NOAA's end products, and to the achievement of NOAA Goals. The observing requirements from the Consolidated Observation Requirement List (CORL) (numbering 89) and the upper air systems (89) from the NOAA Observing System Architecture—(NOSA) that address the three study parameters will be refined to a more manageable number through SME discussions. A second workshop was conducted in April 2011 to review the spectrum of observing system portfolio analysis capabilities within and outside of NOAA. These capabilities ranged from pure numerical model assessment methods (Observing System Experiments-OSE and Observing Systems Simulation Experiments-OSSE) to product verification analyses. A Portfolio Decision Support method called the Portfolio Analysis Machine, or PALMA, was reviewed during the workshop. The Technology Planning and Integration for Observations Office (TPIO) recommended and the NOSIA team selected PALMA as the portfolio assessment tool for this pilot study. Unlike other analysis capabilities considered, PALMA permits the input of both quantitative model outputs and the qualitative assessments from SMEs to represent relative value roll-up rules which provides a balanced and robust capability. For those systems or products without numerical assessments from an OSE, OSSE, etc., PALMA scoring/weighting will be based on SME assessments and data previously collected for observation requirements in the CORL. These CORL data and related observing system definitions (NOSA) produce a Requirement Gap Assessment score for each system and requirement set. Collecting the needed data and refining the PALMA model requires extensive interaction and cooperation with SMEs from NOAA's critical MSAs. A series of targeted SME sessions at key NOAA facilities has been completed to provide the focused time needed to understand the product-oriented detail to populate the PALMA model. The study team visited the National Hurricane Center/Joint Hurricane Test bed, the Aviation Weather Center, the Storm Prediction Center/Hazardous Weather Test-bed, the Hydrological Weather Test-bed/NCAR and the Climate Test-bed. Additional meetings with National Center for Environmental Prediction/Environmental Modeling Center are still needed to focus on assimilating information from numerical models used in product preparation. Efforts to ensure atmospheric research, remote sensing calibration/validation, and climate monitoring, assessment, and modeling are being developed to ensure accurate information is collected to populate the PALMA model. The PALMA model will be populated and initial output will be validated for consistency. The SMEs will be consulted during PALMA model iterations to optimize the systems portfolio evaluated. Potential solutions will be validated, run through sensitivity analysis, analyzed for positive/negative external effects, and reviewed for unintended consequences. SMEs will be consulted throughout these reviews. This process will bound its effort to analyze the needs and satisfaction of upper air temperature, moisture and winds. It is fully recognized that all NOAA missions require additional information to successfully complete their missions. The need for surface temperature, moisture and winds as well as many additional parameters is critically important. While this study will not examine these additional parameters the need for them is not degraded.