Improving the Science of Science Policy through Climate Services Evaluation

Evaluation of climate services draws upon tried and true evaluation methods, however it has also necessitated innovating new approaches and tweaking existing ones. The Climate Assessment for the Southwest (CLIMAS), a program funded by NOAA's Climate Program Office, is currently undertaking a 5-year evaluation project, with an end-goal of improving the CLIMAS program as well as the climate services it provides. Evaluation of climate services also produces insight into the growing science of science policy field, both in terms of policy for science (considerations of priorities for research funding) and science for policy (the use of research in decision processes). While standard economic metrics are often sought to justify research funding, these metrics fail to reflect the multiple types of impacts that result from CLIMAS research and outreach. Similarly, traditional measures of the value of scientific knowledge (e.g., peer-reviewed publications) reveal very little about how that knowledge is used to address complex problems like vulnerability to climate variability and change. The challenge for climate service programs is to develop broad indicators of societal value, plus metrics to monitor those indicators, that allow funders and policy makers to assess the investment in publicly funded science beyond traditional economic measures. This paper provides examples from the CLIMAS program's current evaluation that may help expand developing field of the science of science policy.