Evaluation of Regional Climate Services: Learning from Seasonal-Scale Examples across the Americas

A growing body of work is focused on the evaluation of climate services, including at the regional scale. This literature has helped to characterize the relative success of individual services; it has also helped weigh the pros and cons of particular evaluation methods in context. It has done less to reveal the extent to which current climate service providers have engaged in evaluation, however. This study addresses this gap by reviewing the evaluation practices associated with 19 climate services, focused on seasonal outlooks and produced regionally by climate services providers within the Americas. All but one of the climate services included in the study have undergone some form of evaluation, though the evaluation practices vary significantly from case to case. The variation in evaluation practices is traced to the different purposes that climate service providers identify for engaging in evaluation, and to the challenges they experience in achieving their evaluation goals. The results of this analysis suggest that providers of regional climate services should consider a three-phase approach to evaluation, beginning with (1) rigorous efforts to understand who accesses their information, moving to (2) a nuanced understanding of how that information is used, and culminating with (3) a characterization of the value of the information in context. This study also identifies the role that long-term partnerships might play in helping climate service providers access the capacity needed to advance evaluation with respect to these goals, and concludes by discussing the implications of this work for those organizations seeking to build climate service capacity at regional scales.