3A.3 Improving Recruiting for Impactful RCTs: Outcomes from Two Studies in Climate Frontline Communities

Monday, 29 January 2024: 2:15 PM
Holiday 4 (Hilton Baltimore Inner Harbor)
Natalie Herbert, Stanford Univ., Stanford, CA; and C. Cannedy, T. Harrison, J. Jorns, M. C. Lemos, and G. Wong-Parodi

Frontline communities are already experiencing the worst and most costly effects of extreme weather including hurricanes, inland flooding, heat, and wildfire smoke, and there is urgent need to implement interventions that decrease exposure and increase resilience. Randomized controlled trials (RCTs) and field experiments can test interventions in context, but what can often be unclear in the scientific literature is how large-scale studies design, recruit, and retain samples when using non-professional panels (e.g., AmeriSpeak, YouGov). What are the resources required to recruit samples experiencing the first and worst of climate change? How can community members drive the success of recruitment and engagement in science on the extreme weather they face?

We describe study recruitment and lessons learned for two case studies of frontline communities engaged in RCTs regarding extreme weather. Study 1 (recruitment December 2020 – July 2021) enrolled N=219 frontline decision makers who manage stormwater and local planning for 60 small- and medium-sized communities along the U.S. Gulf Coast. Study 2 (recruitment June 2022 – August 2023) enrolled N=303 frontline community members from low-income, mostly Spanish speaking households in four communities in the San Francisco Bay Area, California. Participants in each study trialed interventions to support decision making – for Study 1, local adaptation planning for storm surge and inland flooding, and for Study 2, individual protective actions for wildfire smoke and extreme heat.

We describe the resources required to meet research goals and support ethical engagement with frontline communities. In particular, we emphasize the human resources – multidisciplinary teams, relationships with communities, and dynamic team building – needed to engage frontline decision makers and community members. Results include comparisons of when multimodal remote recruitment (Study 1) works and when in-person recruitment (both studies) is needed, noting ethical concerns around community health and capacity for each. We also describe approaches to partnerships with boundary organizations (both) and community members via promotoras de salud models (Study 2). Additionally, we describe two modes of developing and iterating on protocols that improve our ability to recruit hard-to-reach populations and maintain relationships with communities: phased recruitment (Study 1) and piloting (Study 2).

Finally, we detail ways of measuring “recruitment failure.” Across both studies, research teams collected empirical data during recruitment, including outcomes from a database of 700+ phone calls (Study 1) and questionnaires from eligible decision makers (Study 1) and community members (Study 2) who did not enroll. Using this data along with geographic, Census, and historic hazard data, we demonstrate who may be “missed” in recruitment, including places where people may have both lower capacity and higher barriers to participate (e.g., recent hazard experience, immigration status). We demonstrate how adding research team capacity to efforts measuring “failure” can improve long-term project success. In particular, we argue this matters for applying study results to policy decisions that matter most for frontline communities.

Identifying effective interventions for supporting adaptation in frontline communities requires implementing studies in context now. However, few social science researchers can reach these communities, namely because they are often not members of them. Research with professional samples is often the status quo due to the reliability of using a set budget to achieve sufficient statistical power. Yet this greatly limits what we can learn about what interventions work where and for whom. Increasing the transparency and replicability of study designs that recruit samples in frontline communities can decrease the hidden costs of this work, providing blueprints for other researchers to follow.

Increasingly, agencies and funders are seeking to make impact in work focused on environmental justice. More attention is needed on the care and resources needed to make this work successful for research outcomes and, more importantly, for the people for whom interventions are intended to generate benefits.

- Indicates paper has been withdrawn from meeting
- Indicates an Award Winner