A Human Over the Loop Forecast Paradigm at The Weather Company (TWC)

A Human Over the Loop Forecast Paradigm at The Weather Company (TWC)

Despite the emergence of full forecast automation for a few weather providers, there would appear to be some consensus in the weather prediction community that a fully automated end-to-end forecast system remains approximately 10-20 years away. Despite this reality, the crucial lag of time inherent in the human editing of objective guidance prior to publishing of the weather forecasts is increasingly problematic. In the case of TWC, the legacy human in the loop (HITL) man-machine process added at least 2 hours delay in the numerical weather prediction, human forecast editing, and forecast publishing assembly line [and this delay notwithstanding the versioning or provenance of the underlying guidance]. In late 2012 it became clear to TWC that a Human-Over-the-Loop (HOTL) or lag-less forecast process was desirable to bridge the “Age” of the traditional human-directed forecast process and the coming “Age” of a true end-to-end automated forecast process. As summarized in Neilley et. al. (2015), The Weather Company 24 x 7 global forecast operations shifted from a HITL process to a HOTL process in Summer of 2014. The human and machine migration is described in this extended abstract. The HOTL software tool is built atop a powerful graphics engine that allows for rich visualization of most current and forecast weather information that exists in the public cloud and within TWC's weather infrastructure. Concurrently, forecasters draw polygonal areas and assign a rich set of properties to these objects that either [a] constrain parts of the digital hourly forecast in a predefined but relative manner, [b] add value to the computer worded forecast via direct human oversight, [c] expound on local and hyper-local weather conditions of high impact or extraordinary nature that would be impossible for the machine to circumscribe or identify, and [d] alter or manage weights that control the blend of precipitation forecasts in the upstream guidance system. Forecaster responsibility [a] are termed Weather Filters, [b] are a legacy function called Qualifiers, [c] is a functionality called Blurbs, and [d] is a background weight-changing paradigm in the Digital Forecast system that is called Short-term Weight Management or SWiM. In all cases, the application of these human-directed forecast changes and forecast amplifications are done post hoc or just in time – eliminating all human-forecaster time lag from the end-to-end process. This paper will discuss the philosophy of the new forecast paradigm, and the responsibilities of the new HOTL forecaster, including the experiences of transition from the HITL to the HOTL environment. An overview of the HOTL tool and results from the first year of HOTL usage will be detailed, and we will document highlights of a survey of TWC forecasters on HOTL usage and experience.