The Significant Role of Verification in Achieving More Automated Routine Forecast Production in Australia

The Australian Bureau of Meteorology has been aiming to change the balance of work for operational meteorologists so that they spend less time on routine gridded and text forecast production. This gives them more time for high value interactions with the community, enabling better weather-sensitive decisions, and gives them greater opportunities to participate in improvement of the forecast production tools.

The first forays into forecaster-supervised use of an automated forecast for 5 to 7 days lead time, took place a decade ago in the New South Wales State Office. Although widening this approach to other offices has been proposed periodically since then, it is only since 2019 that this approach has been adopted for all 7 of the Bureau's Australian forecast offices and extended to cover 3 to 7 days lead time. Verification has played a key role in achieving this change.

Since 2015, we have developed the tools and gathered the data to enable comparative verification of the quality of the issued forecasts and of automated alternatives. Elements verified have progressively expanded to cover precipitation, maximum and minimum temperature, wind speed and direction, dewpoint temperature and fire danger indices. The verification measures adopted have been guided by discussions with forecasters and service representatives regarding which aspects of forecast quality are considered important for our services. We have included verification metrics such as Relative Economic Value that provide a simple consideration of societal benefit. The verification results have shown in many cases that the automated forecasts have comparable quality to the issued forecasts. Where deficiencies have been exposed in the automated forecasts, work has been undertaken to improve the automation.

Evidence from verification has given the organization the confidence to move to wider use of automated forecasts in routine forecast production. National standard operating procedures have been formulated. These have used verification and insights from forecaster experience to determine in what circumstances the automated forecast will be used and in what situations it is valuable for the forecaster to intervene. Processes have been put in place to coordinate any interventions between forecast offices and to gather forecaster reasoning and relevant verification when non-standard interventions are performed. Systematic ‘operations to research’ feedback from operational meteorologists on issues with the automated forecast, as well as continued monitoring of verification, will be important to enable further optimization of forecast production.