The Met Office has a track record in using data driven, statistical techniques combined with physical models and understanding. For example it has developed operational and planning tools for the wind energy sector combining data from our observations network, local observations from site managers, complex model simulations and reanalysis using statistical techniques in an 'intelligence function' to provide accurate wind assessments for new sites, even in regions with complex terrain. It has also used emulators in climate prediction. These emulate the behaviour of extensive complex models using statistical techniques. Multiple emulations were combined with a limited number of more complex model realisations to provide global information to UK government to support their climate policy development. More detailed UK projections were also published f in 2009 or everyone to access. They were used in the last 2 UK climate change risk assessments. Upgraded projections will be published in 2018.
The weather services sector needs to understand the strengths and weaknesses of different data driven different approaches and machine learning and whether or not they should link to physical models and understanding. This is becoming increasingly important as a wider range of suppliers are providing weather services, often with very little understanding of whether science, based on good data analytics capability. Building on its existing experience the Met office is well placed to test the value of data driven approaches for a wide range of applications.
This paper will set out the main drivers for exploring machine learning and some of the challenges that lie ahead in the context of what has already been achieved. In particular, it will ask the question about what the NMS role should be in providing quality standards for data driven approaches, or at the very least exemplars, in addition to their core activities. A related paper by Rachel Prudden will talk about some of the specific techniques being employed in weather forecasting and early results.