Verifying the Performance of the Colorado Fire Prediction System

The Colorado Fire Prediction System (CO-FPS) consists of a coupled atmospheric and wildfire model within a decision support framework to assist the Colorado Division of Fire Prevention and Control (DFPC) in their response to wildfires. The National Center for Atmospheric Research (NCAR) and the DFPC’s Center of Excellence for Advanced Technology Aerial Firefighting (CoE) have been working together to iteratively develop the system to meet end-user needs. The CAWFE® (Coupled Atmosphere-Wildland Fire Environment) model, which was integrated with the Weather Research and Forecasting (WRF) model to develop WRF-Fire, drives the CO-FPS predictions. CO-FPS adapts WRF-Fire for end-users by surrounding it with decision support modules that allow fire personnel to run simulations and use the model output to aid response to wildfires.

In order to successfully deploy a decision support system like CO-FPS, users must be confident in the system’s ability to accurately predict and support good decisions. In addition to the weather that helps drive the model, fire spread and related variables (smoke, flame length, etc.) must be at a level of accuracy that is useful for fire response. However, obtaining fire-related observations to determine the accuracy of the model presents a major challenge. Observations of this type are often collected in the midst of fire response and specifically for day to day operations – this means they may lack accuracy, timeliness, or relevance, and they are stored in disparate databases, in physical copies in filing cabinets, or not at all. It is also important to determine the level of accuracy and types of useful output required for different applications - air quality forecasters use the model differently than firefighters or emergency managers issuing evacuation notices.

Four years into this five year project, this presentation focuses on the assessment methods that have been used to determine and inform improvement of the performance of the system. These include traditional methods, such as contingency statistics typically used in precipitation assessment, along with personal interactions with those using the system. Difficulties with the availability and quality of observations and how this is being handled is also covered, along with the use of observations of opportunity including social media posts and conversations with responders.