Evaluation of the BlueSkyRAINS Smoke Prediction System

BlueSky is a framework linking together weather prediction systems with the latest available fire science and air quality models to predict smoke concentrations from prescribed fire, wildland fire, and agricultural fire. The EPA Region 10 developed the sophisticated Rapid Access Information System (RAINS) user interface. Partnering of these two systems led to the BlueSkyRAINS (BSR) decision support system (DSS). Evaluation of the BlueSkyRAINS smoke prediction system is a complex but critical task which is being supported by the Joint Fire Sciences Program (JFSP). Many problems and issues must be taken into account: paucity of observational data, background PM2.5 concentrations from other anthropogenic and biogenic sources, secondary aerosol formation not taken into account by the Lagrangian model, uncertainty in plume rise calculations and dispersion methods, uncertainty in the predicted weather, and uncertainty in the emission estimation and fuel loadings. Using traditional air quality statistical comparison methods such as bias and mean error only compare a single point in time and space and are not a fair representation of model success or failure. Therefore, different Measure of Success (MOS) methods are investigated as well as different statistical evaluation measures using the spring 2005 prescribed burn season and summer 2005 wildfire season.