Verification of High-resolution WRF-ARW Forecasts for Vermont Utility Applications

Verification is a critical component of any modeling system. A variety of skill scores may be generated, depending on the application of interest. In the case of Vermont, there are renewable (solar, wind) and demand considerations, which can lead to specific, tailored metrics. The Vermont Electric Power Company (VELCO) has partnered with IBM Research to address these applications, which are all driven by the underlying weather predictions. For the meteorological modeling, IBM Research has deployed Deep Thunder, a state-of-the-art high spatial- and temporal-resolution weather forecasting system, customizable to meet the needs of specific weather-sensitive business decisions. It is being run operationally twice daily (initialized at 00 and 12 UTC) at 1-km horizontal resolution (297 x 300 mesh) for 48 hours. The configuration is based, in part, on the ARW core of the Weather Research and Forecasting (WRF) model, version 3.5.1. There are two additional nests at 3-km and 9-km, the latter of which is driven by North American Model (NAM) boundary conditions. The initial conditions are based on the 13-km Rapid Refresh (RAP) and various other input datasets (NASA, USGS, etc.). Three-dimensional variational data assimilation is being performed around each analysis time using MADIS and EarthNetworks WeatherBug observations. Our goal is to couple the weather model to several “impact” models (solar, wind, demand), in order to create a reliable, integrated framework, with a focus on day-ahead lead time. The 48-hour integration allows for additional buffer, given the latency of some of the input data. The current 1-km configuration has been operational since 20 April 2015. Prior to that, a three-nested 2-km configuration was being run, initialized at 06 and 18 UTC.

The verification of the weather model is done using a combination of proprietary and open source code, the latter being the NCAR Developmental Testbed Center (DTC) Model Evaluation Tools (MET), version 4.1. Given the focus on near surface conditions for renewable/demand applications, we validate against all available ground stations, which include several hundred mesonet and metar locations within the 1-km domain, covering all of Vermont and New Hampshire, along with parts of New York, Maine, and Massachusetts. Variables of interest include 10-m wind speed and direction (W), 2-m temperature (T), 2-m dew point (Td), and surface accumulated precipitation (APCP). Scores are generated based on the continuous (T, Td, W) or categorical (APCP) nature of the fields and include: mean absolute error (MAE), root mean square error (RMSE), mean error (ME), critical success index (CSI), Heidke skill score (HSS), odds ratio (OR), accuracy (ACC), probability of detection (POD), etc. This variety of metrics is chosen to create a more comprehensive, robust approach to validation.

Given the under-reporting of traditional weather stations during heavy rainfall events, among other limitations, the liquid water precipitation forecasts are validated against both the National Centers for Environmental Prediction (NCEP) gauge-corrected Stage IV and the Multi-Radar/Multi-Sensor (MRMS) gridded data, at 4.7-km and 1-km resolution, respectively. Thresholds and accumulation intervals for the contingency table analysis can then be defined according to specific applications.

To complement the quantitative verification measures, we also consider more qualitative assessment. This can include comparisons to county flood reports and visible satellite and radar imagery. Feedback is also obtained from the end users.

We will discuss the ongoing work, data, challenges, and preliminary results from the quantitative and qualitative weather model verification.