Monday, 13 January 2020: 11:45 AM
260 (Boston Convention and Exhibition Center)
A common research problem in numerical weather prediction (NWP) is estimating net skill loss/gain from updating an operational model to a new version. In the information-theory framework, gain of information reaches zero when the end-user’s event expectation is unchanged upon receipt of a given message. Herein, we employ analogues between chaos and information theories, combined with elements from the traditional evaluation techniques (first developed in psychophysics), to yield two verification scores that can be used across many scales. These sibling scores both stem from interpreting atmospheric verification as a Shannon-model communication channel, with NWP as a decoder.
The authors submit that verification of numerical weather prediction (NWP) should be (1) mathematically sound, (2) appropriate for the scientific question, and (3) intuitive. We seek an optimal balance of these criteria, whilst extending the traditional paradigm of Lorenzian predictability (and analogous verification) to a more general notion of multivariable (or derived) object and event characteristics.
To address (1) and (2), we introduce two verification methods. The first (Object-specific Information Gain; OSIG) identifies objects in reflectivity, and employs object-matching and the diagnosis of object characteristics. The second (Fractional Information Gain; FIG) builds on the logic of the Fractions Skill Score. Both schemes use a continuous form of ignorance to align with the information-theory paradigm. Both scores are therefore (a) tolerant of small phase errors in location and timing; (b) mathematically proper and fully probabilistic, and (c) in units of bits: the fundamental measure of information, and one which better rewards rare and/or extreme events. Both schemes can also be decomposed into Brier-type reliability (REL), resolution (RES), and uncertainty (UNC). Further, the choice between the pair of methods allows verification when no objects are identified, or vice versa, estimates of object-based hazards that more closely match the mental filtering of forecasters. We conclude by attempting to buttress point (3) by presenting a comparative evaluation of Warn-on-Forecast System (WoFS) 3-km forecasts, for numerous hazards, and at a range of spatial scales.
The authors submit that verification of numerical weather prediction (NWP) should be (1) mathematically sound, (2) appropriate for the scientific question, and (3) intuitive. We seek an optimal balance of these criteria, whilst extending the traditional paradigm of Lorenzian predictability (and analogous verification) to a more general notion of multivariable (or derived) object and event characteristics.
To address (1) and (2), we introduce two verification methods. The first (Object-specific Information Gain; OSIG) identifies objects in reflectivity, and employs object-matching and the diagnosis of object characteristics. The second (Fractional Information Gain; FIG) builds on the logic of the Fractions Skill Score. Both schemes use a continuous form of ignorance to align with the information-theory paradigm. Both scores are therefore (a) tolerant of small phase errors in location and timing; (b) mathematically proper and fully probabilistic, and (c) in units of bits: the fundamental measure of information, and one which better rewards rare and/or extreme events. Both schemes can also be decomposed into Brier-type reliability (REL), resolution (RES), and uncertainty (UNC). Further, the choice between the pair of methods allows verification when no objects are identified, or vice versa, estimates of object-based hazards that more closely match the mental filtering of forecasters. We conclude by attempting to buttress point (3) by presenting a comparative evaluation of Warn-on-Forecast System (WoFS) 3-km forecasts, for numerous hazards, and at a range of spatial scales.
- Indicates paper has been withdrawn from meeting
- Indicates an Award Winner