1222 Are the Discrimination Ability and Maximum Economic Value of a Forecast System Affected by Forecast Biases?

Wednesday, 25 January 2017
4E (Washington State Convention Center )
Hui-Ling Chang, Central Weather Bureau, Taipei, Taiwan; and S. C. Yang

Handout (715.2 kB)

Most forecast systems possess systematic biases because of the crude representation of model physics and dynamics to the real atmosphere. Focusing on short-range probabilistic quantitative precipitation forecasts (PQPFs) for typhoons, this study explores the effect of calibration on the discrimination ability and maximum economic value (EVmax) based on the Local Analysis and Prediction System (LAPS) ensemble prediction system (EPS) operated at Central Weather Bureau in Taiwan.

Results show that the discrimination ability, which can be assessed by relative operating characteristic (ROC), and EVmax of an EPS are insensitive to forecast biases. This implies that improving reliability via calibration cannot increase the discrimination and EVmax of a forecast system. However, the optimal probability threshold (Pt) for users to take preventive action and obtain their EVmax is different when adopting calibrated or uncalibrated forecasts. In other words, biased forecasts will not prevent users from obtaining their EVmax if an appropriate Pt is adopted. When uncalibrated forecasts are adopted, the optimal Pt should be determined based on the past long-term statistics of EV.

Experiments have been conducted to verify that ROC is insensitive to the linear property of a calibration method or even the accuracy of the calibration results. That is, the discrimination ability almost remains the same after a linear or nonlinear calibration, even though forecast biases cannot be properly corrected during the calibration process. This is because calibration only corrects the precipitation amount instead of modifying the rainfall pattern, which is controlled by the model physics/dynamics processes and is associated more with the discrimination of the forecasts.

- Indicates paper has been withdrawn from meeting
- Indicates an Award Winner