Tuesday, 13 January 2009: 3:45 PM
Probabilistic hedging of the True Skill Statistic
Room 125A (Phoenix Convention Center)
The AI competition this year is based around the deterministic classification of polarimetric radar data points into one of three classes. The winner is to be judged as the one yielding the highest True Skill Statistic. We are using a technique which produces probabilistic classification, which serves better to honestly represent our state of knowledge. These probabilities can be turned into a deterministic classification by a standard approach such as choosing the most likely class. However, given reliable (if this is achievable) probabilistic classifications for a test set of data, it is a relatively straight-forward task to calculate how to assign deterministic classes in order to (say) maximise the expected value of some multi-category deterministic score. We apply this approach to the data for the AI competition. The True Skill Score seems particularly sensitive to this hedging approach, at least for this problem and for the technique we are using.
Supplementary URL: