A categorical forecast is a prediction of an event yes or no without any accompanied uncertainty or a probability that an event will occur.
A value forecast is a prediction of a single number from a continuous set of possible numbers e.g. 38 degrees F or wind speed of 22 knots.
Based on the verifying observations a 2 by 2 contingency table is found. For any 2 values or any score that uses 2 values there is a unique ρ which can be used to calculate the other values in the 2 by 2 table and thus one score can be converted to another score. From these four numbers a vast number of scoring rules have been developed such as probability of detection, threat score etc.
FPD is a frequency of each probability class in a probability forecast. An FPD is uniquely determined by ρ. A Brier score (1960) is a measure of the accuracy of a probability forecast. The Brier Score algebraically can be divided into two terms: The first is sharpness which depends only on FPD and thus uniquely on ρ. The second Murphy termed weighted reliability which is a weighted mean reliability of each class in the FPD. The sharpness term in general outweighs greatly the weighted reliability term. A critical probability based on cost of taking preventive action and preventable loss. A probability above the critical probability converts a probability forecast into an event forecast. Working the math backwards allows an event score to estimate an FPD and thus a probability score. Cost and loss in general are very difficult to obtain from a user but in a military context are almost impossible. An ultimate approach is to plot the FPD and vary the critical probability so that the number of correct forecasts and false alarms is shown for a long period. When the user is satisfied with that number of correct forecasts and false alarms then that is the critical probability that you should use.
From the critical probability the frequency of the associated event forecast is known. Conversely giving the frequency of an event forecast part of the FPD is known. Thus it can be shown that the critical probability allows conversion of an event score to a probability score and vice versa. END is the value that has the same cumulative probability in the normal curve as the climatological probability of a weather variable END. Given a suitable climate period of record there is a 1 to 1 correspondence between END and the weather variable.
Ornstein-Uhlenbeck process (OU) is a first order auto regression process in the END. The OU has a redder spectrum then a Markov process of the weather variables. Boehm (1973) showed that the END can provide an accurate and reliable probability forecast. Thus the OU can provide a set of observations and accompanying probability forecasts which can be used to check and tune a verification system. Most important it can provide the sampling distribution of a verification score thus determining the power of a particular score. For example; the number of forecasts needed to determine that the forecast varies in a significant way statistically different from no skill. This is very important in setting up a proper experimental design.
Bivariate normal distribution is defined by the mean and variants of each individual component and what can be termed as the eccentricity parameter ρ - (this term comes from the fact that the bivariate normal has an isopleth of equal density that are ellipses. These ellipses are long and narrow when ρ is near 1. When ρ is near 0 they are circular. Pearson before 1900 found that the product moment correlation is an unbiased estimate of ρ. But not when one variable is dichotomous, in that case he developed a biserial formula to calculate an unbiased estimate of ρ. For two dichotomous variables one must use a tetrachoric correlation to get an unbiased estimate of ρ.
In summary ρ can be used to convert one event score into another and critical probability can be used to convert an event score into a probability score.
- Indicates paper has been withdrawn from meeting
- Indicates an Award Winner