Hypothesis Tests of Approximate Equality for Evaluating Forecasts as Functional Objects

Traditional statistical methods that were derived to handle low-dimensional multivariate data face mathematical and statistical barriers that limit their applicability for high-dimensional or functional data. One key mathematical difficulty is that when the dimension (or number of variables) of the data is larger than the number of observations, the estimate of the matrix of variances and covariances will not be invertible, which is a property that is needed for traditional statistical methods to test hypotheses about means. In addition, since it is often not practical to determine what probability distribution high-dimensional or functional data come from, distribution-free approaches based on large sample sizes are often used. However, when the number of variables is large, the number of observations needed for the asymptotic results to apply is often impractically large.

In this paper, we present variations on a statistical procedure for testing whether the mean of high-dimensional or functional data is approximately equal to a claimed value. By relaxing the requirement that the mean is exactly equal to the claimed value, we will demonstrate that the mathematical and statistical problems described above are solved, which allows for this procedure to be used for a wide variety of problems. Our method also provides a way to determine a measure of effect size, namely how close to equal the mean and the claimed value need to be in order to find a statistically significant difference between them. We will demonstrate the effectiveness of this methodology for evaluating forecasts that provide predictions as high-dimensional vectors, functions, or surfaces by comparing these predictions to observed data.