The authors begin by reviewing several promising approaches, including Kiefer-Wolfowitz algorithms, neural networks, genetic algorithms, and reinforcement learning. In addition, it is shown that linear classification and support vector machines may be used to optimize the combination of multiple predictive modules. The objective function, or performance metric, to be optimized may be derived from comparison of the algorithm's output with a human-truthed value, an objective measurement, data from a different instrument, a simulation, or even from some measure of consistency. However, simple metrics such as mean-squared error or common skill scores are typically inadequate; instead, a suitable training set and performance metric must be constructed that incorporate the selective sampling required to properly treat rare events and take into account the operational significance of the algorithm's output (e.g., warning or failure to warn). In addition, an FL algorithm design that allows modules to be tuned independently greatly facilitates the optimization process. Practical application of these concepts is illustrated using FL radar detection and aircraft data quality control algorithms.
Supplementary URL: