Extreme Learning Machines for Environmental Problems

Artificial neural networks (ANN), support vector machines and random forests (RF) have been successfully used in various aspects of machine learning problems, such as classification, regression, and ranking. However the learning speed of these methods can be far slower than linear techniques, especially when dealing with large datasets.A major challenge is to develop nonlinear machine learning methods which run much faster than the current generation.

Recently the extreme learning machine (ELM) for the single hidden layer feedforward ANN architecture has appeared. The ELM randomly chooses hidden nodes and analytically determines the output weights of the ANN, producing good generalization performance while running several orders of magnitude faster than conventional ANNs.

In this work, we applied the ELM to three atmospheric forecast problems -- surface air temperature (TEMP), precipitation (PRECIP) and sulphur dioxide (SO2) concentration. The first two problems each had 106 predictors and the third problem, 27 predictors. We used multiple linear regression (MLR) as benchmark and a variety of machine learning techniques, including bootstrap-aggregated ensemble ANN, support vector regression with evolutionary strategy (SVR-ES), random forest (RF) and ELM. It is also possible to make the modeling process less time consuming and sometimes more accurate by removing predictors that are irrelevant or redundant with respect to the task to be learned. Therefore, we also tested all methods with stepwise linear regression (SLR) applied first to screen out irrelevant predictors.

In terms of forecast accuracy, ELM outperforms the linear method in all tested datasets and is comparable to the other nonlinear methods. However, ELM is indeed extremely fast compared to the other nonlinear methods, often faster by several orders of magnitude.