Online Learning Using Extreme Learning Machines with Self-Adapting Complexity

While nonlinear machine learning methods have been widely used in environmental forecasting, in situations where new data arrive continually, the need to make frequent model updates can become cumbersome and computationally costly. To alleviate this problem,  the online sequential extreme learning machine (OSELM), an online learning algorithm for the one-hidden-layer feedforward neural network model (with random weights in the hidden layer), is automatically updated inexpensively as new data arrive. OSELM was applied to forecast daily streamflow at two small watersheds in British Columbia, Canada, at lead times of 1-3 days. Predictors used were weather forecast data generated by the NOAA Global Ensemble Forecasting System (GEFS), and local hydro-meteorological observations. Using the online sequential multiple linear regression (OSMLR) as benchmark, we found that the nonlinear OSELM easily outperformed OSMLR in forecast accuracy.

OSELM does have a major limitation, namely the number of hidden nodes (HN), which controls the model complexity, cannot be changed from the initial model as online learning proceeds. Usually, as more data become available, the longer time scale behavior can be learned by using more HN in the model. A new variable complexity online sequential extreme learning machine (VC-OSELM) is proposed, which automatically adds or removes HN as the online learning proceeds, so the model complexity self-adapts to the growing available data. The performance of VC-OSELM was compared with OSELM in daily streamflow predictions at a lead time of one day, where VC-OSELM outperformed OSELM when the number of initial HN turned out to be smaller or larger than optimal.