Assessing Advances in Streamflow Forecasting Methods using Hindcasts and Probabilistic Verification (Invited)

The hydrologic community is moving towards the use of probabilistic estimates of streamflow, primarily through the implementation of Ensemble Streamflow Prediction (ESP) systems, ensemble data assimilation methods, or multi-modeling platforms. However, evaluation of probabilistic outputs has not necessarily kept pace with ensemble generation. Much of the modeling community is still performing model evaluation using standard deterministic measures, such as error, correlation, or bias, typically applied to the ensemble mean or median. Probabilistic forecast verification methods have been well developed, particularly in the atmospheric sciences, yet few have been adopted for evaluating uncertainty estimates in hydrologic model simulations. Likewise, advances in hydrologic modeling are seldom tested for forecast applications, even though hindcast analysis has been shown to be useful in model comparison studies and typically requires no information beyond that already used in the model testing phase. In the current presentation, we overview common probabilistic forecast verification methods and apply the methods to evaluate model ensembles produced using an integrated data assimilation and uncertainty framework developed for the National Weather Service operational rainfall-runoff models. Seasonal water supply ESP hindcasts are generated for the North Fork of the American River Basin (NFARB) in California, using a range of parameter sets. We include data uncertainty analysis in the forecasting framework through a DiffeRential Evolution Adaptive Metropolis (DREAM) algorithm which is part of a recently developed Integrated Uncertainty and Ensemble-based data Assimilation framework (ICEA). Extensive verification of all tested approaches (parameter sets, data assimilation, etc.) is undertaken using traditional forecast verification measures, including root mean square error (RMSE), Nash-Sutcliffe efficiency coefficient (NSE), volumetric bias, as well as probabilistic methods such as joint distribution, rank probability score (RPS), and discrimination and reliability plots. By using tools and techniques relevant to the operational forecast community (ensemble verification, hindcasting, models), we aim to expedite the transfer of research-level advances in hydrologic modeling into operations.