Overview
As water demand increases and water management operations become more constrained, forecast users will require more and better information about the uncertainty associated with the forecasted streamflow. Current water supply forecasts for the western United States are based largely on statistical regression equations that are developed mostly from monthly precipitation, recent snow-water equivalent, and past streamflow observations. These forecasts lack information about the uncertainty of the predictions, which limits the potential for large improvements in forecast accuracy.
To provide an objective means with which to generate streamflow forecasts with uncertainty, the National Weather Service (NWS) Ensemble Streamflow Prediction (ESP) method was developed. The ESP system uses conceptual hydrologic models and historical data to generate a set, or ensemble, of possible streamflow scenarios conditioned on the initial states of a given basin.
The purpose of the ensemble streamflow predictions evaluations for water supply forecasting was to examine the applicability of traditional statistical techniques and distribution-oriented measures for the evaluation of ESP forecasts, and to give insight into potential operational forecast performance based on simulated historical ESP forecasts for the Colorado River basin.
Methods
ESP forecasts encompass the uncertainty in the hydrologic models, watershed conditions, future meteorology, and streamflow estimates. Information about the state of the watershed (e.g., snowpack water content) at the beginning of the forecast period is used to start the hydrologic model. ESP then uses past meteorology events as possible representations of future events. Thus, historical temperature and precipitation data are used as hydrologic model input to generate streamflow estimates for the future forecast period. Each set of meteorologic data from the different years, and spanning the forecast period, produces a single streamflow trace. The resulting ensemble of traces is used to create a probability distribution of streamflows that constitutes the forecast.
In cooperation with the Colorado Basin River Forecast Center (CBRFC), we evaluated their experimental ESP forecasts for 14 locations in the Colorado River Basin where flows have not been affected by regulations or diversions. ESP output has not been systematically archived until recently, making forecast evaluation difficult. Thus we used the NWS Ensemble Streamflow Prediction Verification System to simulate (hindcast) water supply outlooks for the 14 study locations discussed above. A variety of statistics for measuring forecast quality were used to evaluate the probabilistic forecasts, including mean absolute error, ranked probability score, reliability, and discrimination.
Results
We evaluated the ESP hindcasts using a mix of verification criteria, including traditional statistics and distribution-oriented measures. Selected key results from our study:
- Examination of specific ensemble traces (e.g., minimum-error and historical traces) can provide insight about the limitations of the forecast system and process, including proper model identification and parameterization, and the respective roles of initial conditions and meteorological uncertainty in affecting basin response.
- Headwater locations showed different forecast performance behavior across the upper and lower Colorado basins, but common behavior within the basins. Overall, forecasts are better for locations in the upper basin, and forecasts issued March 15 and later are generally the best.
- We recommend that forecasting agencies issue probabilistic forecast products that describe the entire forecasted distribution, or at least several portions of the distribution that have meaning for practical applications.
- Because probabilistic forecasts and their verification criteria are different in character than traditional forecast products, we recommend education efforts focused on the proper interpretation, evaluation, and application of new products.