Overview
The purpose of the seasonal climate outlook evaluation project was to develop multiple evaluation criteria, in order to meet the needs of a wide variety of forecast users and a broad spectrum of user technical sophistication, to examine stakeholders’ forecast needs and the potential application of forecast skill to guide forecast use, and to develop guidelines for the effective communication of seasonal climate outlooks.
Methods
The Climate Prediction Center (CPC) outlooks predict the probability of seasonal average temperatures or seasonal total precipitation falling into one of three categories or terciles (warm, neutral, and cool for temperature and wet, neutral, and dry for precipitation), based on the distribution of observed data from 1971–2000. We evaluated forecast skill using:
- Categorical measures, such as probability of detection and false-alarm rate, which track how often forecasts are correct or incorrect, respectively;
- Brier scores, which consider the strength of the probability given to a specific category (e.g., warm or cold);
- Multi-category scores, such as ranked probability skill score, which evaluate the full range of conditions;
- Conditional distribution diagrams, such as discrimination and reliability diagrams, which can identify forecast skill for specific situations.
We also developed a set of diagrams to show CPC seasonal outlooks and their skill for all lead times leading up to a particular target forecast date.
We conducted extensive interviews and convened small workshops, in order to examine stakeholders’ needs and to learn about impediments to forecast communication.
Results
The results of our study have been published in a 2002 article in the Bulletin of the American Meteorological Society (see the Forecast Evaluation project page for a downloadable copy). Some key findings with regard to the Southwest include the following:
- Forecast users frequently interpret the probabilities shown on NOAA CPC outlook maps as deterministic quantities. Consequently, great care must be used when communicating probabilistic forecasts to users;
- NOAA CPC seasonal temperature outlooks exhibit considerable skill in the Southwest;
- NOAA CPC seasonal precipitation outlooks are skillful in the Southwest for the winter months, but show almost no skill during the summer months. In fact, very few forecasts are issued for summer precipitation in the Southwest;
- From the perspective of water managers in the Southwest, winter precipitation outlooks made during the fall and winter are better than climatology according to all criteria. However, forecast skill is poor in the upper Colorado River basin—the source of the majority of the runoff into the Colorado River;
- Climate forecasts for the Southwest perform better from the perspective of cattle ranchers making use of winter range, given the greater climate predictability during winter;
- To the extent that fire risks are affected by winter conditions, forecast skill even at long lead times may prove useful to fire managers.