Overview
Based on stakeholder interest in evaluation and verification of forecasts, we have devoted significant effort to the development of an online interactive Forecast Evaluation Tool (FET). Earlier stakeholder interactions made clear that long-term acceptance of a web tool by resource managers as a useful system to support decision making required significant commitment to development of a “commercial quality” website. To address this issue, we began a partnership with the Hydrologic Data and Information System (HyDIS) project now at UC Irvine as the Center for Hydrometeorology and Remote Sensing (CHRS), and a project under the GEWEX America Prediction Project (GAPP). Coordination with these projects brought into play a software engineering approach to achieve reliable support of multiple simultaneous users on a variety of browsers, as well as an expert web programming team to develop Java-based tools and implement a system for managing incremental website enhancements. We have now developed an Internet-based interactive Forecast Evaluation Tool that decision makers can customize to their specific interests. The ongoing decision-support tool project aims to improve the capabilities of the forecast evaluation tool and to develop decision-support capabilities for other forms of hydroclimatic information.
Methods
We develop improved decision-support tools using programming methods and software that meet industry standards. We are committed to open-source program code. Our methods allow the FET and other decision-support tools to be transferable to other regions and entities, and scalable to larger and smaller spatial domains, and greater numbers of users. We have used guided interactive workshops, key informant interviews, and structured survey instruments to garner information on the usefulness and usability of forecast evaluation formats and the FET website.
Results
We developed the Forecast Evaluation Tool. Based on feedback from workshops and other formal and informal interactions with website users, we've added features to the FET, including:
A tutorial on correct interpretation of probabilistic seasonal climate outlooks;
A feature that allows forecast comparisons with observed climate data;
A feature to explore historical data and select segments of those data as analogs of possible future conditions;
The modified Heidke Skill Score as a measure for evaluating forecast performance;
Evaluation of seasonal climate outlooks for Alaska;
Evaluation of NOAA-NWS downscaled local temperature outlooks.
The FET is now used by the NOAA-National Weather Service Climate Services Division to train NWS climate focal point personnel. Additionally, our forecast assessment web tools are designed to help decision makers get the most out of a variety of different forecasts. One such tool, the Climate Information Delivery and Decision Support System (CLIDDSS), provides information and decision support based off a paradigm of systematically providing improved forecast and information products. These products support the broadest range of decisions in an equitable manner, such as accommodating different levels of technical abilities. CLIDDSS allows users to:
Use project folders to store selected products for multiple applications and efficient access over repeated site visits;
Share their product suites with identified groups for discussion and consensus interpretation;
Generate PDF reports that ensure inclusion of ancillary information combined with the ability to add customized interpretive comments (e.g., for delivery of information by intermediaries such as extension agents or state climatologists).
CLIDDSS also tracks web tool usage to provide ongoing feedback to operational agencies, science managers, and researchers about which products are preferred by various types of users and applications. CLIDDSS was designed explicitly for transfer to operations and to be scalable to serve intense usage such as hundreds of users at once.