Upload
season
View
41
Download
0
Tags:
Embed Size (px)
DESCRIPTION
User studies: contributing to developing regional decision support for climate services. Andrea J. Ray, Roger S. Pulwarty and Robert S. Webb NOAA-CIRES Climate Diagnostics Center Climate Prediction Assessments Workshop Tallahassee, FL 9-11 March 2004. - PowerPoint PPT Presentation
Citation preview
User studies: contributing to developing regional decision support for climate
services
Andrea J. Ray, Roger S. Pulwarty and
Robert S. Webb
NOAA-CIRES Climate Diagnostics CenterClimate Prediction Assessments Workshop
Tallahassee, FL
9-11 March 2004
“The timely production and delivery of useful climate data, information, and knowledge to decision makers.”
National Research Council, 2001
• NRC recommends that a regional focus is needed to address societal needs, developed through regional decision support systems.
• Pillars of climate services include:
–Regional decision support networks,
–Research,
–Observing systems, and
–Mechanisms to enable user-centric design and improvement.
To be effective, regional decision support systems must not only provide data and forecasts, but also determine:
–what problems and questions are important and critical, and why, to be determined by consensus between users and providers of products;
–what information is needed to solve problems, on short and long-term time scales;
–what transaction costs are involved in developing and maintaining long term partnerships and interactions.
• Many “regional decision support,” networks exist,
–NOAA weather forecast offices,data centers, river forecast centers,
–Regional climate centers,
–State climatologist offices,
–Agricultural extension services
–Some private companies
• These networks provide data and forecasts, but in order to meet the NRC user-centric goals of climate services, these organizations would also:
–adapt products to meet specific user needs,
–support major user groups,
–encourage dialogue with users, and
–develop a regional focus to address societal needs.
• These networks generally are not set up to do effective user-studies
Effective user-centric design and implementation
• To achieve user-centric design and implementation these interactions must:
–identify entry points for information within existing decisionmaking and the space for communicating this information;
–identify research needed, including strategic, applied, and adaptive;
–create nodes for users to engage with researchers and providers, to help determine what products are to be developed and tested;
–Characterize needed products: e.g., scale, units, formats, how accessible, when issued.
–Inform the development of alternative decisionmaking strategies as needed
• Mechanisms for studying users need to be built in to regional networks.
Effective user-centric design and implementation
• Sustained and systematic communication is required on three fronts:
–between users and providers (NOAA and regional);
–between researchers and providers, and
–between researchers and users.
• Why user studies?
–More than consultancy
–More than a simple conversation or communication (not anecdotal)
–Understand the complexity of user context and differences in perspectives between users and scientists
Designing effective user studies
• User-study approach
• Based on case studies
• Based on
–need to get past differences in perspective;
–the innovation-decision process;
–definitions of “use”
• Dimensions to characterize about a user-case
Differences in perspectiveFactor Scientist’s perspective Water Manager’s Perspective
Identifying a critical issue
Based on a broad understanding of the nature of water management
Based on experience of a particular system
Time frame Variable Immediate (operations)
Long-term (infrastructure)
Spatial resolution Defined by data availability or funding Defined by institutional boundaries or authorities
Goals Prediction
Explanation
Understanding of natural system
Optimization of multiple conditions and minimization of risk
Basis for Decisions Generalizing multiple facts and observations
Use of scientific procedures and methods
Availability of research funding
Disciplinary perspective
Tradition; Procedure
Professional judgment; Training
Economics; Politics
Job risks
Expectation Understanding
Prediction
Ongoing improvement (project is never actually complete)
Statistical significance of results
Innovations in methods/theory
Accuracy of information
Appropriate methodology
Save money and time; Protect the public;
Protect their jobs, agendas or institutions
Product Characteristics
Complex
Scientifically defensible
As simple as possible without losing accuracy
Importance of context
Frame Physical (atmospheric, hydrologic, etc.) conditions as drivers
Dependent on scientific discipline
Safety and well being
Profit
Consistency with institutional culture, policy, etc.
Nature of Use Conceptual Applied
Innovation-Decision process: conceptual steps in deciding whether to adopt an innovation
• Knowledge phase: learns about the existence of an innovation, perhaps thru an outreach function
• Persuasion phase: forms a favorable/unfavorable opinion• Decision phase: engages in activities leading to adopting or
rejecting• Implementation: puts the innovation into use • Confirmation phase: decision to adopt is reinforced, or could be
reversed if experience is adverse (Rogers,1995)
Path towards adoption and use: studies and mechanisms for developing user-centric products should take advantage of this
Definitions of “use”
• Consult: the product is consulted, e.g., looked up on a web page or received as a briefing or from other source (type1);
• Consider: after consulting the product, it is considered in management deliberations as a factor potentially influencing decisions, but not necessarily in operational models (type 2);
- forecasts may be used in this way when they are not in appropriate forms for use in operational models
• Incorporate: some form of the forecast is incorporated into an operational model that is utilized in operational decisions (type 3).
• Dialogue about risks: communication of risk, i.e. the forecast is used to communicate with other managers and stakeholders about the risk of certain conditions and about the need to take actions, or to justify actions (type 4).
User Study Approach
• One mechanism for assessing user needs• Dimensions to characterize about the user-context:
– Identify critical problems which are sensitive to climate– Identify decisionmakers and their key stakeholders – Assess how climate variability interacts with their critical
problems• Decision calendar may help organize recurring decisions and
show when products are needed
– Identify user groups who may be willing partners in testing and prototyping this new technology
• Potential users in a rapidly evolving phase of their critical problem may be more open to interacting as partners
• Social changes, legal or policy change, or a climate event such as drought or flood
User group is the reservoir managers in the Colorado headwaters and Gunnison basins who operate reservoirs originally built to provide reliable irrigation supply, hydropower, and recreation
Redlands Diversion dam and fish ladder, Gunnison river near Grand Junction, CO
Case study: Multi-purpose Reservoir Management in the Upper Colorado River Basin
Dimensions for Aspinall Unit Case
• Critical problem sensitive to climate– Evolving reservoir management in which demands now are closely
balanced with supply but new uses are being legally proscribed– More intensely managed water system is more sensitive to
shortage because there is less buffer in the system
• Decisionmakers and their key stakeholders– USBR has authority to manage, but USFWS, NPS, have interacting
legal authorities, other stakeho
• How climate variability interacts w/ the critical problem– Decision calendar for annual operating plan helped organize
recurring decisions and ID potential climate
• User groups who may be willing partners/adopters– Deadlines to write ESA recovery plans and formal filing for a federal
reserve water right for a national park are requiring USBR to accommodate these in their operations
– 2002 drought as a climate event (not forecasted) also resulted in finding flexibility and new ways of operating
Potential Uses in the Aspinall case
• Identified a number of potential uses of seasonal climate forecasts in their operational models, as well as shorter term forecasts
• However, seasonal forecasts are currently not available in a form that can be incorporated into their operational models. Current use is to consult these products and consider them in management deliberations
• Aspinall decisionmakers use climate information (e.g. during drought) in a dialogue with their stakeholders about the risk of certain conditions and about the need to take actions, or to justify actions.
• Projects with Martyn Clark (WWA) and Dave Brandon (CBRFC) are working to translate both MRF forecasts and CPC seasonal forecasts
Conclusions I
• Regional decision support networks exist, but are not designed to carry on the systematic study of and interaction with users to ensure the development of user-centric products and services
• Mechanisms need to be built into the regional decision support networks
• These mechanisms could ensure user-centric design and implementation by:
–identifying entry points for information within existing decisionmaking and the space for communicating this information, e.g., through basin water management groups, water availability task forces, emergency management meetings
–identifying research needed, including strategic, applied, and adaptive;
–enabling RDS networks to function as nodes for users, researchers and providers to engage in dialogue and to help determine what products are to be developed and tested;
–effectively characterizing needed products and synthesize user needs from among many interactions (beyond consultancy)
Conclusions II
• User studies are a means to create and sustain the communication and learning required for user-centric design and development – and also a cost.
–create a dialogue about climate risks and not just provide information;
–create a dialogue about the types of climate products desired by a broad spectrum of users by integrating knowledge about what climate products are
needed.
• NOAA’s mechanisms for developing user-centric products and services need to:
–Recognize that differences in perspective exist and limit scientists ability to create user-centric products – unless we deliberately work to see other perspectives
–“Use” of products is broader than just in operational models: recognize that there is a range of uses and create products for all types. Evaluation of “use” should consider all types.
–Adoption is a process: develop mechanisms to draw potential users along this path and evaluate whether activities support evolution of use
• Training needed across the RDSS networks in how to conduct user-assessments
Summer season forecast for Peak Augmentation planning
Jan Feb Mar Apr May Jun July Aug Sep OctOct Nov DecAug Sep
Provide for late Summer/early Fall irrigation while maintaining target flows
Next water year runoff unknown, reserve water until February snowpack observations
Winter season precipitation forecast for Fall release decisions
Winter releases based on January/February snowpack observations
Operational issuesOperational issues
Potential use of forecastsPotential use of forecasts
Legend:
Planning ProcessPlanning Process
Winter/Spring forecast for Winter release decisions
Peak Flow Augmentation - fill curve
Week 2 forecasts for Peak Augmentation
Peak Flow Augmentation releases
Plan releases for Summer irrigation & hydropower
Week 2 forecasts for Summer irrigation & hydropower release decisions
Reservoir Management Decision Calendar
Provide for Summer irrigation & hydropower needs while maintaining target flows
Water Year Planning Next Water Year Planning
Ray
et
al.
(200
0)
Figure 4-28
Differences in perspective
• Identifying a critical issue
• Time frame
• Goals
• Basis for decisions
• Expectiation
• Product characteristics
• Frame
Desired information not available
Understanding Barriers New Services
How to they understand climate? What do they use now?
Can we produce forecasts in a manner related to how they think about climate?
Are the problems critical to the user influenced by climate? If so, are there
products available to address these?
Are there circumstances that minimize constraints?
User studies seek answers to the right hand questions
Other factors more
important
Constraints prevent flexible response
Fail tounderstand