Upload
joseph-melton
View
217
Download
0
Tags:
Embed Size (px)
Citation preview
Amanda ReckerJamie Pina, MSPH, PhD
Barbara L. Massoudi, MPH, PhD
RTI International
2013 International Symposium on Human Factors and Ergonomics in Health Care:
Advancing the CauseMarch 11, 2013
Long-term Usability Testing for Public Health Information Technology:
BioSense 2.0
BioSense 1.0: Web-based syndromic surveillance• Mandated in the Public Health Security and
Bioterrorism (BT) Preparedness and Response Act of 2002
• Nationwide integrated system for early detection and assessment of potential BT-related illness
• 2003 Funding provided by Congress to CDC• Development of BioSense infrastructure started, initial
focus on:– VA and DoD– Direct reporting to CDC of detailed clinical data by
civilian hospitals
• Began soliciting more limited data from health departments (HDs) that had already established automated systems for ED-based syndromic surveillance– By 2007, 8 state/local HDs connected
Recommendations from Prior Evaluations• Strengthen state and local public health engagement
– Enhance state/local HD syndromic surveillance capacity– Increase participation of state/local HD syndromic
surveillance systems (improve coverage)– Share data with HDs from hospitals reporting directly to
CDC– Share governance with public health community
• Leverage investments in electronic health records (EHRs)
• GAO, 2008: Adopt an “open, distributed computing model”
• Improve utility of the data and data sources• Preparedness role: Greater “all hazards” emphasis• Expand uses for broader spectrum of public health
concerns
BioSense 2.0: Timeline• June 2010: Redesign begins• November 2011: Opened for business• November 2011 – June 2013: Onboarding new
jurisdictions – 35 jurisdictions signed the Data Use Agreement (DUA)– 17 fully onboarded
• April 2012: Retired BioSense 1.0
BioSense 2.0: Approach• Shift from a need-to-know to a need-to-share and co-
create approach• User-centered design
– Stakeholders engaged in every step of the redesign– HDs fully control “their data” at the level of granularity
they choose– More options for data sharing with other jurisdictions
and CDC
• Alignment with ONC and Meaningful Use– Agreed-upon core syndromic surveillance data elements– Collaborations with public health professional
associations– Funding to states: Meaningful Use syndromic
surveillance adoption, build capacity, join BioSense 2.0
• Cloud technology: distributed, easy to adopt, cost effective, secure
Why Long-term Usability Testing?
• Long user-centered design lifecycle• Expectation management• Stakeholder ownership• BioSense 2.0 continuously changing and growing• Longitudinal usability testing
– User satisfaction– Efficiency
• Functions not changing: building a query, viewing results, analyzing the data, sending and saving information
• Two approaches to testing
1. User-Centered Design• Qualitative data collection methods• The user knows best• Test the right participants• Broad range of public health professionals
– Public health generalist – less sophisticated users– Syndromic surveillance epidemiologist – more
sophisticated users
2. Activity-Centered Design
• Quantitative data collection methods• Behavior vs. opinions• Based on empirical data
– Time-on-task analysis– Mouse-click analysis– Pathway analysis
1. Establish requirements2. Design alternatives3. Develop prototype4. Conduct evaluation
Basic Activities in Interaction Design
(Rogers, Sharp, & Preece, 2011)(Rogers, Sharp, & Preece, 2011)
How to Choose Users
• Interact directly with the system– Epidemiologists, state and local public health
professionals
• Manage direct users – decision-makers– Public health directors
• Use similar syndromic applications
• User-centered design – Open question response– Focus groups– Expectation testing– SUS
• Expert evaluation– Usability heuristics (Nielsen)
• Activity-centered design– Morae software– Scenarios and tasks– Closed/open
question response– Time on tasks– Mouse-click analysis– Pathway analysis– Critical incidents
Generating Design Alternatives
Scenarios and Tasks
Scenario: “Over dinner at [a public health conference] an argument has erupted, but luckily as a BioSense 2.0 user you can settle this dispute. Health authorities in Virginia suspect that the flu season was more severe than it was in Michigan.”
Task: “Please determine which state, Virginia or Michigan, had more cases of influenza-like illness (ILI) starting in October 1, 2010 through March 1, 2011.”
How to Choose Among These Alternatives?• If one person says something is a problem, do you
change the design?• Resolved conflicting alternatives• Conducted feasibility analysis• In the end…relied on face validity
What Happens After Design Changes?
• User training through webinars and videos• Expert user testing• Focus group sessions• Continually comparing SUS scores