Upload
dave-kreimer
View
148
Download
3
Tags:
Embed Size (px)
DESCRIPTION
Citation preview
March 2013
TESTING TECHNOLOGY PRODUCTS
Websites, Apps, Software etc…
OVERVIEWMETHODOLOGIES
- EXPLORATORY TESTING- USABILITY TESTING- “REAL-WORLD” TRIALS
CONCLUSION
Reasons to Test Tech Products
Ensure efficient development of useful and appealing products– Begin with a well-conceived concept– Add features and functions systematically – Conduct periodic “reality checks” with users– Avoid “spider web” syndrome with changes– Increase confidence in reliability, relevance and
appeal of product as release approaches– Be prepared for an effective roll-out: sales story,
user training, competitive awareness etc…3
Reasons for Not Testing Tech
4
Reasons for Not Testing Tech
Lack of time Lack of money Overconfidence
– Unaware of potential problems– Disconnected from users true needs, attitudes or
skill level Sometimes not necessary
– minor changes, simple product, previously well researched, based on something similar, etc…
5
Types of Tech Product Testing
EXPLORATORY: – What digital content or functionality do users need
and/or desire? USABILITY:
– How easy or hard is it to use the digital product? “REAL-WORLD” TRIALS:
– How well does the digital product actually fulfill the users needs?
6
OVERVIEW
METHODOLOGIES- EXPLORATORY TESTING- USABILITY TESTING- “REAL-WORLD” TRIALS
CONCLUSION
7
Methodologies
Exploratory Testing
Usually groups – Live or Online groups– Online Bulletin Board
Typical flow of discussion:– Current technology practices– Speculate on future needs– Response to technology product ideas & options
Concept statements Wireframes or other mock-ups of ideas Existing websites or applications8
OVERVIEW
METHODOLOGIES- EXPLORATORY TESTING- USABILITY TESTING- “REAL-WORLD” TRIALS
CONCLUSION
9
Methodologies
Usability Testing
Most complex but not more important Common practices:
– Multiple iterations– Each consisting of 8-12 In-depth interviews
Depending on types users– 45-60-minute sessions– Conducted live in the early phases
Computer lab provides stable digital environment– Conducted online in later stages
Real-world digital environment - varied OS and browsers
10
Methodologies
Usability Testing
Four “phases” of usability testing– Wireframe– Alpha – Beta– Pre-release or Post-release
Dependent on level of completion and functioning of test digital product
Phases may not be clearly defined Nice but not necessary to test each phase
11
Methodologies
Usability – Wireframe Testing
Participants navigate through hyperlinked screen shots with only minimal content and sketchy design elements
Demonstrates the degree to which the website is organized and functions logically
Best conducted live, in professional market research facilities or computer lab
12
Methodologies
Usability – Alpha Testing
Product functions roughly and is populated with just enough data for participants to complete the most essential tasks
Design is evident but unrefined Evaluate how well the key navigational and
basic design elements align with users expectations
Best conducted live, in professional market research facilities or computer lab
13
Methodologies
Usability – Beta Testing
Product is reasonably stable and approaching full functionality with at least one complete segment of content
Gauges extent to which navigation, design and other conventions will work when product is complete
Good time to test “optional” features This phase of research is typically conducted
online to experience “real-world” conditions14
Methodologies
Usability – Pre or Post Release
Product is fully functional and content is complete
Simulates “real world” use to isolate bugs, impact of combined actions, identify minor refinements to improve user experience, uncover training needs
Usually conducted online– Tasks are more open-ended to see how new
users learn and problem-solve15
Methodologies
Usability
16
Methodologies
Usability – When Are You Done?
Digital products are tools…– Should fulfill the user’s needs in an efficient and
simple manner that is often “unremarkable”– “The absence of feedback is feedback” – High praise = “It worked just as I expected”
Ready for release when… almost all test participants successfully complete the mission critical tasks with minimal problems
17
OVERVIEW
METHODOLOGIES- EXPLORATORY TESTING- USABILITY TESTING- “REAL-WORLD” TRIALS
CONCLUSION
18
Methodologies
"Real-World" Trials
Multiple phase process– Individual trial and reporting of experience
Online Journal– Can use video, audio or text (smartphone saves time)
– Group discussion Live or Online or through Online Bulletin Board
Options for depth and duration– Task specific – 2-3 weeks– Short-term panel – 1-2 months– Long-term panel – 3 or more months1
9
Methodologies
"Real-World" Trials
Options for testing– Individual product– Comparison of competing products
Options for participants (Consider comparisons)– First users of new product– New users of established product– Veteran users of established product– Pre-trained users– Self-trained users– Users of competitive products2
0
Methodologies
Trials vs. Usability Testing
"Real-World" Trials Usability Testing
Documented by posts of users recollections to an asynchronous online bulletin board
Directly observed in real-time during one-on-one interviews
Macro view - Tasks are broad “create a lesson”- Feedback is non-specific
Micro view -Tasks are more specific “find a lesson about cell division”- Feedback is specific
Exposes common problems with only vague or speculative insight into users expectations
Exposes all problems and provides precise insight into users expectations
Helps to determine general functions that are problematic
Specifically identifies the problems related to each tested function
21
Methodologies Learning Curve vs Usability Issues
Learning Curve Issues Usability Issues
Challenge is learning the convention or action
Challenging to learn and to repeat
Once learned – repeated completion of task is easy
Users don’t learn – process is flawed
Improved with tweaks to labeling, design, button position etc… Also, training, tutorials etc…
Requires redesign of process. Sometimes creates a “ripple effect” that impacts other parts of platform.
Can be identified in both usability and "Real-World" Trials
Requires usability testing.Trial testing feedback is generally not specific enough to provide insight into these issues
22
Methodologies
"Real-World" Trials (related)
23
Corporate “Online Presence” Testing– Harvest available data
Online comments Customer communications Technical support data Website metrics
– Enhance and deepen insights with qualitative Live or online groups – primed with pre-group task Online Bulletin Board – users respond to web presence
of various companies and then “discuss”
OVERVIEWMETHODOLOGIES
- EXPLORATORY TESTING- USABILITY TESTING- “REAL-WORLD” TRIALS
CONCLUSION
24
About Dave Kreimer & Next Step Consulting
25
Next Step Consulting consists of a team of professionals supporting the efforts of Dave Kreimer, the company's principal.
Dave Kreimer's background in psychology, and years of facilitation and marketing experience enhance his market research skills. Since 1993, Dave has successfully designed, conducted, and documented roughly 500 research projects involving thousands of focus groups and personal interviews. The company’s most loyal clients include medical, educational, publishing, technology, and finance firms.
Contact Us
26
eMail: [email protected] Phone: (206) 782-6002 NSC Website NSC Blog LinkedIn Facebook Twitter
27
Graphic- Thank You