Upload
erinleebrady
View
262
Download
4
Embed Size (px)
DESCRIPTION
Presentation for paper "Investigating the Appropriateness of Social Network Question Asking as a Resource for Blind Users" at CSCW 2013. We discuss a survey of blind people's social network use, their thoughts on social networking sites as a resource for question asking, and how financial incentives affected their use of social networking sites for question asking.
Citation preview
Investigating the Appropriateness of Social Network Question Asking
as a Resource for Blind Users
Questions or comments during the talk? Tweet @erinleebradyPhoto used via CC Liscense from
http://www.flickr.com/photos/loneblackrider/315302588/
Erin Brady, Yu Zhong, Meredith Ringel Morris, Jeffrey P. Bigham
University of Rochester || Microsoft Research
Usefulness of SNSs for Blind People
Have visual questions that need to be answered [Brady 2013]
Introduction | Survey | Field Experiment
Why use SNSs?
FreeHigh saturation
[Pew 2012]
Personalized and trusted [Morris 2010]
Matches existing model [Kane 2009, Burton 2012]
Low-costAlways-available workers
Anonymous
Images used from Facebook, Twitter, and http://imaginarywitness.files.wordpress.com/2010/03/mechanical-turk.jpg
Introduction | Survey | Field Experiment
Primary Questions
How do blind people use SNSs?How do blind people view SNSs for question
asking?How do blind people ask questions on SNSs in
practice?
Introduction | Survey | Field Experiment
Primary Questions
Survey of social networking site useHow do blind people use SNSs?How do blind people view SNSs for question
asking?
Field experiment with VizWiz SocialHow do blind people ask questions on SNSs in
practice?
Introduction | Survey | Field Experiment
Survey Design
Accessible online surveyAvailable for 3 weeks (Jan/Feb 2012)Advertised via email to NA organizations for
the blindAsked participants not to spread survey via
SNSRaffled gift card for incentiveSurvey focused on all types of question-asking
Introduction | Survey | Field Experiment
Demographics
203 completed survey191 self-reported as blindAges skewed older, which may reflect onset of
blindnessGenerally had significant internet experience
Introduction | Survey | Field Experiment
Networks Used
At 92%, general SNS use higher than average of 66%
[Pew
2012]
Face
book
Twitt
er
Link
edIn
Googl
e Plus
MyS
pace
Yam
mer
Inclus
ive
Plan
et
Orkut
0102030405060708090 80
52
40
154 3 3 1
* χ2(1, N=191) = 55.88, p < .001
Introduction | Survey | Field Experiment
Networks Used
85% used Facebook, Twitter, or bothTwitter use higher than general population
Facebook Twitter0
10
20
30
40
50
60
70
80
9080
5261
15
Blind UsersPew Study
* 61% Facebook is calculated from 66% of online adults using SNSs, 92% of those using Facebook
Introduction | Survey | Field Experiment
Network Size
Smaller than average network sizes Facebook median 100
vs. Pew median 111, Facebook stats 130 Twitter median 45
vs. reports of Twitter around 126
Introduction | Survey | Field Experiment
SNS for Question Asking
Logged in frequentlyGeneral behavior tended toward “lurking”Status message question asking was
infrequentAnswer rates were low
Introduction | Survey | Field Experiment
SNS for Question Asking
55% thought SNS question-asking could be effective
Few users of SNSs felt comfortable posting questions
Introduction | Survey | Field Experiment
Findings
High adoption rate of SNSs Despite accessibility challenges Asynchronous communication with physically remote
contacts Twitter in particular (text-based)
Status posting and question asking infrequent Poor response rates Smaller-than-average network sizes Concerns about social costs
Introduction | Survey | Field Experiment
Primary Questions
Survey of social networking site useHow do blind people use SNSs?How do blind people view SNSs for question
asking?
Field experiment with VizWiz SocialHow do blind people ask questions on SNSs in
practice?
Introduction | Survey | Field Experiment
VizWiz Social
Mobile phone application that answers visual questions
Based on concept presented by Bigham et al.5,000+ blind users asked over 40K questions
in first year
“Which can is the corn?”
Introduction | Survey | Field Experiment
Identification
Description Reading Unanswerable
44% 26% 23% 7%
General Question TypesIntroduction | Survey | Field Experiment
Answer Sources - Anonymous/Crowdsourced
Questions can be sent to: Web workers
Recruited from Mechanical Turk Answers in 98 seconds VizWiz team pays 5¢ per answer
IQ Engines Human-backed object recognition VizWiz team pays approximately 1¢ per answer
Introduction | Survey | Field Experiment
Answer Sources - Social/Friendsourced
Questions can be sent to: Broadcast to Social Networking Sites
Facebook/Twitter Posts made by the VizWiz application Free to VizWiz team
Email to Individual Contacts Free to VizWiz Team
Introduction | Survey | Field Experiment
Use of Social Sources
In month-long period preceding study: 702 users asked 3116 questions (average 4.44/user) 15% of users had tried social sources, 10.7% excluding
email 5% of questions (156) were sent to social sources
94 to Twitter 47 to email 26 to Facebook
Only 3 received answers, with median response time of 2:55:00
Introduction | Survey | Field Experiment
Experimental Design
Test the impact of financial restrictionsMirror the existing costs of the VizWiz
serviceEach participant was given $25 balance for
a monthSplit into conditions:
Cheap: 1¢ for IQ Engines, 5¢ for web workers Expensive: 5¢ for IQ Engines, 25¢ for web workers
Remaining balance, plus $10 gratuity, paid at end
Introduction | Survey | Field Experiment
Modified ApplicationIntroduction | Survey | Field Experiment
Experimental Design
Recruited from 207 active VizWiz users30 were recruited, 23 asked 1 or more
questionsSplit evenly into cheap, expensive
conditionsAnalyzed pre-study behaviors:
Asked 217 questions total (average 9.86/user) 81% of questions sent to web workers 93% of questions sent to IQ Engines 14% to social sources (10% to FB, 3% to Twitter, 1%
to email)
Introduction | Survey | Field Experiment
In-Study Behaviors
23 participants asked 170 questions (average 7.3/user)
Web
Wor
kers
IQ E
ngin
es
Face
book
Twitt
er
Emai
l0
20
40
60
80
10081%
93%
10%3% 1%
83%
45%
1% 0% 0%
Pre-StudyIn-Study
Introduction | Survey | Field Experiment
Post-Study Questionnaire
12 participants completed the questionnaireDemographics:
7 male, 5 female Four aged 20-29, six aged 30-39, two aged 50-59 Most had used internet for 10+ years All used at least one social networking site
Introduction | Survey | Field Experiment
Post-Study Questionnaire
All chose crowdsourcing as preferred answer source
“Humans are much more reliable, in my opinion, and Web workers are entirely
anonymous. They might necessarily not even know that they're dealing with an
accessibility application if Amazon Turkit [sic] is involved.”
Introduction | Survey | Field Experiment
Post-Study Questionnaire
Crowdsourcing was preferred to friendsourcing 9/12 “much preferred” web workers 1/12 “somewhat preferred” web workers 2/12 had no preference between web workers or
friendsourcing
Introduction | Survey | Field Experiment
Post-Study Questionnaire
Reasons were both technical and personal Speed of response Accuracy Feedback
Preferred anonymity Didn’t want to broadcast Didn’t like SNSs in general
“…because there's no guarantee that a facebook or twitter post would get you an immediate answer. When I need something identified like a can or TV Dinner I am going to use it now, not whenever my friends get around to telling me what it is. :) ”
Web-workers are completely anonymous, and there is sometimes no reason to think they are actually assisting
with a disability related question.
Introduction | Survey | Field Experiment
Post-Study Questionnaire
Restricted question-asking behaviors for social sources 8/12 chose not to ask 1+ questions to social sources 4/12 chose not to ask 1+ questions to crowdsourced
sourcesSocial costs played a role in restricted
question-asking“Not my friend's job to tell me that stuff. Plus it clutters up people's timelinesand [sic] they
might not like it.”
Introduction | Survey | Field Experiment
Primary Questions
Survey of social networking site useHow do blind people use SNSs?How do blind people view SNSs for question
asking?
Field experiment with VizWiz SocialHow do blind people ask questions on SNSs in
practice?
Introduction | Survey | Field Experiment
Findings
0. Friendsourcing can be a valuable resource for Q&A1. Blind people are heavy users of SNSs2. Blind people infrequently use SNSs for QA3. In practice, even practical & financial incentives can’t motivate blind users to use SNSs for QA
Investigating the Appropriateness
of Social Network Question Asking
as a Resource for Blind Users
Erin Brady, Yu Zhong, Meredith Ringel Morris, Jeffrey P.
Bigham
University of Rochester Microsoft
Research
Get in touch via email:
or Twitter: @erinleebrady
Interested in questions asked by VizWiz users? Come see our talk at CHI! Visual Challenges in
the Everyday Lives of Blind People
Supported by NSF Awards #IIS-1149709
and #IIS-1049080