Crowdsourcing for Business: An Emerging Paradigm Shourya Roy
Area Manager, Human Computation Xerox Research Centre India,
Bangalore [email protected] Workshop on Social Computing, IIT
Kharagpur 5 th Oct, 2012
Slide 2
Crowdsourcing : What is it? The act of taking a task
traditionally performed by an employee or contractor, and
outsourcing it to an undefined, generally large group of people, in
the form of an open call Digitization, image labeling, user
studies, machine translation evaluation, logo design, EDA
simulation, innovation contests,...
Slide 3
Services Thrust Xerox Confidential Handwriting Recognition
Problem* Many tasks are easy/feasible/doable for humans, but
difficult/challenging/impossible for computer programs 1.Make
progress towards deciphering this handwriting 2.Put words which you
are unsure about in parenthesis Instructions to Crowd
Slide 4
Services Thrust Examples (1/5)
Slide 5
Services Thrust Examples (2/5)
Slide 6
Services Thrust Examples (3/5)
Slide 7
Services Thrust Examples (4/5)
Slide 8
It has been Existing Humans were the first computers,
computers, used for math computations 9 examples of crowdsourcing,
before crowdsourcing existed : http://bit.ly/mXFdRp
http://bit.ly/mXFdRp
Slide 9
Xerox Confidential Internet and Mobile Have Made it More Common
and Promising
Slide 10
Services Thrust Increasing Activities and Popularity Page 10 2M
contributors who does more than 4PY of work on an average day! --
Increasing Popularity as Depicted by Google Trends Crowdsourcing on
Google Scholar Over the Last Few Years CEO
Slide 11
Services Thrust Changing Demographics
Slide 12
What is the Problem Given a computational problem, design a
solution using human computers and automated computers Xerox
Confidential
Slide 13
Human in the loop (and not Guinea Pigs) Main doer is Human (and
not Machines as in Assembly Lines) Humans are actively
computing(not merely carrier of sensors) The outcome is determined
by an algorithm (and not the natural dynamics of the crowd) Why is
it Different?
Slide 14
Where is Research? Xerox Confidential Quality Estimation and
Assurance ( Redundancy and voting; Gold data; joint estimation of
worker quality and task difficulty; Symbiosis with Machine Learning
) Complex Tasks ( No discrete answer; Exploration and exploitation;
crowd workflows ;) Task Design ( Optimize cost, quality and time;
infinite completion time; Real time ) Incentive and Motivation (
Payment vs. non-payment; Optimal payment; Payment and quality ; )
Market Design ( Reputation Mechanism; Monitoring and feedback; Task
Discovery ; Behavioral Aspects ( Noisy behaviour; Non-reproducible
;)
Slide 15
An Emerging Research Field
Slide 16
An Interdisciplinary Research Field
Slide 17
Thats Alright but Xerox!!?
Slide 18
We have transformed into the worlds leading enterprise for
Business Process and Document Management Revenue Market Opportunity
2011 Services-led ~50% Services Document Outsourcing Business
Process Outsourcing Information Tech Outsourcing ~$23 billion $500
billion + Services Leadership In $15.2 billion 2009 Technology-led
~25% Services $132 billion Document Outsourcing 18
Slide 19
Xerox Revenue by Business Segment* *
http://www.fastcompany.com/magazine/161/ursula-burns-xerox
Slide 20
Page 20 Is Crowdsourcing a Viable Alternative to Outsourcing?
Outsourcing is Focus on the core business while partnering with 3rd
party vendors to tackle the non-core operations Tasks requiring
human intelligence and skills Data and process migration by smart
use of technology Heavily human intensive; typically with the help
of computing technologies Large distributed workforce enabled by
technology executing tiny pieces of work requiring human
intelligence
Slide 21
Page 21 Data Entry by Crowd We started by considering a typical
outsourced process (Data Entry) Objective is to understand a
process in detail and identify implications for crowdsourcing
Digitisation of insurance forms and medical records for US based
insurance companies Typing in, validation/ correction of
information from scanned forms Outsourced, distributed process
Slide 22
Features that make Form Digitization process amenable to crowd
sourcing Page 22 Relatively low skill data entry work, known as key
what you see Already an outsourced process requiring a low level of
interactivity between sequential steps Strong workflow tool to
manage work, which flows through a series of system and human steps
Between sites Between sequential tasks Between agents (given their
known skill set)
Slide 23
Findings from Work-Practice Study (1/2)
Slide 24
Findings from Work-Practice Study (2/2) Page 24 Workplace
Ecology : Data security is physical, technical & social
Crowdsourcing: lose physical and social enforcement, reduced
control of workforce. Need technical solutions. Skills and
Knowledge 1)key what you see data entry actually involves extensive
rule set. 2) Form difficulty is situational. 3) Non-standard means
non- standard. Crowdsourcing: Situational-based incentives and
supporting learning Being a Corporate Employee Pay alone not enough
to achieve SLA. Agents made accountable. Crowdsourcing: reduced
accountability could increase rejections of difficult work. Making
the Workflow Work: Push model of work Crowdsourcing: Pull model of
work raises coordination and completion issues. Collaborative
Working: Work is not collaborative at workflow level; but it is at
claim level (floorwalkers & colleagues). Crowdsourcing:
building collaboration in? Pull models of supervision?
Slide 25
Conclusion Page 25 Crowdsourcing is an emerging Research area
It requires expertise and research competencies from a number
disciplines Crowdsourcing can be applied in various domains to
solve problems in a more effective manner Finally, a large fraction
of the crowd comes from India Focused research and technologies
will be highly relevant
Slide 26
Services Thrust References TurKit: Tools for Iterative Tasks on
Mechanical Turk; Greg Little, Lydia B. Chilton, Robert C. Miller,
and Max Goldman Matt Lease Tutorial Soylent A cr Fold.it S. Cooper
et. al
