AN INTRODUCTION TO
HUMAN COMPUTATION &
GAMES WITH A PURPOSE
ALESSANDRO BOZZON DELFT UNIVERSITY OF TECHNOLOGY
LUCA GALLI POLITECNICO DI MILANO
ABOUT THE TUTORIAL • Crowdsourcing, Human Computation, and GWAPs are hot topics
• “Human Computation” => more than 3000 papers • 400 in 2013
• “Crowd Sourcing” => more than 36000 papers • 4800 in 2013
• “Games With A Purpose” => more than 1400 papers • 162 in 2013
• This short tutorial is necessarily shallow, but • Concrete Examples • Lot of references and links • An outlook on the future
• Slides and additional materials available • http://hcgwap.blogspot.com
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 2
ABOUT THE SPEAKERS ALESSANDRO BOZZON Assistant Professor - TU Delft http://www.alessandrobozzon.com
LUCA GALLI Ph.D. Student - Politecnico di Milano http://www.lucagalli.me
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 3
• RESEARCH BACKGROUND AND INTERESTS
• Web Data Management • Crowdsourcing and Human Computation • Game Design • Web Engineering and Model Driven Development
AGENDA
4 ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
AGENDA • PART 1 => CrowdSourcing and Human Computation
• Introduction • Design of Human Computation Tasks • Frameworks For/With Human Computation • The Future of Human Computation
• PART 2 => Games With a Purpose • Play vs Games • Introduction to Game Design • Games with a Purpose Design Guidelines
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 5
PART 2 AN INTRODUCTION TO GAMES WITH A PURPOSE
6 ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
7 ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
Something really bad is going to happen…
8 ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
Or not?
9 ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
MAGIC CIRCLE
“All play moves and has its being within a playground marked off beforehand materially or ideally, deliberately or as a matter of course… The arena, the card-table, the magic circle, the temple, the stage, the screen, the tennis court, the court of justice, etc., are all in form and function play-grounds, i.e., forbidden spots, isolated, hedged round, hallowed, within which special rules obtain. All are temporary worlds within the ordinary world, dedicated
to the performance of an act apart.”
Johan Huizinga, “Homo Ludens”, (1938-1950)
WHAT IS PLAY?
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 10
Autonomy: play is a voluntary ac2vity.
Safety: during play there are radically reduced serious consequences in what we do
A<unement: being interested in the same things temporarily while playing together
Explora2on: the possibility to experiment and try out new things
Mastery: the will to improve one’s own skills
WHICH ARE THE BENEFITS?
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 11
“Play is an enjoyable ac/vity that allows us to train and test our skills in a safe environment.”
Fosters crea2vity, flexibility, and learning
An2dote to loneliness, isola2on, anxiety, and depression Teaches perseverance
Hints on how to cooperate with others Increases energy and prevents burnout
Stuart Brown, “Play: How it Shapes the Brain, Opens the Imagination, and Invigorates the Soul”, 2009.
GAMES VS PLAY
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 12
Games are dis/nguished from play
– Play is free-‐form
– Games are rule-‐based A game is a closed, formal system that ·∙ Engages players in structured conflict and ·∙ Resolves its uncertainty in an unequal outcome.
Fullerton, T.; Swain, C. & Hoffman, S. Game Design Workshop: A playcentric approach to creating innovative games, 2008
CLASSIC GAME DEFINITION
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 13
1. Rules • Games are rule-‐based.
2. Variable, quan/fiable outcome
• Games have variable, quan/fiable outcomes.
3. Valoriza/on of outcome
• The different poten/al outcomes of the game are assigned different values, some posi/ve and some nega/ve
CLASSIC GAME DEFINITION
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 14
4. Player effort
• The player exerts effort in order to influence the outcome (games are challenging).
5. Player aVached to outcome
• The player is emo/onally aVached to the outcome of the game in the sense that a player will be winner and “happy” in case of a posi/ve outcome, but a loser and “unhappy” in case of a nega/ve outcome.
6. Nego/able consequences
• The same game [set of rules] can be played with or without real-‐life consequences
Juul, J. “Half-R
eal: Video Gam
es Betw
een Real
Rules and Fictional W
orlds”, 2005
INCENTIVES
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 15
Money, Passion, Ease of Participation, Altruism, Appeal to one’s own knowledge, Reputation, Fun
Organisciak, Piotr. Why Bother?: Examining the Motivations of Users in Large-scale Crowd-powered Online Initiatives. Diss. University of Alberta, 2010.
INCENTIVES
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 16
Intrinsic mo2va2on is defined as doing an ac2vity for its inherent sa2sfac2ons rather than for some separable consequence.
Extrinsic mo2va2on is defined as doing an ac2vity to a<ain some separable outcome
Deci, Edward L., and Richard M. Ryan. Self‐Determination. ,1985.
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 17
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 18
GAMES WITH A PURPOSE
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 19
Games with a Purpose (GWAPs) are digital games that generate useful data as a by-‐product of play. [vA06,LvA09]
The design of a GWAP requires to create a game so that its structure encourages computa2on, correctness of the output and players reten2on.
WHAT THEY ARE USED FOR
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 20
Krause, M
. & S
meddinck, J. (2011b). H
uman C
omputation
Gam
es: A Survey. In: P
roceedings of the 19th European
Signal P
rocessing Conference (E
US
IPC
O-2011).
Intui2ve Decisions
Aesthe2c Judgment
Contextual Reasoning
Embodiment Issues
ARE THEY WORKING?
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 21
ESP Game
Purposes: Let players determine the contents of images by submitting meaningful labels they can agree on.
Results: A total of 13,630 people played the game during the test phase, generating 1,271,451 labels for 293,760 different images.
Luis von Ahn and Laura Dabbish. 2004. Labeling images with a computer game.
ARE THEY WORKING?
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 22
Foldit
Purposes: The objective of the game is to fold the structure of selected proteins to the best of the player's ability, using various tools provided within the game to discover new structural configurations.
Results: In 2011, players of Foldit helped to decipher the crystal structure of an AIDS-causing monkey virus. While the puzzle was available to play for a period of three weeks, players produced an accurate 3D model of the enzyme in just ten days. The problem of how to configure the structure of the enzyme had stumped scientists for 15 years.
Khatib, F.; Dimaio, F.; Cooper, et al. (2011). "Crystal structure of a monomeric retroviral protease solved by protein folding game players"
GAMIFICATION
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 23
“The use of game design techniques and game mechanics to enhance non-‐game contexts”
S. Deterding, M. Sicart, L. Nacke, K. O’Hara, and D. Dixon, “Gamification. Using game-design elements in non-gaming contexts”
Actually ques/onable, we will see why
Sebastian Deterding
Miguel Sicart
WHAT IS USED FOR
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 24
IS IT WORKING?
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 25
Samsung Nation
Purposes: Samsung Nation, a social loyalty program that lets users earn badges for activities as writing reviews and watching videos and compete for rewards.
Results: 500% increase in customers product reviews 66% increase in site visitors 30% increase in comments reduced marketing costs reduced product support costs
IS IT WORKING?
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 26
MTV Italia MyChart
Purposes: User generated video chart based on various “game dynamics” like avatars, points and leaderboards to drive users from Facebook to TV and to loyalize existing TV audience. Results: More than 500.000 votes in less then three months More than 150.000 videos viewed
COMMONALITIES AND DIFFERENCES
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 27
One or more objec2ves or tasks that users have to accomplish.
A gaming experience, defining challenges to overcome and rewards for their solu2on.
One or more players, the users of the applica2on, who are profiled and monitored in their ac2vi2es.
But...
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 28
INTRODUCTION TO GAME DESIGN
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 29
Mancala, 7th century AD
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
30
Unreal Tournament 3, Epic Games, 2007
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 31
Turn based boardgame vs Real /me ac/on shooter
Handmade physical board vs Personal Computer
Public domain rules vs Copyrighted
1 - PLAYERS
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 32
Number of players Roles of the players Interac2ons among players and the game:
Single Player/Mul2 Single Unilateral/Mul2lateral Compe22on
Coopera2ve Team compe22on
2 - OBJECTIVES
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 33
Objec2ves: specific goals for the player to accomplish
Capture: capture or destroy a resource Chase: catch an opponent or elude one Race: reach a goal before the other players Alignment: arrange game objects in a spa/al or conceptual
configura/on Rescue\Escape: bring a unit to safety Construc2on: build, maintain, manage objects Explora2on: explore the environment where the game take place Solu2on: solve a problem or puzzle before (or more accurately) the compe//on or certain constraints Outwit: gain and use knowledge to defeat other players
2 – OBJECTIVES: EXAMPLES
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 34
Alignment: arrange game objects in a spa2al or conceptual configura2on
Tetris, Alexey Pajitnov, 1984 Puzzle Bobble, Taito Corporation, 1994
2 – OBJECTIVES: EXAMPLES
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 35
Solu2on: solve a problem or puzzle before (or more accurately) than the compe22on or following certain constraints
Connect Four, Milton Bradley, 1974
Professor Layton and the Miracle Mask, Level-5, 2011
3 - PROCEDURES
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 36
Procedures: the methods of play and the ac/ons that players can take to achieve the game objec/ves. They are used to define who does what, where and how.
Typically a game is composed of: Star%ng ac%on: How to put a game into play. Progression of ac%on: Ongoing procedures a_er the star/ng ac/on. Special ac%ons: Available condi/onal to other elements or game state. Resolving ac%ons: Bring gameplay to an end.
3 – PROCEDURES: EXAMPLE
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 37
Star%ng ac%on: Choose a player to go first. Each player chooses a color: red or black.
Progression of ac%on: On each turn, a player drops one colored checker down any of the slots in the top of the grid.
Resolving ac%ons: The play alternates un/l one of the players gets four checkers of one color in a row. The row can be horizontal, ver/cal, or diagonal.
Connect Four
4 - RULES
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 38
• Define Objects • Restrict Ac/ons • Determine Effects (ECA rules) Chess: A player cannot move her king into check. Poker: A straight is five consecu8vely ranked cards; a straight flush is five consecu8vely ranked cards of the same suit. WarCrad II: To create knight units, a player must have upgraded to a keep and built a stable.
5 - RESOURCES
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 39
Resources: valuable objects that can help the players to achieve their goals but are scarce in the system.
Lives: number of “trials” that can be aVempted to reach a goal Units: mul/ple objects that has to be managed instead of lives Health: represent the status of loss or near loss of lives and units Points: numerical value that represents a measure of the skill and progression of a player Ac2ons: number of possible dis/nct choices that a player can make within a defined /meframe Power-‐ups: object that gives a boost to the player Items: used by the player to accomplish an objec/ve, made scarce by the system Turns: the number of game phases within which a player must accomplish the objec/ves Time: restricts player ac/ons or phases in periods of /me.
5 – RESOURCES: EXAMPLE
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 40
Card Hunter, Blue Manchu Pty Ltd, TBR
Time
Points
Ac/ons
6 - CONFLICTS
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 41
Conflicts: emerges from the players trying to accomplish the goals of the game within its rules and boundaries, since procedures and rules tend to deter players from accomplishing goals directly or make players work against each other. The most common conflicts are generated by: Obstacles, objects or rules that limit the freedom of the players Opponents, since they are usually trying to achieve an objec/ve faster than us Meaningful Choices, players have to make choices that will influence the outcome of the game
7 - BOUNDARIES
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 42
Boundaries are what separate the game from everything that is not the game and defines the physical or virtual scopes in which the game is performed.
Example: Football would not be the same game if the boundaries of the football field were not defined.
8 - OUTCOME
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 43
Outcome: the outcome of the game (if present) is uncertain, since it is not possible to predict the results ahead. If an outcome can be obtained it has to be quan/fiable with respect to the defined goals. It is different than the objec/ve since all the player can reach the same objec/ve.
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 44
Derived by modeling the guidelines and best practices found in: Fullerton, T.; Swain, C. & Hoffman, S.
Game Design Workshop: A playcentric approach to creating innovative games, 2008
Crowford, C. The Art of Computer Game Design, 1984
GAME DEVELOPMENT PROCESS
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 45
GWAPS AS SERIOUS GAMES
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 46
Ben
Saw
yer,
Pet
er S
mith
: S
erio
us G
ames
Ta
xono
my,
200
8
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 47
Katharina Siorpaes and Martin Hepp. 2008. OntoGame: weaving the semantic web by online games.
GAMES WITH A PURPOSE: CURRENT ISSUES
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 48
● “Ad-‐Hoc” Design
● Engagement and reten/on mechanisms.
● Mapping from task to game mechanics
● Valida/on techniques and strategies
E.G. Real user comments about OntoGalaxy
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 49
1) PURPOSE
2) TASK DEFINITION
3) GAME MECHANICS DESIGN
4) INSTANCE MATCHING
5) OUTPUT VALIDATION
6) PERFORMANCE EVALUATION
7) ENGAGEMENT SRATEGY
GWAPS DIMENSIONS
GWAP DEVELOPMENT PROCESS
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 50
REQUIREMENTS SPECIFICATION
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 51
uTask Task Description
Input Objects
Output Objects
Execution Interface
Operations Output Validation
INPUT OBJECTS
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 52 Adapted from“Mathematical Modeling of social games, IrwinKing, SIAG2009”
An input object is an object with data t23e and a set of finite a67ibutes denoted as
tex9, images, audio seg=ents, video seg=ents, other unst7@ct@red data, st7@ct@red data
where the data t23e is the media t23e presented by
Each a67ibute has a relationship to a set of values . Each is an object with its own data t23e that represent the metadata of
DEFINING OPERATIONS
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 53
uTask Task Description
Input Objects
Output Objects
Execution Interface
Operations Output Validation
OPERATION TYPES
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 54
They fall in two broad categories: Genera/ve tasks Decision tasks
A possible (non-‐exhaus/ve) list of human computa/on tasks may include: • Object Recogni/on/Iden/fica/on/Detec/on • Sor/ng (Clustering/Ordering) • Natural Language Processing • State Space Explora/on • Content Genera/on/Submission • User preference/opinion elicita/on
ENSURE OUTPUT QUALITY
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 55
uTask Task Description
Input Objects
Output Objects
Execution Interface
Operations Output Validation
Twofold purpose: ensure output correctness and counter player collusion.
OUTPUT VALIDATION
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 56
Player Tes2ng: compare the output provided by the user with known correct outputs
Repe22on:collect more than one solu/on for the same input object, then aggregate the solu/ons
Taboo Output: limit the possible solu/ons that can be submiVed
Random Pairing: pairing players at random
TASK EXECUTION
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 57
uTask Task Description
Input Objects
Output Objects
Execution Interface
Operations Output Validation
TASK TO GAME MECHANICS MATCHING
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 58
Does a game with game mechanics similar to the task exists? If so, integrate the task within the game If not, custom game mechanics has to be implemented. Clear and Transparent game mechanics
SOLUTION MECHANICS
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 59
Pa<ern Recogni2on
Wikipedia stands to knowledge as BoardGameGeek stands to games.
Benng/Wagering
Line-‐Drawing
Tile-‐Placement Pa<ern Building
Memory
Hand Management
Ac2on Programming
Auc2on/Bidding
Partnership
Possible, meaningful game mechanics:
Take Inspira/on!
TILE PLACEMENT
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 60
Tile Placement games feature placing a piece to score points, based on adjacent pieces or pieces in the same group/cluster, taking into consideration non-spatial properties like color, "feature completion", cluster size etc.
PATTERN RECOGNITION APPLIED TO GWAPS
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 61 Kawrykow A, Roumanis G, Kam A, Kwak D, Leung C, et al. (2012) Phylo: A Citizen
Science Approach for Improving Multiple Sequence Alignment
Purpose: Used to arrange the sequences of D.N.A, R.N.A or proteins to iden/fy regions of similarity
Game Mechanic: align the sequence contained in each row in order to obtain the greatest number of columns with matching colors.
LINE DRAWING
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 62
Games that make use of this mechanics involve drawing drawing of lines in one way or another.
LINE DRAWING APPLIED TO GWAPS
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 63
Purpose: Segment images Game Mechanic: draw the shapes of objects in a provided image in order to make the other players guess the underlying object.
MEMORY
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 64
Games that use the Memory mechanic require players to recall previous game events or information in order to reach an objective.
MEMORY APPLIED TO GWAPS
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 65
Purpose: Find similar Images Game Mechanic: Exploit the visual memory of players to find images which are similar.
PARTNERSHIP
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 66
Games with partnerships offer players a set of rules for alliances and teams. Partners are o_en able to win as a team, or penal/es are enforced for not respec/ng alliances.
PARTNERSHIP APPLIED TO GWAPS
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 67
Purpose: Collect “common-‐sense” fact for specific words. Game Mechanic: provide hints to the partner in order to let him guess the secret word
GWAPS VALIDATION MECHANICS
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 68
Bad solu/ons due to: incompetence mischief behavior plain mistakes
Find or adapt game mechanics in order to validate the results.
GWAP FORMAL MODEL
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 69 Based on Mathematical Modeling of social games, Irwin King, SIAG2009
is the set of uTask that are associated with the problem, with the uTask defined as we have previously seen.
is the answer domain; it contains the set of possible cer9ain answers fi (based on the input D) if the out3ut is contained within a defined domain, or empt2 if the answers are provided by the players
is a f@nction that deter=ines if a par9icular out3ut is a valid solution to the problem
is the GWAP Domain, that maps the input to the out3ut of a GWAP
MULTIPLAYER: INPUT AGREEMENT
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 70
Same or different input Based on each other’s hints, player must agree if they are dealing with the same input
Given a correct answer set for the problem
players have a set of poten/al outputs which cardinality influence the probability of agreeing upon the input
Results Submission: One Trial
Suggested for: Subjective Information
Same or different input, same roles
INPUT AGREEMENT: TAG A TUNE
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 71
Same Input: Audio file
Two different shared sets of descrip/ons, one for each player
Result: same input or different input
Results Submission: One Trial
MULTIPLAYER: OUTPUT AGREEMENT
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 72
Same Input Agreeing on common output without communica/on
Same Input, Same Roles.
Given a correct answer set for the problem,
players have a set of poten/al outputs
the probability to obtain an agreement depends on the shared output
Results Submission: Mul/ple Trials Suggested for: Objec/ve Informa/on
OUTPUT AGREEMENT: ESP GAME
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 73
Same Input: Image
List of possible guesses, hidden to the other players
Result: Meaningful tag for the provided image
Results Submission: Mul/ple Trials
MULTIPLAYER: INVERSION PROBLEM
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 74
One players knows the whole problem and gives hints to the other players. The others try to unveil the secrets based on the hints.
Input to just one player
the output of one player is the input for the others
The correct answer set for the problem is unknown
The probability of guessing depends on the cardinality of the hints
Result: User generated content, solu/ons for complex data structures
Results Submission: Mul/ple Trials
INVERSION PROBLEM: IMAGE LABELING GAME
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 75
Describer: Image
Seekers: Textual description given by the describer Results Submission: Multiple trials Results: Meaningful descriptions for the image
INSUFFICIENT PLAYERS: PRERECORDED GAMES
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 76
Fake players based on the logs of previous matches Pros: The gwap can be played anytime Cons: The actions in the game has to be modeled and stored Inversion problem games can be difficult to simulate Requires a bootstrapping phase to acquire the initial traces
SINGLE PLAYER: ALGORITHMIC EVALUATION
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 77
Automa/c algorithm able to par2ally assess the quality of the provided results.
For each instance of the game the input is provided to just one player
The correct answer set for the problem is defined
The probability to obtain meaningful results is applica/on specific,based on the algorithm that is used to make the evalua/on
ALGORITHMIC EVALUATION: TYPE ATTACK
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 78
Jovian
, L.T.; Am
prim
o, O., "O
CR Correc8on
via Hum
an
Compu
ta8o
nal G
ame," S
ystem Sciences (HICSS), 2011 44th
Hawaii Interna
8ona
l Con
ference on
, vol., no., pp.1,10, 4-‐7
Jan. 2011
Improving the results of OCR in digitalizing ar/cles old newspapers.
Human players transcribe snippet of text and their performance is measured by comparing the number of similar words in the player’s output and the text from the library’s OCR transcrip/on.
HYBRID
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 79
Combines the previous approaches Input data based on previous results or user generated content
Inputs to one or more players with same or different roles
Answer set depending on the nature of the problem or provided by a player
The probability of obtaining a solu/on depends on the combina/on of the strategies
HYBRID: SKETCHNESS
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 80
Input: One image to a player Hints to the others
Segments traced by one player, possible tags submiVed by the others
Result: Meaningful tag + segmenta/on
Results Submission: Mul/ple Trials
INSTANCE MATCHING
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 81
How to assign tasks (instances) to players? • Random Strategy: assign a random task
Pros: Easy to implement Cons: Does not take into considera/on player skills
• Past History: assign a task based on past performances
Pros: Can improve the quality Cons: Keeping track of gaming history Performance measures must be defined
• Selec2ve Assignment: assign a specific task based on skills and past performances
Pros: Can improve the quality Cons: Same as Past History Needs profiling of the player to assess skills
PERFORMANCE EVALUTION
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 82
Luis von Ahn and Laura Dabbish. 2008. Designing games with a purpose. Commun. ACM 51, 8
Throughput = average number of problem instances solved per human hour
ALP = average (across all people who play the game) overall amount of time the game will be played by an individual player
Expected contribution = throughput multiplied by ALP.
ENGAGEMENT STRATEGY
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 83
Core activity must be already engaging for the user. Gamification involves defining clear and definitive objectives that last even past the gaming experience, and are either set by, or negotiated between the user and the game along with a progressive path of short and intermediate goals leading to it. Onboarding Retention Virality Social Dimensions Additional Mechanics
POINTS
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 84
Points or Player Scores are a numerical value that represents a measure of the skill of a player. • Immediate and las/ng feedback • External display of progression • May determine the win state • Connec/on between progress in the game and rewards
Galli, L., Fraternali, P. “Achievement Systems Explained“ SGSC2012, Singapore
Werbach, K. & Hunter, D. For the Win: How Game Thinking Can Revolutionize Your Business Wharton Digital Press, 2012
LEADERBOARDS
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 85
A Leaderboard is an ordered list of players based on the scores they have obtained in a specific game or system. • Relates the performance of a player to the others • Fosters compe//on and par/cipa/on • Risky: May be demo/va/ng.
Galli, L., Fraternali, P. “Achievement Systems Explained“ SGSC2012, Singapore
Werbach, K. & Hunter, D. For the Win: How Game Thinking Can Revolutionize Your Business Wharton Digital Press, 2012
ACHIEVEMENTS AND BADGES
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 86
An Achievement is a set of tasks, defined by a designer, for the player to fulfill so to achieve a milestone and track the progress in a system. A Badge is an ar/fact associated to the comple/on of an achievement and given to a player a_er its comple/on, or, in gaming terms, a_er “unlocking the achievement”. • Define goals • Onboarding tool • Visual markers for reputa/on, • Provide las/ng rewards
Galli, L., Fraternali, P. “Achievement Systems Explained“ SGSC2012, Singapore
THIS IS JUST A GLIMPSE OF WHAT GAMIFICATION IS...
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 87
EXAMPLE: ZAMZEE GAMIFICATION
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 88
Personalization Rewards
Challenges Social Status
CASE STUDY: FASHION TREND MINING
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 89
Problem statement: segment fashion images for mining trends based on visual features of garments (e.g. color and texture)
Use case: identifying trends in collections of images of people and garments Applications: retrieving similar garments, inspect clothing trends in image collections, analyzing trends change in the years
Color descriptors
Texture descriptors
coarse (sub-)image similarity
THE FASHION TREND MINING PIPELINE
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 90
Male, 24 Female, 22 Female?, ??
REQUIREMENTS SPECIFICATION
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 91
Task Description: Recognize if a particular garment is present within a picture or define a new one and outline its contours.
Input Objects: A fashion image, an optional tag defining the garment to identify.
Aggregation Strategy: assign a value of 0 to each pixel outside the contour and 1 to each pixel contained within the contour, sum all the contribution and apply a threshold based on the number of players.
Output Data: For each submitted task the game has to provide the contour of the garment within the image (Polyline) and a tag defining the garment that has been segmented
GWAP DESIGN
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 92
Solution Mechanics: Pattern Recognition
Established genre: Draw and Guess
Inversion Problem Mechanic
PLAYERS:Number >=2
Multilateral Competition
Two different roles: Sketcher: has to draw the contours of the stated garment Guesser: has to guess the garment drawn by the sketcher
PLAYER ROLE: SKETCHER
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 93
● The only player to see the low confidence image
● “May” be asked to provide a tag for the image
● Is asked to draw the contour of the object for which the tag is provided within the allotted time
● Goal of the Sketcher is to let the other players guess the tag within a time slot without providing any other hints than the contour
PLAYER ROLE: GUESSER
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 94
● Any other player in the game
● His/Her goal is to guess the object for which the Sketcher has provided the contour
● Not allowed to draw on the whiteboard, just to type in the chat box the probable answer as fast as possible
● Scoring:
● Sketcher: 10 pts + 1 for each guesser
● Guesser: 10 pts to the first, then decreasing down to five
FRAMEWORKS: WEB GAMES
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 95
FRAMEWORKS: GAMIFICATION
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 96
Google Play Game Services
Mozilla Open Badges User Infuser
THE CUBRIK PROJECT
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 97
● CUbRIK is a research project financed by the European Union
● Goals:
● Advance the architecture of multimedia search
● Exploit the human contribution in multimedia search
● Use open-source components provided by the community
● Start up a search business ecosystem
● http://www.cubrikproject.eu/