Upload
flora-doyle
View
219
Download
0
Tags:
Embed Size (px)
Citation preview
ResearchResearch
Virtual CurrencyCase Study: Prediction Markets
David M. Pennock
Yahoo! Research
ResearchResearch
Virtual Currency
• What actions can be motivated with virtual currency?
• To what degree of competence?• Case study: Prediction markets• Epilogue: Designing a better virtual
currency
ResearchResearch
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
ResearchResearch
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
ResearchResearch
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.worse... i would not payit would make thingsworse...I would not pay
Not many people wouldbuy it... less answers...
I would die immediately.Nobody will pay...
Keep... the Internet free
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
ResearchResearch
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.QuickTime™ and a
TIFF (LZW) decompressorare needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.worse... i would not payit would make thingsworse...I would not pay
Not many people wouldbuy it... less answers...
I would die immediately.Nobody will pay...
Keep... the Internet free Better (if done right)
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
ResearchResearch
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
ResearchResearch
Things People Do “For Free”
• Provide information• Write software• Label images• Answer questions QuickTime™ and a
TIFF (LZW) decompressorare needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
ResearchResearch
Things People Do “For Free”
• Provide information• Write software• Label images• Answer questions
WIRED MAGAZINE: ISSUE 16.03
ResearchResearch
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.QuickTime™ and a
TIFF (LZW) decompressorare needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
Things People Do “For Free”
• Provide information• Write software• Label images• Answer questions
• Provide predictionsQuickTime™ and a
TIFF (LZW) decompressorare needed to see this picture.
WIRED MAGAZINE: ISSUE 16.03
ResearchResearch
Ask The Crowd
• Choose what you’d like to predict:
• Let people bet on iti.e., Buy or sell share of
$1 if $0 if
I am entitled to:
Bird Flu Outbreak US 2008?(Y/N)
Bird FluUS ’08
Bird FluUS ’08
ResearchResearch
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
http://intrade.com
Screen capture 2008/03/17
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
ResearchResearch
The Wisdom of CrowdsBacked in dollars• What you can say/learn
% chance that• Hillary wins• GOP wins Texas• YHOO stock > 30• Duke wins tourney• Oil prices fall• Heat index rises• Hurricane hits Florida• Rains at place/time
• Where
• IEM, Intrade.com• Intrade.com• Stock options market• Las Vegas, Betfair• Futures market• Weather derivatives• Insurance company• Weatherbill.com
ResearchResearch
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
The Wisdom of CrowdsWith Money Without
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
ResearchResearch
The Widsom of CrowdsBacked in “Points”• HSX.com• Newsfutures.com• InklingMarkets.com• Foresight Exchange• CasualObserver.net• FTPredict.com• Yahoo!/O’Reilly Tech Buzz• ProTrade.com• StorageMarkets.com• TheSimExchange.com• TheWSX.com• Alexadex, Celebdaq, Cenimar, BetBubble, Betocracy, CrowdIQ,
MediaMammon,Owise, PublicGyan, RIMDEX, Smarkets, Trendio, TwoCrowds
• http://www.chrisfmasse.com/3/3/markets/#Play-Money_Prediction_Markets
ResearchResearch
The Wisdom of CrowdsWith Money Without
IEM: 237 Candidates HSX: 489 Movies
1 2 5 10 20 50 100estimate
1
2
5
10
20
50
100
actual
ResearchResearch
The Wisdom of CrowdsWith Money Without
ResearchResearch
Real markets vs. market gamesHSX FX, F1P6
probabilisticforecasts
forecast source avg log scoreF1P6 linear scoring -1.84F1P6 F1-style scoring -1.82betting odds -1.86F1P6 flat scoring -2.03F1P6 winner scoring -2.32
ResearchResearch
Does money matter? Play vs real, head to headExperiment• 2003 NFL Season• ProbabilitySports.com
Online football forecasting competition
• Contestants assess probabilities for each game
• Quadratic scoring rule• ~2,000 “experts”, plus:• NewsFutures (play $)• Tradesports (real $)
• Used “last trade” prices
Results:• Play money and real
money performed similarly• 6th and 8th respectively
• Markets beat most of the ~2,000 contestants• Average of experts
came 39th (caveat)
Electronic Markets, Emile Servan-Schreiber, Justin Wolfers, David Pennock and Brian Galebach
ResearchResearch
0
25
50
75
100
TradeSports Prices
0 20 40 60 80 100NewsFutures Prices
Fitted Value: Linear regression
45 degree line
n=416 over 208 NFL games.Correlation between TradeSports and NewsFutures prices = 0.97
Prices: TradeSports and NewsFutures
Prediction Performance of MarketsRelative to Individual Experts
020406080
100120140160180200220240260280300
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Week into the NFL season
Rank
NewsFutures
Tradesports
0
10
20
30
40
50
60
70
80
90
100
Observed Frequency of Victory
0 10 20 30 40 50 60 70 80 90 100Trading Price Prior to Game
TradeSports: Correlation=0.96NewsFutures: Correlation=0.94
Data are grouped so that prices are rounded to the nearest ten percentage points; n=416 teams in 208 games
Market Forecast Winning Probability and Actual Winning ProbabilityPrediction Accuracy
ResearchResearch
Does money matter? Play vs real, head to head
Probability-Football Avg
TradeSports(real-money)
NewsFutures(play-money)
DifferenceTS - NF
Mean Absolute Error
= lose_price
[lower is better]
0.443
(0.012)
0.439
(0.011)
0.436
(0.012)
0.003
(0.016)
Root Mean Squared Error
= ?Average( lose_price2 )
[lower is better]
0.476
(0.025)
0.468
(0.023)
0.467
(0.024)
0.001
(0.033)
Average Quadratic Score
= 100 - 400*( lose_price2 )
[higher is better]
9.323
(4.75)
12.410
(4.37)
12.427
(4.57)
-0.017
(6.32)
Average Logarithmic Score
= Log(win_price)
[higher (less negative) is better]
-0.649
(0.027)
-0.631
(0.024)
-0.631
(0.025)
0.000
(0.035)
Statistically:TS ~ NFNF >> AvgTS > Avg
ResearchResearch
Discussion
• Are incentives for virtual currency strong enough?• Yes (to a degree)• Conjecture: Enough to get what people already know;
not enough to motivate independent research• Reduced incentive for information discovery possibly
balanced by better interpersonal weighting
• Statistical validations show HSX, FX, NF are reliable sources for forecasts• HSX predictions >= expert predictions• Combining sources can help
ResearchResearch
A Problem w/ Virtual CurrencyPrinting Money
Alice1000
Betty1000
Carol1000
ResearchResearch
A Problem w/ Virtual CurrencyPrinting Money
Alice5000
Betty1000
Carol1000
ResearchResearch
YootlesA Social Currency
Alice0
Betty0
Carol0
ResearchResearch
YootlesA Social Currency
I owe you 5
Alice-5
Betty0
Carol5
ResearchResearch
YootlesA Social Currency
credit: 5 credit: 10
I owe you 5
Alice-5
Betty0
Carol5
ResearchResearch
YootlesA Social Currency
credit: 5 credit: 10
I owe you 5 I owe you 5
Alice-5
Betty0
Carol5
ResearchResearch
YootlesA Social Currency
credit: 5 credit: 10
I owe you 5 I owe you 5
Alice3995
Betty0
Carol5
ResearchResearch
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
YootlesA Social Currency• For tracking gratitude among friends• A yootle says “thanks, I owe you one”
ResearchResearch
New Prediction Game
ResearchResearch
• A. Ghosh, M. Mahdian, D. Reeves, D. Pennock, R. Fugger (2007). Mechanism design on trust networks, Workshop on Internet and Network Economics
• E. Servan-Schreiber, J. Wolfers, D.M. Pennock, B. Galebach (2004). Does money matter?, Electronic Markets
• D.M. Pennock, S. Lawrence, C.L. Giles, F.Å. Nielsen (2001). The real power of artificial markets. Science, 291(5506), pp. 987-988
• D.M. Pennock, S. Lawrence, F.Å. Nielsen, C. Lee Giles (2001). Extracting collective probabilistic forecasts from web games, Conference on Knowledge Discovery and Data Mining, pp. 174-183
Talk Bibliography