Upload
lyngoc
View
223
Download
5
Embed Size (px)
Citation preview
1
HARNESSING THE WISDOM OF
CROWDS
Z h i D a , U n i v e r s i t y o f N o t r e D a m e
X i n g H u a n g , M i c h i g a n S t a t e U n i v e r s i t y
S e c o n d A n n u a l N e w s & F i n a n c e C o n f e r e n c e
M a r c h 8 , 2 0 1 7
2
Many important decisions in l i fe are
made in a group
F O M C m e e t i n gB o a r d m e e t i n g
J u r y
“Who wants to be a millionaire”
A s k t h e a u d i e n c e
3
Wisdom of crowdsJ e l l y b e a n s i n t h e j a r e x p e r i m e n t
Ask a group of people to guess:
How many jelly beans in the jar?
A large group’s average answer to a question involving
quantity estimation is generally as good as, and often
better than, the answer provided by any individual in
that group
Law of large number requires independence
94
13599
77
67
97
112
102
59161
68
78
47
126
80
106
95 jelly beans
4
CONS: “MEN THINK IN HERDS”
Herding can reduce the accuracy of a group’s average answer
“The more influence we exert on each other, the more likely it is that we will believe the same things and make the same mistakes. That means it’s possible that we could become individually smarter but collectively dumber.”
---James Surowiecki, The Wisdom of Crowds.
Sequential sett ing: Pros vs. Cons
95
5“None of us is as dumb as al l of us.”
6
CONS: “MEN THINK IN HERDS”
Herding can reduce the accuracy of a group’s average answer
“The more influence we exert on each other, the more likely it is that we will believe the same things and make the same mistakes. That means it’s possible that we could become individually smarter but collectively dumber.”
---James Surowiecki, The Wisdom of Crowds.
Additional information production may improve the accuracy of private signals.
We become both individually smarter and collectively smarter.
PROS:GENERATE ADDITIONAL INFORMATION PRODUCTION
Sequential sett ing: Pros vs. Cons
95
7
WE STUDY THE QUESTION:
WE QUANTIFY THE IMPACT OF HERDING ON ECONOMIC OUTCOMES
Do individuals herd in a sequential setting and reduce the usefulness of information aggregated across individuals (i.e., the wisdom of crowds)?
The empirical challenge is that individual’s information set is usually unobservable
We overcome the empirical challenge by directly measuring and randomizing on individual’s information set
In this paper
8
WHY IT’S A GOOD SETTING?
• Forecasts and realizations are clearly measured and easily observable• The forecasters do not have direct influence on realizations• Corporate earnings are of crucial importance
Our sett ingA c r o w d - b a s e d e a r n i n g s f o r e c a s t p l a t f o r m
( E s t i m i z e . c o m )
9
M A I N A N A LY S I S
• Estimize.com related• Sample statistics
1D AT A A N D S T AT I S T I C S
2
• Herding behavior• The influence of herding on
forecast accuracyo Individual forecasto Consensus forecast
I N F L U E N T I A L U S E R A N D H E R D I N G
• Herding behavior• The influence of herding on
forecast accuracyo Individual forecasto Consensus forecast
3R A N D O M I Z E DE X P E R I M E N T
4
• Are users herding more with “influential” users?
• Does herding lead to return predictability
Roadmap
10
11
Open web-based platform founded in 2011, where users can make earnings forecasts
Diverse user group: buy-side | sell-side | independent analysts | other working professionals | students
Estimize consensus is more accurate than WS | IBES consensus, also complementary
• Jame et. al. (2016) | Adebambo and Bliss (2015)• Now available on Bloomberg
Various incentives for making forecasts
• Competition, monetary and professional prizes• Reputation, useful track record• Altruism, social preference
Estimize.com
Scoring system [-25, 25]:- Positive if more accurate than WS
consensus- Awarded on an exponential scale,
which encourages independent opinion
12
Sample period for :
• Main analysis: 2012/03 - 2015/03• Randomized experiment: Q2 and Q3 in 2015
Ticker-quarter-forecast panel:
• 2147 quarterly earnings • 2516 users covering • 730 stocks: mostly large-growth
An average release (a sequence of forecasts for a ticker-quarter):
• 20 forecasts from 16 users
User activity on Estimize.com is tracked by Mixpanel
• Events: release page, estimate page, submit estimates, etc.• Timestamp, location, device, etc.
Sample statistics
13Release page view
Viewing activity = 1: If a user spent more than 5 seconds on the release page before making her own forecast
14
M A I N A N A LY S I S
• Estimize.com related• Sample statistics
1D AT A A N D S T AT I S T I C S
2
• Herding behavior• The influence of herding on
forecast accuracyo Individual forecasto Consensus forecast
I N F L U E N T I A L U S E R A N D H E R D I N G
• Herding behavior• The influence of herding on
forecast accuracyo Individual forecasto Consensus forecast
3R A N D O M I Z E DE X P E R I M E N T
4
• Are users herding more with “influential” users?
• Does herding lead to return predictability
Roadmap
15
Note: Fixed effects subsume the need to control for stock or user characteristicsClustered standard errors account for autocorrelations in forecast errors
Herding behavior
Information weighting regression, Chen and Jiang (2006, RFS)
FE 0Dev 0 0 : overweight on public information0 0 : overweight on private information
I n d i v i d u a l s h e r d m o r e w h e n t h e y v i e w o t h e r s ’ f o r e c a s t s
More weight on the consensus forecast after viewing the release page
16
Influence of herding on forecast accuracy• Herding makes individual forecast more accurate • but reduces the accuracy of the consensus forecast
17
Note: Nonzero Views = 1: if a user spent more than 5 seconds on the release page before making her own forecast
CTA dummy = 1: if the forecast is submitted during the last three days before announcements
Individual’s absolute forecast error decreases after viewing release page
Influence of herding on forecast accuracyH e r d i n g m a k e s i n d i v i d u a l f o r e c a s t m o r e a c c u r a t e
18
Note: LnNumView = ln (1+ the percentage of forecasts with release views)Std Dev(FE)/Abs(Median(FE)) controls for uncertainty
Consensus’s absolute forecast error increases as there are more viewing activities within a release
Magnitude: 0.0551 X ln(1+1) = 3.82 centsThis is more than the distance between the perfect forecast and the forecast with median Abs(FE) (3 cents)
The consensus of group without viewing activity wins more than 50% with statistical significance
Influence of herding on forecast accuracyH e r d i n g m a k e s c o n s e n s u s f o r e c a s t l e s s a c c u r a t e
19
Note: Consistent bias indicator = 1: if the bias in both early period and CTA period are in the same direction.
LnNumView = ln (1+ the percentage of forecasts with release views)
More viewing activity in the close-to-announcement period, the biases in these two periods are more likely to be consistent.
Influence of herding on bias persistence
Early period Close-to-announcement period
20
VIEWING ACTIVITY MAY BE ENDOGENOUS
• Less informed users are more likely to view others’ forecasts.• Consistent with less accurate consensus.• But inconsistent with more accurate individual forecast.
ADDRESS THIS CONCERN:
• randomizing users’ information sets
Endogeneity concern
21
M A I N A N A LY S I S
• Estimize.com related• Sample statistics
1D AT A A N D S T AT I S T I C S
2
• Herding behavior• The influence of herding on
forecast accuracyo Individual forecasto Consensus forecast
I N F L U E N T I A L U S E R A N D H E R D I N G
• Herding behavior• The influence of herding on
forecast accuracyo Individual forecasto Consensus forecast
3R A N D O M I Z E DE X P E R I M E N T
4
• Are users herding more with “influential” users?
• Does herding lead to return predictability
Roadmap
22
RANDOMIZED EXPERIMENTS
• Pilot round (Q2 in 2015): 13 stocks are randomly selected• Second round (Q3 in 2015): 90 stocks are randomly selected
WHAT WE DO
• Randomly select users and disable the release page • Ask them to make earnings forecasts (blind forecasts)• Afterwards, the release page is restored • They can immediately revise their forecasts (revised forecasts)• Others not selected still view the original release page and make forecasts (default forecasts)
Bl ind experiments
23Blind view
24
Note: Estimate view, Avg #releases, #tickers, and abs(FE) are based on users’ forecasts before the experiment
Users in blind and default group are similar in observable user characteristics.
Blind vs. DefaultU s e r c h a r a c t e r i s t i c s
25
HERDING BEHAVIOR
Default forecasts put more weight on Estimizeconsensus relative to blind forecasts
Blind vs. DefaultH e r d i n g a n d c o n s e n s u s a c c u r a c y
CONSENSUS ACCURACY (HORSE RACE)
• Blind consensus significantly beats default consensus 56.3% of the time• Default consensus significantly beats blind consensus 36.9% of the time
Placebo test with concurrent WS consensus
26
Note: This test only includes releases in the pilot round, because only in the pilot round, most users revise their forecasts immediately after the release page is restored.
The blind consensus is much more accurate than the revised consensus
Blind vs. RevisedC o n s e n s u s f o r e c a s t a c c u r a c y
Information weighting regression
27
M A I N A N A LY S I S
• Estimize.com related• Sample statistics
1D AT A A N D S T AT I S T I C S
2
• Herding behavior• The influence of herding on
forecast accuracyo Individual forecasto Consensus forecast
I N F L U E N T I A L U S E R A N D H E R D I N G
• Herding behavior• The influence of herding on
forecast accuracyo Individual forecasto Consensus forecast
3R A N D O M I Z E DE X P E R I M E N T
4
• Are users herding more with “influential” users?
• Does herding lead to return predictability
Roadmap
28
Note: page rank is also how Google ranks websites.
Measuring user influenceP a g e r a n k a l g o r i t h m b a s e d o n v i e w i n g a c t i v i t i e s
D views A’s estimate
B and C are influential users in this network
• Viewed by many users in general• Viewed by another influential user
Factors behind user influence:
29
Note: Influenced = 1 if the number of influential users ahead of the observed user is above the 80th percentile across all observations
Individuals put more weight on the consensus if it contains the forecasts of influential users
Influential user and herdingI n f o r m a t i o n w e i g h t i n g r e g r e s s i o n
Robustness check using data from the experiment
30
Predict ing forecast errors of inf luent ial users
SENTIMENT-BASED MEASURE
• Influential users’ revision: upward | downward • Contemporaneous firm returns: positive | negative
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
31
Market does not seem to undo the optimism bias completely and is negatively surprised on average at the earnings announcement
Predicting earnings announcement returns
32
Herding improves the accuracy of individual forecast but reduces the accuracy of consensus forecast
Wisdom of crowds can be better harnessed by encouraging independent voice
Conclusions
Aftermath:
Our experiment results convinced Estimize.com to
switch to a “blind” platform in November 2015