Paul Ormerod

Centre for the Study of Decision Making Uncertainty, University College London and Non-Equilibrium Social Science (NESS)

www.paulormerod.com

A policy maker’s perspective Jean-Claude Trichet, Governor ECB, November 2010:

“When the crisis came, the serious limitations of existing economic and financial models immediately became apparent. Macro models failed to predict the crisis and seemed incapable of explaining what was happening to the economy in a convincing manner. As a policy-maker during the crisis, I found the available models of limited help. In fact, I would go further: in the face of the crisis, we felt abandoned by conventional tools.

We need to develop complementary tools to improve the robustness of our overall framework. In this context, I would very much welcome inspiration from other disciplines: physics, engineering, psychology, biology.

Bringing experts from these fields together with economists and central bankers is potentially very creative and valuable”.

Background information Multi-disciplinary group at the Centre for the Study of

Decision Making Uncertainty at University College London: psychology, maths, computer science – and economics!

Partly financed by George Soros and the Institute for New Economic Thinking (INET)

Prototype software in use at the Bank of England

Tuckett et al. ‘Conviction Narrative Theory and Directed Algorithmic Text Analysis: Predicting the Evolution of the US and UK Economies’, 2014

Nyman et al. ‘Narratives and emotions in financial systems; Exploiting big data for systemic risk assessment’, 2014

Some key uncertainties in macroeconomics What is the size of the fiscal multiplier?

Neither time series econometrics nor DSGE models have resolved this fundamental policy question

Laury, Lewis, Ormerod (Nat Inst Ec Review,1978) for the UK, the range is 0.5 to 1.2; Ramey (J Ec Lit 2011) range for the US is 0.8 to 1.5; Barro and Redlick QJE 2011 argue it is even less

Why did companies and individuals run down savings in the global financial crisis of the 1930s, but build them up in the next global crisis in the 2000s?

What will be the reaction to phasing out quantitative easing?

A hypothesis to account for this lack of progress ‘The fundamental tool of neoclassical economics is an

objective function that maps the space of all relevantdecision variables into a real scalar’ Harstad and Selten,‘Bounded rationality models’, J. Ec. Literature, June 2013

Models do not capture emotions

Mental states (narratives) remove the one-to-onecorrespondence between input and output

People need to imagine the uncertain future, and theirvisions, their narratives, may differ

A few economic papers try and consider sentiment (e.g.Baker et al. 2013, Romer and Romer 2010, but they usenaive text analysis methodologies)

Making the concept of narratives operational: an illustration (1) The Reuters news archive is preserved for analysis in the form of CSV

files of monthly collections of articles from a variety of publishers. Atthe time of our analysis it contained over 17 million English articlesspanning the period from 1996 onwards

We remove articles with tags SPO (sports), ODD (human interest) orWEA (weather) within the ‘tags’ field

Other ‘tags’ enable UK and US to be identified separately

For each daily collection of articles we compute two emotionalsummary statistics, one for excitement and one for anxiety by applyinga simple word count methodology. We use standard algorithmic textsearch procedures

Making the concept of narratives operational: an illustration (2) Two sets of emotion words, each of size approximately 150,

indicative of the relevant emotions have been defined(Tuckett et al. 2013, Strauss 2013) and validated inpsychology experiments

The word list is invariant across different contexts and databases

In other words, we do not use machine learning techniquesto try and ‘optimise’ the word lists in any given context -the search is directed by the psychological theory

There are often problems of generalisation outside theparticular sample period with optimising approaches

Note that Reuters news stories are meant to be written asneutrally as possible – news not opinions

Reuters

DataData Range Description Abbreviation

Internal MarketCommentary(Bank of England)

January 2000 through July 2010

Daily comments on market events

MCDAILY

BrokerCirculars (Macro view)

January 2008 through June 2013

Low volume prior to June 2010. Primarily weekly economic research reports

BROKER

Reuters News January 1996 through October 2014

Reuters news published in English

RTRS

Some results with the Reuters time series for the US

The RTRS series Granger causes the Baker et al. PolicyUncertainty Index, but not vice versa

The RTRS series Granger causes quarterly real US GDPgrowth, but not vice versa

The RTRS series improves the explanatory power of atime series model for ‘nowcasting’ GDP growth

Clustering (Kaufman and Rousseeuw 1990) of theRTRS and GDP series identifies two clusters (Burnsand Mitchell 1946, Kim and Nelson 1998)

Further illustrative results The algorithms can be ‘pointed’ at a word or phrase

(e.g. ‘liquidity’) and sentiment around the word orphrase computed (different metrics of nearness can beused e.g. same sentence, same paragraph)

The BROKER series Granger causes the changes in theMichigan Consumer Sentiment Index, andoutperforms consensus economic forecasts in out ofsample prediction

Changes in MCDAILY Granger cause changes in the VIX

Liquidity in the US

MCI Predictions – BROKER vs. consensus

Market Commentary (inverted) & VIX

Theoretical considerations (1) In many situations, the rational expectations approach lacks traction

because of model uncertainty

Model uncertainty is pervasive (e.g. Chatfield JRSS 1995)

Onatski and Williams, ECB 2002 “The most damaging source ofuncertainty for a policy maker is found to be the pure model uncertainty,that is the uncertainty associated with the specification of the referencemodel”)

Gilboa et al. J Ec Persp 2008 “the standard expected utility model, alongwith Bayesian extensions of that model, restricts attention to beliefsmodeled by a single probability measure, even in cases where no rationalway exists to derive such well-defined beliefs”

if the environment is changing fairly rapidly, it may not be possible to useBayesian techniques in a meaningful way (Alchian J Pol Ec1950, Rendell etal. Science 2010)

Theoretical considerations (2) Radical uncertainty exists when agents face a high level of model uncertainty.

They do not know how to distinguish between models and so cannot formrational expectations.

An implication of this is that they do not know the probability distributionwhich they are facing

Narratives and emotions are key drivers behind economic and financial activity (e.g., Akerlof & Shiller, 2009; Tuckett, 2011)

Within the context of radical uncertainty, agents act by gaining conviction through the use of narratives

Such conviction narratives (Chong & Tuckett, 2014) must have emotional support to be acted upon – excitement about gain, suppressing doubt and anxiety about loss

Potential pitfalls of Big Data sentiment analysis and prediction Computer science technology has made dramatic advances which

enables very large text data bases to be analysed rapidly

Big Data is big! So from a standard probability theory perspective,there are huge numbers of seemingly significant relationships out there

Algorithmic searches need to be directed by validated theories ofhuman decision making – this reduces the risk of spuriously‘significant’ results

An example is Google Flu (Ormerod et al. arXiv:1408.0699[physics.soc-ph], 2014) – internet searches arise from differentmotivations

Potential advantages of Big Data sentiment analysis and prediction In many situations, especially in the policy world, the past may not be a

very good guide to the future

The future needs to be imagined e.g. ‘We will make money by building theChannel Tunnel’; ‘Building Crossrail will have major benefits for the UKeconomy’; ‘Joining the Euro will be good for us’; ‘the UK will be better offoutside the EU’

Relevant narratives and their emotional content can now be identified witha combination of algorithmic text analysis and social psychological theoryof decision making under uncertainty

A number of potential early warning signals are provided by the directedalgorithmic search of the kinds of unstructured text data bases described

Can be extended to other data bases e.g. Emails, memoranda, minutes,interviews

We are developing a Financial Strain Index