35
Advanced Analytics at the Bank of England Presentation to the Riksbank conference on Big Data: Building Strategies for Central Banks in Light of the Data Revolution 9 September 2015

Advanced Analytics at the Bank of England...• More data, increased computing power, technical advances –Change of circumstances • Lessons from the financial crisis –Change

  • Upload
    others

  • View
    1

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Advanced Analytics at the Bank of England...• More data, increased computing power, technical advances –Change of circumstances • Lessons from the financial crisis –Change

Advanced Analytics at the Bank of

England

Presentation to the Riksbank conference on Big Data: Building Strategies

for Central Banks in Light of the Data Revolution

9 September 2015

Page 2: Advanced Analytics at the Bank of England...• More data, increased computing power, technical advances –Change of circumstances • Lessons from the financial crisis –Change

Outline

• Why are we interested?

• What are we interested in?

– Matched micro data sets

– Text mining

– Visualisation

– Machine learning

• But… there are no free lunches, so what’s the bill?

Advanced Analytics at the Bank of England

2

Page 3: Advanced Analytics at the Bank of England...• More data, increased computing power, technical advances –Change of circumstances • Lessons from the financial crisis –Change

Why are we interested in Big Data?

• What do we mean by the term?

– Fuzzy meaning, covering data, techniques and attitude

• Why are we interested?

– Change of responsibilities

• The arrival of the PRA

– Change of opportunity

• More data, increased computing power, technical advances

– Change of circumstances

• Lessons from the financial crisis

– Change of philosophy

• Inductive vs deductive reasoning

Advanced Analytics at the Bank of England

3

Page 4: Advanced Analytics at the Bank of England...• More data, increased computing power, technical advances –Change of circumstances • Lessons from the financial crisis –Change

Why are we interested in Big Data?

• What do we mean by the term

– Very loose meaning, covering data, techniques and attitude

• Why are we interested?

– Change of responsibilities

• The arrival of the PRA

– Change of opportunity

• More data, increased computing power, technical advances

– Change of circumstances

• Lessons from the financial crisis

– Change of philosophy

• Inductive vs deductive reasoning

Advanced Analytics at the Bank of England

4

Page 5: Advanced Analytics at the Bank of England...• More data, increased computing power, technical advances –Change of circumstances • Lessons from the financial crisis –Change

Why are we interested in Big Data?

• What do we mean by the term

– Very loose meaning, covering data, techniques and attitude

• Why are we interested?

– Change of responsibilities

• The arrival of the PRA

– Change of opportunity

• More data, increased computing power, technical advances

– Change of circumstances

• Lessons from the financial crisis

– Change of philosophy

• Inductive vs deductive reasoning

Advanced Analytics at the Bank of England

5

Page 6: Advanced Analytics at the Bank of England...• More data, increased computing power, technical advances –Change of circumstances • Lessons from the financial crisis –Change

Why are we interested in Big Data?

• What do we mean by the term

– Very loose meaning, covering data, techniques and attitude

• Why are we interested?

– Change of responsibilities

• The arrival of the PRA

– Change of opportunity

• More data, increased computing power, technical advances

– Change of circumstances

• Lessons from the financial crisis

– Change of philosophy

• Inductive vs deductive reasoning

Advanced Analytics at the Bank of England

6

Page 7: Advanced Analytics at the Bank of England...• More data, increased computing power, technical advances –Change of circumstances • Lessons from the financial crisis –Change

Why are we interested in Big Data?

• What do we mean by the term

– Very loose meaning, covering data, techniques and attitude

• Why are we interested?

– Change of responsibilities

• The arrival of the PRA

– Change of opportunity

• More data, increased computing power, technical advances

– Change of circumstances

• Lessons from the financial crisis

– Change of philosophy

• Inductive vs deductive reasoning

Advanced Analytics at the Bank of England

7

Page 8: Advanced Analytics at the Bank of England...• More data, increased computing power, technical advances –Change of circumstances • Lessons from the financial crisis –Change

Is Correlation the New Causality?

Karl Popper (Source: http://en.wikipedia.org/wiki/Karl_Popper)

Hal Varian (Source: http://en.wikipedia.org/wiki/Hal_Varian)

15

Advanced Analytics at the Bank of England

Page 9: Advanced Analytics at the Bank of England...• More data, increased computing power, technical advances –Change of circumstances • Lessons from the financial crisis –Change

What are we interested in?

• Gaining a richer understanding of the phenomenon of interest

– Can help disentangle cause and effect…

– …and identify the underlying issue that needs to be addressed

• Getting a speedier reading of developments in the economy and

financial system

– ‘Nowcasting’ and ‘nearcasting’

– This might be particularly important when the system is undergoing

rapid changes

• Quantifying previously purely qualitative data

– Eg text

Advanced Analytics at the Bank of England

9

Page 10: Advanced Analytics at the Bank of England...• More data, increased computing power, technical advances –Change of circumstances • Lessons from the financial crisis –Change

Matched micro data sets

Advanced Analytics at the Bank of England

10

Page 11: Advanced Analytics at the Bank of England...• More data, increased computing power, technical advances –Change of circumstances • Lessons from the financial crisis –Change

Loan-to-income multiple ≥ 4.5

Source: Data are based on the Bank of England’s internal Product Sales Database collected by the FCA.

Advanced Analytics at the Bank of England

Page 12: Advanced Analytics at the Bank of England...• More data, increased computing power, technical advances –Change of circumstances • Lessons from the financial crisis –Change

Sources: WhenFresh (Zoopla listings), Land Registry Price Paid, Land Registry Cash/Mortgage data, FCA Product Sales Data on mortgages, ONS

Postcode Directory.

Advanced Analytics at the Bank of England

Page 13: Advanced Analytics at the Bank of England...• More data, increased computing power, technical advances –Change of circumstances • Lessons from the financial crisis –Change

Sources: WhenFresh (Zoopla listings), Land Registry Price Paid, Land Registry Cash/Mortgage data, FCA Product Sales Data on mortgages, ONS

Postcode Directory.

Advanced Analytics at the Bank of England

Page 14: Advanced Analytics at the Bank of England...• More data, increased computing power, technical advances –Change of circumstances • Lessons from the financial crisis –Change

Advanced Analytics at the Bank of England

•14

Page 15: Advanced Analytics at the Bank of England...• More data, increased computing power, technical advances –Change of circumstances • Lessons from the financial crisis –Change

Highly granular data sets

Advanced Analytics at the Bank of England

15

Page 16: Advanced Analytics at the Bank of England...• More data, increased computing power, technical advances –Change of circumstances • Lessons from the financial crisis –Change

Advanced Analytics at the Bank of England •16

EMIR Data

Positions in

outstanding CHF-

denominated FX

derivatives

positions on

15/1/15

Page 17: Advanced Analytics at the Bank of England...• More data, increased computing power, technical advances –Change of circumstances • Lessons from the financial crisis –Change

Text Analytics

Advanced Analytics at the Bank of England

17

Page 18: Advanced Analytics at the Bank of England...• More data, increased computing power, technical advances –Change of circumstances • Lessons from the financial crisis –Change

4

5

6

7

8

9

10

Google index of JSA

Unemployment rate

Percent of

labour force

Source: ONS; Google. Notes: The Google indices are mean and variance adjusted to put on the same scale as the unemployment rate and wage growth. The Google

indices are drawn from searches containing the terms “salaries” and “job seekers allowance”. See Mclaren and Shanbhogue (2011) for further details.

Googling the Labour Market

-8

-6

-4

-2

0

2

4

6

8

10

Google index of salaries

Wage growth

Percent year

on year

20

Advanced Analytics at the Bank of England

Page 19: Advanced Analytics at the Bank of England...• More data, increased computing power, technical advances –Change of circumstances • Lessons from the financial crisis –Change

Advanced Analytics at the Bank of England

19

Page 20: Advanced Analytics at the Bank of England...• More data, increased computing power, technical advances –Change of circumstances • Lessons from the financial crisis –Change

Advanced Analytics at the Bank of England

20

Page 21: Advanced Analytics at the Bank of England...• More data, increased computing power, technical advances –Change of circumstances • Lessons from the financial crisis –Change

Advanced Analytics at the Bank of England 21

Page 22: Advanced Analytics at the Bank of England...• More data, increased computing power, technical advances –Change of circumstances • Lessons from the financial crisis –Change

Advanced Analytics at the Bank of England

22

Page 23: Advanced Analytics at the Bank of England...• More data, increased computing power, technical advances –Change of circumstances • Lessons from the financial crisis –Change

Advanced Analytics at the Bank of England

23

Page 24: Advanced Analytics at the Bank of England...• More data, increased computing power, technical advances –Change of circumstances • Lessons from the financial crisis –Change

Visualisation

Advanced Analytics at the Bank of England

24

Page 25: Advanced Analytics at the Bank of England...• More data, increased computing power, technical advances –Change of circumstances • Lessons from the financial crisis –Change

Asset class

correlation

heatmap

Advanced Analytics at the Bank of England

•26

Page 26: Advanced Analytics at the Bank of England...• More data, increased computing power, technical advances –Change of circumstances • Lessons from the financial crisis –Change

BoE communication

21

Advanced Analytics at the Bank of England

Page 27: Advanced Analytics at the Bank of England...• More data, increased computing power, technical advances –Change of circumstances • Lessons from the financial crisis –Change

Machine learning

Advanced Analytics at the Bank of England

28

Page 28: Advanced Analytics at the Bank of England...• More data, increased computing power, technical advances –Change of circumstances • Lessons from the financial crisis –Change

Issues with analysing ‘Big Data’

• Example: CPI micro-data

• The ONS has produced a data set comprising:

– 215 months (Feb 1996-Dec 2013)

– ~110,000 prices collected per month (not the same number each

month)

– 1,113 items (not the same items each year)

– 71 COICOP classes

– various other meta-data (eg type of shop, region etc)

– in total: 24,442,988 records with 25 fields

– 611,074,700 pieces of data

Advanced Analytics at the Bank of England

29

Page 29: Advanced Analytics at the Bank of England...• More data, increased computing power, technical advances –Change of circumstances • Lessons from the financial crisis –Change

Issue 1: the stability of annual inflation

21

0

1

2

3

4

5

6

1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2009 2010 2011 2012 2013 2014

UK CPI inflation 12m ratePercentage change over 12 months

Advanced Analytics at the Bank of England

Page 30: Advanced Analytics at the Bank of England...• More data, increased computing power, technical advances –Change of circumstances • Lessons from the financial crisis –Change

Issue 2: over-fitting: what to do?

• T is normally >> N; here N >>T (indeed this is occasionally used

as a definition of ‘big data’ by statistically-minded analysts)

• Aggregate the data? – eg by type of good

– But that tends to obviate the point of using the micro data

• Shrink the dimensionality of the matrix of explanatory variables:

– PCA/factor models extract combinations of the variables that explain

most of the variance of the dependent variable

– So you can keep much of the information that is contained in the

data set without getting into large statistical problems

– A key issue here though is how to interpret the resulting model, and

the components/factors may be unstable over time

• Penalised regressions

Advanced Analytics at the Bank of England

31

Page 31: Advanced Analytics at the Bank of England...• More data, increased computing power, technical advances –Change of circumstances • Lessons from the financial crisis –Change

Issue 3: explaining non-linear functions

21

PLUMBERDAYTIME_HOURLY_RATE

≤106.9386 >106.9386

ORANGECLASS_1EACH

≤104.6805 > 104.6805

CANNED_FISHTUNA180200G

≤102.279 > 102.279

WOMENS_NIGHTDRESSPYJAMAS

≤93.3806 > 93.3806

WINDOWCLEAN_3BED_SEMI

≤101.5122 > 101.5122

WASHING_POWDER_AUTOMATIC

≤99.0384 > 99.0384

DOOR_HANDLEPACK

≤109.8218 > 109.8218

Advanced Analytics at the Bank of England

• Try explaining the intuition behind this relationship to busy policy

makers…

Page 32: Advanced Analytics at the Bank of England...• More data, increased computing power, technical advances –Change of circumstances • Lessons from the financial crisis –Change

Issue 4: Stability

21

0%

2%

4%

6%

8%

10%

12%

14%

16%

18%

20%

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29

% o

f th

e t

ota

l nu

mb

er

of

test

cas

es

Run number

% false positives over 30 random samples

0%

10%

20%

30%

40%

50%

60%

70%

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29

% o

f tr

ue

po

siti

ves

Run number

Positives correctly identified over 30 random samples

• An issue that is closely linked to over-fitting is the stability of the

models

• This is a particularly important issues when there is no strong a

priori reason to think that the world works in this way

• (Though a priori thinking can also be misleading at times)

Advanced Analytics at the Bank of England

Page 33: Advanced Analytics at the Bank of England...• More data, increased computing power, technical advances –Change of circumstances • Lessons from the financial crisis –Change

Issue 5: Confidentiality / ‘Big Brother’ state

Advanced Analytics at the Bank of England

• This was not relevant to the CPI work

• In general, the more detailed and granular the data set is, the

more likely it is to contain confidential information

• We must ensure that:

– we only use data for appropriate reasons

– the minimum number of people are able to see any confidential data

given the needs of the situation

– data are stored securely and professionally

35

Page 34: Advanced Analytics at the Bank of England...• More data, increased computing power, technical advances –Change of circumstances • Lessons from the financial crisis –Change

Issue 6: Practical issues

Advanced Analytics at the Bank of England

• Hiring!

• IT!

36

Page 35: Advanced Analytics at the Bank of England...• More data, increased computing power, technical advances –Change of circumstances • Lessons from the financial crisis –Change

Conclusion

• Do these issues mean that ‘Big Data’ is likely to be a passing

fad?

– No!

• The data exist and the BoE has the responsibility and opportunity

to use them to help us understand economic developments and

the structure of the economy and financial system

• But the issues do mean that this is no panacea

• Just as with any other empirical work the data need to be

cleaned and understood (both of which are more difficult with

larger data sets) prior to analysis

• And then analysed carefully using appropriate methods

Advanced Analytics at the Bank of England

37