Content Analysis of the IMF Article IV Staff Reports for Euro Area

Countries

Content Analysis of the IMF Article IV Staff Reports for Euro Area

Countries

Lena Golubovskaja, NUIM

April 14, 2011

IEA ANNUAL CONFERENCE

Outline

• Introduction

• Theoretical Developments

• Data and Methodology

• Analysis of the Data

• Concluding Remarks

• In the United States “core commercial and investment banks are in a sound financial position, and systemic risks appear low” (IMF, 2007:14)

• “Favorable global economic prospects, particularly strong momentum in the Euro area and in emerging markets led by China and India, continue to serve as a strong foundation for global financial stability…” (GFSR)

Problem

The IMF’s Judgement

EMU Fiscal Policies

Was the miracle a ‘mirage’?• Some northern countries (Germany,

Austria, the Netherlands) experienced competitiveness gains and accumulated huge external surpluses.

• Some southern countries (Spain, Portugal, Greece) accumulated huge external deficits under imbalanced high growth strategies driven by strong negative real interest rates.

Case Study: Ireland

• Large transitory tax receipts used to fund tax cuts and permanent spending commitments

• Budget surpluses were insufficiently large

• Excessive current spending growth

• Narrowed tax base

• Policy stance was inappropriately procyclical

Chart plots change in budget balance (x-axis) and in gross debt (y-axis), as %of GDP between 2007 and 2009. Source: Public Finances in EMU, EC 2009

-10.0

0.0

10.0

20.0

30.0

40.0

-15.0 -10.0 -5.0 0.0 5.0

IRE

LAT

ESP

UK

Deterioration in Fiscal Positions: EU Members, 2007-09

-10.0

0.0

10.0

20.0

30.0

40.0

-15.0 -10.0 -5.0 0.0 5.0

IRE

LAT

ESP

UK

Theoretical Developments

•Corporate earnings releases (Rogers at al., 2009)

• Accounting policy disclosures (Levine and Smith, 2006)

•Multiple sources of financial text (Kothari, Li, and Short, 2008)

•Asymmetric Effect of the Beige Book Language (Armesto et al., 2006)

Textual analysis using computer algorithms

Economic implications of the external policy

reports

The accuracy of information (Batchelor, 2001)

Pellechio and Cady (2005)

‘Anecdotical’ Evidence: Opinions of the IMF executive Board, Ireland

• “The continued impressive performance of the Irish economy, which is based on sound economic policies, provides useful lessons for other countries” (2004)

• “Impressive performance is due in significant measure to sound economic policies, including prudent fiscal policies… The housing market is likely headed for a soft landing, but there is a risk of a sharp decline.” (2005)

• “A number of directors, however, saw less merit in fiscal tightening at the current juncture, pointing to the need for further increases in spending to achieve social goals” (2006)

• “The directors commended Ireland’s continued impressive economic performance. this performance has been underpinned by outward-oriented policies, prudent fiscal policy, low taxes and labour market flexibility” (2007)

Content Analysis

1999 2000 2001 2002 2003 2004 2005 2006 2007 2009

Inappropriate fiscal stance

X X X X X X

Inflexibility of NWAs X X X

Declining competitiveness

X X X X X

House price overvaluation

X X X

Unwinding of construction boom

X X X X

Unbalanced growth X X

Vulnerability to external shocks

X X X

Vulnerability of banking system

X X X

Score 4 1 2 2 2 1 3 4 3 7

Concerns/Risk Factors Identified by IMF (IV), Ireland

Limitations*Author calculations

DICTION 5.0

• Dictionary-based program that counts types of words most frequently encountered in contemporary American public discourse and is designed to capture the linguistic style.

• Uses 10,000 search words assigned to 35 theoretically-based linguistic categories to characterize text on several dimensions.

• Advantages: pre-existing search rules, efficient analysis of large number of text, partial correction for the context.

• Disadvantage: assumes that higher frequency usages of a word or phrase mean the concept is more meaningful.

Content Analysis

DICTION 5.0: thematic categories and sub-categories

Content Analysis

DICTION 5.0: Hardship Dictionary

• Bias, Corrupting, Crime, Crises, Deficit, Depress, Deteriorate, Difficulties, Distress, Disturbance, Folly, Problem, Recession, Risk, Shock, Weakness, Worry, Worse…

Content Analysis

Data and Methodology

Data• 16 euro area countries• Staff Reports, IV (2005-2007)• Executive Board Assessments , PIN (2005-

2007)

Sample construction• Only textual part of the documents

Mean of 5.86 and standard deviation of 4.64 based on 122 runs of a variety of financial news stories.

Validation of WARNING measure

Year Score Z-score t-test p-value

2005 4.750 -0.240 2.6423 0.0093

2006 7.210 0.290 -3.2136 0.0017

2007 6.630 0.160 -1.8330 0.0693

2008 6.090 0.050 -0.5475 0.5850

2009 13.94 1.750 -19.2342 0.0000

Case of U.S., Staff Reports (IV)

WARNING: Euro AreaYear IV PIN t-test p-value

1999 4.480 3.669 0.9665 0.3448

2000 5.644 2.481 4.6893 0.0001

2001 5.352 3.178 2.5647 0.0167

2002 6.923 3.814 2.4581 0.0216

2003 6.111 3.430 2.7135 0.0117

2004 4.584 3.000 2.3867 0.0249

2005 5.404 3.662 2.4122 0.0235

2006 7.110 3.897 3.4015 0.0027

2007 6.309 2.975 3.6082 0.0013

2008 7.146 2.758 4.2968 0.0006

2009 9.423 4.395 6.1878 0.0000

Total 6.155 3.399 10.4324 0.0000

WARNING, Staff Reports

Average WARNING scores for the euro area: Staff Reports, 2005-2007

Insignificantly different results for Finland, France, Germany, Ireland, the Netherlands, and Slovakia

0.000

1.000

2.000

3.000

4.000

5.000

6.000

7.000

8.000

9.000

Austri

a

Belgium

Cypru

s

Finla

nd

Fran

ce

Germ

any

Greec

e

Irelan

dIta

ly

Luxe

mbo

urg

Malt

a

Nethe

rland

s

Portu

gal

Slovak

ia

Sloven

iaSpa

in

Sc

ore

Did WARNING score change over time for EMU? Staff Reports, 1999-2009

Year N Mean Z-score Std deviation t-test p-value

1999 9 4.480 -0.298 1.138

2000 14 5.644 -0.047 2.447 -1.3294 0.1980

2001 13 5.352 -0.110 2.459 0.3091 0.7598

2002 13 6.923 0.228 3.893 -1.2302 0.2306

2003 14 6.111 0.053 2.853 0.6215 0.5399

2004 13 4.584 -0.276 2.055 1.5846 0.1256

2005 14 5.404 -0.099 1.538 -1.1797 0.2492

2006 11 7.110 0.268 2.235 -2.2604 0.0336

2007 14 6.309 0.096 2.076 0.9261 0.3640

2008 9 7.146 0.276 2.809 -0.8224 0.4201

2009 10 9.423 0.767 1.877 -2.0982 0.0511

Total 134 6.155 2.661

WARNING, Staff Reports

WARNING for Ireland is not statistically different from the euro area averages only for the years 2002, 2003, and 2006

0

2

4

6

8

10

12

1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009

Sco

re WARNING, euro area

WARNING, Ireland

Summary

•Do not allow editing by member-country

•The language of surveillance needs to be unambiguous

•More weight on analysis, less on diplomacy: strengthen the role of staff/secretariat.

•Standard-based approach for staff interactions.