The Recovery Theorem©

March 25, 2014

Chicago Booth School

Morgan Stanley, New York City

Steve Ross

Franco Modigliani Professor of Financial Economics

MIT

Page 2

Disclaimer

This document is confidential and may not be reproduced without the written consent of Ross, Jeffrey & Antle LLC (RJA).

It is not intended for distribution and may not be distributed without the written consent of RJA.

The ideas and material contained herein are the intellectual property of RJA. Please note that the results in this

presentation are preliminary and are subject to revision.

Page 3

Forecasting the Equity Markets

• For equities we don’t even ask the market for a single point forecast like we do with

forward interest rates

• Here is what we do in the equity markets:

– We use historical market returns and the historical premium over risk-free

returns to predict future returns

– We build a model, e.g., a dividend/yield model to predict stock returns

– We survey market participants and institutional peers

– We use the martingale measure or some ad hoc adjustment to it as though it

was the same as the natural probability distribution

• What we want to do in the equity markets – and the fixed income markets – is find

the market’s subjective distribution of future returns

• At the least this would open up a host of different empirical possibilities

• Lets look at the derivatives market and option pricing models

Page 4

The Binomial Model

• Arguably, the binomial model and its close cousin, Black-Scholes-Merton, are the

most successful models in economics if not in all of the social sciences:

aS

f

S b < 1 + r < a

1 - f bS

where f is the natural jump probability and r is the risk free interest rate

• The absence of arbitrage implies the existence of positive (Arrow Debreu (AD)) prices

for pure contingent claims, i.e., digitals that pay $1 in state a or b:

𝑝 𝑎 = 1

1 + 𝑟π 𝑎𝑛𝑑 𝑝 𝑏 =

1

1 + 𝑟(1 − 𝜋)

where

𝜋 = 1 + 𝑟 − 𝑏

𝑎 − 𝑏 𝑎𝑛𝑑 1 − 𝜋 =

𝑎 − (1 + 𝑟)

𝑎 − 𝑏

Page 5

The Binomial Model

• The price, D, of any derivative with payoffs D(a) and D(b) is simply

𝐷 = 𝑝 𝑎 𝐷 𝑎 + 𝑝 𝑏 𝐷 𝑏

• This is the simplest expression of the risk neutral theory of no arbitrage where prices

are the discounted expected value of payoffs under the risk neutral or martingale

measure

• Equivalently, no arbitrage and spanning imply that there is a unique portfolio of the

stock and the bond that replicates the payoff of the option and the cost of this

portfolio is simply the price of the derivative

• As is well known and striking, the natural probability plays no role whatsoever in this

theory for pricing derivatives

• Conversely, there is no way to use derivatives prices to find any information about

the underlying natural distribution, f

Page 6

Black-Scholes-Merton

• The binomial model converges to a lognormal diffusion as the step size gets smaller

𝑑𝑆

𝑆= 𝜇𝑑𝑡 + 𝜎𝑑𝑧

• The binomial formula for the price of an option also converges to the famous Black-

Scholes risk neutral differential equation

1

2𝜎2𝑆2𝜕2𝑃

𝜕𝑆2+ 𝑟𝑆𝜕𝑃

𝜕𝑆 − 𝑟𝑃 +

𝜕𝑃

𝜕𝑡= 0

yielding the equally famous Black-Scholes solution for a call option

𝑃 𝑆, 𝑡 = 𝑆𝑁 𝑑1 − 𝐾𝑒−𝑟 𝑇−𝑡 𝑁(𝑑2)

where N(◦) is the cumulative normal, K is the strike, and

𝑑1 =ln𝑆𝐾+ (𝑟 +

12𝜎2)(𝑇 − 𝑡)

𝜎 𝑇 − 𝑡 𝑎𝑛𝑑 𝑑2 = 𝑑1 − 𝜎 𝑇 − 𝑡

Page 7

Black-Scholes-Merton

• The Black-Scholes solution is the discounted expected value of the payoff on a call

under the risk neutral measure, i.e., under the assumption that the stock drift is the

risk free rate

• This produces the well known result that the expected return on the stock, μ,

doesn’t enter the formula, and option prices are independent of the drift of the

natural distribution of the stock

• This is both a blessing and a curse;

• It means that we can price derivatives without knowing the very difficult to measure

parameters of the underlying process

• But, conversely, the market for derivatives is of no use to us for determining the

natural distribution

• The challenge, then, is whether it is possible to build a different class of model that

will have the ability to link option prices to the underlying parameters of the stock

process while at the same time being consistent with risk neutral pricing

Page 8

The Options Market

• A put option is like insurance against a market decline with a deductible; if the

market drops by more than the strike, then the put option will pay the excess of the

decline over the strike, i.e., the strike acts like a deductible

• The markets use the Black-Scholes formula to quote option prices, and volatility is an

input into that formula

• The implied volatility is the volatility that the stock must have to reconcile the

Black-Scholes formula price with the market price

• Notice that the market isn’t necessarily using the Black-Scholes formula to price

options, only to quote their prices

Page 9

0.5

1.5

2.5

3.5

4.5

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

50% 60% 70% 80% 90%100%

110%120%

130%140%

150%

Maturity (years)

Volatility (% p.a.)

Moneyness (% of spot)

0%-5% 5%-10% 10%-15% 15%-20% 20%-25% 25%-30% 30%-35% 35%-40% 40%-45% 45%-50%

The Volatility Surface

Surface date: January 6, 2012

Page 10

Implied Vols, Risk Aversion, and Probabilities

• Like any insurance, put prices are a product of three effects:

Put price = Discount Rate x Risk Aversion x Probability of a Crash

• But which is it – how much of the high price comes from high risk aversion and how

much from a higher chance of a crash?

• The binomial model and Black-Scholes-Merton not only provide no answer to this

question, they also suggest that no answer is possible and, to some extent, we’ve

lost the intuitions of insurance in derivative theory

• In fact, much is made of the distinction between the risk neutral martingale measure

which we can infer from option prices and the latent natural measure and research

carefully and rightfully separates these different concepts

• Nonetheless, one often encounters a careless blurring of this distinction

• For example, risk aversion is sometimes ignored and the skew of the vol surface is

casually interpreted as proof that tail probabilities are large

Page 11

Contingent Forward Prices

• Implicit in option prices are the prevailing forward prices for securities that pay off if

the market is in a given range, e.g., between 1700 and 1750 in one year

• To recover the true distribution of stock returns we use a new concept, contingent

forward prices (CFP’s), i.e., the forward prices that prevail from any value of the

market and not just the current value

• Suppose the market is in some state i (for simplicity we will index states by the stock

price alone) and we want to know the price of a digital security that pays $1 if the

market is in state j in a year, i.e., the contingent forward price, pij

• Like forward rates in the fixed income market, CFP’s are prices that we can ‘lock in’

today for buying a digital paying, say, if the market is at 1700 in one year and drops

to 1400 in the next six months

• First, though, we have to find the CFP’s and since the current state is c = 1600,

rather than some arbitrary state i and since we don’t have forward markets we have

to demonstrate that this is possible

• To do so we apply the forward equation to the risk neutral distribution

Page 12

Contingent Forward Prices

End of 3rd Quarter

P(1300,1450,4)

P(1600,1900,3) P(1900,1450)

Today

1900

End of Year

P(1600,1300,3)

P(1300,1450)

1800

1700

1600

1500

1400

1300

1450

current

state,

S&P = 1600

Page 13

The Recovery Theorem: The Model

• A state is a description of the information we use to forecast the market, e.g., the

current market level, last month’s returns, current implied volatility

• The probability that the system moves from state i to state j in the next quarter is fij

F = [fij] – the natural probabilities embedded in market prices

• P = [pij] is the matrix of CFP’s

• The CFP’s are the Arrow-Debreu prices conditional on the current state of nature and

they are proportional to the risk neutral or martingale probabilities

• The next slide describes the relation between the CFP’s, pij , and the natural

probabilities, fij

• The subsequent slides demonstrate how we can use this relation to find the natural

probabilities, fij , from the contingent prices, pij

Page 14

The Kernel (Risk Aversion) and Contingent Prices

• The contingent forward price, pij , depends on both the probability that a transition

from state i to state j will occur and on the pricing kernel, i.e., market risk aversion:

𝑝𝑖𝑗 = 𝛿𝜑𝑖𝑗𝑓𝑖𝑗

where δ is the market’s average discount rate and ϕ(i,j) is the pricing kernel, and fij

is the natural probability of a transition from state i to state j

• The absence of arbitrage alone implies the existence of both the risk neutral

measure and a positive pricing kernel

• This is a very different approach than that of Black-Scholes-Merton, the binomial

model, and risk neutrality

• The following slides show how to find the discount rate and risk aversion and isolate

the natural probabilities from the market’s risk aversion

Page 15

The Recovery Theorem

• Keep in mind that we observe P and we are trying to solve for F

• The pricing equation above has the form:

𝑝𝑖𝑗 = 𝛿𝜑𝑖𝑗𝑓𝑖𝑗

• Knowing p there is no way to disentangle the kernel from the probabilities

• Simply, there are 2m2 + 1 unknowns and only m2 + m equations

• Suppose, though, that the kernel is transition independent, i.e., that it has the form

δφ(j)/φ(i)

• For example, for a representative agent with additively separable preferences

𝜑𝑖𝑗 = 𝛿𝑈´(𝐶 𝑗 )

𝑈´(𝐶 𝑖 )= 𝛿𝜑(𝑗)

𝜑(𝑖)

• Now we can separate φ from f

Page 16

The Recovery Theorem

• Letting D be the diagonal matrix with the m kernel values, ϕ(j), on the diagonal, the

pricing equation can be written as:

P = δD-1FD

• Since the probabilities in each row have to add up to one (m additional equations), if

e is the vector of ones, then we have:

Fe = e

• Rearranging the pricing equation we have:

F = (1/δ) DPD-1

and since Fe = e,

DPD-1e = δe,

or

Px = δx,

where x = D-1e is a vector whose elements are the inverses of the kernel

Page 17

The Recovery Theorem

• From the Perron-Frobenius Theorem (if P is irreducible), we know it has a unique

positive eigenvector solution, x, and an associated positive eigenvalue δ

• From x = (1/ϕ(i)) we have the kernel and, given δ, we can now solve for the natural

probabilities, F

𝑓𝑖𝑗 = (1

𝛿)𝜑(𝑖)

𝜑(𝑗)𝑝𝑖𝑗

Page 18

A Simple Binomial Example

• If m = 2 we have a simple 2x2 example:

𝜑1𝑝11 = 𝛿𝑓1𝜑1

𝜑1𝑝12 = 𝛿(1 − 𝑓1)𝜑2

𝜑2𝑝21 = 𝛿(1 − 𝑓2)𝜑1 and

𝜑2𝑝22 = 𝛿𝑓2𝜑2

• This is four equations in five unknowns, ϕ1, ϕ2, f1, f2, and δ which, with some

manipulation reduces to a quadratic in δ with a unique positive solution

• Without state dependence we would have

𝑝1 = 𝛿𝑓𝜑1 𝑎𝑛𝑑 𝑝2 = 𝛿(1 − 𝑓)𝜑2

two equations in the four unknowns with no unique positive solution

Page 19

Interest Rate Models (joint with Ian Martin)

• The setup is the same as before except that now we will explicitly consider bonds as

the underlying assets and let P be the matrix of AD prices

• As before and by the same technique, we can recover the pricing kernel, φ, as a

function of the state, but now we can offer some interesting interpretations that

appear to be not well known

• Letting B(i,T) be the price of a T period zero coupon bond if the current state is i,

the continuously compounded yield is given by

𝑦𝑇 𝑖 = −1

𝑇ln ( 𝑃𝑇(𝑖, 𝑗))

𝑗

• and the yield on the long bond,

𝑦∞ 𝑖 = − lim𝑇→∞

1

𝑇ln ( 𝑃𝑇(𝑖, 𝑗))

𝑗

Page 20

Interest Rate Models (joint with Ian Martin)

• The kernel, φ, is interpreted as the inverse of marginal utility for the defined

representative agent and the recovered probabilities are those of the agent, but the

recovered kernel can be thought of as characterizing a pseudo agent

• It is useful to think of the kernel as a one period asset paying φ(i) in state I

• It follows that

𝑅𝑒𝑡𝑢𝑟𝑛 𝑜𝑛 𝑡ℎ𝑒 𝑙𝑜𝑛𝑔 𝑏𝑜𝑛𝑑 = 𝑅∞ 𝑖, 𝑗 = (1

𝛿)𝜑(𝑗)/𝜑(𝑖)

• In other words, the kernel is the return on the long bond (Kazemi (1992) observed

this in a diffusion model)

• Importantly, it can be shown that the return on a finite T period bond

𝑅𝑇 𝑖, 𝑗 = 𝑅∞ 𝑖, 𝑗 + 𝑂 ∈𝑇 , 0 < ∈ < 1

and if γ is the second largest eigenvalue then ϵ is arbitrarily close to γ/δ

• This and subsequent result follow from well known theorems which have the limiting

properties of PT determined by the eigenvector and the maximal eigenvalue

Page 21

Interest Rate Models (joint with Ian Martin)

• A host of results on expected forward rates and the yield curve can be found, e.g.,

• The yield curve cannot always slope up or down across all states but it must slope up

on average across states

• The difference between the T period yield and the infinite yield is bounded,

𝑦𝑇 𝑖 − 𝑦∞ 𝑖 < 𝑄/𝑇

where

𝑄 = 𝑙𝑛𝑚𝑎𝑥𝑘(

1𝜑 𝑘)

𝑚𝑖𝑛𝑘(1𝜑 𝑘)

• These results on long yields and the returns on long but finilte lived assets are

particularly important because they assure us that we can closely approximate the

kernel from observable asset returns

• They might also say something inherent about the impact of far off information??

Page 22

Some Extensions

• If we attempt to apply the Recovery Theorem to a Black-Scholes-Merton environment

we discover that there are multiple solutions; any exponential is an eigenvector of

the state price transition function

• There are two sufficient conditions that permit extensions to continuous spaces

• First, if the state space is bounded we can apply the Krein-Rutman Theorem which is

a direct extension of Perron-Frobenius to continuous states if the space has an

interior (e.g., closed, convex, and with a boundary and the transition function is a

compact linear operator and strongly positive (i.e., Pv >> 0 if v > 0)

• Compactness is a good assumption for interest rate processes

• Alternatively, we can prove that there is a unique positive eigenvalue with an

associated positive eigenfunction if the kernel is bounded above and below,

0 < 𝑎 ≤ 𝜑 ≤ 𝑏 < ∞

Page 23

Bounded Eigenfunctions

• In a continuous space the matrix operators are integrals

𝐴(𝑔 ∙) = 𝑒−𝛿𝑇1

𝜑(𝑥) 𝜑 𝑦 𝑔 𝑦 𝑓 𝑥, 𝑦 𝑑𝑦

Since the kernel is positive, A is a positive linear operator. Notice that

𝐴1

𝜑= 𝑒−𝛿𝑇

1

𝜑(𝑥) 𝜑 𝑦

1

𝜑(𝑦)𝑓 𝑥, 𝑦 𝑑𝑦 = 𝑒−𝛿𝑇

1

𝜑(𝑥)

• Theorem 1: The operator, A, has a unique positive eigenvalue, e-δT , associated with

all positive eigenvectors bounded both from above and away from zero.

• Theorem 2: The kernel is the unique eigenfunction of A that is bounded away from zero

and of bounded variation.

• Theorem 3 The Continuous State Space Recovery Theorem

Under the assumptions outlined above, it is possible to recover both the pricing

kernel, φ, and the natural probability density, f(x,y) from the conditional state

price density.

Page 24

Compact Domain and Mean Reversion

• Carr has shown that recovery is possible in some interest rate models and, more generally,

Carr and Yu (2012) prove recovery with a bounded diffusion under certain boundary

conditions – not, of course, the Black-Scholes-Merton model

• Dubinskiy and Goldstein (2013) argue for the need to impose boundary conditions that, in

effect, characterize the behavior of the representative agent – behavior which we would

hope to recover

• In a major extension Walden (2013) derives necessary and sufficient conditions for

recovery with an unbounded diffusion process and shows that mean reversion is a

sufficient condition – intuitively the process cannot drift off to infinity to rapidly and

mean reversion satisfies that condition

• Walden also extends the above result that recovery is possible whenever the kernel is

bounded above and below and shows that if recovery is possible on an infinite domain

then an efficient truncation method will recover it – independent of boundary conditions

– nicely extending earlier work validating truncation with bounded kernels

Page 25

Applying the Recovery Theorem – A Three Step Procedure

• For each date:

• Step 1: Use option prices to determine the forward price surface, i.e., the risk

neutral or martingale density

• Step 2: Use the forward price surface to determine the contingent forward

prices, i.e., the transition probabilities

• Step 3: Apply the Recovery Theorem to recover the kernel and the market’s

natural probability distribution of equity returns on that date

Page 26

Forward (Digital) Prices

• Options ‘complete the market’, i.e., from options, we can create a pure Arrow-

Debreu security, a digital option that only pays off iff the market is between, say,

1800 and 1810, one year from now (see Ross (1976) and Breeden and Litzenberger

(1978))

• A one year bull butterfly spread, composed of one call with a strike of 1550, one with

a strike of 1560, and selling two calls with strikes of 1555 pays off iff the market is

between 1550 and 1560

• Notice that the prices are peaked near the current value for the market and fall off

and spread at longer maturities

• Notice, too, that the prices are lowest for large moves, higher for big down moves

than for big up moves, and that after an initial fall prices tend to rise with tenor

Page 27

Forward (Digital) Prices

Priced using the SPX volatility surface from December 2, 2013

Market Level 1 month 2 month 3 month 6 month 1 year

0.50 0.0007 0.0014 0.0034 0.0169 0.0169

0.60 0.0002 0.0005 0.0006 0.0027 0.0069

0.70 0.0014 0.0027 0.0042 0.0149 0.0261

0.80 0.0337 0.0611 0.0973 0.1143 0.1143

0.90 0.3785 0.3537 0.2832 0.2090 0.2090

1.00 0.5704 0.5433 0.4560 0.3163 0.3163

1.10 0.0084 0.0258 0.0985 0.1769 0.1769

1.20 0.0000 0.0005 0.0065 0.0435 0.0435

1.30 0.0000 0.0000 0.0000 0.0010 0.0111

1.40 0.0000 0.0000 0.0000 0.0001 0.0024

1.50 0.0000 0.0000 0.0000 0.0001 0.0001

Page 28

Risk Neutral Density

December 2, 2013

1 month

2 month

3 month

6 month

1 year

0

0.02

0.04

0.06

0.08

0.1

1.0

0

6.0

0

11

.00

16

.00

21

.00

26

.00

31

.00

36

.00

41

.00

46

.00

51

.00

56

.00

61

.00

66

.00

71

.00

76

.00

81

.00

86

.00

91

.00

96

.00

10

1.0

0

10

6.0

0

11

1.0

0

11

6.0

0

12

1.0

0

12

6.0

0

Page 29

Steps 2 and 3:

• Use the forward equation technique described earlier to determine the contingent

forward prices, pij from the forward prices

• Apply the Recovery Theorem and solve for the unknown probabilities and the kernel

(risk aversion)

Page 30

Recovered Kernels

0.00

2.00

4.00

6.00

8.00

10.00

12.00

14.000

.60

0.6

2

0.6

4

0.6

7

0.6

9

0.7

1

0.7

4

0.7

7

0.7

9

0.8

2

0.8

5

0.8

8

0.9

1

0.9

4

0.9

7

1.0

1

1.0

4

1.0

8

1.1

2

1.1

6

1.2

0

1.2

4

1.2

8

1.3

3

1.3

8

1.4

2

1.4

7

MU 5/1/2009

MU 6/1/2010

MU 6/1/2011

MU 6/1/2012

MU 6/3/2013

Page 31

Recovered Density

December 2, 2013

1 month

3 month

1 year

0.00E+00

2.00E-02

4.00E-02

6.00E-02

8.00E-02

1.00E-01

1.20E-01

1.40E-01

0.5

0

0.5

2

0.5

5

0.5

7

0.6

0

0.6

2

0.6

5

0.6

8

0.7

1

0.7

4

0.7

7

0.8

1

0.8

4

0.8

8

0.9

2

0.9

6

1.0

0

1.0

4

1.0

9

1.1

4

1.1

9

1.2

4

1.3

0

1.3

5

1.4

1

1.4

7

Page 32

Recovered Probabilities vs. Risk-Neutral Probabilities

December 2, 2013

0

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0.04

0.50 0.55 0.60 0.65 0.71 0.77 0.84 0.92 1.00 1.09 1.19 1.30 1.41

Risk Neutral Density

Recovered Density

Page 33

Recovered Cumulative vs. Historical Cumulative

December 2, 2013

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

0.50 0.55 0.60 0.65 0.71 0.77 0.84 0.92 1.00 1.09 1.19 1.30 1.41

Recovered 1 year Cumulative

Historical Cumulative

Page 34

Recovered and Historical Statistics

5/9/2009 2/3/2014

Historical Recovered Recovered

mean 9.53% 11.65% 5.17%

excess return 4.19% 10.57% 4.94%

sigma 15.30% 30.31% 15.91%

Sharpe 0.27 0.35 0.31

rf 5.34% 1.08% 0.23%

divyld 3.25% 2.67% 1.94%

Page 35

Predictive Return Regressions

Residuals from R(3)

on lags

Residuals from R(3)

on lags R(1) R(1) R(3) R(3)

Constant -0.01 -0.02 -0.07 0.02 -0.05 -0.06

-0.74 -0.99 -2.33 1.96 -1.33 -2.02

ER(1) 4.86 -57.95 -30.73

1.64 -2.49 -1.10

ER(2) 77.31 65.26

2.32 1.63

ER(3) -30.15 6.37 4.64 -29.32

-1.96 3.73 2.51 -1.60

R(1)

R(3, t-1) 0.61

4.30

R(3,t-2) -0.13

-0.90

Adj R^2 0.03 0.14 0.19 0.27 0.09 0.24

(t-stats under coefficients)

Monthly data from May, 2009 to February, 2014

Page 36

3m Returns and 3m Expected Returns

-25.00%

-20.00%

-15.00%

-10.00%

-5.00%

0.00%

5.00%

10.00%

15.00%

20.00%

5/1/2009 5/1/2010 5/1/2011 5/1/2012 5/1/2013 Realized 3m Return

Expected 3m Return

Page 37

Volatility

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

5/1/2009 5/3/2010 5/2/2011 5/1/2012 5/1/2013

Realized Vol

Recovered vol

VIX

Page 38

Volatility

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

5/1/2009 5/3/2010 5/2/2011 5/1/2012 5/1/2013

Realized Vol

VIX

Page 39

Volatility

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

5/1/2009 5/3/2010 5/2/2011 5/1/2012 5/1/2013

Realized Vol

Recovered vol