Martingale Test for Alpha

7/30/2019 Martingale Test for Alpha

1/36

1

AMartingaleTestforAlpha

DeanP.Foster*,RobertStine*,H.PeytonYoung**

Thisversion:January31,2010

*DepartmentofStatistics,WhartonSchool,UniversityofPennsylvania

**DepartmentofEconomics,UniversityofOxford


2/36

2

Abstract

Wepresentanewmethodfortestingwhetherthereturnsfromafinancialasset

aresystematicallyhigherthanaperfectlyefficientmarketwouldallow.Suchan

asset is said tohavepositivealpha. The standardwayofestimatingalpha is to

regresstheassetsreturnsagainstthemarketreturnsoveranextendedperiodof

timeandtoapplythettest. Thedifficultyisthattheresidualsoftenfailtosatisfy

independenceandnormality.Infact,portfoliomanagersmayhaveanincentive

to employ strategieswhose residualsdepart bydesign from independence and

normality.Toaddress theseproblemsweproposearobusttestforalphabased

on the martingale maximal inequality. Unlike the ttest, our test places no

restrictions on thedistribution of returnswhile retaining substantial statistical

power.Weillustratethemethodforfourassets:astock,amutualfund,ahedge

fund,andafabricatedfundthatisdeliberatelydesignedtofoolstandardtestsof

significance.


3/36

3

1.Estimatingalpha

An asset that consistently delivers higher returns than abroadbasedmarket

portfolioissaidtohavepositivealpha.Thisistheexcessreturnthatresultsfrom

the asset managers superior skill in exploiting arbitrage opportunities and

judgingtherisksandrewardsassociatedwithvariousinvestments. Buthowcan

investors (and statisticians) tell from historicaldatawhether a givenportfolio

actually is generating positive alpha relative to the market? To answer this

question one must address four issues: i) multiplicity; ii) trends; iii) cross

sectional correlation; iv) robustness. We begin by reviewing standard

adjustments for the first three; thiswill set the stage for our approach to the

robustness issue,which involvesanovelapplicationof themartingalemaximal

inequality(Doob,1953).

Thefirststepinevaluatingthehistoricalperformanceofafinancialassetrequires

adjustingformultiplicity. Assetsareseldomconsidered in isolation: investors

canchooseamonghundredsorthousandsofstocks,bonds,mutualfunds,hedge

funds,andotherfinancialproducts.Withoutadjustingformultiplicity,statistical

testsofsignificancecanbeseriouslymisleading.Totakeatrivialexample,ifwe

weretotestindividuallywhethereachof100mutualfundsbeatsthemarketat

level =0.05,wewouldexpecttofindfivestatisticallysignificantpvalueswhen

infactnoneofthembeatsthemarket.

Statisticiansarewellversedinthedangersofsearchingforthemoststatistically

significanthypothesesindataandhavedevelopedawidevarietyofprocedures


4/36

4

tocorrectformultiplecomparisons.Thesimplestandmosteasilyusedoftheseis

theBonferronirule. When testingmhypothesessimultaneously,onecompares

the observed pvalues to an appropriately reduced threshold. For example,

insteadofcomparingeachpvalueip toathresholdsuchas =0.05,onewould

comparethemtothereducedthreshold /m.

ModernalternativestoBonferronihaveextendeditintwodirections. Thefirst,

calledalphaspending,allows thesplittingof thealpha intounevenpieces;see

forexamplePocock(1977)andOBrienandFleming(1979). Thesecondgroupof

extensions providesmore powerwhen several different hypotheses arebeing

tested. These canbemotivated frommany perspectives: false discovery rate

(BenjaminiandHochberg1995),Bayesian(GeorgeandFoster,2000),information

theory (Stine,2004)and frequentist risk (Abramovich,Benjamini,Donoho,and

Johnstone, 2006). One can even apply several of these approaches

simultaneously (Foster and Stine 2007). In the case of financial markets,

however,thenaturalnullhypothesis isthatnoassetcanbeat themarketforan

extended period of time because this would create exploitable arbitrage

opportunities. Thus, in thissetting, thekey issue iswhetheranythingbeats the

market,letalonewhethermultipleassetscanbeatthemarket.

Asecondkeyissueinevaluatingthehistoricalperformanceofdifferentassetsis

theneedtodetrendthedata.Thisisparticularlyimportantforfinancialassets,

whichgenerallyexhibitastrongupwardtrendduetocompounding. Consider,

forexample, thepriceseriesshown inFigure1forfourquitedifferenttypesof

assets. The top two panels represent the value over time of a longrunning


5/36

5

generalmutual fund (FidelityPuritan),and the stockofadiversifiedcompany

(BerkshireHathaway) that is famous for its superiorperformance over a long

periodoftime.Thetwoassetsinthelowerpanelsrepresentmanagedfundsthat

followdynamicinvestmentstrategiesthatweshalldescribeinmoredetaillater

on. TheTeamFund isbasedonadynamicrebalancingalgorithm thatseeks to

reducevolatilitywhilemaintaininghighreturns(Gerth,1999;Agnew,2002). The

PiggybackFund isbasedonanoptionstradingstrategythat isdesignedtofool

investors into believing that the fund is generating excess returns when in

expectation it isnot (FosterandYoung,2007; forarelatedconstructionseeLo,

2001).

Figure1. Valueoffourassetsovertime


6/36

6

Thesimplest

way

to

de

trend

such

data

isto

study

the

period

by

period

returns

ratherthanthevalueoftheassetitself.Thatis,insteadoffocusingonthevalue,

tV ,onestudiesthesequenceofreturns 1 1( ) /t t tV V V oversuccessivetimeperiods

t. The hope is that the returns are being generated by a process that is

sufficientlystationaryforstandardstatisticalteststobeapplied. Asweshallsee

lateron, thishopemaynotbewellfoundedgiven that financialportfoliosare

oftenmanagedinawaythatproduceshighlynonstationarybehaviorbydesign.

Figure2. Monthlyreturnsseriesforthefourassets


7/36

7

The third issue, crosssectional correlation, arisesbecause returns on financial

assetsoften exhibitahighdegreeofpositive correlation.The standardway to

deal with this problem is the Capital Asset Pricing Model (CAPM), which

partitionsthevariationinassetreturnsintotwoorthogonalcomponents:market

risk,which isnondiversifiable andhenceunavoidable, and idiosyncratic risk.

Byconstruction,idiosyncraticriskisorthogonaltomarketriskandmeasuresthe

rewardsandrisksassociatedwithaspecificasset.It is themeanreturnon this

idiosyncraticrisk,knownasalpha,thatdrawsinvestorstospecificstocks,mutual

funds,andalternativeinvestmentvehiclessuchashedgefunds.

Thestandardwaytoestimatethealphaofaparticularassetorportfolioofassets

istoregressitsreturnsagainstthereturnsfromabroadbasedmarketindexsuch

as theS&P500aftersubtractingout theriskfreereturn. Specifically, lettm be

thereturngeneratedby themarketportfolio inperiod t,and let tr be therisk

freerateofreturnduringtheperiod,thatis,thereturnavailableonasafeasset

suchasUSTreasurybills. Theexcessreturnofthemarketduringthe tht periodis

t tm r . Let ty be the return inperiod t fromaparticularasset (orportfolioof

assets)thatwewishtocomparewiththemarket. Theportfoliosexcessreturnin

period t is defined as t ty r. CAPM isolates the idiosyncratic return of the

portfoliofromtheoverallmarketreturnbyestimatingtheregressionequation

( ) t t t t t y r m r . (1)

Themarketadjustedreturnisdefinedtobetheinterceptplustheresidual,thatis,

t .


8/36

8

The crucialquestion from the investors standpoint iswhether the intercept is

positive ( 0 ). If it is, then the investor could increasehisoverall returnby

investing a portion of his wealth in this portfolio instead of in the market.

Moreover, this increased returncouldbeachievedwithoutexposinghimself to

muchadditionalvolatility,providedheputsasufficientlysmallproportionofhis

wealthintheportfolio.Thustherelevantquestioniswhetherthereexistsanyasset,or

portfolio of assets, that systematically beats the market in the sense that it exhibits

positivealphaatahighlevelofsignificance.

Aswehavealreadynoted,thestandardwaytoanswerthisquestionistoapplya

ttest to the estimateof the intercept. But this isonlyjustified if the residuals

satisfy quite restrictive assumptions, including independence and normality.

Whenwegraphtheresidualsfromourfourcandidateassetsovertime,however,

it becomes apparent that two of them (Team and Piggyback) exhibit highly

erratic,nonstationarybehavior(seeFigures34).


9/36

9

Figure3.Marketadjustedreturnsversusmarketexcessreturns.

Figure4. Timeseriesofmarketadjustedreturnsforthefourassets.


10/36

10

In the case of Team, the market-adjusted returns are negatively correlated with

the market returns (see Figure 3), and they exhibit a cyclic pattern with bursts of

high volatility followed by periods of low volatility immediately thereafter (see

Figure 4). This is a direct consequence of Teams dynamic rebalancing strategy.

At any given time the fund is invested in a mixture of cash (earning the risk-free

rate of return) and the S&P 500. At the end of each year funds are moved from

cash to stock if the risk-free rate in that period exceeded the market rate of

return, and from stock to cash if the reverse was true. The target proportions tomaintain between stock and cash are adjusted at the end of each five-year

interval i. (The five-year intervals and target proportions -- 70% stock, 30% cash -

- are chosen purely for purposes of illustration.)

Gerth (1999) proves that if stocks have returns that are independent and

identicallydistributed,and if the riskfree rateof return is constant, then there

existsa choiceof targets such that this strategyyieldshigher returns than the

marketwithoutincreasingthelevelofvolatility. Ourpurposeisnottocritique

this approach, but to take it as a concrete example showing why a natural

portfoliomanagementstrategycouldeasilyproducereturnsthatbydesigndiffer

substantiallyfromtheassumptionsneededtoapplythettest.

The returns from thePiggyback Fund exhibit evenmore erraticbehavior.The

reasons for thiswillbediscussed insection3. Suffice it tosayhere that these

returnsareproducedbyastrategythatisdesignedtoearntheportfoliomanager

alotofmoneyratherthantodeliversuperiorreturnstotheinvestors.However,

since thenaturalaimofportfoliomanagers is toearn largeamountsofmoney,

statisticaltestsofsignificancemustaccommodatethissortofbehavior.


11/36

11

Thepurposeofthispaperistointroducearobusttestforalphathatisimmuneto

this type ofmanipulation.Our test assumes nothing about thedistribution of

marketadjusted returns except that they form amartingaledifference:neither

independence,constantvariance,nornormalityarerequired. Thetestissimple

to compute and does not depend on the frequency of observations. A

particularlyimportantfeatureofthetestisthatitcorrectsforthepossibilitythat

themanagersstrategyconcealsasmallriskofalargelossinthetail. Thisisnot

a hypotheticalproblem: empirical studiesusingmultifactor risk analysishave

shown thatmany hedge funds have negatively skewed returns and that as a

resultthettestsignificantlyunderestimatesthelefttailrisk(AgarwalandNaik,

2004). Moreover,negativelyskewedreturnsaretobeexpected,becausestandard

compensation arrangements give managers an incentive to follow just such

strategies(FosterandYoung,2009).

Theplanof thepaper isas follows. In section2wederiveour test in itsmost

basicform. Thebasicideaisfirsttocorrectformarketcorrelationbycomputing

themarketadjusted returns as in (1), then compound these returns and test

whether they formamartingale.Section3 showswhy it isvital tohavea test

that,unlike the ttest, is robust todistributionalassumptions. Inparticular,we

showthatthettestleadstoafalsedegreeofconfidenceinthePiggybackFund,

whichisconstructedsothatitlookslikeitproducespositivealphawhenthisis

notactuallythecase.Section4extendstheapproachtotestwhetheroneormore

assetsoutofagivenpopulationofassets isproducingpositivealphaatahigh

levelofsignificance.


12/36

12

Insection5weshowhow torampup thestatisticalpowerof thisapproachby

leveraging theasset. Weshow, inparticular,that ifthereturnsof theassetare

lognormally distributedwith known variance, thenwe can choose a level of

leveragesuch thatthe loss inpower isquitemodestrelativetotheoptimaltest

(whichin thiscase isthettest).Infact,forapvalueof .01the loss inpower is

less than 30%, and for apvalue of .001 the loss inpower is only about 20%.

Insection6weconsiderthemoregeneralsituationinwhichweknownothingex

anteabout theassets truedistribution. Inspiteof thiswecanuse leverage to

advantagebyapplyingdifferentlevelsofleveragetotheassetinquestion.This

resultsinapopulationofleveragedassetstowhichweapplyPERT.Theessential

pointisthatthebestlevelofleverage(whichwedonotknowinadvance)results

inexponentiallyhigherratesofgrowth thanother levels.Hence thecompound

excessreturn testapplied to thepopulation (PERT)cancomequiteclose to the

compoundexcessreturntestappliedtothebestlevelofleverage.(Thisresultis

closely related to Covers work on universal portfolios (Cover, 1991.)

Wecallthistheexponentialpopulationexcessreturnstest(EXPERT).

Inthefinalsectionweapplythistesttoourfourcandidateassets. Itturnsout

that,aftercorrectingformultiplicity,marketcorrelation,andunrealized

downsiderisk,theonlyassetthatplausiblydeliverspositivealphaisBerkshire

Hathaway,eventhoughtwooftheothers(Fidelity, and Piggyback) look quite

impressive at first sight.


13/36

13

2. The compound excess returns test (CERT)

Considerafinancialasset,suchasastock,amutualfund,orahedgefundwhose

performancewewishtocomparewith thatof themarket. Thedataconsistof

returns generatedby the asset over a series of reporting periods 1,2,...,t T .

Denotethemarketreturninperiodtbytherandomvariablet

M andtheassets

returnbytherandomvariablet

Y ,wherebydefinition , 0t t

M Y . Inapplications,

tM wouldbethereturnonabroadbasedportfolioofstockssuchastheS&P500.

Thefirststepinouranalysisistosubtractofftheriskfreerateofreturnineach

period,thatis,therateavailableonasafeassetsuchasTreasurybills.Lettingt

r

denotetheriskfreerateinperiodt,definetherandomvariables

,t t t t t t

M M r Y Y r . (2)

Thesecondstepistocorrectforcorrelationwiththemarket.Let

( , )

( )

Cov Y M

Var M

. (3)

In practice can either be estimated directly from historical data or by

analyzingthecompositionoftheassetinquestion. Animportantpointtonotice

isthat canbeestimatedaccuratelyusingshorttermdata(e.g.,dailyreturns)

because it measures the extent to which the returns are correlated with the

market,not theirmagnitude. (Thuswith sufficientlyhigh frequencydataone


14/36

14

could estimate the correlation coefficient period by period, i.e., monthly or

quarterly.) Themarketadjustedreturnoralphainperiodtis

t t tA Y M . (4)

Thenullhypothesisisthattheconditionalexpectationoft

A iszeroineveryperiod

t,thatis,

1, 1, [ | ..., ] 0t tt T E A A A . (5)

This isequivalent to saying that the compoundgrowthof themarketadjusted

returnsformsanonnegativemartingale

NullHypothesis:

1

(1 )t ss t

C A

isanonnegativemartingale. (6)

CompoundExcessReturnTest (CERT). For each t, 1 t T , let 0tC be the

compoundmarketadjusted returnofa candidateasset throughperiod t.TheCERTp

valuefortestingthenullhypothesisis

1min (1/ )t T tp C . (7)

Theproofisastraightforwardapplicationofthemartingalemaximalinequality,

which for convenience we shall derive here (Doob, 1953). Under our

assumptions, tC isanonnegativemartingalewith conditional expectation1 in

everyperiod. Givenarealnumber 0 ,definetherandomtime ( )T tobethe


15/36

15

firsttime t T such that tC ifsucha timeexists,otherwise let ( )T T .By

the optional stopping theorem,( )

[ ] 1TE C . (See Doob, 1953, Theorem 2.1.)

Clearly, if 1max s t sC then ( )TC , hence 1 ( )(max ) ( )s t s T P C P C .

Since ( )TC isnonnegative, ( ) ( )( ) [ ] / 1/T TP C E C . Itfollowsthat

1(max ) 1/s t sP C . (8)

It follows that the sequence 1 2, , ..., TC C C of compound excess returns was

generatedwithprobabilityatmost 1min (1/ )t T tC .

Notice that the lengthof the series is immaterial:whatmatters is themaximum

compound return thatwasachievedatsomepointduring theseries. Moreover,

thecompoundreturnsmustbeverylargeforthenullhypothesistoberejected.

Forexample,

to

reject

the

null

at

the

5%

level

of

significance

requires

that

the

assetgrow twentyfold after subtractingoff the riskfree rateand correcting for

correlationwiththemarket. Twoofthefourfundsmeetthistest(Fidelityand

Berkshire)asmaybeseen fromFigure5,whichshows thecompoundvalueof

eachassetsmarketadjustedreturns.

Table1comparesthepvaluesfromCERTwiththosefromthettest. According

to the latter, Berkshire, Fidelity, and Piggyback have positive alpha at a high

levelofsignificanceandPiggybackisparticularlyimpressive.Bycontrast,CERT

saysthatweshouldbeconfidentinBerkshire,whichhasaCERTpvalueequalto

.011,butnotinanyoftheothers.


16/36

16

Figure5.Compoundvalueofmarketadjustedreturnsforthefourassets

Asset Regressiont Regressionpvalue CERTpvalue CERTt

Fidelity 3.23 0.0013 0.37 0.33

Berkshire 4.03 56.7 10 0.011 2.30

Team 0.23 0.82 0.96 1.77

Piggyback 16.79 411.3 10 0.15 1.03

Table1.RegressionpvaluesversusCERTpvaluesforthefourassets.

[CERTtisthevalueofthetstatisticthatcorrespondstothegivenpvalue.]


17/36

17

3. Gamingthettest.

At first it might seem counterintuitive that one would need to see a fund

outperformthemarketbyatleasttwentyfoldinordertobereasonablyconfident

(atthe5%levelofsignificance)thattheoutperformanceisforreal.Thereason

isthatanapparentlystellarperformancecanbedrivenbystrategiesthatleadtoa

totallosswithpositiveprobabilitybutaverylongtimecanelapsebeforetheloss

materializes.Indeed,itiseasytoconstructstrategiesofthisnatureforwhichthe

CERT boundis tight. Choose a number 1 , and consider the following

nonnegativemartingalewithconditionalexpectation1

0 1,C 1/ 1/

1( ) ,T T

t tP C C

1/( 0) 1 TtP C

for1 t T . (9)

Ineachperiodthefundcompoundsbythefactor 1/ T withprobability 1/ T and

crasheswithprobability 1/1 T .Thustheprobabilitythatthefundscompound

excessreturn tC exceeds isprecisely1/ overanynumberofperiodsT.1

ThePiggybackFundwasconstructedalongjust these lines. Namely, the fund

was invested in theS&P500,and thereturns reportedeverymonth.However,

once every sixmonths the returnswere artificiallyboostedby the factor 1.02.

ThiswasdonebytakinganoptionspositionintheS&P500thatwouldbankrupt

the fund if theoptionswereexercised.Thestrikepricewaschosen so that the

probabilityofthiseventwas 1/1.02=.9804,sothefairvalueoftheoptioniszero.2

1Returnsserieswiththispropertycanbeconstructedusingstandardoptionscontracts(Foster

andYoung,2009).2Withprobability1/1.02thefundgrowsbythefactor1.02andwithprobability.02/1.02itloses

everything,whichisalotterywithexpectationzero.


18/36

18

This explains thebizarrepattern of themarketadjusted residuals in Figure 3:

onesixthof the time theyare+2%,and fivesixthsof the time theyare 2% .

With less than25yearsofdata there isasizableprobability that thedownside

riskwillneverberealized,andinvestorswillbelulledintothinkingthatthefund

isgeneratingpositivealpha.ThisiswhyCERTattachesaverymodestpvalueto

thereturnsgeneratedbythePiggybackFund.

Ofcourse,evenacasualinspectionoftheresidualsinFigure3suggeststhatthe

ttest should notbe used in this case.However this is not the essence of the

problem,because it is easy to construct piggyback strategieswhose returns

look i.i.d.normal. Namely, suppose thateverymonth themanagerboosts the

fundsreturnsbythefactor1.0033 where isalognormallydistributederror

withmean1andsmallvariance. (Thepurposeoftheerroristolendaplausible

amountofvariabilitytotherealizedreturns.) Asbefore,theboostcomesatthe

cost of going bankrupt with probability 1/(1.0033 ) .9967 / each month.

Assuming that thevariance of is small, this schemewill run for about 300

months (25 years)before the fund goesbankrupt, and the residualswill look

veryconvincing. Thus,inthiscasethettestwouldseemtobeappropriate,and

the estimate of alpha will be about 4% per year at a very high level of

significance. This ismisleading,however,because inreality thedistributionof

returns

is

not

approximately

normal

there

is

a

large

potential

loss

hidden

in

the

tail. One of the main virtues of CERT is that it corrects for this hidden

volatility:underCERT thepvalue of this scheme after 25 yearswill onlybe

about 300(1.0033) .37p .


19/36

19

4.Multiplicity,Bonferroni,andtheportfoliotest.

The testdescribedabove is for returnsgeneratedbya single fund. Inpractice,

investorswouldliketoidentifythosefundsfromagivenpopulationthatgenerate

positive alphawith high probability. This requires amore demanding test of

significance. Toillustrate,supposethatwecanobservethereturns foreachofn

fundsover the same time frame 1,2,...,t T . Let it

C be the compoundexcess

returnforfund i throughperiod t. Supposethatweobserveaparticularfund,

say *i ,whosereturnsaresufficientlyhigh thatwewouldreject thenullat the

5%levelusingCERT. Thisdoesnotmeanwehave95%confidencethatfund *i

isgeneratingpositivealpha. Suppose,infact,thatnoneofthenfundsisableto

generate positive alpha and that the excess returns are stochastically

independent. It is straightforward to construct n independent nonnegative

martingales, eachwith expectation 1, such that on average therewillbe .05n

fundsthatexceedthecriticalthreshold .05c .Forexample,outof1000fundsthere

will,on average,be 50 funds thatpass the singlefund threshold even though

noneofthefundsisactuallygeneratingpositivealpha.

Moregenerally,supposethatweobservennonnegativemartingales ,1it

C i n ,

overtheperiods1 t T .Letthenullhypothesis 0H bethat 1 1[ | , ..., ] 1i i i

t tE C c c

forall i andforall t,wherethe itC areassumedtobeindependentacross i for

each t. Bythemartingalemaximalinequalityweknowthat

0,c 1(max )i

t T tP C c c . (10)

Hence


20/36

20

0,c 1 1(max max ) /i

i n t T t P C c n c . (11)

Itfollowsthat,toreject 0H at levelp>0,theremustexistoneormorefunds i such

that

1max /i

t T tC n p . (12)

WecallthistestBonferroniCERT. (The idea isquitegeneralandappliestoany

situation inwhichmultiple hypothesis tests arebeing conducted; seeMiller,

1981.) Inthepresentcasethetestsuggeststhatweshouldnotbeconfidentthat

anyofthefourassetsexhibitspositivealpha. Thereasonisthattheonlyonethat

passesmusteratan individual level isBerkshire,whichwecherrypickedfrom

theS&P500. Tobeconfidentthatthebestof500stocksgeneratedpositivealpha

at the 5% levelof significance,would require that the stocksmarketadjusted

returnscompoundbyafactorofatleast500x20=10,000.Whilethismightseem

likean impossiblyhighhurdle,weshallshow inthenextsectionthatthereare

morepowerfulversionsofthetestthatmakesuchvaluesachievable.

Beforeconsideringthisissue,however,letusobservethatthereisasimpleand

powerful alternative to BonferroniCERT thatworkswhen onewants to know

whether one ormore assets in a given population has positive alphawithout

identifyingwhichparticularasset (orassets) thatmightbe.This test, called the

Portfolio Excess Returns Test (PERT), relies on the fact that any weighted

combinationofassetsthathavezeroalphamustalsohavezeroalpha.


21/36

21

PortfolioExcessReturnsTest (PERT). Consider afamily of n funds,where each

fund i generates a series of compound excess returns 1 2, , ...,i i i

TC C C over T periods.

Createaportfolioconsistingofequalamountsinvestedinitiallyineachofthefunds. The

portfolios compound return series isgiven by1

(1/ ) it ti n

C n C

, 1 t T . The null

hypothesisthatnoneofthefundshaspositivealphahaspvalue 1min (1/ )t T tC .

The proof is more or less immediate. Each of the n funds generates a

nonnegative martingale of excess returnsi

tC that may nor may not be

independentacrossfunds.Theinvestorsportfolioisthenonnegativemartingale

1

(1/ ) it ti n

C n C

. Theassumption thatnoneofthefundsexhibitspositivealpha

implies that the conditional expectation of tC equals 1 in every period. The

conclusionfollowsatoncefromthemartingalemaximalinequality.

NotethatthistestisatleastaspowerfulastheBonferronitest:thelatterrejects

onlyif /itC n p ,butinthisevent 1/tC p ,whichimpliesthattheportfoliotest

rejectsalso.

5. Powerandleverage

WhileCERTandPERTarerobust, theyarealsoveryconservative. In thenext

twosectionsweshallshowthatmuchmorepowerfulvariantsofthesetestscan

bedevisedby leveragingtheassetunderscrutiny. Letusbeginbyconsidering

thespecialcase inwhichthereturnsareknowntobe i.i.d. lognormal,inwhich

casetheoptimaltestofsignificanceisthettestappliedtotheloggedreturns.We

shall show thatby leveraging the asset at an appropriate levelwe obtain an


22/36

22

exponentiated form of CERT (EXCERT) that involves only a modest loss of

powercomparedtotheoptimaltest.

Tobeconcrete,consideramanagerwhosefundisgeneratingcompoundreturns

0tC relative to the riskfree rate and suppose for simplicity that there isno

correlationwith themarket ( 0) . We shall assume thatt

C is lognormally

distributed:

2 2log (( / 2) , )t

C N t t .3 (13)

Whentheassetisleveragedbythefactor 0 ,thecompoundreturnsattimet,

( )tC ,aredescribedbytheprocess

2 2 2 2log ( ) (( / 2) , )tC N t t .4 (14)

Supposethatis 2 isknownand isnot. Thenullhypothesisisthat 0 and

the alternative hypothesis is that 0 .Choose apvalue 0p and a time t at

whichatestofsignificanceistobeconducted. CERTrejectsthenullatlevelpif

andonlyif

log log(1 / )tC p . (15)

3ThisisconsistentwiththetraditionalrepresentationofassetreturnsasageometricBrownian

motionincontinuoustimet t t t

dC C dt C dW (Berndt,1996;Campbell, Lo,andMacKinlay,

1997).4 Toleverageanassetbythefactoroneborrows 1dollarsattheriskfreerateandinvestsdollarsintheasset.If


23/36

23

Underthenullhypothesis,

2log ( / 2)tt

C tZ

t

is (0,1)N . (16)

HenceCERTrejectsthenullifandonlyif

2log(1 / ) ( / 2)t

p tZ

t

. (17)

Tomaximizethepowerofthetestwechoosetheleveragesothattheprobability

of rejection is maximized. This occurs when the righthand side of (17) is

minimized,thatis,when

2log(1/ )

*

p

t . (18)

Definition.EXCERT(exponentialCERT)isCERTappliedtotheassetleveragedby

theamount * .

Noticethat * dependsonthevarianceoftheprocess,thetimeatwhichthetest

is conducted, and the level of significancep. However, the corresponding z

valuedependsonlyonp,thatis,EXCERTrejectsifandonlyif

2log(1/ )t pZ p c , (19)


24/36

24

that is,p

c is the criticalvalue forEXCERTat significance level p Letuscompare

thiswiththecriticalvalueofthettest,whichrejectsatlevelpifnullatlevel p if

andonlyif

1(1 )t pZ p z . (20)

Thepower loss at significance levelp, ( )L p , is themaximumprobability that for

somecombinationof , , t , the ttestrejectsthenullat levelpwhenEXCERT

accepts.

Proposition1. ( ) 2 (.5( )) 1p p

L p c z and0

lim ( ) 0p

L p .

TheproofisgivenintheAppendix. Thefirstexpressioninthepropositioncan

beusedtonumericallycompute ( )L p ,andtheresultsareillustratedinFigure6.

The

loss

in

power

is

around

20

25%

for

p

in

the

range

3

10

to

5

10

,

which

is

the

relevant rangewhenwe test for thebest of n assets and n is on the order of

severalhundredorseveralthousand.

Figure6.Powerlossfunction ( )L p forEXCERTcomparedtothettest.


25/36

25

6.LeveragingCERTwhenthevarianceisunknown

The situation considered in the preceding section is quite special in that the

distributionofreturnswasassumedtobei.i.d.lognormalwithknownvariance.

Herewe introduceanalternativeversionof the test that ismore robustand is

asymptoticallyjustaspowerful. Toapply this test,allweneed toknow is the

maximumleveragethatcanbeappliedtotheassetunderconsiderationwithout

causingnegative realizations; inotherwords it suffices toknow themaximum

downsidelossthattheassetcansufferinanygivenperiod.

Consider an asset whosemarketadjusted returnst

A arebounded belowby

1 . Then the randomvariable (1 / )t t

M A isnonnegative, that is, the

maximum permissible level of leverage is 1/ . The null hypothesis is that

[ ] 0tE A ,which implies that [ / ] 0tE A . Indeed thenullhypothesisholdsfor

everylevelofleverageintherange 0 1/ :

Nullhypothesis: , 0 1/ , 1

(1 )t s

s t

C A

isanonnegativemartingale.

Letusnow constructafamilyof funds thatarebasedon thegivenasset { }tA ,

where each fund operates under a different level of leverage 0 1/ .

Suppose,forexample,thatweweightallfeasiblelevelsofleverageequally.We

thenobtainapopulationoffundswhosetotalvalueattimeisgivenby

1/

01

(1 )t s

s t

C A d

. (21)


26/36

26

Moregenerally,consideranydensity ( )f that isboundedawayfromzeroon

an interval [ , ] [0,1 / ]a b where ( ) 1b

af d . We can construct a family of

fundswithcompoundreturns

1

( ) (1 )b

t sa

s t

C f A d

. (22)

Underthenullhypothesis,anysuchfundhascompoundreturnsthatformanonnegative

martingale. Hencewecanrejectthenullhypothesisatlevelpif

1

( ) (1 ) 1/b

t sa

s t

C f A d p

. (23)

Any test of this formwillbe called an exponentialpopulation excess returns test

(EXPERT). This type of test is very general and assumes nothing about the

distribution of returns of the underlying asset { }tA except that they form a

martingaledifferenceandareboundedawayfrom 1 .

Given itsgeneralityonemightexpectthatthepowerofsuchatest isvery low.

In fact,however, ithas thesamepowerasymptoticallyasdoesEXCERTwhere

thevariance isassumed tobeknown.Theonlyrequirement is thatthe interval

[ , ]a b contains the actual value * that optimizes EXCERT. By choosing the

interval to be as wide as possible, namely [0,1/ ] , this requirement will

automaticallybesatisfied.


27/36

27

Proposition 2. Consider an asset whose returns are lognormally distributed

2 2log (( / 2) , )tC N t t . SupposethatEXPERTisappliedattimettoapopulation

offundsleveragedaccordingtoadensitythatisboundedawayfromzeroonaninterval

[ , ] [0,1 / ]a b that contains the optimal leverage level.Thenforall sufficiently large

timestthelossinpowerrelativetothettestiswellapproximatedbyL(p).

Thisresultfollowsfromamoregeneraltheoremonuniversalportfoliosdueto

Cover(1991). Lett

C bethesizeoftheEXPERTportfolioattimet. Let *t

C bethe

value at t of the portfolio that is leveraged at the optimal level * using

EXCERT.(Thisdepends,ofcourse,on t,p,and ). Coverstheoremcompares

thesevalueswiththevalue **tC ofathirdportfolioinwhichtheassetisleveraged

atalevel ** thatischosenexposttomaximizethevalueofthefundattimetgiventhat

the realized values of the returnsfrom the underlying assetup through t are known.

Clearly ** *t t

C C because **t

C isoptimized expostwhereas *t

C isoptimized ex

ante. Coverstheoremimpliesthat,foreverysmall 0 ,thereisatimeT such

that **( / 1 ) 1t tP C C forall t T .Itfollowsthat*( / 1 ) 1t tP C C for

all t T . By construction, EXCERT rejects at level p if* 1/t

C p , whereas

EXPERTrejectsatlevelpif 1/t

C p .ItfollowsthatEXPERThasapproximately

thesamepowerlossasdoesEXCERT,namely ( )L p ,atallsufficientlylargetimes

t.

5

5Coverconsidersportfoliosthatareconvexcombinationsofagivenfinitesetofassets.This

conditionissatisfiedbecauseanyleveragedportfolio1t

A canberepresentedasaconvex

combinationofthemaximallyleveragedportfolio1t

bA andtheminimallyleveragedportfolio

1t

aA .Coveralsoassumesthatthereturnsfromeachassetinanygivenperiodarenonnegative

andboundedabove. Tomeetthisconditionwecantruncatethelognormaldistributionof

returnsfromthemaximallyleveragedfundatalevel thatisseveralordersofmagnitudesmaller

thanthelevelpwearetesting.


28/36

28

7. Empiricalanalysisofthefourassets

In this concluding sectionwe shallapply thepreceding framework to the four

assetsshowninFigures14. Aswehavealreadyseen,CERTwithoutleveraging

andwithout correcting formultiplicity leads to thepvalues shown inTable1.

These suggest thatBerkshirehaspositivealphawhenviewed in isolation (p =

0.011),butnotwhenculledfromfivehundredstocks(theS&P500). However,

wehaveadditional informationabout the compositionofBerkshire (aswellas

FidelityandTeam) thatallowsus torampup the leverage.Namely, ineachof

these cases they were primarily invested in common stocks and cash and

(according to their annual reports) did not leverage their holdings to any

appreciableextent.6Thisallowsustoestimatetheamountofleveragethatcanbe

appliedwithoutthefundgoingbankrupt.WecanthenapplyEXPERT.

More generally we propose the following fourstep procedure for testing

whetheragivenassethaspositivealpha:

1.Estimatethecorrelationcoefficientbetweentheassetsreturns tY andthe

marketsreturnstM .7

2.Foreachtcomputethemarketadjustedreturns ( ) ( )t t t t t A Y r M r .

6BerkshireHathawayisadiversifiedholdingcompanythatinvestsmainlyinthecommonand

preferredstockofothercompanies.TheFidelityPuritanFundinvestsinamixtureofstocksand

bonds,andrebalancestheproportionsperiodically.Byconstruction,Teamholdsacombination

ofcashandtheS&P500ateachpointintimeandisnotleveraged.7Alternatively,onecanestimatearollingcorrelationcoefficient

t usingatrailingsubsampleof

thedata.


29/36

29

3. Estimate the maximum amount of leverage that can be applied to the

market-adjusted returns without going bankrupt. For an asset consisting of

liquid common stocks, this can be estimated either from the cost of a put option,

or from the size of the buffer needed to implement a stop-loss order. Here we

shall assume that a stop-loss order on stocks can be executed within 3% of the

limit price, so that one could leverage up to about 33 times without going

negative. (It is not crucial to estimate the upper bound precisely; what matters is

the optimal level of leverage, which will usually be lower than the maximum

possible leverage.)

4.Computethemaximumcompoundvalueoftheleveragedassetforaselection

ofleveragelevelsbetween0and,andlet Cbetheaverage.Theestimatedp

valueoftheasset is1/ C. Although thisestimatewilldependontheassumed

distributionofleveragelevels,therewilltypicallybeanarrowbandofleverage

levelsthatyieldvastlyhighervaluesofCthandotheothers.Theaverage Cwill

depend largely on these critical levels and not on the particular form of the

distribution.

Forpurposes of illustrationwe shall evaluate each of the four funds at seven

leverage levels: 1/2, 1, 2, 4, 8, 16, 32.8 As noted above, the choice of these

particularvalues isnot important,whatmatters is that theycover therangeof

possiblevaluesreasonablywell(inthiscase0 33)andthesamedistributionof

values is applied to all the assetsbeing evaluated. The results for Berkshire,

Fidelity,andTeamareshowninTable3.(Notethatwecannotapplythismethod

tothePiggybackFund,becausethemaximumlevelofleverageisnot33;indeed,

8Thisapproximatesthedistributioninwhichlogleverageisuniformlydistributed..


30/36

30

the fund isalready leveraged to thehiltbecause it isconstructed fromoptions

thatwillbankruptthefundwithpositiveprobability.)

Leverage MaxCvalue

Fidelity Berkshire Team

0.5 1.7 12 1.0

1 2.7 92 1.0

2 6.7 2,400 1.1

4 31 110,000 1.2

8 290 230 1.3

16 1,200 3,600 1.5

32 44 200 1.5

AvgC 225 16,648 1.22

EXPERTp=1/C .0044 .00006 0.82

EXPERTt

3.29 4.41 0.89

Regressiont 3.23 4.03 0.23

Regressionpvalue .0017 .00007 0.41

Table 3. EXPERTapplied to three assets at leverage levels (1/2, 1/,2, 4, 8, 16, 32)under the assumption that none of the leveraged assets can go negative atthese levels.

Noticethatthepvaluesestimatedbyourtestandbythettestareinquiteclose

agreement even though our test makes none of the regularity assumptions

requiredforthettest.


31/36

31

While thesepvalueswouldbehighly significant for a fund that isviewed in

isolation, however, we need to correct formultiplicity using Bonferroni. In

particular, Berkshirewas cherrypicked from the S&P 500, so its EXPERTp

value,adjustedformultiplicity,is.00006x500=.012. Thisisstillsignificantbut

not impressivelyso.(AndifweconsiderthatBerkshireisjustoneofthousands

oflistedstocks,thentheadjustedpvaluewouldnotbesignificant.)Fidelitywas

selected fromapoolofhundredsof stockmutual funds,sowhenadjusted for

multiplicityitspvalueisnotevenclosetobeingsignificantunderourtestorthe

ttest. Weconclude thatBerkshirehassomeclaim todeliveringpositivealpha

aftercorrectingformultiplicity,butevenitmustbeviewedasaborderlinecase.

Theotherthreeassetsdonotevencomeclose.

Appendix:ProofofProposition1

Weneed to show that

( ) 2 (.5( )) 1p pL p c z and 0lim ( ) 0p L p .Under the

nullhypothesis( 0 ),

22 ln .5ln .5( * )

*

t ptt

p

C cC tZ

ct

is (0,1)N . (A1)

Hencethettestrejectsthenullatlevelpifandonlyif

ln.5tp p

p

Cz c

c . (A2)

Bycontrast,EXCERTacceptsthenullifandonlyif 2ln ln(1/ ) .5t pC p c ,thatis,


32/36

32

ln

.5t pp

Cc

c . (A3)

Powerlossoccursintheregionwhereboth(22)and(23)hold.Assumenowthat

ln.5t p

p

Cc

c isdistributed ( ,1)

tN

forsome 0 . For this t the loss in

powerisgivenbytheprobabilityoftheevent

ln .5tp p p

p

Ct t tz c cc

. (A4)

ThemiddletermisdistributedN(0,1),sotheprobabilityofthiseventis

( ) ( )p p

t tc z

. (A5)

Thisprobability ismaximizedwhen pt

z

and p

tc

are symmetrically

situatedaboutzero.Itfollowsthatthepowerlossfunctionis

( ) 2 (.5( )) 1p pL p c z . (A6)

Tocompletetheproofweneedtoshowthat 0p pc z as 0p . Tocomplete

theproofweneedtoshowthat 0p pc z as 0p . Recallthatwhenzislarge

therighttailofthenormaldistributionhasthefollowingapproximation[Feller,

1957,p.193]:


33/36

33

2 / 2

( )2

ze

P Z zz

. (A7)

Thereforewhen p isreasonablysmall,say .01p ,wehavetheapproximation

2 log 2 log(2 / )p pz z p . (A8)

Combining(A6)and(A8)wefind,aftersomemanipulation,that

log .5 log(2 ) log .5 log(2 )

2 2 log(1/ ) 2

p p

p p

p

z cc z

p c

. (A9)

Hence 0p p

c z as 0p ,fromwhichweconcludethat ( ) 0L p as 0p .

References

Abramovich, F., Benjamini, Y., Donoho, D. L., and Johnstone, I. M. (2006),

Adaptingtounknownsparsitybycontrollingthefalsediscoveryrate,Annalsof

Statistics,34,584653.

Agarwal,V., andNaik,N.Y., (2004), Risks andportfoliodecisions involving

hedgefunds,ReviewofFinancialStudies,17,6398.

Agnew, R. A. (2002), On the TEAM approach to investing, American

MathematicalMonthly,109,188192.


34/36

34

Benjamini, Y. and Hochberg, Y., (1995), Controlling the false discovery

rate: a practical and powerful approach to multiple testing, Journal

oftheRoyalStatist.Soc.,Ser. B,57,289300.

Berndt,E.R. (1996),ThePractice ofEconometrics:Classic andContemporary,New

York,AddisonWesley.

Campbell,J. C., Lo, A.W., andMacKinlay, A. C. (1997), The Econometrics of

FinancialMarkets.PrincetonNJ,PrincetonUniversityPress.

Cover,ThomasM.(1991),Universalportfolios,MathematicalFinance,1,129.

Doob,J.L.(1953),StochasticProcesses,NewYork,JohnWiley.

Feller, W. (1971), AnIntroduction

to

Probability

Theory

and

Its

Applications,

PrincetonUniversityPress.

Foster, D. P. and Stine, R. A. (2008),Alphainvesting: sequential

controlofexpectedfalsediscoveries,JournaloftheRoyalStatisticalSocietySeries

B,70,.

Foster,D.P.,andYoung,H.P. (2009),GamingPerformanceFeesbyPortfolio

Managers,WorkingPaper08041,FinancialInstitutionsCenter,WhartonSchool

of Business, University of Pennsylvania. Available at

fic.wharton.upenn.edu/fic/papers/09/0909.pdf


35/36

35

George, E. and Foster, D. P. (2000), Empirical Bayes Variable

Selection,Biometrika, 87,731 747.

Gerth, F. (1999), The TEAM approach to investing, American Mathematical

Monthly,106,553558.

Lo,AndrewW., 2001, Riskmanagement for hedge funds: introduction and

overview,FinancialAnalystsJournal,Nov/DecIssue,1633.

Miller,R.G.(1981),SimultaneousStatisticalInference,2nded.,NewYork:Springer

Verlag.

OBrien,P.C.,andFlemingT.R.(1979),Amultipletestingprocedureforclinical

trials,Biometrics,35,549556.

Pocock, S.J. (1977), Group sequentialmethods in the design and analysis of

clinicaltrials,Biometrika,64,191199.

Stine, R. A.(2004), Model selection using information theory and

theMDLprinciple. SociologicalMethods&Research,33,230260.


36/36

Documents

Martingale Test for Alpha