Estimating the Predictive Distribution for Loss Reserve ModelsGlenn MeyersISO Innovative AnalyticsCAS Annual MeetingNovember 14, 2007

S&P Report, November 2003Insurance Actuaries A Crisis in CredibilityActuaries are signing off on reserves that turn out to be wildly inaccurate.

Background to Methodology - 1Zehnwirth/Mack Loss reserve estimates via regressiony = ax + e GLM E[Y] = f(ax) Allows choice of f and the distribution of YChoices restricted to speed calculationsClark Direct maximum likelihoodAssumes Y has an Overdispersed Poisson distribution

Background to Methodology - 2Heckman/MeyersUsed Fourier transforms to calculate aggregate loss distributions in terms of frequency and severity distributions.HayneApplied Heckman/Meyers to calculate distributions of ultimate outcomes, given estimate of mean losses

High Level View of PaperCombine 1-2 aboveUse aggregate loss distributions defined in terms of Fourier transforms to (1) estimate losses and (2) get distributions of ultimate outcomes.Uses other information from data of ISO and from other insurers.Implemented with Bayes theorem

Objectives of PaperDevelop a methodology for predicting the distribution of outcomes for a loss reserve model.The methodology will draw on the combined experience of other similar insurers.Use Bayes Theorem to identify similar insurers.Illustrate the methodology on Schedule P dataTest the predictions of the methodology on several insurers with data from later Schedule P reports. Compare results with reported reserves.

A Quick Description of the MethodologyExpected loss is predicted by chain ladder/Cape Cod type formulaThe distribution of the actual loss around the expected loss is given by a collective risk (i.e. frequency/severity) model.

A Quick Description of the MethodologyThe first step in the methodology is to get the maximum likelihood estimates of the model parameters for several large insurers.For an insurers dataFind the likelihood (probability of the data) given the parameters of each model in the first step.Use Bayes Theorem to find the posterior probability of each model in the first step given the insurers data.

A Quick Description of the MethodologyThe predictive loss model is a mixture of each of the models from the first step, weighted by its posterior probability.From the predictive loss model, one can calculate ranges or statistics of interest such as the standard deviation or various percentiles of the predicted outcomes.

The DataCommercial Auto Paid Losses from 1995 Schedule P (from AM Best)Long enough tail to be interesting, yet we expect minimal development after 10 years.Selected 250 Insurance GroupsExposure in all 10 yearsBelievable payment patternsSet negative incremental losses equal to zero.

16 insurer groups account for one half of the premium volume

Look at Incremental Development Factors Accident year 1986Proportion of loss paid in the Lag development yearDivided the 250 Insurers into four industry segments, each accounting for about 1/4 of the total premium.Plot the payment paths

Incremental Development Factors - 1986 Incremental development factors appear to be relatively stable for the 40 insurers that represent about 3/4 of the premium.They are highly unstable for the 210 insurers that represent about 1/4 of the premium. The variability appears to increase as size decreases

Do Incremental Development Factors Differ by Size of Insurer?Form loss triangles as the sum of the loss triangles for all insurers in each of the four industry segments defined above.Plot the payment paths

There is no consistent pattern in aggregate loss payment factors for the four industry segments.Segment 1Segment 3Segment 2Segment 4

Expected Loss ModelPaid Loss is the incremental paid loss in the AY and LagELR is the Expected Loss RatioELR and DevLag are unknown parametersCan be estimated by maximum likelihoodCan be assigned posterior probabilities for Bayesian analysisSimilar to Cape Cod method in that the expected loss ratio is estimated rather than determined externally.

Distribution of Actual Loss around the Expected LossCompound Negative Binomial Distribution (CNB)Conditional on Expected Loss CNB(x | E[Paid Loss])Claim count is negative binomialClaim severity distribution determined externallyThe claim severity distributions were derived from data reported to ISO. Policy Limit = $1,000,000Vary by settlement lag. Later lags are more severe.Claim Count has a negative binomial distribution with l = E[Paid Loss]/E[Claim Severity] and c = .01See Meyers - 2007 The Common Shock Model for Correlated Insurance Losses for background on this model.

Claim Severity DistributionsLag 1Lag 2Lag 3Lag 4Lags 5-10

Where

Likelihood Function for a Given Insurers Losses where

Maximum Likelihood EstimatesEstimate ELR and DevLag simultaneously by maximum likelihoodConstraints on DevLagDev1 Dev2 Devi Devi+1 for i = 2,3,,7Dev8 = Dev9 = Dev10Use Rs optim function to maximize likelihoodRead appendix of paper before you try this

Maximum Likelihood Estimates of Incremental Development Factors Loss development factors reflect the constraints on the MLEs described in prior slideContrast this with the observed 1986 loss development factors on the next slide

Incremental Development Factors - 1986(Repeat of Earlier Slide) Loss payment factors appear to be relatively stable for the 40 insurers that represent about 3/4 of the premium.They are highly unstable for the 210 insurers that represent about 1/4 of the premium. The variability appears to increase as size decreases

Maximum Likelihood Estimates of Expected Loss Ratios Estimates of the ELRs are more volatile for the smaller insurers.

Testing the Compound Negative Binomial (CNB) AssumptionCalculate the percentiles of each observation given E[Paid Loss].55 observations for each insurerIf CNB is right, the calculated percentiles should be uniformly distributed.Test with PP PlotSort calculated percentiles in increasing orderVector (1:n)/(n+1) where n is the number of percentilesThe plot of the above two vectors against each other should be on the diagonal line.

Interpreting PP PlotsTake 1000 lognormally distributed random variables with m = 0 and s = 2 as dataIf a whole bunch of predicted percentiles are at the ends, the predicted tail is too light.If a whole bunch of predicted percentiles are in the middle, the predicted tail is too heavy.If in general the predicted percentiles are low, the predicted mean is too high

Testing the CNB AssumptionsInsurer Ranks 1-40 (Large Insurers)This sample has 5540 or 2200 observations. According to the Kolomogorov-Smirnov test, D statistic for a sample of 2200 uniform random numbers should be within 0.026 of the 45 line 95% of the time.Actual D statistic = 0.042. As the plot shows, the predicted percentiles are slightly outside the 95% band. We are close.

Testing the CNB AssumptionsInsurer Ranks 1-40 (Large Insurers)Breaking down the prior plot by settlement lag shows that there could be some improvement by settlement lag.But in general, not bad!pp plots by settlement lag

Testing the CNB AssumptionsInsurer Ranks 41-250 (Smaller Insurers)This is bad!pp plots by settlement lag

Using Bayes TheoremLet W = {ELR, DevLag, Lag = 1,2,,10} be a set of models for the data.A model may consist of different models or of different parameters for the same model.For each model in W, calculate the likelihood of the data being analyzed.

Using Bayes TheoremThen using Bayes Theorem, calculate the posterior probability of each parameter set given the data.

Selecting Prior ProbabilitiesFor Lag, select the payment paths from the maximum likelihood estimates of the 40 largest insurers, each with equal probability.For ELR, first look at the distribution of maximum likelihood estimates of the ELR from the 40 largest insurers and visually smooth out the distribution. See the slide on ELR prior below.Note that Lag and ELR are assumed to be independent.

Prior Distribution of Loss Payment Paths Prior loss payment paths come from the loss development paths of the insurers ranked 1-40, with equal probabilityPosterior loss payment path is a mixture of prior loss development paths.

Prior Distribution of Expected Loss Ratios The prior distribution of expected loss ratios was chosen by visual inspection.

Predicting Future Loss PaymentsUsing Bayes TheoremFor each model, estimate the statistic of choice, S, for future loss payments.Examples of SExpected value of future loss paymentsSecond moment of future loss paymentsThe probability density of a future loss payment of x,The cumulative probability, or percentile, of a future loss payment of x.These examples can apply to single (AY,Lag) cells, of any combination of cells such as a given Lag or accident year.

Predicting Future Loss PaymentsUsing Bayes Theorem forSums over Sets of {AY,Lag}If we assume losses are independent by AY and LagActually use the negative multinomial distributionAssumes correlation of frequency between lags in the same accident year

Predicting Future Loss Payments Using Bayes TheoremCalculate the Statistic S for each model.Then the posterior estimate of S is the model estimate of S weighted by the posterior probability of each model

Sample Calculations for Selected InsurersCoefficient of Variation of predictive distribution of unpaid losses.Plot the probability density of the predictive distribution of unpaid losses.

Predictive DistributionInsurer Rank 7Predictive Mean = $401,951 KCV of Total Reserve = 6.9%

Predictive DistributionInsurer Rank 97Predictive Mean = $40,277 KCV of Total Reserve = 12.6%

CV of Unpaid Losses

Validating the Model on Fresh DataExamined data from 2001 Annual StatementsBoth 1995 and 2001 statements contained losses paid for accident years 1992-1995.Often statements did not agree in overlapping years because of changes in corporate structure. We got agreement in earned premium for 109 of the 250 insurers.Calculated the predicted percentiles for the amount paid 1997-2001Evaluate predictions with pp plots.

PP Plots on Validation DataKS 95% critical values = 13.03%

FeedbackIf you have paid data, you must also have the posted reserves. How do your predictions match up with reported reserves? In other words, is S&P right?Your results are conditional on the data reported in Schedule P. Shouldnt an actuary with access to detailed company data (e.g. case reserves) be able to get more accurate estimates?

Response Expand the Original Scope of the PaperCould persuade more people to look at the technical details.Warning Do not over-generalize the results beyond commercial auto in 1995-2001 timeframe.

Predictive and Reported ReservesFor the validation sample, the predictive mean (in aggregate) is closer to the 2001 retrospective reserve.Possible conservatism in reserves. OK?% means % reported over the predictive mean.Retrospective = reported less paid prior to end of 1995.

Predictive Percentiles of Reported ReservesConservatism is not evenly spread out.Conservatism appears to be independent of insurer sizeExcept for the evidence of conservatism, the reserves are spread out in a way similar to losses.Were the reserves equal to ultimate losses?

Reported Reserves More Accurate?Divide the validation sample in to two groups and look at subsequent development.1. Reported Reserve < Predictive Mean2. Reported Reserve > Predictive MeanExpected result if Reported Reserve is accurate.Reported Reserve = Retrospective Reserve for each group Expected result if Predictive Mean is accurate?Predictive Mean Retrospective Reserve for each group There are still some outstanding losses in the retrospective reserve.

Subsequent Reserve ChangesGroup 1 50-50 up/downUps are bigger

Group 2 More downs than ups

Results are independent of insurer sizeGroup 1Group 2

Subsequent Reserve ChangesThe CNB formula identified two groups where:Group 1 tends to under-reserveGroup 2 tends to over-reserveIncomplete agreement at Group level Some in each group get it rightDiscussion??

Main Points of PaperHow do we evaluate stochastic loss reserve formula? Test predictions of future loss paymentsTest on several insurersMain FocusAre there any formulas that can pass these tests?Bayesian CNB does pretty good on CA Schedule P data.Uses information from many insurers Are there other formulas? This paper sets a bar for others to raise.

Subsequent DevelopmentsPaper completed in April 2006Additional critiqueDescribe recent developmentsDescribe ongoing research

PP Plots on Validation DataClive Keatinges ObservationDoes the leveling of plots at the end indicate that the predicted tails are too light?The plot is still within the KS bounds and thus is not statistically significant.The leveling looks rather systematic.

Alternative to the KSAnderson-Darling TestAD is more sensitive to tails.Critical values are 1.933, 2.492, and 3.857 for 10, 5 and 1% levels respectively.Value for validation sample is 2.966Not outrageously bad, but Clive has a point.Explanation Did not reflect all sources of uncertainty??

Is Bayesian Methodology Necessary?Thinking Outside the TrianglePaper in June 2007 ASTIN ColloquiumWorks with simulated data on a similar modelCompares Bayesian with maximum likelihood predictive distributions

Maximum Likelihood Fitting MethodologyPP Plots for Combined FitsPP plot reveals the S-shape that characterizes overfitting.The tails are too light

Bayesian Fitting MethodologyPP Plots for Combined FitsNailed the Tails

IN THIS EXAMPLEMaximum Likelihood method understates the true variabilityI call this overfitting i.e. the model fits the data rather than the populationNine parameters fit to 55 pointsSPECULATION Overfitting will occur in all maximum likelihood methods and in moment based methodsi.e. GLM and Mack

Expository Paper in PreparationFocus on the Bayesian method described in this paperUses Gibbs sampler to simulate posterior distribution of the resultsComplete algorithm coded in RHope to increase population of actuaries who:Understand what the method meansCan actually use the method