14
Erlang Mixtures The density of a mixture of Erlangs with common scale parameter: h(x |θ,α)= M X i =1 a i x i -1 e -x θ i (i - 1)! where a i is the non-negative weight of the i -th Erlang distribution in the mixture, and θ is the common scale parameter. Theoretical justification (Tijms Approximation): Let h(x |θ)= X j =1 [F (j θ) - F ((j - 1)θ)] x j -1 e -x θ j (j - 1)! . Then, lim θ0 H (x |θ)= F (x ), at all the continuous points.

Erlang Mixtures - University of Torontosheldon/ACT451-2017/Erlang-Fitting.pdf · Erlang Mixtures The density of a mixture of Erlangs with common scale parameter: h(xj ; ) = XM i=1

  • Upload
    lythien

  • View
    254

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Erlang Mixtures - University of Torontosheldon/ACT451-2017/Erlang-Fitting.pdf · Erlang Mixtures The density of a mixture of Erlangs with common scale parameter: h(xj ; ) = XM i=1

Erlang Mixtures

The density of a mixture of Erlangs with common scale parameter:

h(x |θ, α) =M∑i=1

aix i−1e−x/θ

θi (i − 1)!

where ai is the non-negative weight of the i-th Erlang distributionin the mixture, and θ is the common scale parameter.Theoretical justification (Tijms Approximation):Let

h(x |θ) =∞∑j=1

[F (jθ)− F ((j − 1)θ)]x j−1e−x/θ

θj(j − 1)!.

Then,limθ→0

H(x |θ) = F (x),

at all the continuous points.

Page 2: Erlang Mixtures - University of Torontosheldon/ACT451-2017/Erlang-Fitting.pdf · Erlang Mixtures The density of a mixture of Erlangs with common scale parameter: h(xj ; ) = XM i=1

The Standard EM algorithm

Proposed in Dempster, et al (1977), it is an iterative algorithm forfinding the maximum likelihood estimate of the parameters of anunderlying distribution from a set of incomplete data.

Page 3: Erlang Mixtures - University of Torontosheldon/ACT451-2017/Erlang-Fitting.pdf · Erlang Mixtures The density of a mixture of Erlangs with common scale parameter: h(xj ; ) = XM i=1

The EM algorithm for Erlang MixturesLet Φ = (θ, a1, a2, · · · , aM),

p(x , y |Φ) = ayxy−1e−x/θ

θy (y − 1)!

and

q(yi |xi ,Φ(k−1)) =p(xi , yi |Φ(k−1))

p(xi |Φ(k−1)).

Then

a(k)y =

1

n

n∑i=1

q(y |xi ,Φ(k−1)), y = 1, 2, · · · ,M,

and

θ(k) =

n∑i=1

xi/n

M∑y=1

ya(k)y

.

Page 4: Erlang Mixtures - University of Torontosheldon/ACT451-2017/Erlang-Fitting.pdf · Erlang Mixtures The density of a mixture of Erlangs with common scale parameter: h(xj ; ) = XM i=1

Uniform DistributionThe uniform distribution is a benchmark to test whether a fittingalgorithm is of high quality.A set of 1000 data between 1 and 2 were generated uniformly forstudy.

Figure: Histogram of uniform distribution and line for the fitteddistribution

Page 5: Erlang Mixtures - University of Torontosheldon/ACT451-2017/Erlang-Fitting.pdf · Erlang Mixtures The density of a mixture of Erlangs with common scale parameter: h(xj ; ) = XM i=1

Uniform Distribution

Figure: PP and QQ plots for uniform (1,2) and the fitted distribution

Page 6: Erlang Mixtures - University of Torontosheldon/ACT451-2017/Erlang-Fitting.pdf · Erlang Mixtures The density of a mixture of Erlangs with common scale parameter: h(xj ; ) = XM i=1

Mixture of two gamma’s

The underlying distribution is a mixture of 2 gamma distributionswith shape parameters being 2.6 and 6.3. The corresponding scaleparameters are 0.3125 and 0.8333 having weights 0.2 and 0.8respectively.

Page 7: Erlang Mixtures - University of Torontosheldon/ACT451-2017/Erlang-Fitting.pdf · Erlang Mixtures The density of a mixture of Erlangs with common scale parameter: h(xj ; ) = XM i=1

Mixture of two gamma’s, cont’d

Figure: Histogram for a mixture of two gammas and the fitted densityusing a mixture of 3 Erlangs

Page 8: Erlang Mixtures - University of Torontosheldon/ACT451-2017/Erlang-Fitting.pdf · Erlang Mixtures The density of a mixture of Erlangs with common scale parameter: h(xj ; ) = XM i=1

Lognormal distribution

Figure: Histogram for lognormal(0.03,0.2) and the fitted density using amixture of 11 Erlangs

Page 9: Erlang Mixtures - University of Torontosheldon/ACT451-2017/Erlang-Fitting.pdf · Erlang Mixtures The density of a mixture of Erlangs with common scale parameter: h(xj ; ) = XM i=1

PCS Catastrophe Loss Data

A data set of 1271 catastrophe losses in US from 1997 to 2005with all kinds of losses: flood, wind damage, etc..The data was supplied by ISO, a leading source of informationabout property losses in the U.S.Stylized Facts:

1. The maximum value of the data is 247 times of the mean.

2. There is 9.13% of the observations categorized as outliers if1.5 IQR rule is used.

3. The skewness and kurtosis for the data are 23.04 and 619.63.

4. 56% of the data is smaller than 0.1% of the maximum valuewhile 96.6% of the data is smaller than 1% of the maximumvalue.

All points above suggest that the data is heavy-tailed.

Page 10: Erlang Mixtures - University of Torontosheldon/ACT451-2017/Erlang-Fitting.pdf · Erlang Mixtures The density of a mixture of Erlangs with common scale parameter: h(xj ; ) = XM i=1

Fitting to the PCS DataA mixture of 12 Erlangs with a common scale parameter of5830867 fits the data well with each Erlang having significantweight.

Figure: Histogram of observed loss and line for the fitted distribution

Page 11: Erlang Mixtures - University of Torontosheldon/ACT451-2017/Erlang-Fitting.pdf · Erlang Mixtures The density of a mixture of Erlangs with common scale parameter: h(xj ; ) = XM i=1

p-p and q-q plots

Page 12: Erlang Mixtures - University of Torontosheldon/ACT451-2017/Erlang-Fitting.pdf · Erlang Mixtures The density of a mixture of Erlangs with common scale parameter: h(xj ; ) = XM i=1

Moments

Quantities Empirical Fitted Fitted/Empirical Percentage Difference (%)Mean 98.33 million 98.33 million 1.0000 0.00%

Standard Deviation 825.85 million 814.00 million 0.9857 -1.43%Skewness 23.03 23.37 1.0148 1.48%Kurtosis 619.28 643.31 1.0389 3.89%

Page 13: Erlang Mixtures - University of Torontosheldon/ACT451-2017/Erlang-Fitting.pdf · Erlang Mixtures The density of a mixture of Erlangs with common scale parameter: h(xj ; ) = XM i=1

Swedish mortality data

The mortality data for the Swedish cohort of Year 1911. The datais available at www.mortality.org.Reasons for choosing the data:

I Reliability due to the high life expectancy and homogeneity ofthe Swedish population.

I Cohort 1911 presents a “noticeable” accident hump at youthages.

Page 14: Erlang Mixtures - University of Torontosheldon/ACT451-2017/Erlang-Fitting.pdf · Erlang Mixtures The density of a mixture of Erlangs with common scale parameter: h(xj ; ) = XM i=1

Swedish Mortality Data

Figure: Fitted curve for qx