SA3 Tutorial 2

8/12/2019 SA3 Tutorial 2

http://slidepdf.com/reader/full/sa3-tutorial-2 1/32

Count Data Models



Poisson distribution - Examples

• Insurance claims in an year• Number of customers in a waiting line

• Number of defects in a given surface area

• Number of road construction projects in a city at a given tim

• Number of road accidents that occur on a particular stretcha week

Notice that all the above are counts.



Poisson Distribution - Properties

!/)(Pr xe x X ob x

Mean = Variance =









Regression model - I

• Linear regression• Assumptions

• Linear

• Independent

• Normal

• Equal Variance

For each value of the regressor (X), the distribution of the response (Y) is thefor a linear shift





Poisson Regression - Assumptions

• Assumptions

1. Probability function of response (given ) is Poisson withparameter .

2.

3. Observations are independently distributed.

k k X X e

...110



Regression model – II: PoissonRegression

• The mean of the distribution is positive. In poisson regressiothe response is modeled as a linear function of the regresso



Interpretation of β



Interpretation of β (contd)





Linear regression - Results



Poisson Regression - Results





For poisson model,

Deviance,

i

in

i i

y y D

ˆlog

1



Property of deviance

•Deviance has approximately Chi-square distribution with n

degrees of freedom where n is the number of observations the number of parameters.



Interpretation of the outputs

• Consider linear regression.

• The regression equation is:

• predicted number of claims = -1.12 +0.51 numveh +0.02 age.• For numveh = 1 and age = 25, the expected number of claims is -0.22

can never be negative!

• Consider the poisson regression.

• The regression equation is

• predicted number of log claims = -3.20 + 0.74 numveh + 0.03 age

• For numveh = 1 and age = 25, the expected number of claims is

a positive number



Interpretation of the outputs

• Linear regression

• If the number of vehicles increases by 1 (whether it is from 1 to 2 or frthe age remains the same, the number of claims increases by about 0

• Poisson Regression

• If the number of vehicles increases by 1, while the age remains the sanumber of claims get multiplied by =2.1.

• For age =40 and numveh = 2, the predicted number of claims is =

• For age = 40 and numveh =3, the predicted no. of claims is 0.59 X 2.1=

• For age = 40 and numveh =4, the predicted no. of claims is 1.24 X 2.1=

74.0e

52.0e



Which model to use?

• We already saw that linear regression can give negative valupredicted count.

• Poisson Regression always gives a positive value.

• We can look at the plot of Pearson residuals against the regand see which looks better.

• We can also look at the plots of predicted vs actual observeeach case and see which fit is better.



When is poisson regression appropriate?

• The response is count data.

• The conditional distribution of Y given X is poisson.

• Mean of Y is less than 10, preferably between 1 and 5.

• If mean is greater than 10 one can try the linear regression of log Y agregressors.



Caveat

• Heterogeneity in data

• Data collected in clusters

• Missing regressors

• Consequence – Overdispersion (variance > mean)



Overdispersion



Overdispersion

• Overall Poisson look• Different clusters

• Overdispersion: variance > mean







Negative binomial model

• Y follows negative binomial with mean µ and variance µ+k µ

• Now use the poisson type model.

• How to detect overdispersion?

• Fit a poisson model and obtain plots of Pearson residuals vs regres

• If one or more of these show a funnel type shape, then there isoverdispersion. If you find this use negative binomial regression.



Zero-inflated Poisson model

• Prob(Y=0)=p

• Prob(Y ~Poisson(α)=1 – p.

• So Prob(Y=0) =p+ (1-p)e^-α

• And Prob (Y=r)= (1-p)(e^-α)(α^r)/r!





Hurdle model

• Prob(Y=0)=p

• Prob(Y>0)=1-p

• (Y=r|Y>0) has a truncated poisson truncated at 0.

• So ,...2,1),!)1/(()1()(Pr r r e pr Y ob r



Thank You!

Documents

SA3 Tutorial 2