Time Series Models for Event Counts, Ipxb054000/code/count-examples/ECTS-I-2010.pdf · Put the pests.r and the data les you are going to use in the same folder. Patrick T. Brandt

AgendaEvent Count Models

Event Count Time Series

Time Series Models for Event Counts, I

Patrick T. Brandt

University of Texas, Dallas

July 2010

Patrick T. Brandt Time Series Models for Event Counts, I



Agenda

Introduction to basic event count time series

Examples of why we need separate models for these kind ofdata

PEWMA and PAR(p) introduction

Fitting and interpreting PEWMA and PAR(p) models usingPESTS: dynamic inferences

Changepoint models for count data

Some recent extensions and new models




Preface / Getting Started

Get R from your favorite CRAN mirror. The mirror list is at:http://cran.r-project.org/mirrors.html

Get the R source code for PESTS fromhttp://www.utdallas.edu/~pbrandt/code/pests.r

These slides, data, and R code for examples are athttp://www.utdallas.edu/~pbrandt/code/count-examples

Put the pests.r and the data files you are going to use in the samefolder.


http://cran.r-project.org/mirrors.html

http://www.utdallas.edu/~pbrandt/code/pests.r

http://www.utdallas.edu/~pbrandt/code/count-examples



1 Event Count ModelsData ExamplesPoisson ModelsNegative Binomial Models

2 Event Count Time SeriesExisting approachesModels for time series of countsPEWMAPAR(p)




Data ExamplesPoisson ModelsNegative Binomial Models

Example: Mayhew’s Legislation Data





Example: Militarized Interstate Disputes (MIDS)

Time

MIDS

1850 1900 1950

2040

6080

100

120

140

0 5 10 15 20

0.2

0.0

0.2

0.4

0.6

0.8

1.0

Lag

ACF

Series 1





Chinese and Taiwanese MIDS

01

23

4

Chi

na th

reat

s

01

23

45

Chi

na a

ctio

ns

0.0

1.0

2.0

3.0

Taiw

an th

reat

s

01

23

45

6

1950 1960 1970 1980 1990 2000

Taiw

an a

ctio

ns

Time

China and Taiwan threats and actions, MIDS: 1950−2001





Count data models

Count data are the result of a process of measuring thenumber of discrete events over some period of time.

Typically, these models assume that the process thatgenerates the events is independent of time (t). This meansthat they are memoryless.

The times between events are assumed to be independent andexponentially distributed.

This is a very strong set of restrictions: most event dataviolate them in one or more ways.





Event Count Models

Typically, one of several models are used to fit a regression modelto count data:

Poisson regression

Negative binomial regression

Generalized event count

Generalized estimating equations





Poisson Models

Standard approach to modeling event count data is to use aPoisson distribution:

Pr (yt |µt) =µyt

t e−µt

yt !.

Estimation of the mean parameter is accomplished viamaximum likelihood methods.

Note that for the Poisson model, E (yt) = V (yt) = µ.

A Poisson regression model can be created by theparameterization

µt = exp (Xtδ) .





Negative binomial models

This model allows for V (y) > E (y) which is common in countdata.This is known as overdispersion.

The negative binomial distribution is given by

Pr(yt |µt , νt) =Γ(yt + νt)

yt !Γ(νt)

(νt

νt + µt

)νt(

µt

νt + µt

)yt

The νt parameter captures the level of overdispersion, or howmuch larger the variance is than the mean. We can make this aregression model by defining µt = exp (Xtδ).





How the negative binomial captures overdispersion

The conditional mean for the negative binomial (NB) regressionmodel is

E [yt |Xt ] = µt = exp (Xtδ) .

The conditional variance is

V [yt |Xt ] = µt

(1 +

µt

νt

)= exp (Xtδ)

(1 +

exp (Xtδ)

νt

).

This variance will be unidentified since the term νt has a t index.





Identifying the negative binomial

The common assumption is that the variance parameter νt is thesame across all of the observations (this same assumption is usedin the subsequent time series models).If we assume that νt = α−1 and α > 0 then

V [yt |Xt ] = µt

(1 +

µt

α−1

)= µt + αµ2

t

(So now you know how to interpret correctly that α parameterreported from Stata for nbreg.)




Existing approachesModels for time series of countsPEWMAPAR(p)

Standard Approaches Fail

Poisson regression models are limited because they assumeevents are independent

Alternative models assume a particular dependence: negativebinomial and generalized event count (GEC)

OLS / ARIMA models use the wrong (Gaussian) distribution

Including a lagged endogenous count implies a growth ratemodel

These are the motivating arguments for Brandt, Williams,Fordham and Pollins (2000: American Journal of PoliticalScience) and Brandt and Williams (2001: Political Analysis).





Why does this matter?

Little is known about the efficiency properties of event countmodels in the presence of dynamic mis-specification.

If count data demonstrate serial dependence, how can wemodel this dependence?

Further, if we fail to model this dependence, how biased /inconsistent / inefficient are the estimates we get?

Can’t we just fix all of this with a lagged dependent variablelike we do in most other models?





Limits of a lagged count model

Exponentiated coefficient on the lagged variable is no longeran autocorrelation coefficient.

It is a growth rate.

Model is only appropriate for non-stationary or trending eventcounts, since the mean is an exponential function of time.





Just say NO to the lagged count model!

Let zt ∼ Po (µt), µt = exp (Xtδ + ρzt−1) and Xt are i.i.d.The growth rate of this lagged Poisson regression model is thedifference the logged mean counts:

ln (µt)− ln (µt−1) = Xtδ − Xt−1δ + ρzt−1 − ρzt−2.

Taking expectations gives

E [ln (µt)− ln (µt−1)] = ρE [zt−1 − zt−2] .

Unless ρ = 0 or E [zt−1 − zt−2] = 0, this model implies anon-zero growth rate for the conditional mean.

The coefficient ρ is a growth rate rather than anautocorrelation or discounting coefficient.





Desiderata of a count time series model

Need models that can deal with trends in counts (PEWMA)

Need models that can deal with cycles in counts (PAR(p))

Diagnostics for model selection

When are Gaussian-based models OK?





Models for count time series

There are two frameworks for count time series models:

Observation driven models: past counts predict currentcounts.

PEWMAPAR(p)

Parameter driven models: parameters change over time.

Changepoint modelsLatent dynamic parameter or factor models

This list of examples is by no means exhaustive.Count time series models fit into one of these approaches andthere is a fair amount of observational equivalence across thesemodeling strategies.





Simple Diagnostics for Count Time Series

We most often want to know if the count time series are seriallycorrelated. Cameron and Trivedi (1998) show that one can usestandard time series diagnostics for serial correlation to determinewhether counts should be modeled with a time series.

1 Standardize the count time series: for each observationsubtract off the mean and divide by the standard deviation ofthe series (so just like finding a z-score).

2 Compute the autocorrelation function of the standardizedcounts.





Example: using autocorrelation functions to diagnosecount serial correlation

This example is based on the data in StraitsMIDS-example

load("StraitsMIDS.RData")

# Make d into a ts() objectd <- as.ts(d)

# Compute the ACFs for the standardized data

acf(apply(d, 2, scale))





ACF for China-Taiwan MIDS

0 2 4 6 8

−0.

20.

41.

0

Lag

AC

F

China threats

0 2 4 6 8

−0.

20.

41.

0

Lag

Chnt & Chna

0 2 4 6 8

−0.

20.

41.

0

Lag

Chnt & Twnt

0 2 4 6 8

−0.

20.

41.

0

Lag

Chnt & Twna

−10 −6 −2

−0.

20.

41.

0

Lag

AC

F

Chna & Chnt

0 2 4 6 8

−0.

20.

41.

0

Lag

China actions

0 2 4 6 8

−0.

20.

41.

0

Lag

Chna & Twnt

0 2 4 6 8

−0.

20.

41.

0

Lag

Chna & Twna

−10 −6 −2

−0.

20.

41.

0

Lag

AC

F

Twnt & Chnt

−10 −6 −2

−0.

20.

41.

0

Lag

Twnt & Chna

0 2 4 6 8

−0.

20.

41.

0

Lag

Taiwan threats

0 2 4 6 8

−0.

20.

41.

0

Lag

Twnt & Twna

−10 −6 −2

−0.

20.

41.

0

Lag

AC

F

Twna & Chnt

−10 −6 −2

−0.

20.

41.

0

Lag

Twna & Chna

−10 −6 −2

−0.

20.

41.

0

Lag

Twna & Twnt

0 2 4 6 8

−0.

20.

41.

0

Lag

Taiwan actions





ACF interpretation

Plots on the diagonal give the autocorrelation functions.

Plots off the diagonal are the cross-correlation functions.

First autocorrelation value is always 1 (why?)

See evidence of serial correlation in the China actions, Taiwanthreats, and Taiwanese action series.





Two Observation Driven Models for Event Count TimeSeries

PEWMA Poisson Exponentially Weighted Moving Average.Models a moving mean for persistent event countdata. This is used for time-varying or random walkcount data.

PAR(p) Poisson Autoregressive Model of Order p. Models alinear autoregressive, mean reverting event countseries.





PEWMA: A Model for Persistent Counts

Model for persistent count data: the Poisson exponentiallyweighted moving average (PEWMA)

The PEWMA is a structural time series model. The modeland method of estimation were originally proposed by Harveyand Fernandes (1989)

Easily implemented: our implementation modifies the originalto correct the transition equation as proposeed by Shephard(1994).





The PEWMA model details

Measurement Equation:

Pr (yt |µt) =µyt

t e−µt

yt !

Transition Equation:

µt = µ∗t−1 exp (Xtδ + rt) ηt ,

where ηt ∼ Beta (ωat−1, (1− ω) at−1)

Conjugate Prior:

µ∗t−1 ∼ Γ (at−1, bt−1) .





PEWMA parameters

The hyperparameter ω ∈ (0, 1], measures the dependence.µ∗t−1 is the period t − 1 conditional mean.

Small values indicate more dynamics and dependence in thedata, while values near one indicate independence (i.e.,Poisson model).

Values of the regressor coefficients, δ, can be interpreted as ina standard Poisson model.

PEWMA nests Poisson: can use standard ML tests toevaluate dependence v. independence.

Transition equation differs from Harvey and Fernandes. It isbased on the gamma distributed transition results of Shephard(1994). It allows for a separate growth rate (rt) in eachperiod. The mean growth rate is zero.





PEWMA Forecast Function

Based on repeated substitutions of a and b the forecastfunction for the one-step ahead prediction is:

yT+1|T =exp (XT+1δ + rT+1)

∑T−1j=0 ωjyT−j∑T−1

j=0 ωj exp(X ′T−jδ + rT−j

) .

This is an exponentially weighted moving average.

When T is large, yT+1|T approaches

µT = ωyT |T−1 + (1− ω) yT for T = T + 1, ...,T + h.





PEWMA Interpretation

Latent variable is a random walk Latent mean is a random walk.

Coefficients of regressors Can use standard methods for Poissonregression. No impact multipliers because this is anEWMA.

Nests the Poisson Test whether ω = 1 to see if you can just use aPoisson regression.





PAR(p) model

An event count series can also be modeled using a “linearautoregressive process.”

This process can be used to define transition equation for astate space or non-linear filter model.

In this specification, the counts today will depend on the ppast values via an autoregression.





Why develop the PAR(p) model?

Alternative specification for stationary count data series.

Ease of interpreting predictive distribution is based on a linearfunction.

Generalization: The AR model can be account for higher order,finite lag structures.

Diagnostics: ACF and PACF routines can be used for diagnostics.





PAR(p) model

Measurement Equation:

Pr (yt |mt) =myt

t e−mt

yt !,

Transition Equation:

mt =

pXi=1

ρiyt−i +

1−

pXi=1

ρi

!µt

µt = exp (Xtδ)

Conjugate Prior:

Pr (mt |Yt−1) = Γ (σt−1mt−1, σt−1)

mt−1 = E [yt |Yt−1]

σt−1 = Var [yt |Yt−1]

mt−1 > 0, σt−1 > 0





PAR(p) likelihood

The forecast density for the one-step ahead distribution is

Pr (yt |Yt−1) =

∫θ

Pr (yt |θt)Measurement

· Pr (θt |Yt−1)Transition

dθ

=Γ(σt|t−1mt|t−1 + yt

)Γ (yt + 1) Γ

(σt|t−1mt|t−1

) (σt|t−1

)σt|t−1mt|t−1

·(1 + σt|t−1

)−(σt|t−1mt|t−1+yt) .

This is a negative binomial distribution.





PAR(p) Interpretation

Short run impacts

SR =(

1−∑

ρi

)exp (Xtδ)δ

Long run multipliers

LR = exp (Xtδ)δ

See why the static Poisson model is biased?





Application: Hospital Deaths

Question: Was Orville Lynn Majors’ responsible forsuspicious deaths at the local hospital?

Data: Number of monthly deaths in Vermillion County(Indiana) Hospital, January 1991-December 1995.

Independent variables: Two temporary interventions,1 Accused nurse working in hospital: captures the epidemic

effects.2 Post-nurse period: captures the post-epidemic effects.





Deaths data

Dea

ths

1991 1992 1993 1994 1995 1996

510

1520 Intervention Period

0 5 10 15

0.2

0.2

0.6

1.0

AC

F

2 4 6 8 10 12 14

0.2

0.1

0.3

Parti

al A

CF





Where is Vermillion County, Indiana?

In the middle of nowhere (the nearest place you might know is Terre Haute, Indiana)





Models considered

Models: Estimate interventions’s effects using,

1 PEWMA

2 PAR(1)

3 PAR(2)

4 Poisson

5 Lagged Poisson

6 Negative Binomial

7 Lagged Negative Binomial





Results

Model PEWMA PAR(1) PAR(2) Poisson Lagged Negative Lagged Neg.Poisson Binomial Binomial

Nurse 0.632 0.714 0.832 0.696 0.610 0.696 0.608Intervention (0.164) (0.206) (0.201) (0.111) (0.137) (0.130) (0.162)Post-Nurse -0.722 -0.819 -0.927 -0.671 -0.646 -0.724 -0.693Intervention (0.185) (0.269) (0.319) (0.147) (0.151) (0.180) (0.183)Constant 2.178 2.144 2.157 2.063 2.162 2.066

(0.109) (0.117) (0.053) (0.116) (0.062) (0.138)ω 0.929

(0.125)ρ1 0.381 0.374 0.012 0.012

(0.062) (0.069) (0.011) (0.014)ρ2 0.018

(0.198)γ 0.391 0.383

(0.637) (0.656)N 60 60 60 60 59 60 59

Final LLF -156.61 -153.87 -151.65 -156.36 -153.59 -154.41 -151.76AIC 317.21 313.73 309.29 318.72 315.19 314.83 311.53





Interpretation

For the Poisson, PEWMA, or negative binomial regressions,the percentage change in the odds for a unit change in aregressor is

100 · (exp (β)− 1)

For the PAR(p), where Xt = [xt , zt ] and β = [β1, β2] are theregression parameters for xt and zt respectively, the estimatedpercentage change in the counts for ∆zt can be found by,

100

[(1−

∑pi=1 ρi

)exp (xtβ1) (exp (∆ztβ2)− 1)∑p

i=1 ρiyt−i +(1−

∑pi=1 ρi

)exp (xtβ1)

]





Computing the percentage changes for the interventions

PAR(1) nurse intervention coefficient is 0.714 and ρ1 is 0.381. Sothe percentage change is

100(1− 0.381)(exp(0.714)− 1) = 65

The post-nurse intervention coefficient is −0.819. So thepercentage change after is

100(1− 0.381)(exp(−0.819)− 1) = −35





Predicted Percentage Changes

InterventionModel Nurse Working Post-Nurse

PEWMA 88% -51%PAR(1) 65% -35%PAR(2) 79% -37%Poisson 101% -48%Lagged Poisson 84% -48%Neg. Binomial 101% -52%Lagged Neg. Binomial 84% -50%





Predictive PDFs for the Models





Conclusions: Hospital Deaths Series

PAR(p) models are more consistent with the data.

Static event count models predict a constant mean, ratherthan capturing the change in the number of deaths over time.

Only event count time series models capture the effects of thenurse’s arrival and departure.

Orville Lynn Majors was convicted of 6 murders and sentencedto 360 years in prison in 1999.





























Documents

Time Series Models for Event Counts, Ipxb054000/code/count-examples/ECTS-I-2010.pdf · Put the pests.r and the data les you are going to use in the same folder. Patrick T. Brandt