27
Multinomial Processing Tree Models

Multinomial Processing Tree Models. Agenda Questions? MPT model overview. –MPT overview –Parameters and flexibility. –MPT & Evaluation Batchelder & Riefer,

  • View
    233

  • Download
    2

Embed Size (px)

Citation preview

Page 1: Multinomial Processing Tree Models. Agenda Questions? MPT model overview. –MPT overview –Parameters and flexibility. –MPT & Evaluation Batchelder & Riefer,

Multinomial Processing Tree Models

Page 2: Multinomial Processing Tree Models. Agenda Questions? MPT model overview. –MPT overview –Parameters and flexibility. –MPT & Evaluation Batchelder & Riefer,

Agenda

• Questions?• MPT model overview.

– MPT overview– Parameters and flexibility.– MPT & Evaluation

• Batchelder & Riefer, 1980.• Assignment 1 solution(s).• Math for next week.• Assignment 2 problem statement.

Page 3: Multinomial Processing Tree Models. Agenda Questions? MPT model overview. –MPT overview –Parameters and flexibility. –MPT & Evaluation Batchelder & Riefer,

Uses of MPT Models

• Data-analysis tool capable of disentangling and measuring separate contributions of different cognitive processes.– Provides a means for separately

measuring latent processes that are confounded in observable data.

• Framework for developing and testing quantitative theories.

Page 4: Multinomial Processing Tree Models. Agenda Questions? MPT model overview. –MPT overview –Parameters and flexibility. –MPT & Evaluation Batchelder & Riefer,

Purview of MPTs

• Categorical data: each observation falls into one and only one of a finite set of categories.– Example 1: Correct or incorrect.– Example 2: Reaction time < 100, 100 ≥ RT

< 200, 200 ≤ RT.

Page 5: Multinomial Processing Tree Models. Agenda Questions? MPT model overview. –MPT overview –Parameters and flexibility. –MPT & Evaluation Batchelder & Riefer,

Multinomial Distribution

• Consider a die with sides: 1, 1, 1, 2, 2, 3.

• If we roll the die 10 times, what is the probability we get five 1’s, three 2’s, and two 3’s?

10!

5!3!2!

1

2

⎝ ⎜

⎠ ⎟5

1

3

⎝ ⎜

⎠ ⎟3

1

6

⎝ ⎜

⎠ ⎟2

Page 6: Multinomial Processing Tree Models. Agenda Questions? MPT model overview. –MPT overview –Parameters and flexibility. –MPT & Evaluation Batchelder & Riefer,

Multinomial Distribution

• In general,

• Inverse goal is to determine the pi’s given the ni’s.

n!

n1!n2!n3!p1( )

n1 p2( )n2 p3( )

n3

Page 7: Multinomial Processing Tree Models. Agenda Questions? MPT model overview. –MPT overview –Parameters and flexibility. –MPT & Evaluation Batchelder & Riefer,

Statistical vs Explanatory

• Multinomial models (log-linear, logit) are statistical.– The parameters are used to explore main

effects and interactions.

• MPT models are explanatory.– The parameters of the MPT model

represent the underlying psychological processes.

Page 8: Multinomial Processing Tree Models. Agenda Questions? MPT model overview. –MPT overview –Parameters and flexibility. –MPT & Evaluation Batchelder & Riefer,

MPT Models

Root

ProcessState 2

ProcessState 1

ProcessState 3

ProcessState 4

Response 1

Response 2

Response 1

Page 9: Multinomial Processing Tree Models. Agenda Questions? MPT model overview. –MPT overview –Parameters and flexibility. –MPT & Evaluation Batchelder & Riefer,

MPT Models• The probability of each

process state change is represented by a parameter.

• The parameters range from 0 to 1.

• The parameters are independent.

• There are other restrictions…

Page 10: Multinomial Processing Tree Models. Agenda Questions? MPT model overview. –MPT overview –Parameters and flexibility. –MPT & Evaluation Batchelder & Riefer,

MPT Models

• The response probabilities are given by polynomials, e.g., P(E1)=c·r.

• The parameter estimators (and other important properties) are easy to find.

Page 11: Multinomial Processing Tree Models. Agenda Questions? MPT model overview. –MPT overview –Parameters and flexibility. –MPT & Evaluation Batchelder & Riefer,

Estimators

• A parameter is a descriptive measure in a model– E.g., a is a parameter the describes how quickly the line

increases in y=ax+b.

• An estimator is a function on the data that gives a parameter estimate, usually denoted with a hat, â.– E.g., given some data, get an estimate of how quickly the

linear trend increases.

• Estimators are usually picked to minimize the discrepancy between the predictions and the data.

Page 12: Multinomial Processing Tree Models. Agenda Questions? MPT model overview. –MPT overview –Parameters and flexibility. –MPT & Evaluation Batchelder & Riefer,

Parameters and Flexibility

• As a rule of thumb, a model should have fewer parameters than degrees of freedom in the data.

• This model is saturated.

Cond Data Model

1 .2 .2

2 .4 .4

3 .9 .9

p1

p2

p3

C1

C2

C3

Page 13: Multinomial Processing Tree Models. Agenda Questions? MPT model overview. –MPT overview –Parameters and flexibility. –MPT & Evaluation Batchelder & Riefer,

Parameters and Flexibility

• In general, the more parameters, the more flexible the model.

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

y=ax+b

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

y=b

Page 14: Multinomial Processing Tree Models. Agenda Questions? MPT model overview. –MPT overview –Parameters and flexibility. –MPT & Evaluation Batchelder & Riefer,

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

But…

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

1 parameter 2 parameters

Page 15: Multinomial Processing Tree Models. Agenda Questions? MPT model overview. –MPT overview –Parameters and flexibility. –MPT & Evaluation Batchelder & Riefer,

Further…

• It is not always easy to count parameters.

y = ax + b

y = d ⋅ex + f g2

Page 16: Multinomial Processing Tree Models. Agenda Questions? MPT model overview. –MPT overview –Parameters and flexibility. –MPT & Evaluation Batchelder & Riefer,

Parameters and Flexibility

• A more restricted a model is– usually simpler.– usually easier to

interpret.– usually more

falsifiable.

PossibleEvents

ObservedEvents

Model 1 Model 2

Page 17: Multinomial Processing Tree Models. Agenda Questions? MPT model overview. –MPT overview –Parameters and flexibility. –MPT & Evaluation Batchelder & Riefer,

Nested Models

Model 1: y = ax + b

Model 2 : y = ax

Model 3 : y = b

•Models 2 and 3 are submodels (nested in) Model 1.•Models 2 are 3 are NOT nested.

•The main benefit of nested models is that it makes it relatively easy to compare the goodness-of-fit of the two models.

Page 18: Multinomial Processing Tree Models. Agenda Questions? MPT model overview. –MPT overview –Parameters and flexibility. –MPT & Evaluation Batchelder & Riefer,

Parameters and Flexibility

• What is important is the flexibility of the model relative to– the data.– competing models.

Page 19: Multinomial Processing Tree Models. Agenda Questions? MPT model overview. –MPT overview –Parameters and flexibility. –MPT & Evaluation Batchelder & Riefer,

Identifiability• The parameters in a model are

identifiable if there is a unique set of parameters that give rise to the model predictions.

• Identifiability is especially desirable if the parameter values are to be interpreted.

vs

Page 20: Multinomial Processing Tree Models. Agenda Questions? MPT model overview. –MPT overview –Parameters and flexibility. –MPT & Evaluation Batchelder & Riefer,

Batchelder and Modeling

“The assumption is … clearly an approximation, but one that greatly simplifies the analysis of the model and still allows the model to reflect the main processing stages of the task” (p. 59).

Page 21: Multinomial Processing Tree Models. Agenda Questions? MPT model overview. –MPT overview –Parameters and flexibility. –MPT & Evaluation Batchelder & Riefer,

Batchelder and Modeling

“…there is usually a large number of parameters used to account for a small number of categories, leaving few, if any, degrees of freedom for testing the model’s fit…However, it is the measurement of the cognitive processes in the form of parameter estimates, and not the data-fitting capacity, that characterizes the usefulness of MPT models” (p. 81-82).

Page 22: Multinomial Processing Tree Models. Agenda Questions? MPT model overview. –MPT overview –Parameters and flexibility. –MPT & Evaluation Batchelder & Riefer,

Batchelder and Modeling

“… there will often be psychologically uniterpretable MPT models that nevertheless fit a given set of data well. Thus, the process of developing a valid model requires that one fit a number of data sets in the same paradigm and that the resulting parameter estimates be interpretable in terms of the underlying processing assumptions” (p. 76).

Page 23: Multinomial Processing Tree Models. Agenda Questions? MPT model overview. –MPT overview –Parameters and flexibility. –MPT & Evaluation Batchelder & Riefer,

Batchelder and Modeling

“Each MPT model is at best an approximation to a complete process description of categorical data, and the task of the modeler is to select the most important processes and capture them in a valid way” (p. 75).

Page 24: Multinomial Processing Tree Models. Agenda Questions? MPT model overview. –MPT overview –Parameters and flexibility. –MPT & Evaluation Batchelder & Riefer,

Batchelder and Modeling

“… the process of developing a valid model requires that one fit a number of data sets in the same paradigm and that the resulting parameter estimates be interpretable in terms of the underlying processing assumptions” (p. 76).

Page 25: Multinomial Processing Tree Models. Agenda Questions? MPT model overview. –MPT overview –Parameters and flexibility. –MPT & Evaluation Batchelder & Riefer,

Batchelder and Modeling

“…an even more crucial test of a model’s validity is to show that the model performs well under basic experimental manipulations. If the model’s parameters behave in a psychologically interpretable fashion, then the model gains credence as a valid measurement tool” (p. 82).

Page 26: Multinomial Processing Tree Models. Agenda Questions? MPT model overview. –MPT overview –Parameters and flexibility. –MPT & Evaluation Batchelder & Riefer,

Batchelder and Modeling

“…an acceptable MPT model must not only be able to fit data but its parameters must be globally identifiable, must be psychologically interpretable, and must pass appropriate validation experiments” (p. 78).

Page 27: Multinomial Processing Tree Models. Agenda Questions? MPT model overview. –MPT overview –Parameters and flexibility. –MPT & Evaluation Batchelder & Riefer,

Batchelder and Modeling

“…there are many models for categorical data that are not in the MPT class. If one of these models accounts for data in a a particular paradigm, then, technically one can infer that the MPT class is falsified in that paradigm. Of course, it may be possible to design an MPT model that closely mimics or approximates the successful fits of the non-MPT model; thus, it may be difficult to argue that the MPT framework is falsifiable in practice” (p. 78).