Decision making as a model

3. Heavy stuff: derivation of two important theorems

Recap: two types of measures of sensitivity (independent of criterion:)

2. Area under ROC-Curve: A

1. Distance between signal and noise distributions

cf. d'

Recap: four types of measures for criterion:

1. Likelihood ratio LRc = p(xc|S)/p(xc|N) = h/f (vgl β)

hf

2. Position on x-axis (c)

3. Position in ROC-plot (left down vs. right up)

4. Slope of tangent at point of ROC (S)

c

Relationships between measures of sensitivity

Relationships between measures of criterion

Interpreting A

Area theorem:

A is equivalent with proportion correct answers in 2AFC-experiment:

Given:1 noise stimulus1 signal (+noise) stimulus,Which is which?

Makes sense. Important

Produce a formula for proportion correct in a 2AFC-experiment (Pc)

Produce a formula for area under ROC-curve (A)

Show that the formula for A looks like the formula for Pc

Show that formulas are identical.

Approach:

PFA

PH

ffnn ffss

xx0 0 λλ

∞ PH = ∫ fs(x)dx

λ ∞

PFA = ∫ fn(x)dx λ

= H(λ)

= FA(λ)

λ = FA-1(PFA)

ROC-curve: PH = H(λ) = H[FA

-1(PFA)]

Recap: In general:

Specific model depends on fn and fs

ffnnffss

xx0 0 λλ

∞ PH = ∫ fs(x)dx

λ ∞ PFA = ∫ fn(x)dx

λ

= H(λ)

= FA(λ)

Reinterpretation for 2A FC experiment:

Two alternatives correspond with two points on the x-axis. Suppose λ is noise stimulus:

if xn = λ PC = p(xs>xn), p(xs>xn) = H(λ)

“summate” H(λ) for every λ ,weighted for density of λ [= fn(λ)]:

∞ PC = ∫ H(λ)fn(λ)dλ

-∞

PFA

PH

ffnn ffss

xx0 0 λλ

∞ PH = ∫ fs(x)dx

λ ∞

PFA = ∫ fn(x)dx λ

= H(= H(λλ))

= F= FAA((λλ))

ROC-curve: ROC-curve: PPHH [= H( [= H(λλ)] as a )] as a

function of Pfunction of PFAFA [ [= F FAA((λλ)])]

1 A = ∫ H(λ)dFA(λ) 0

Area under Roc-curve:

∞ PC = ∫ H(λ)fn(λ)dλ

-∞ A looks like PC ; A is PC ; can be proved

proof (optional):

dFA(λ) d(λ)------- = -fn(λ)

∞ PC = ∫ H(λ)fn(λ)dλ

-∞

dFA(λ) = -fn(λ)dλ

Still two small chores:Limits of integration and minus sign

ffnn(x)dx = 1 - f(x)dx = 1 - fnn(x)dx(x)dx∞

λ∫

-∞

λ

∫

1 A = ∫ H(λ)dFA(λ) 0 -f-fnn((λλ)d)dλλ

Limits: if FA(λ)=PFA= 0 then λ = ∞if FA(λ)=PFA= 1 then λ = -∞

reverse: -H(λ)fn(λ) H(λ)fn(λ)

∞ PC = ∫ H(λ)fn(λ)dλ

-∞

∫

∫ 0

1 -∞

∞

∞

∫

∫ -∞

-∞

∞

ffnnffss

xx0 0 λλ-∞ ∞ 1 A = ∫ H(λ)dFA(λ) 0 -f-fnn((λλ)d)dλλ

fromIntegration over FA

tointegration over λ ?

∫ -H(λ)fn(λ)dλ ?

PFA

PH

ROC-curve: PH as a function of PFA

Every point of ROC-curve gives criterion/bias at that sensitivity

Slope tangent at that point as measure for bias/criterium S = .49

dPHSlope S = ------ dPFA

Measures for criterion

ffnn ffss

xx0 0 λλ

∞ PH = ∫ fs(x)dx

λ ∞

PFA = ∫ fn(x)dx λ

dPH dPH dPFA ----- = ----- • ------.dx dPFA dx

d(1-PFA) dPFA dPFA dPH---------- = - ----- = fn , ----- = - fn , also: ------ = - fs …dx dx dx dx

(chain rule)

dPH dPH/dx S = ----- = -----------

dPFA dPFA/dx

- fs fs= ----- = ----- - fn fn

= LRc

Measures for criterion

dPH dPHfrom ----- to ------dx dPFA

Intermezzo.

A category of (older) alternative models.

Finite state models:Measuring sensitivity.

Finite State models

High threshold: Yes 1 α detect signal

1-α η Yes uncertain

1-η No

1 η Yes noise uncertain

1-η No

HitHit

Hit Hit

MissMiss

FAFA

crcr

PH = α +η(1-α)PFA = η

Hits

Hits

False AlarmsFalse Alarms

αα

PH = α +η(1-α)PFA = η

ηη

Theoretical ROC curve

detect: Yes

uncertain: η Yes1-η No

α

“high threshold”

PH = α +η(1-α) PFA = η

PH = α + PFA(1-α)

PH = α + PFA - αPFA

α – αPFA = PH - PFA

α(1- PFA) = PH - PFA

PH – PFAα = -------------

1 - PFA

PH = α +η(1-α)

Pm = (1-η)(1-α)

Pm (1-α) = -------

(1-η)

Pm PH =α + η ------- (1-η)

η α = PH - ---- Pm (1-η)

Cf correction for guessing MC-questions:

N-AFC G Hits, F misses

η Sc = G - ------ F (1-η)

Analogously: a low threshold model :Signal leads always to uncertain state

noise leads with P = β to nondetect state (always NO) and else to uncertain state.

hits

False Alarms

1-β

β

Nondetect: No

Uncertain: η Yes1-η No

hitshits

False AlarmsFalse Alarms

A combined three state modelA combined three state model

N U D