1 Finite Population Inference for Latent Values Measured with Error from a Bayesian Perspective Edward J. Stanek III Department of Public Health University

1

Finite Population Inference for Latent Values Measured with Error from

a Bayesian Perspective

Edward J. Stanek IIIDepartment of Public HealthUniversity of Massachusetts

Amherst, MA

2

Collaborators

Parimal Mukhopadhyay, Indian Statistics Institute, Kolkata, IndiaViviana Lencina, Facultad de Ciencias Economicas, Universidad Nacional de Tucumán, CONICET, ArgentinaLuz Mery Gonzalez, Departamentao de Estadística, Universidad Nacional

de Colombia, Bogotá, ColombiaJulio Singer, Departamento de Estatística, Universidade de São Paulo, BrazilWenjun Li, Department of Behavioral Medicine, UMASS Medical School,

Worcester, MARongheng Li, Shuli Yu, Guoshu Yuan, Ruitao Zhang, Faculty and Students

in the Biostatistics Program, UMASS, Amherst

3

Outline

• Review of Finite Population Bayesian Models1. Populations, Prior, and Posterior2. Notation 3. Example4. Exchangeable distribution

• Addition of Measurement Error1. Latent values and Response2. Heterogeneous variances3. Prior distribution of response for a prior point (vector of latent values)4. Prior and Data- matching the subjects in the data to random variables in the

population5. Subsets of prior points:

i. for populations not including some subjects in the data ii. for populations including subjects in data, where the sample doesn’t

include the subjects in the dataiii. for populations and samples that include subjects in data.

6. Posterior points (corresponding to 5iii)7. Marginal posterior points (over measurement error among remaining

subjects)

4

Bayesian Model

General Idea

Populations Populations

# Posterior Populations: H

DataPrior Posterior

# Prior Populations: H

Prior

Probabilities

Posterior

Probabilities

L

x

1 2

1 2

1 2

1 2

H

H

H

H

L L L

p p p

1 2

1 2

1 2* * *1 2

H

H

H

H

L L L

p p p

Review

5

Bayesian Model

Population Notation

Population

; 1,...,h jy j N

; 1,...,h jL j N

0

0

1

L

yN

y

LabelLatent Value

Labels

Parameter

0hλ

Vector

0hy

Data

; 1,...,s

x s n IxVector

Iλ ; 1,...,s

L s n

1

L

x xn

Review

6

Exchangeable Prior Bayesian Model- Example: H=3, N=3, n=2

Populations

Data

Prior ;

1,...,s

x

s n

1

2

3

,10 , ,5 , ,3

,10 , ,5 , ,6

,10 , ,2 , ,12

Rose Lily Daisy

Rose Lily Daisy

Rose Daisy Violet

10 5 3 31 2

31 2

31 2

, , , , , ,

86 7

0.20.2 0.6 pp p

10 5 6 , ,

10 122 , ,

, , 10 5 3

10 5 6 10 122

Posterior

31 2

31 2

31 2

, , , , , ,

86 7

?? ? pp p

10 5 3 10 5 6 10 122

Review

7


Populations

Data

Prior

Posterior

;

1,...,s

x

s n

10 5 3 31 2

31 2

31 2

, , , , , ,

86 7

0.20.2 0.6 pp p

1

2

3

10 5 6 , ,

10 122 , ,

, , 10 5 3

10 5 6 10 122

31 2

31 2

31 2

, , , , , ,

86 7

?? ? pp p

10 5 3 10 5 6 10 122

1

2

3

6

7

8

1

2

3

0.2

0.6

0.2

p

p

p

, 10 5

Suppose the Data is

Prior

Review

8


Populations

Data

Prior

Posterior

10 5 3 31 2

31 2

31 2

, , , , , ,

86 7

0.20.2 0.6 pp p

1

2

3

10 5 6 , ,

10 122 , ,

, , 10 5 3

10 5 6 10 122

31 2

31 2

31 2

, , , , , ,

86 7

?? ? pp p

10 5 3 10 5 6 10 122

1

2

3

6

7

8

1

2

3

0.2

0.6

0.2

p

p

p

, 10 5

Prior

Review

9

Exchangeable Prior Bayesian Model-Example: H=3, N=3, n=2

Populations

Data

Prior

Posterior10 5 3

31 2

31 2

31 2

, , , , , ,

86 7

0.20.2 0.6 pp p

, 10 5

10 5 6 10 122

1 2

1 2

* *1 2

, , , ,

6 7

0.2 0.6

0.8 0.8p p

10 5 3 10 5 6

1

2

3

6

7

8

1

2

3

0.2

0.6

0.2

p

p

p

PosteriorPrior

1

2

6

7

1

2

0.25

0.75

p

p

Review

10

Exchangeable Prior Populations

General Idea When N=3

11

1 1 12

13

Y

Y Y

Y

Y

p pYY

1p

Each Permutation p of subjects in L(i.e. each different listing)

1p Y

Joint Probability Density

2 11 1

2 2 2 12 3

2 13 2

Y Y

Y Y Y

Y Y

Y 2p

2p Y

6 11 3

6 6 6 12 2

6 13 1

Y Y

Y Y Y

Y Y

Y

6

!

p

N

6p Y

Must beidentical

1

2

3

i

Y

Y Y

Y

YExchangeableRandomVariables

p YThe commondistribution

GeneralNotation

Assigns (usually) equal probability to eachpermutation of subjectsin the population.

Review

11

Exchangeable Prior Populations N=3

Potential Response for Each Listing of subjects

1p

2p

Listings

11 0

Rose

Lily

Daisy

y

y

y

y u y

22 0

Rose

Daisy

Lily

λ u λ

11 0

Rose

Lily

Daisy

λ u λ

22 0

Rose

Daisy

Lily

y

y

y

y u y

Latent Values for Listing

Lilyy

Rosey

Lilyy

Daisyy

Rosey

Rosey

Rosey Rosey

Rosey

Lilyy

Lilyy Lilyy

Lilyy

LilyyDaisyy

Daisyy

Daisyy

Daisyy Daisyy

Latent Values for permutations of listing

11u y

12u y

15u y

13u y

14u y

16u y

Review

12

Exchangeable Prior Population

Permutations

Lilyy

Rosey

Lilyy

Daisyy

Rosey

LilyyDaisyy

11u y

12u y

13Y

Rose

Daisy

Lily

11 0

10

5

2

Rose

Lily

Daisy

y

y

y

y u y

Listing p=1

11Y

12Y

11u y

12u y

11

1 12

13

Y

Y

Y

Y

Review

13


Permutations

Lilyy

Rosey

Lilyy

Daisyy

Rosey

Rosey

Rosey Rosey

Rosey

Lilyy

Lilyy Lilyy

Lilyy

LilyyDaisyy

Daisyy

Daisyy

Daisyy Daisyy

11u y

12u y

15u y

13u y

14u y

16u y

11Y

12Y

13Y

1u

2u

3u

4u

5u

6u

Rose

Daisy

Lily

11 0

10

5

2

Rose

Lily

Daisy

y

y

y

y u y

Listing p=1

11

1 12

13

Y

Y

Y

Y

Review

14


2

10

2

5

Rose

Daisy

Lily

y

y

y

y

Rose

Daisy

Lily

22 0

Rose

Daisy

Lily

y

y

y

y u y

21u y

22u y

25u y

23u y

24u y

26u y

Daisyy

Lilyy

Rosey Daisyy

Lilyy

RoseyDaisyy

LilyyRosey

Daisyy

Lilyy

Rosey Daisyy

Lilyy

Rosey

Daisyy

LilyyRosey

Listing p=2

21Y

22Y

23Y

1u

2u

3u

4u

5u

6u

21

2 22

23

Y

Y

Y

Y

Review

15


Permutations of Listings

11Y

12Y

13Y

1u

2u

3u

4u

6u

5u

Listing p=1

Listing

21Y

22Y

23Y

1u

2u

3u

4u

5u

6u

Listing p=2

31Y

32Y

33Y

1u

2u

3u

4u5u

6u

Listing p=3

41Y

42Y

43Y

1u

2u

3u

4u

5u

6u

Listing p=4

51Y

52Y

53Y

1u

2u3u

4u

5u

6u

Listing p=5

61Y

62Y

63Y

1u

2u

3u

4u

5u

6u

Listing p=6

Review

16



Listings

Latent Value Vectors for permutations of listing

* * * * * *

11 12 13 14 15 16

21 22 23 24 25 26

31 32 33 34 35 36

41 42 43 44 45 46

51 52 53 54 55 56

61 62 63 64 65 66

1 2 3 4 5 6

1

2

3

4

5

6

p p p p p p

p

p

p

p

p

p

y y y y y y

y y y y y y

y y y y y y

y y y y y y

y y y y y y

y y y y y y

1

2

3

p p

Y

Y Y

Y

Y

Potential response for Random VariablesFor Listing p

*

* 0

pp

ppy u u y

Circled points are equal and have equal probability,for different listings.

Listing

Review

17


Permutations of Listings

11Y

12Y

13Y

1u

2u

3u

4u

6u

5u

Listing p=1

Same Point in Listing

21Y

22Y

23Y

1u

2u

3u

4u

5u

6u

Listing p=2

31Y

32Y

33Y

1u

2u

3u

4u5u

6u

Listing p=3

41Y

42Y

43Y

1u

2u

3u

4u

5u

6u

Listing p=4

51Y

52Y

53Y

1u

2u3u

4u

5u

6u

Listing p=5

61Y

62Y

63Y

1u

2u

3u

4u

5u

6u

Listing p=6

Review

18



Listings

Latent Value Vectors for permutations of listing

* * * * * *

11 12 13 14 15 16

21 22 23 24 25 26

31 32 33 34 35 36

41 42 43 44 45 46

51 52 53 54 55 56

61 62 63 64 65 66

1 2 3 4 5 6

1

2

3

4

5

6

p p p p p p

p

p

p

p

p

p

y y y y y y

y y y y y y

y y y y y y

y y y y y y

y y y y y y

y y y y y y

Potential response for Random VariablesFor Listing p

*

* 0

pp

ppy u u y

Circled points are equal and have equal probability,And are the same point for different listings.

Same Point in Listing

1

2

3

i

Y

YY

Y

Y

Review

19

Bayesian Model

Link between Prior and Data

Populations

DataPrior


;

1,...,s

x

s n

1 2

1 2

1 2

1 2

H

H

H

H

L L L

p p p

N=3

Supposen=2

1

2

3

i

Y

YY

Y

Y

Realizations of

1 2,Y Y are the Data

Review

20

Bayesian Model Exchangeable Prior Populations N=3

11Y

12Y

1u

2u

3u

4u

5u

6u

10

10

5

5

2

2

Listing p=1Sample Space n=2

Prior

11Y

12Y

13Y

1u

2u

3u

4u

6u

5u

11

1 12

13

Y

Y

Y

Y

Review

21

Bayesian Model Exchangeable Prior Populations N=3: Sample Point n=2

31Y

32Y

1u

2u

3u

4u

5u

6u

10

10

5

5

2

2

Listing p=3

41Y

42Y

1u

2u

3u

4u

5u

6u

10

10

5

5

2

2

Listing p=4

11Y

12Y

1u

2u

3u

4u

5u

6u

10

10

5

5

2

2

Listing p=1

Listing p=2

21Y

22Y

1u

2u

3u

4u

5u

6u

10

10

5

5

2

2

51Y

52Y

1u

2u

3u

4u

5u

6u

10

10

5

5

2

2

Listing p=5

61Y

62Y

1u

2u

3u

4u

5u

6u

10

10

5

5

2

2

Listing p=6

Review

22

Exchangeable Prior Populations N=3 Sample Points

Lilyy

Rosey

Lilyy

Daisyy

Rosey

Rosey

Rosey Rosey

Rosey

Lilyy

Lilyy Lilyy

Lilyy

LilyyDaisyy

Daisyy

Daisyy

Daisyy Daisyy

11u y

12u y

15u y

13u y

14u y

16u y

11Y

12Y

13Y

1u

2u

3u

4u

5u

6u

Rose

Daisy

Lily

11 0

10

5

2

Rose

Lily

Daisy

y

y

y

y u y

Listing p=1

11

1 12

13

Y

Y

Y

Y

,L Rose Daisy Review

23


Sample Points When

11Y

12Y

1u

2u

3u

4u

5u

6u

10

10

5

5

2

2

Listing p=1Sample Space n=2 when

PriorListing p=1

11Y

12Y

13Y

1u

2u

3u

4u

5u

6u

11

1 12

13

Y

Y

Y

Y

,L Rose Daisy

,L Rose Daisy

Review

24

Exchangeable Prior Populations N=3: Sample Points n=2

31Y

32Y

1u

2u

3u

4u

5u

6u

10

10

5

5

2

2

Listing p=3

41Y

42Y

1u

2u

3u

4u

5u

6u

10

10

5

5

2

2

Listing p=4

11Y

12Y

1u

2u

3u

4u

5u

6u

10

10

5

5

2

2

Listing p=1

Listing p=2

21Y

22Y

1u

2u

3u

4u

5u

6u

10

10

5

5

2

2

51Y

52Y

1u

2u

3u

4u

5u

6u

10

10

5

5

2

2

Listing p=5

61Y

62Y

1u

2u

3u

4u

5u

6u

10

10

5

5

2

2

Listing p=6

Positive Prob.Review

25

Exchangeable Prior Populations N=3 Sample Points with Positive Probability n=2

31Y

32Y

4u

6u

10

10

5

5

2

2

Listing p=3

41Y

42Y

4u

6u

10

10

5

5

2

2

Listing p=4

11Y

12Y

2u

5u

10

10

5

5

2

2

Listing p=1

Listing p=2

21Y

22Y

1u

3u

10

10

5

5

2

2

51Y

52Y

1u

3u

10

10

5

5

2

2

Listing p=5

61Y

62Y

2u

5u

10

10

5

5

2

2

Listing p=6

,L Rose Daisy

Review

26

Exchangeable Prior Populations N=3 Posterior Random Variables

1 2 H

* * * * * *

11 12 13 14 15 16

21 22 23 24 25 26

31 32 33 34 35 36

41 42 43 44 45 46

51 52 53 54 55 56

61 62 63 64 65 66

1 1 2 3 4 5 6

1

2

3

4

5

6

h p p p p p p

p

p

p

p

p

p

y y y y y y

y y y y y y

y y y y y y

y y y y y y

y y y y y y

y y y y y y

Prior

Data

Rosey

Daisyy Rosey

Daisyy

,

? , ?

I

ph

hII

Y

Y

Y

Rosey

Daisyy Rosey

Daisyy

If permutations of subjects in listing p are equally likely: * *kkk k

p p p

1

II H

h hIIIIh

I

VxY

YY

Random variables representing the data are independent of the remaining random variables.

The Expected Value of random variables for the data is the mean for the data.

Review

*

x nI

h hII N nIIh H

Ep

1Y

1Y

2

22 2var

x n n N n

I

II N n n N n II N n

N

N n

P 0 0Y

Y 0 0 J P

*

2 2II h hII

h H

p

where

*h h

h H

p

*

22

h hh H

p

and

27

Data without Measurement Error

Data (set)

Vectors

1

2I s

n

x

xx

x

x

1

2I s

n

λ

Ik k Iλ v λ Ik k Ix v x

; 1,...,s

x s n

; 1,...,s

L s n

permutation matrix, k=1,…,n!

and 1 nv I

kv n nto be anLet

, ; 1,..., !Ik Ik k n λ x

Data (set of vectors)

sx

Latent Value

For simplicity, denote by

s

x s sx

28

Data with and without Measurement Error

No Measurement Error

1

2I

n

x

x

x

x

1

2I

n

λ

Ik k Iλ v λ Ik k Ix v x

; 1,...,s

x s n

; 1,...,s

L s n

, ; 1,..., !Ik Ik k n λ x

Latent Values

Data

Data

With Measurement Error

1

2

t

tst It

nt

x

xx

x

x

Ik k IX v XVectors

Sets Data

Data

; 1,...,t t sx s n

; 1,...,s

L s n * 2 ; 1,...,e s

X s n

1

2s I

n

X

XX

X

X

s s sX E

Realization at t

* 2, , ; 1,..., !Ik Ik ek k n λ X σ

Potential Response

1

2I

n

μ

21 1

22 22

2

var

var

var

R e

R eeI

R n ne

X

X

X

σ

2 2ek k eIσ v σ

29

Data with Measurement Error

ste the realization of sE

0R sE E

2varR s esE

*

*

*

0R s sE E E

s s

st s stx e on occasion tThe realization of sX

Sets Data

s s sX E

un-observed latent value R s sE X Assume:

Measurement errors are independentbetween any two subjects

*

*

0

R s s

s s

E E E

E E

Measurement errors are independentwhen repeatedly measured on a subject

; 1,...,t t sx s n

; 1,...,s

L s n * 2 ; 1,...,e s

X s n

30

Measurement Error Model

The Data

Ik k Iλ v λ

1

2I s

n

λ

Vectors

Ik k IX v X

1

2I s

n

X

XX

X

X

Potential response

1

22

1

1

1

1

n

x ss

n

x s xs

n

n

Ik k Iμ v μ

Define

Latent Values

1

2I

n

μ

* 2, , ; 1,..., !Ik Ik ek k n λ X σ

2 2ek k eIσ v σ

Response Error Variance

21

22 2 2

2

e

eeI es

en

σ

31


Prior Random Variables

Populations

Prior


1 2

1 2

1 2

1 2

H

H

H

H

L L L

p p p

Population h

1

1 N

h hjjN

Labels:

Parameter

Prior Probabilities

*

* 0

pp

h p hpλ u u λ

; 1,...,h jj N

Assume Random Variables representinga population are exchangeable

*

* 0

pp

h p hpμ u u μ

Defines the axes for points in the prior and the measurement variance

indicates initial order

vector

0h hjλ

Latent Values: 0h hjμ

; 1,...,h jL j N

Measurement Variance:

2 20eh ehjσ

*

*

2 20

pp

eh p ehpσ u u σ

32


Rose

Daisy

Lily

21Y

22Y

23Y

*

* 0

pp

h p hpλ u u λ

0p hu λ

SinglePoint

*

* 0

pp

h p hpμ u u μ

*

*

2 20

pp

eh p ehpσ u u σ

33



Population h, Prior


; 1,...,h jj N

; 1,...,h jL j N

*

* 0

pp

h p hpλ u u λ

Vectors

Assume Random Variables representinga population are exchangeable

When p=1,define

Sets Prior

11 0h hu λ λ1 Nu I

hp * * * 2

*

, , , ;

, 1,..., !

pp pp pp

h h h ehh

p

p p N

λ μ σ1,...,h H

*

* 0

pp

h p hpμ u u μ

11 0h hu μ μ

*

*

2 20

pp

eh p ehpσ u u σ

1 221 0eh ehu σ σ

34

Prior Random Variables and Datawith Measurement Error

If permutations of subjectsin listing p are equally likely:

Assume Random Variables representinga population are exchangeablein each population

*

*

*

!

1

Nppp

h hpp

I

Y μ

*

* 0

pp

h p hpμ u u μSince

1

H

h hh

I

Y Y

10h hμ μ

or

* *

*

!1

1 1

H Np

h p hp ph p

I I

Y u u μ

initial listing p=1

Prior

Data * 2, , ; 1,..., !Ik Ik ek k n λ X σ

* * * 2

*

, , , ;

, 1,..., !

pp pp pp

h h h ehh

p

p p N

λ μ σ1,...,h H

35


1

2

1

2

n

n

n

N

Y

Y

Y

Y

Y

Y

Y

* *

*

!1

1 1

H Np

h p hp ph p

I I

Y u u μ


Prior Random Variablesthat will correspond to Latent values for subjectsIn the data

Remaining Prior Random Variables

Prior


* * * 2

*

, , , ;

, 1,..., !

pp pp pp

h h h ehh

p

p p N

λ μ σ1,...,h H

36


1

2

1

2

n

n

n

N

Y

Y

Y

Y

Y

Y

Y

* *

*

!1

1 1

H Np

h p hp ph p

I I

Y u u μ


Prior Random Variablesthat will correspond to Latent values for subjectsIn the data

Remaining Prior Random Variables

Prior


* * * 2

*

, , , ;

, 1,..., !

pp pp pp

h h h ehh

p

p p N

λ μ σ1,...,h H

* *

*

!1

1 1

H Np

h p hp ph p

I I

L u u λ

* *

*

!1 22

1 1

H Np

e h p ehp ph p

I I

σ u u σ

37

Consider Possible Populationsin the Posterior

Set of Subjects in Population: hL

Set of Subjects in the Data: L

Populations possible in the posterior:

Only Populations that include the set of subjects in the data: | hh L L

* 2, , ; 1,..., !Ik Ik ek k n λ X σ

||; 1,...,

|h h

h h

L Lh H

L L

* * * 2

*

, , , ;

, 1,..., !

pp pp pp

h h h ehh

p

p p N

λ μ σPrior

Data

38

Possible Points for Possible Populationsin the Posterior

Set of Subjects in Population: hL

Set of Subjects in the Data: L

Not in Posterior

Populations possible in the posterior:

Only Populations that include the set of subjects in the data: | hh L L

Also: Only points wherethe set of subjects in the data match the subjects representing the point corresponding to the data in the prior

For possible population, possible points in the posterior:

* 2, , ; 1,..., !Ik Ik ek k n λ X σ

||; 1,...,

|h h

h h

L Lh H

L L

* * * 2

*

, , , ;

, 1,..., !

pp pp pp

h h h ehh

p

p p N

λ μ σPrior

Data

39

1

1

1

hI

h

hII

λλ

λIk k Iλ v λ

Ik k IX v X

Initial Order k=1

1

1

1

hI

h

hII

μμ

μ

1I Iλ λLabels

LatentValues

Data

1I IX X

Potential Response

Also:the set of subjects in the data must match the set ofsubjects representing the point corresponding to the data in the prior

* 2, , ; 1,..., !Ik Ik ek k n λ X σ

||; 1,...,

|h h

h h

L Lh H

L L

* * * 2

*

, , , ;

, 1,..., !

pp pp pp

h h h ehh

p

p p N

λ μ σPrior

Data

*

*

2 1 2pp

eh p ehpσ u u σ

*

*

1pp

h p hpμ u u μ

*

*

1pp

h p hpλ u u λ

MeasurementVariances

1 2

1 2

1 2

ehI

eh

ehII

σσ

σ

Possible Points for Possible Populations with Measurement Error

Possible Populations in the Posterior| hh L L

40


Ik k Iλ v λ

Ik k IX v X

Initial Order k=1

*

*

1pp

h p hpμ u u μ

1

1

1

hI

h

hII

μμ

μ

1I Iλ λ *

*

1pp

h p hpλ u u λLabels

LatentValues

Data

1I IX X

Potential Response

Define 1hI Iλ λ

1

1

1

hI

h

hII

λλ

λ

*

*

2 1 2pp

eh p ehpσ u u σ

1 2

1 2

1 2

ehI

eh

ehII

σσ

σ

* 2, , ; 1,..., !Ik Ik ek k n λ X σ

||; 1,...,

|h h

h h

L Lh H

L L

* * * 2

*

, , , ;

, 1,..., !

pp pp pp

h h h ehh

p

p p N

λ μ σPrior

Data

Initial Listing


The set of subjects in the data must match the subjects representing the point corresponding to the data in the prior


41


Ik k Iλ v λ

Ik k IX v X

Initial Order k=1

1

1

1

hI

h

hII

μμ

μ

1I Iλ λ *

*

1pp

h p hpλ u u λLabels

LatentValues

Data

1I IX X

Potential Response

Define 1hI Iλ λ

11

I

hhII

λλ

λ

*

*

2 1 2pp

eh p ehpσ u u σ

1 2

1 2

1 2

ehI

eh

ehII

σσ

σ

*

*

1pp

h p hpμ u u μ

* 2, , ; 1,..., !Ik Ik ek k n λ X σ

||; 1,...,

|h h

h h

L Lh H

L L

* * * 2

*

, , , ;

, 1,..., !

pp pp pp

h h h ehh

p

p p N

λ μ σPrior

Data


The set of subjects in the data must match the subjects representing the point corresponding to the data in the prior


42


Ik k Iλ v λ

Ik k IX v X

Initial Order k=1

*

*

1

1

pp hI

h pphII

μμ u u

μ

1I Iλ λ

Labels

LatentValues

Possible Populations in the Posterior Data

1I IX X

Potential Response

| hh L L

*

*

1 22

1 2

pp ehI

eh ppehII

σσ u u

σ


*

* 1

Ipp

h pphII

λλ u u

λ

* 2, , ; 1,..., !Ik Ik ek k n λ X σ

||; 1,...,

|h h

h h

L Lh H

L L

* * * 2

*

, , , ;

, 1,..., !

pp pp pp

h h h ehh

p

p p N

λ μ σPrior

DataThe set of subjects in the data must match the subjects representing the point corresponding to the data in the prior

43


Ik k Iλ v λ

Ik k IX v X

*

*

1

1

pp hI

h pphII

μμ u u

μ

Possible Populations in the Posterior Data

Potential Response

| hh L L

*

*

1 22

1 2

pp ehI

eh ppehII

σσ u u

σ

*

* 1

Ipp

h pphII

λλ u u

λ

* 2, , ; 1,..., !Ik Ik ek k n λ X σ

||; 1,...,

|h h

h h

L Lh H

L L

* * * 2

*

, , , ;

, 1,..., !

pp pp pp

h h h ehh

p

p p N

λ μ σPrior

DataThe set of subjects in the data must match the subjects representing the point corresponding to the data in the prior

*

*

k

ppk

v 0u u

0 w*

*

k

ppk

v 0u u

0 w

Possible Points in the Population

*

*

* *

1

k Ipp

hk h II

v λλ

w λ

*

* *

*

1

Ipp

ph ph II

λλ u u

λ

Points

in Prio

r whe

re

Subjec

ts m

atch

Dat

aor

Points in the Posterior

Labels

LatentValues


44


Ik k Iλ v λ

Ik k IX v X

*

*

1

1

pp hI

h pphII

μμ u u

μ

Data

Potential Response

*

*

1 22

1 2

pp ehI

eh ppehII

σσ u u

σ

*

* 1

Ipp

h pphII

λλ u u

λ

*

*

k

ppk

v 0u u

0 w

*

*

*

1

k Ipp

h kkhIIk

h

v λλ

w λλ

*

*

pp

h hkkμ μ

*

*

pp

h hkkλ λ

*

*

2 2pp

eh ehkkσ σ


Possible Populations in the Posterior| hh L L Possible Points in the Population

Points Notin the Posterior

||; 1,...,

|h h

h h

L Lh H

L L

*

* * *

* *

* 2 *

2

*

, , ,

; , 1

, , , ;

1,..., !; 1,.., !

,..., !

pp pp

h h hhk

h

k

pp

e

h

h

kk hk

h

k ehkkp

k n k

p

p p N

N n

λ λ μ

σ

λ μ σ

2* , , ;

1,..., !Ik Ik ek

k n

λ X σ

Labels

LatentValues


45


Ik k Iλ v λ

Ik k IX v X

*

*

1

1

pp hI

h pphII

μμ u u

μ

Possible Populations in the Posterior

Data

Potential Response

| hh L L

*

*

1 22

1 2

pp ehI

eh ppehII

σσ u u

σ

*

* 1

Ipp

h pphII

λλ u u

λ

Possible Points in the Population

*

*

*

1

k Ipp

h hkkhIIk

v λλ λ

w λ

Labels

LatentValues


||; 1,...,

|h h

h h

L Lh H

L L

*

* * *

* *

* 2 *

2

*

, , ,

; , 1

, , , ;

1,..., !; 1,.., !

,..., !

pp pp

h h hhk

h

k

pp

e

h

h

kk hk

h

k ehkkp

k n k

p

p p N

N n

λ λ μ

σ

λ μ σ

2* , , ;

1,..., !Ik Ik ek

k n

λ X σ

*

*

pp

h hkkμ μ

*

*

2 2pp

eh ehkkσ σ

*

*

k

ppk

v 0u u

0 w


46


*

*

k

ppk

v 0

0u u

w


Populationsin the Posterior

2* , , ;

1,..., !Ik Ik ek

k n

λ X σ|h hL L

Prior|

|; 1,...,|

h h

h h

L Lh H

L L

*

* * *

* *

* 2 *

2

*

, , ,

; , 1

, , , ;

1,..., !; 1,.., !

,..., !

pp pp

h h hhk

h

k

pp

e

h

h

kk hk

h

k ehkkp

k n k

p

p p N

N n

λ λ μ

σ

λ μ σ

Data

Posterior|

1,...,h

h

L L

h H

* * *

* * *

1 1 2

21 1 1 2

*

, , , ;

1,..., !; 1,.., !

,

,

I hI ehI

h hkk hkk ehkkhII hII ehIIk k

Ik Ik

k k k

k

p

k n k N n

λ μ σλ μ σ

w λ

λ X

v

w σ

v

μ w

v

*

*

k

ppk

v 0

0u u

w

47




2* , , ;

1,..., !Ik Ik ek

k n

λ X σ|h hL L

Prior|

|; 1,...,|

h h

h h

L Lh H

L L

*

* * *

* *

* 2 *

2

*

, , ,

; , 1

, , , ;

1,..., !; 1,.., !

,..., !

pp pp

h h hhk

h

k

pp

e

h

h

kk hk

h

k ehkkp

k n k

p

p p N

N n

λ λ μ

σ

λ μ σ

Data

Posterior Random Variables for subject labels(assuming permutationsare equally likely in the prior)

* *

*

!

,1

!1

,1 1

k

n

I k Ik

N nH

h hIIII k kh k

I

I I

λ

w λ

v

*

*

k

ppk

v 0

0u u

w

Posterior|

1,...,h

h

L L

h H

* * *

* * *

1 1 2

21 1 1 2

*

, , , ;

1,..., !; 1,.., !

,

,

I hI ehI


Ik Ik

k k k

k

p

k n k N n

λ μ σλ μ σ

w λ

λ X

v

w σ

v

μ w

v

*

*

k

ppk

v 0

0u u

w

48




2* , , ;

1,..., !Ik Ik ek

k n

λ X σ|h hL L

Prior|

|; 1,...,|

h h

h h

L Lh H

L L

*

* * *

* *

* 2 *

2

*

, , ,

; , 1

, , , ;

1,..., !; 1,.., !

,..., !

pp pp

h h hhk

h

k

pp

e

h

h

kk hk

h

k ehkkp

k n k

p

p p N

N n

λ λ μ

σ

λ μ σ

Data

Posterior Random Variables for latent values (assuming permutationsare equally likely in the prior)

* *

*

!

,1

!1

,1 1

n

I I k Ik

N nH

II h hIIII k kh k

kI

I I

Y μv

Y w μ

*

*

k

ppk

v 0

0u u

w

Posterior|

1,...,h

h

L L

h H

* * *

* * *

1 1 2

21 1 1 2

*

, , , ;

1,..., !

, ,

; 1,.., !

I hI ehI

h hkk hkk ehkkhII hI

Ik R Ik

k k k

I ehIIk k k

p

k n k N n

E

λ X

v λ μ σλ μ σ

w λ w μ

v v

w σ

*

*

k

ppk

v 0

0u u

w

49


*

*

k

ppk

v 0u u

0 w



2* , , ;

1,..., !Ik Ik ek

k n

λ X σ|h hL L

Prior|

|; 1,...,|

h h

h h

L Lh H

L L

*

* * *

* *

* 2 *

2

*

, , ,

; , 1

, , , ;

1,..., !; 1,.., !

,..., !

pp pp

h h hhk

h

k

pp

e

h

h

kk hk

h

k ehkkp

k n k

p

p p N

N n

λ λ μ

σ

λ μ σ

Data

Posterior Random Variables for Measurement Variance (assuming permutationsare equally likely in the prior)

* *

*

!1 2

,1

!1 2

,1 1

n

I k ehIk

N nH

h ehIIII kk

k

kh

I

I I

v σ

w σ

*

*

k

ppk

v 0

0u u

w

Posterior|

1,...,h

h

L L

h H

* * *

* * *

1 1 2

21 1 1 2

*

, , , ;

1,..., !;

, var

1,.., !

,

I hI ehI

h hkk hkk ehkkhII hII eh

Ik R Ik

k k k

IIk k k

p

k n k N n

λ μ σλ μ σ

w λ w μ w σ

λ X

v v v

*

*

k

ppk

v 0

0u u

w

50

Posterior Random Variableswith Measurement Error

1

1

II H

h hIIIIh

I

VλL

WλL

*

* 1*

1

II H

h hIIIIh

I

VXY

WYY

1

1

II H

h hIIIIh

I

VμY

WμY

Subjects

Latent values

Response

* *

*

!

,1

!1

,1 1

n

I I k k Ik

N nH

II h hIIII k kh k

I

I I

Y v μ

Y w μ

* *

*

!

,1

!1

,1 1

n

I I k k Ik

N nH

II h hIIII k kh k

I

I I

L v λ

L w λ

* *

*

!*

,1

!* 1*

,1 1

n

I I k k Ik

N nH

II h hIIII k kh k

I

I I

Y v X

Y w Y

51

Posterior Random Variableswith Measurement Error

1

1

II H

h hIIIIh

I

VλL

WλL

*

* 1*

1

II H

h hIIIIh

I

VXY

WYY 1

1

II H

h hIIIIh

I

VμY

WμY

Subjects Latent values Response

Observed (in the data)

Define Response Marginal over Measurement Error forSubjects that are not in the data

*

* 1| *|

1

II H

R Lh R L hIIII

h

EI E

VXY

W YY Since * 1 1

|R L hII hIIE Y μ

* *

| *I I

R L

II II

E

Y Y

Y Y

Condition on to obtain the posteriordistribution

L

52

Posterior Random Variables with Measurement Error


*

1

II H

h hIIIIh

I

VXY

YY

*x nI

RII N nII

E

1Y

1Y

2 2

*2

2 2varx n e n n N n

IR

N n n N n II N nII

N

N n

P I 0 0Y

0 0 J PY

where

Posterior|

1,...,h

h

L L

h H

* * *

* * *

1 1 2

21 1 1 2

*

, , , ;

1,..., !; 1,.., !

,

,

I hI ehI


Ik Ik

k k k

k

p

k n k N n

λ μ σλ μ σ

w λ

λ X

v

w σ

v

μ w

v

*

*

k

ppk

v 0

0u u

w

53



*

1

II H

h hIIIIh

I

VXY

YY

*x nI

RII N nII

E

1Y

1Y

2 2

*2

2 2varx n e n n N n

IR

N n n N n II N nII

N

N n

P I 0 0Y

0 0 J PY

where

Posterior|

1,...,h

h

L L

h H

* * *

* * *

1 1 2

21 1 1 2

*

, , , ;

1,..., !; 1,.., !

,

,

I hI ehI


Ik Ik

k k k

k

p

k n k N n

λ μ σλ μ σ

w λ

λ X

v

w σ

v

μ w

v

*

*

k

ppk

v 0

0u u

w

1

1 n

x ssn

1

1 N

hII hjj nN n

*II h hII

h H

p

54



*

1

II H

h hIIIIh

I

VXY

YY

*x nI

RII N nII

E

1Y

1Y

2 2

*2

2 2varx n e n n N n

IR

N n n N n II N nII

N

N n

P I 0 0Y

0 0 J PY

2 21e e

Ln

where

22

1

1

1

n

x s xsn

22

1

1

1

N

hII hj hIIj nN n

* | 1hH h d

*

2 2II h hII

h H

p

*

22

h hh H

p

1 if

0 otherwiseh

hd

*h h

h H

p

Posterior|

1,...,h

h

L L

h H

* * *

* * *

1 1 2

21 1 1 2

*

, , , ;

1,..., !; 1,.., !

,

,

I hI ehI


Ik Ik

k k k

k

p

k n k N n

λ μ σλ μ σ

w λ

λ X

v

w σ

v

μ w

v

*

*

k

ppk

v 0

0u u

w

55


*

1

II H

h hIIIIh

I

VXY

YY

*x nI

RII N nII

E

1Y

1Y

Finite Population Mixed Modelfor the subjects in the Data:

where

* *I x n I Y 1 a E

i I x na a V x 1

random subject effect

*R I x nE Y 1

* 2 2var R I x n e n Y P I

Use this model to obtainthe best linear unbiased predictor of the latent valuefor a subject in the data (which we call the BLUP for a realized subject)

2 2

*2

2 2varx n e n n N n

IR

N n n N n II N nII

N

N n

P I 0 0Y

0 0 J PY

Posterior|

1,...,h

h

L L

h H

* * *

* * *

1 1 2

21 1 1 2

*

, , , ;

1,..., !; 1,.., !

,

,

I hI ehI


Ik Ik

k k k

k

p

k n k N n

λ μ σλ μ σ

w λ

λ X

v

w σ

v

μ w

v

*

*

k

ppk

v 0

0u u

w

56

Finite Population Mixed Model (FPMM) for Subjects in the Data based on the Posterior Random Variables

* *I x n I Y 1 a E

What is the Latent valuefor a subject in the data?

Data (set)

Vectors

1

2I s

n

μ

1

2I s

n

λ

; 1,...,ss n

s

Latent Valuefor subject swith label

FPMM

I x n Vx 1 a

Ik k Iμ v μData (set of vectors)

, ; 1,..., !Ik Ik k n λ μ

s

; 1,...,t t sx s n

; 1,...,s

L s n * ; 1,...,

sX s n

Latent Values

57

Finite Population Mixed Model (FPMM) for Subjects in the Data based on the Posterior Random Variables

* *I x n I Y 1 a E

What is the Latent valuefor a subject in the data?

Data

sLatent Valuefor subject swith label

FPMM

I x n Vx 1 a

s

1

2I

n

x

x

x

x

Example (n=2)Rose

ILily

μ

1,..., ! 2k n

1 1

1 0

0 1Rose Rose

I ILily Lily

μ v μ

2 2

0 1

1 0Rose Lily

I ILily Rose

μ v μ

In the data, the permutation matricesare not random- they just differ.

kv

FPMM

I x n Vμ 1 a

In the posterior distribution, thepermutation matrices are random!… a consequence of using the prior

*

!

1

np

kpk

I

V v

, ; 1,..., !Ik Ik k n λ μ

Ik k Iμ v μ

58



*

1

II H

h hIIIIh

I

VXY

YY

*x nI

RII N nII

E

1Y

1Y

2 2

*2

2 2varx n e n n N n

IR

N n n N n II N nII

N

N n

P I 0 0Y

0 0 J PYwhere

Posterior|

1,...,h

h

L L

h H

* * *

* * *

1 1 2

21 1 1 2

*

, , , ;

1,..., !; 1,.., !

,

,

I hI ehI


Ik Ik

k k k

k

p

k n k N n

λ μ σλ μ σ

w λ

λ X

v

w σ

v

μ w

v

*

*

k

ppk

v 0

0u u

w


where

* *I x n I Y 1 a E

i I x na a V x 1


*R I x nE Y 1



59


FPMM to Predict Latent Values


where

* *I x n I Y 1 a E

i I x na a V x 1


*R I x nE Y 1



*

1

II H

h hIIIIh

I

VXY

YY *

x nIR

II N nII

E

1Y

1Y

, ; 1,..., !Ik Ik k n λ μ * , ; 1,..., !Ik Ik k n λ X

; 1,..., !IkL k n λ| ;

h

h

h

p

* , ,h h hL

| ;

h

h

h

p

*, ,h h hL

*

1 *

1,..., !,;

1,..., !

Ik

hhIIk

k nL

k n

λ

w λ

* *

*1 * 1 *

1,..., !,, ;

1,..., !

Ik Ik

hhII hIIk k

k n

k n

λ X

w λ w Y

* *

1 1 *

1,..., !,, ;

1,..., !

Ik Ik

hhII hIIk k

k n

k n

λ μ

w λ w μ

| hh

2 2

*2

2 2varx n e n n N n

IR

N n n N n II N nII

N

N n

P I 0 0Y

0 0 J PY

Documents

1 Finite Population Inference for Latent Values Measured with Error from a Bayesian Perspective Edward J. Stanek III Department of Public Health University