demography of unmarked individuals

Background Model definitions Tools Tests

Estimating demographic parameters fromsamples of unmarked individuals

Ben Bolker

Departments of Mathematics & Statistics and Biology, McMaster University

19 November 2010

Ben Bolker Departments of Mathematics & Statistics and Biology, McMaster University

Estimating unmarked individuals


Outline

1 BackgroundEcological/statistical motivationsStudy system

2 Model definitions

3 ToolsState-space modelsMarkov chain Monte CarloGibbs/block sampling

4 TestsPerfect observation, trivial dynamicsPerfect observation, non-trivial dynamicsImperfect observation




Ecological/statistical motivations

Why do ecologists care about demography?

Determines “distribution and abundance of organisms”

Basic:what regulates populations?effects of population density & individual characteristics?

Applied:predict population dynamicsuse this information to manage them sustainablyeffects of interventions (e.g. attraction-productioncontroversy)





Why do ecologists care about demography?

Determines “distribution and abundance of organisms”

Basic:what regulates populations?effects of population density & individual characteristics?

Applied:predict population dynamicsuse this information to manage them sustainablyeffects of interventions (e.g. attraction-productioncontroversy)





Data

Repeated samples of individual animals

. . . Multiple times, observers, locations

Presence and size (imperfect)

Estimate demographic parameters:

birth/immigrationdeath/emigrationchange (growth)

. . . as a function of size, (population density)





Data

Repeated samples of individual animals

. . . Multiple times, observers, locations

Presence and size (imperfect)

Estimate demographic parameters:

birth/immigrationdeath/emigrationchange (growth)

. . . as a function of size, (population density)





Statistical motivations

Challenging (=fun)

General problem (individual identification is often difficult)

Impossible with current methods

Novel application of standard building blocks




Study system

Natural history

photo: Leonard Low, Flickr via species.wikimedia.org

Thalassoma hardwicke(sixbar wrasse)

Settlement: ≈ 5–10 mm

Growth: a few mm per month

Death: by predator(competition for refuges)

Immigration/emigration:rare below 40–50 mm




Study system

Where they live

Patch reefs, FrenchPolynesia (andthroughout the southPacific)




Study system

What they look like to me

Date

Est

imat

ed fi

sh s

ize

(mm

)

20

40

60

80

Feb Mar Apr May Jun Jul




Study system

What they look like to me

Date

Est

imat

ed fi

sh s

ize

(mm

)

20

40

60

80

Feb Mar Apr May Jun Jul




Study system

Outline


2 Model definitions






Outline


2 Model definitions






Demographic parameters as f (size)

Fish size (mm)

Probability(per day)

0.0

0.2

0.4

0.6

0.8

1.0

Growth

l.grow

1020304050607080

Appearance

a.settleb.settle

tot.settle

1020304050607080

Persistence

a.mortc.mort

1020304050607080




Observation model

Fish size (mm)

0.2

0.4

0.6

0.8

Detection(probability)

10 20 30 40 50 60 70 80

1.0

1.5

2.0

2.5

3.0

3.5

Measurement(standard deviation)

10 20 30 40 50 60 70 80

Measurement errorDouble−countZero−count




Outline


2 Model definitions






State-space models

State-space models

Problem: estimating parameters of systems with un- orimperfectly-observed states?

Easy if no feedback (measurement error models)

With feedback (dynamic models), brute force approachesbecome infeasible . . .




State-space models

Directed acyclic graph (DAG)

parameters

N1 N2 N3 N4 N5

obs1 obs2 obs3 obs4 obs5




Markov chain Monte Carlo

Bayesian statistics, in 30 seconds

computational (convenience) Bayesian statistics(forget all that stuff about philosophy of statistics, subjectivity, incorporating prior information . . . )

have a likelihood, L(θ) = Prob(data|θ)

want a posterior probability, Ppost(θ) = Prob(θ|data)

Bayes’ rule:

Ppost(θ) =L(θ) · prior(θ)∫∫∫L(θ) · prior(θ) dθ

may want mean, mode, confidence intervals, . . . denominatorvery high-dimensional





Markov chain Monte CarloP

oste

rior

prob

abili

ty

Parameter value

θA MCMC rule:

ifP(θA)

P(θB)=

J(θB → θA)

J(θA → θB)

then stationary distribution =posterior probability(provided chain is irreducible etc.)






oste

rior

prob

abili

ty

Parameter value

θA

● ●

MCMC rule:

ifP(θA)

P(θB)=

J(θB → θA)

J(θA → θB)







oste

rior

prob

abili

ty

Parameter value

θA

● ●●

MCMC rule:

ifP(θA)

P(θB)=

J(θB → θA)

J(θA → θB)







oste

rior

prob

abili

ty

Parameter value

θA

● ●●●

MCMC rule:

ifP(θA)

P(θB)=

J(θB → θA)

J(θA → θB)







oste

rior

prob

abili

ty

Parameter value

θA

● ●●●●●● ●●●

MCMC rule:

ifP(θA)

P(θB)=

J(θB → θA)

J(θA → θB)







oste

rior

prob

abili

ty

Parameter value

θA MCMC rule:

ifP(θA)

P(θB)=

J(θB → θA)

J(θA → θB)







oste

rior

prob

abili

ty

Parameter value

θAθA

θB

MCMC rule:

ifP(θA)

P(θB)=

J(θB → θA)

J(θA → θB)






Metropolis-(Hastings) updatingP

oste

rior

prob

abili

ty

Parameter value

θA

Candidate distribution

Metropolis rule:

accept B with probability

min

(1,

P(θB)

P(θA)

)satisfies MCMC rule . . . Don’tneed to know denominator!





Metropolis-(Hastings) updatingP

oste

rior

prob

abili

ty

Parameter value

θA

Candidate distribution

Metropolis-Hastings rule:

accept B with probability

min

(1,

P(θB)

P(θA)· C (θB ,θA)

C (θA,θB)

)satisfies MCMC rule . . . Don’tneed to know denominator!




Gibbs/block sampling

Gibbs sampling

0.5

0.5

1

1

1.5

1.5

2

2

2.5

2.5

3

3

3.5

3.5 4

4

4.5

Joint distribution ofparameters often difficult

Condition each elementon “known” (i.e. imputed)values of all otherelements: P(a, b, c) =P(a|b, c)P(b, c)

Gibbs sampling: do thisrepeatedly for eachelement, or block ofelements





Gibbs sampling

0.5

0.5

1

1

1.5

1.5

2

2

2.5

2.5

3

3

3.5

3.5 4

4

4.5

●

● ●

● ●

●

●●

●

●●●

●●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●








Gibbs sampling

0.5

0.5

1

1

1.5

1.5

2

2

2.5

2.5

3

3

3.5

3.5 4

4

4.5

●

● ●

● ●

●

●●

●

●●●

●●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●








Back to the DAG

parameters

N1 N2 N3 N4 N5

obs1 obs2 obs3 obs4 obs5




Outline


2 Model definitions






Perfect observation, trivial dynamics

Outline


2 Model definitions







DAG

parameters fates

N1 N2

Demographic parameters|fates:standard M-H

Fates|parameters





Scenario

X

Lsurv = (1 − m)(1 − p)S

Lmort = mp(1 − p)S−1

One fish of the same size observedon two consecutive days . . .was it the same individual?

P(surv|m, p) =(1−m)(1− p)

1−m − p + 2mp





Updating probabilities for parameters

probability

probabilitydensity

0.5

1.0

1.5

0123456

mortality (m)

immigration (p)

0.2 0.4 0.6 0.8

typepriorposterior

If the fish survived, then ourposterior probabilities for theparameters are:

m ∼ Beta(1, 2)(0 mortalities, 1 survival)

p ∼ Beta(1, S + 1)(0 immigrations, Snon-immigrations)





Trivial example: results

value

density

0

1

2

3

4

mortality (m)

2(m + S(1 − m))S + 1

0.0 0.2 0.4 0.6 0.8 1.0

immigration (p)

Beta(p,1,S)

0.0 0.2 0.4 0.6 0.8 1.0





P(mortality & no immigration) ≈ 0.16 = 1/(S + 1);P(survival & immigration) ≈ 0.83 = S/(S + 1);

results are simple and make sense; closed form solution ispossible but ugly . . .

combination of observations and non-observation of otherindividuals provides information (could fail with non-detection)




Perfect observation, non-trivial dynamics

Outline


2 Model definitions







Non-trivial dynamics

Multiple individuals, measured over multiple days

Still simple: no density-dependent rates,immigration/emigration of large individuals

observations still error-free





Parameter/fate sampling

parameters

fates(1,2) fates(2,3)

N1 N2 N3

Parameter sampling: M-Hupdating with simplecandidate distributions

Fate sampling: ???

Enumeratingpossibilities is tedious. . .





M-H fate sampling

6 mm 7 mm dead

5 mm G (5) · (1−M(5)) X M(5)6 mm (1− G (6)) · (1−M(6)) G (6) · (1−M(6)) M(6)absent I (6) I (7) X





M-H fate sampling

6 mm 7 mm dead






M-H fate sampling

6 mm 7 mm dead






M-H fate sampling

6 mm 7 mm dead






M-H fate sampling

6 mm 7 mm dead






M-H fate sampling

6 mm 7 mm dead5 mm G (5) · (1−M(5)) X M(5)6 mm (1− G (6)) · (1−M(6)) G (6) · (1−M(6)) M(6)absent I (6) I (7) X





Sampling test

Probability

Fate

both die

6 grows

5 grows

both grow

6 survives

10−4 10−3 10−2 10−1

wSamplingTrueGibbs





Testbed

Simulate for 80 days, with a settlement rate (tot.settle) of5% day:

29 distinct individuals, size range 5–30, total of 618 fish-days(observations).

Sample fates only (1000 MCMC steps), fixed demographicparameters

Sampled 1000 unique fates (but only 895 unique likelihoods)





Testbed

Simulate for 80 days, with a settlement rate (tot.settle) of5% day:

29 distinct individuals, size range 5–30, total of 618 fish-days(observations).

Sample fates only (1000 MCMC steps), fixed demographicparameters

Sampled 1000 unique fates (but only 895 unique likelihoods)





True demography

0 20 40 60 80

510

1520

2530

day

size





Fate-only results

Log−likelihood

0.00

0.01

0.02

0.03

0.04

0.05

−520 −500 −480 −460

●

Density





Sampling fates, parameters, both . . .

Log−likelihood

Density

0.00

0.05

0.10

0.15

0.20

0.25

−500 −490 −480 −470 −460 −450

typefate onlyparameters onlyboth





Parameter estimates 1

density

0.51.01.52.02.53.0

0.10.20.30.40.50.60.7

lgrow

−1.9−1.8−1.7−1.6−1.5−1.4−1.3−1.2a.settle

11 12 13 14

0.2

0.4

0.6

0.8

0.1

0.2

0.3

0.4

0.5

a.mort

−4.0 −3.5 −3.0 −2.5b.settle

1.01.52.02.53.03.54.0

20406080

100120140

10

20

30

40

50

c.mort

0.0100.0150.0200.0250.030tot.settle

0.020.030.040.050.06






Fish size (mm)

Probability(per day)

0.0

0.2

0.4

0.6

0.8

1.0

Growth

l.grow

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

0 5 10 15 20 25 30

0.00

0.02

0.04

0.06

0.08

Appearance

a.settle

b.settle

tot.settle

●

●

●

●

●

●

●

●

●●●●●●●●●●●●●●●●●●

0 5 10 15 20 25 30

0.92

0.94

0.96

0.98

1.00

Persistence

a.mort

c.mort

●

●

●

●

●

●●

●

●

●

●●●●●●●●●●●●●●●●

0 5 10 15 20 25 30






−4.0

−3.5

−3.0

−2.5

−2.0

0.0050.0100.0150.0200.0250.030

a.mort

●

●

●

●

●

●

●

●

●

●

c.mort

●

●

●

●

●

●

●

●

●

●

10111213141516

−1.8

−1.6

−1.4

−1.2

a.settle

●●

● ● ● ●

●● ●

●

lgrow

●●

●

●

●

●

●

● ●

●

1.01.52.02.53.03.54.0

0.0100.0150.0200.0250.0300.0350.0400.045

b.settle

●

● ●

●

●

●

●

●

●

●

tot.settle

●

● ●

● ●●

●●

●

●




Imperfect observation

Outline


2 Model definitions







Adding errors

parameters

fates(1,2) fates(2,3)

N1 N2 N3

obs1 obs2 obs3

obs parameters

Back to the more typicalgraph: incorporate informationfrom N(t − 1), N(t + 1), andcurrent observation





New algorithm

For each individual:

Pick true state at N(t), from all possible states, conditioningon N(t − 1)

Pick identity at time N(t + 1) from feasible set:update probabilities

Pick corresponding observation from observed(t):update probabilities

Calculate likelihood, M-H update





Open questions, caveats

Will it work ??

Add complexities:

Multiple populationsSpatial variation, correlation among parameters

Simplify/generalize code

Speed?





Conclusions & musings

MCMC as powerful technology

. . . can it be democratized?

Data limitations shift over time:hierarchical models are great for “modern” (high-volume,low-quality) data



Education

demography of unmarked individuals