1

Collaborative Systemwide Monitoring and Evaluation Project (CSMEP)

Presentation to PNAMP Steering Committee

August 28, 2008

2

CSMEP Analyses - Tools

CBFWAESSA Technologies Ltd.

State AgenciesIDFGODFWWDFW

Federal AgenciesNOAAUSFWSEPADFO

Tribal AgenciesCRITFCNez Perce TribeColville TribesYakama NationUmatilla Tribes

ConsultantsEco Logical ResearchQuantitative ConsultantsPERWEST

3

Status and Trends

abundanceproductivity (age structure)spatial structurespatial diversity

population, MPG, ESU scales

Harvest

ESU scales

Hatchery

Hatchery: Wild (straying)relative productivity

program,population,system scales

Hatchery: WildAge structure

Hydrosystem

population, MPG, ESU,system scales

Habitat

watershed,population,scales

• upstream/downstream• estimate• estimate of total survival• mainstem survival• SARS

• onboard monitoring• landed catch monitoring• creel surveys

• dam monitoring

abundanceproductivityspatial structurespatial diversity

• redd counts• weirs• carcasses• MRC• juvenile traps• other methods

PIT Tags

PIT T

ags PIT Tags

PIT

Tag

sP

IT T

ags

4

Status and Trends M&E

Purpose: Assess viability of the population and evaluate overall management strategies.

4H Impacts Actions Monitoring

Harvest √

Hydro √

Habitat √

Hatchery √

Status and Trends will tell you what the population is doing but not why.

5

1) Monitoring Data S&W Assessments (S&T)

SOTR Reporting

2) Salmon Viability Model

3) Integrated Costs Database Tool

Project Specific Planning/Budgeting

CSMEP Analyses/Tools

CSMEP Analyses/Tools

6

Status & Trends Monitoring

Priority Question: Are salmon viable using IC-TRT criteria?

Related Decision: Has there been sufficient improvement in the status of a salmon population/ESU to justify delisting and allow removal of ESA restrictions?

7

What if we make the wrong decision?

De-listing when not warranted

Risk to populations

Not de-listing when warranted Missed fishing opportunities Lost land use opportunities Unjustified cost of ESA protections

8

How likely to make the correct viability decision with different levels of monitoring intensity?

Data Needs:• Abundance• Productivity• Spatial structure• Diversity

Objective: Create a tool to help managers evaluate alternative monitoring designs

Approach: Evaluate existing monitoring data and develop a model to explore the ability of monitoring to correctly assess salmon population viability

Technical Recovery Team

viability criteria

Viability Status:• Not Viable• Maintained• Viable• Highly Viable

9

Salmon Viability Simulation Model

(CSMEP S&T group, D. Pickard, C. Jordan, C. McGrath)

1st step towards model: try to quantify variability/uncertainty in the quality of monitoring data:

• CSMEP S&W assessments• Summary of the statistical properties of different methods for

estimating fish performance measures. D. Pickard - ESSA Report

• Current research on sampling variability by Dan Rawding (WDFW) and Claire McGrath (USFS)

10

Monitoring activities used to assess viability

x = monitoring occurs in at least one Major or Minor Spawning Area; a = PIT-tags scanned at weirs; b = CWT recoveries of hatchery adults at hatchery weirs.

MPG L. Snake Grande Ronde River

Data need Method/Description Aso

tin

Tuc

anno

n

Jose

ph

Wa

llow

a

LGra

nde

Ron

de

U G

rand

e R

ond

e

A1 census weir (number) 1 1 A2 weir w/MR (number) 1 1 2 A3 weir w/o MR (number) 1

Abundance of adults

A4 MR survey, no weir B1 Index-multi x x Abundance and distribution

of redds B2 Index-once x x x C1 Tags (CWT, PIT) a a b a,b C2 Hard parts, scales x x x x C3 Length at age

Age structure of spawners

C4 Basinwide estimate D1 Marks , weirs (number) 1 1 2 3 D2 marks, remote sense Origin of spawners D3 marks, carcasses E1 Carcass survey Sex ratio of spawners E2 Weirs (number) 1 1 2 3 F1 Juvenile trap (number) 1 1 2 3 F2 Electrofish F3 Snorkel survey--random F4 Snorkel survey--fixed

Abundance and spatial distribution of juveniles/smolts

F5 Presence/absence Survival of juveniles/smolts G1 mark-recapture x x

H1 Juvenile trap x x Age structure of juveniles/smolts H2 other in-river sampling

11

Summaries of the quality of Status Quo Monitoring (feeding the viability model)

PopulationRedd count

type Weir

Proportion of spawning area

covered by weirAbundance

AssumptionsSpatial

coverage

Assess diversity metrics

Asotin Creek two-time census ground

yes 100% Unbiased, high precision Good Good

Tucannon River multiple ground-census

yes 70% Unbiased, high precision Good Good

Minam River multiple ground-census

no na unbiased, medium precision

Good Good

Little Salmon River none yes 50% Overall: biased, medium precision

None Poor

SF Salmon River mainstem

index (aerial) yes 25% Biased, low precision None Poor

12

Alternative designs to explore within viability model(example – a ‘Medium” design)

Population Redd count type WeirProportion of spawning area

covered by weirAbundance

AssumptionsSpatial

coverage

Assess diversity metrics

Asotin Creek 3 pass ground index, 3rd pass is spatial census ground count

no 100% Unbiased, med precision

Good Good

Tucannon River 3 pass ground index + 1pass spatial census aerial count

yes 70% Unbiased, high precision

Good Good

Minam River 3 pass ground index, 3rd pass is spatial census ground count

no na Unbiased, med precision

Good Good

Little Salmon River 3 pass ground index, 3rd pass is spatial census ground count

no 50% Unbiased, med precision

Good Good

SF Salmon River mainstem 3 pass air index, 3rd pass is spatial census aerial count

no 25% Unbiased, high precision

Good Poor

13

Model Inputs & Outputs

Model inputs

Strengths & weakness

assessments Spatial coverage

Abundance (CV and bias)

Diversity

Model

Age-structure (smoothing)

Probability of correctly

assessing viability

Model outputs

TRUTH (A, P, SS, D)

14

Model Inputs: Spatial Structure & Diversity

•A probability transition matrix is used to determine the probability of correctly classifying the data in each of the 4 risk categories (H, M, L, VL) given the monitoring in place

15

Model Input: Abundance & Productivity

Abundance

Productivity

16

e.g., Unbiased, low precision

e.g., Unbiased, medium precision

e.g., Biased, medium precision

The monitoring design defines the assumed measurement error (“noise’)

17

Measurement error within the model:

• Derived from CSMEP’s S&W assessments and user inputs

• Dependent on level of effort to assess spawners• Bias depends on how spatially representative the

sampling• Dependent on quality of information from each

identified spawning area• Dependent on the number of samples obtained• Dependent on variable ability to ‘get hands on fish’

18

Test case: Snake Basin Spring/Summer Chinook Evolutionarily Significant Unit (ESU)

• 32 populations• 5 major population groups (MPGs)• Diversity of current monitoring efforts• State biologists are interested in

modifying monitoring designs

• Objective: Test the ability of alternative monitoring designs to correctly assess viability

19

20

21

22

23

24

Status quo

0.60

A single run of the simulation

Medium

0.73

High

0.84

Low

0.41

Pr (correct assessments)

25

Utility of Salmon Viability Model

• Given that we have:

– A framework for decision making– Estimates of uncertainty in data

• The simulation modeling allows us to:

– Evaluate sensitivity of decisions to quality of monitoring data– Test influence of specific types of monitoring data on

decisions– Managers to evaluate alternative monitoring designs

• Viability model is currently coded in R; now converting to more user friendly format and developing associated user guide to allow managers to explore their own alternative M&E designs

26

Integrated Costs Database Tool(CSMEP S&T group, D. Carr - programmer)

• A relational database (MS Access) developed to allow estimation of the cost of integrated monitoring designs

• Combines the costs of equipment, manpower, tagging and analyses required for a suite of survey techniques required across S&T and 4H monitoring

• Also helps identify the particular performance measures that could be captured within a proposed monitoring design

27

Interaction of viability and cost estimates

Probability of correct viability decision

Monitoring Cost ($)

0.0

0.5

1.0 probability

Have we achieved acceptable reliability at acceptable cost?

Yes - Stop

No - Redesign

28

Objectives by Alternatives Matrix:Status & Trends Designs

Status and Trends Subgroup

.94.87.45.55Pr (making the correct viability assessment for Snake ESU)

Statistical Reliability

2,1007101751,170annual cost of design alternatives (x $1,000)

Cost ($)

GoodFairPoorPoor

ability to make viability assessments for each population in the ESU

Inferential ability (Qualitative)

HighMed Low Status Quo

Design examplesPerformance Measures(Abundance and Spatial

Structure)

Design Objectives

0.840.730.410.60Pr (making the correct viability assessment for ESU populations)

Statistical Reliability

2,1257101751,283annual cost of design alternatives (x $1,000)

Cost ($)

ExcellentVery GoodPoorFair

ability to make viability assessments for each population in the Snake Sp/S Chinook ESU

Inferential ability (Qualitative)

HighMed Low Status Quo

Design alternativesPerformance Measures(Abundance and Spatial

Structure)

Design Objectives

29

30

ICDT User Guide

31

Questions?