André E. Punt1School of Aquatic and Fishery Sciences,

UW2CSIRO Marine and Atmospheric

Research

How has Strategic Advice Been Usedin a Global LMR Context: A GlobalPerspective on How to Deal With

Ecosystem Model Uncertainty

Outline Some definitions (to provide context). A process for strategic evaluation.

Assigning plausibility weights Case studies I & II (environmental drivers of

recruitment) Case studies III & IV (trophic interactions & MRMs) Case study V (whole of ecosystem models)

Examples International Whaling Commission:

Aboriginal and commercial whaling. Australia:

Management of the SESSF. South Africa:

Penguins and anchovy. USA:

Evaluation of the GOA pollock harvest control rule.

Definitions andcontext

Tactical Advice -> What is next year’s catch limit for pollock?

Strategic Advice -> How well does the approach we use for determining next year’s catch limit for pollock perform relative to a set of agreed management objectives

Tactical Advice -> A number.

Strategic Advice -> A set of trade-offs.

Strategic Evaluation For this talk a “strategic evaluation” asks the question:

How well does a set of tactics (monitoring, assessment,decision making) achieve a set of (agreed) managementgoals.

Strategic evaluation is not:

• How to determine what the goals should be?• Perfect knowledge analyses• Constant F projections

An “Ecosystem model”:

Anything which is NOT a single-species, single-area,population dynamics model driven by random perturbationsin recruitment and fishery selection.

Standardmodel

Environmentaldrivers

Trophicinteractions

Spatialstructure

Non-fisheriesdrivers

Elements of a Strategic Evaluation (aka MSE):• A set of management goals (appropriately quantified).• A set of candidate strategies to evaluate.• A set of operating models which span the space of possible realities.

Uncertainty arises because of:• model uncertainty (is our model right?).• process uncertainty (are the parameters constant?). • parameter uncertainty (given a model, can we estimate its parameters?).• implementation uncertainty (given a management decision, can we implement it as anticipated?).

The Indirect Approach

Time

Carrying capacity

(or natural mortality)

Simulation trialswere run for changes

in climate in anindirect way

[IWC testing of its revised management procedure]

A Process for Strategic Evaluation

Qualitative managementobjectives (aka the M-S Act)

Quantitative performancemeasures

Hypotheses for systembehaviour

Models of systembehaviour

Data andpriors

Models weights

Candidatestrategies Strategy

ranks

Systemsimulation

A model weighting schemeHow strong is the basis for the hypothesis; in the actual data for the system under

consideration; in the actual data for a similar system; for any system; or in theory.

After Butterworth et al. (1996); Rep. Int. Whal. Comm 46: 637-40.

An IWC interpretation-I Step 1 of the previous scheme requires a

belief in the objective function (aka AIC, DIC, etc.); this is rarely possible.

The IWC approach: Assign each hypothesis (model) a rank of ‘high’,

‘medium’, ‘low’ or ‘no agreement’ using a “Delphi” approach.

Each rank is associated with an agreed (conservation) performance standard.

An IWC interpretation-IIWhat makes a hypothesis “low” plausibility? Obvious conflict with actual data. Obvious conflict with auxiliary information.

Quantitative Tools for Model Weighting

In order of relative ease: Fit diagnostics (observed versus predicted

data; residual plots, q-q plots, etc). Sensitivity tests Variance estimates

Bayesian; Bootstrap; delta method

Case Studies I & II

Environmental Drivers of Recruitment

Incorporating climate forcing(An empirical approach)

1960 1970 1980 1990 2000

-2-1

01

23

Pacific Decadal Oscillation

Year

Link torecruitment

Climate indices

Age-structured operating model

Management Strategy

TACData

“Climate”Decision rule??

Gulf of AlaskaPollock

A’mar et al. (2009); IJMS 66: 1614-32

Data from surveys andthe fishery

Stock assessment model

Target and limitreference points

Stock size, productivity

Fish

ing

mor

talit

y re

lativ

e to

F35

%

Stock size relative to SB47%

Acceptable BiologicalCatch (ABC)

The performance of this approach to setting TACcan be quantified in terms of:

• high stable catches;• low probability of reducing stock size to undesirable (low) levels; and• accurate and precise estimates of biomass (and status relative to target biomass levels). [essentially hindcast skill]

0 2000 4000 6000 8000 10000

020

0040

0060

0080

0010

000

R2 0.36

What drives pollock recruitment?

Kendall et al. Fish Ocean (1996)

Predicted recruitment(with environment)

Est

imat

ed re

crui

tmen

t(fr

om a

sses

smen

t)

0.0

0.1

0.2

0.3

0.4

2010 2020 2030 2040 2050

Spa

wni

ng b

iom

ass

Year

0.0

0.5

1.0

1.5

2010 2020 2030 2040 2050

Spa

wni

ng b

iom

ass

(rel

ativ

e to

targ

et)

Year

050

100

150

200

250

2010 2020 2030 2040 2050

Cat

ch ('

000

t)

Year

Performance when:• the assessment is (almost) correct• recruitment varies about a mean

• the stock is left above the target and the average catch is ~ 150,000t.

0.1

0.2

0.3

0.0

0.4

0.8

1.2

2010 2020 2030 2040 2050

Spawning biomass:• Generally downward• Depends on model for forecasting future climate (two of eight IPCC models)

Year

Spa

wni

ng b

iom

ass

5010

015

020

00

250

500

750

2010 2020 2030 2040 2050

Year

Cat

ch

Spawning biomass:• Generally downward• Depends on model for forecasting future climate (two of eight IPCC models)

Catches:• React faster than abundance, especially for a declining resource.

Uncertainty Model uncertainty

Choice of IPCC model Relationship between environmental indices and

recruitment Process uncertainty

Variation in recruitment about the assumed relationship

Estimation uncertainty Parameter uncertainty (Bayesian analysis)

Eastern North Pacific Gray Whales

Brandon and Punt (2009): IWC Document SC/61/AWMP2

Eastern North Pacific Gray Whale

Ice conditions in the Bering Sea have beenpostulated to impact calf production.

Objectives and Strategies Objectives

Satisfy aboriginal need (Russia and the US) Achieve stock conservation objectives

Management strategy (default) Surveys (of absolute abundance) every 5-10

years. Strike limits based on the IWC’s “Gray whale

SLA”.

Previous Assessment

With climate

Performance Evaluation

Model uncertainty:• Sea-ice impacts calf production• Future catastrophic events are:

• random• related to population density.

Process uncertainty:• Random variation in calf production.

Estimation uncertainty:• Parameters are based on Bayesian estimation.

Other Studies Rock Lobsters off Southern Australia Pacific Sardine off the west coast of the US

Cases StudiesIII and IV

Trophic Interactions(MRMs)

MRM Types Biological interactions

Competition, predation, etc.

Technical interactions Interactions through bycatch.

Anchovy and Penguins

How does penguin breeding success and adult survival depend on the abundance of

pelagic fish?

How does penguin breeding success and adult survival depend on the abundance of

pelagic fish?

Penguins as output statistics

0

1000

2000

3000

4000

5000

6000

1984 1989 1994 1999 2004

thou

sand

tons

Cape Columbine

Robben

Island

Dassen

Island

Dyer Island Cape

Agulhas20°E18°E

33°S

Stony Point

Seal Is.

Cape Town

Malgas, Marcus,

Vondeling &

Jutten Islands

Boulders

Geyser Is.

Cape Columbine

Robben

Island

Dassen

Island

Dyer Island Cape

Agulhas20°E18°E

33°S

Stony Point

Seal Is.

Cape Town

Malgas, Marcus,

Vondeling &

Jutten Islands

Boulders

Geyser Is.

Anchovy and sardinecontrol rule

Uncertainty (sardine and pilchard) Model uncertainty

Stock-recruitment relationships Process uncertainty

Variation in recruitment Variation in bycatch rates

Estimation uncertainty Quantified using bootstrapping

Gulf of AlaskaPollock

A’mar et al. Fish. Res. (Submitted)

GOApollock

Arrowtoothflounder

PacificHalibut

Pacificcod

, , max

, ,,max max1

i a i i y yi y a

i i y

V B B BM

B B B

{Predator functionalrelationship

Pollock harvest policy

Predatorharvestpolicy

(const F)

M really isn’t constant it seems…

Age

Year

24

68

1012

14

1960 1970 1980 1990 2000

Age

Year

24

68

1012

14

1960 1970 1980 1990 2000

Age

Year

24

68

1012

14

1960 1970 1980 1990 2000

Type I

Type II

Type III

Uncertainty Model uncertainty

With / without predation mortality Predator feeding relationship Fishing mortality on the predators

Process uncertainty Variation in recruitment

Estimation uncertainty Parameter uncertainty (Bayesian analysis)

Other Studies Predator-prey interactions:

SSLA for krill management (CCAMLR) Cod and minke whales in the Barents Sea

Technical interactions: Hake off South Africa. Coral trout and red throat emperor off the Great

Barrier Reef, Australia. Prawns off Northern Australia.

Cases StudiesV

Whole of System Models

South East AustraliaWhole of System

Review

Beth Fulton, pers. commn

SE Australian Atlantis-I

EEZ

Claimable shelf

Aim: To rethink management arrangements in the SESSF

Complications:1. Multi-everything2. Relatively data poor3. Many objectives

Atlantis:1. Physical component.2. Biological component.3. Assessment component.4. Management component.5. Social component.6. Non-fishing impacts.

SE Australian Atlantis-II

Advantages:• Considerable “realism”• Appeals to decision makers

Key difficulties:• Driven to an unknown extent by assumptions• Very difficult to calibrate• No variance estimates (ever) relative performance of high level policies (at a PEIS level; perhaps even beyond “strategic”).

Calibration Tests for Atlantis-I

Observed and predicted diet composition for gummy shark

Calibration Tests for Atlantis-II

Forecast basedon Atlantis

Uncertainty Model uncertainty

Productivity / susceptibility – alternative parameterizations.

Structural sensitivity (loop analysis; social network theory).

External forcing scenarios. Process uncertainty

Emergent property of the model. Estimation uncertainty

In a formal sense - N/A (ever?)`

Uncertainty of StrategicEvaluation

(Adoption, Uncertainty, and the State of the Art)

Strategic Evaluations(directly used!)

HakeAnchovySardine

Rock lobster

SardineMackerel

cod

Rock lobster

Toothfish

Minkewhales

Overall Summary(the State of the Art?) Sensitivity tests / model scenarios

IPCC data sets (pollock) Productivity scenarios (Atlantis) Predation functions (pollock)

Process uncertainty Climate-recruitment (pollock) Ice coverage – birth rate (gray whales)

Variance estimation Gray whales, pollock, etc.

How are strategies based ecosystem models used? USA

Pollock A requirement for (continuing) MSC certification. Presented to the NPFMC SSC (but validates current management

strategy). Pacific Sardine

Included in the PFMC CPS FMP IWC

Aboriginal subsistence whaling and commercial whaling management schemes all tested accounting for “ecosystem changes”

The ENP gray whale analysis will form (part of) the basis for the review of the current Strike Limit Algorithm for gray whales in 2010.

How are strategies based ecosystem models used? South Africa (OMPs have legal status)

Penguin model currently “on hold” while it is being refined. Management strategies for sardine and pilchard have

taken technical interactions (and between sector-allocation) into account for over a decade.

Australia Used to “guide” decision making rules. Atlantis provided direction that helped set policy directions

in SESSF (gears, spatial, quota, etc.)

Pre-pre-Implementation Assessment (1)

First Intersessional

Workshop

Agree completed at an Annual Meeting

First Annual Meeting

Pre-Implementation

Assessment (2+)

Second Intersessional

Workshop

Second Annual Meeting

Option or options presented to the Option or options presented to the CommissionCommission

Catch limit?Catch limit?

Commission

2 years

What is actually necessary toprovide strategic advice? The Objective:

How robust are the current / alternative strategies (note that strategies which are “deterministically optimal” will not necessarily be given uncertainty).

Stakeholder Buy-in: Most successful applications are associated with strong

stakeholder involvement: Workshops to identify candidate strategies, hypotheses,

desired trade-offs. Stakeholder involvement is key when “implementation

uncertainty” is important.

What is actually necessary toprovide strategic advice? Think carefully about candidate strategies:

The default strategy should always be the current one. A TAC which is 20% of current biomass will always be

preferred to the outcome of complicated (e.g. ecosystem) model.

Look for a “good enough” solution which is easily explained rather than “complex perfection”.

Avoidance of “unrealistic” scenarios Avoid scenarios which “while interesting” are not strongly

supported by the data (IWC “rejects” all scenarios which are “low” plausibility).

What is actually necessary toprovide strategic advice? Capture the major uncertainties, but avoid 1,000

scenarios: Consider when to “integrate” (process error) and

“scenario”. A balance here is key. Always include assessment error (at realistic levels). Keep the scenarios “balanced” (e.g. high vs low

productivity) Combinations for factors are nice, but usually just add

confusion.

University of Washington Teresa A’mar John Brandon

CSIRO• Beth Fulton • Eva Plaganyi-Lloyd

UCTEva Plaganyi-Lloyd

KEYAcknowledgements