This article was downloaded by:[University of Leeds]On: 18 January 2008Access Details: [subscription number 773557615]Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Coastal ManagementPublication details, including instructions for authors and subscription information:http://www.informaworld.com/smpp/title~content=t713626371

Performance Indicator Importance in MPA ManagementUsing a Multi-Criteria ApproachAmber H. Himes aa CEMARE, University of Portsmouth, Portsmouth, United Kingdom

Online Publication Date: 01 October 2007To cite this Article: Himes, Amber H. (2007) 'Performance Indicator Importance inMPA Management Using a Multi-Criteria Approach', Coastal Management, 35:5, 601- 618To link to this article: DOI: 10.1080/08920750701593436URL: http://dx.doi.org/10.1080/08920750701593436

PLEASE SCROLL DOWN FOR ARTICLE

Full terms and conditions of use: http://www.informaworld.com/terms-and-conditions-of-access.pdf

This article maybe used for research, teaching and private study purposes. Any substantial or systematic reproduction,re-distribution, re-selling, loan or sub-licensing, systematic supply or distribution in any form to anyone is expresslyforbidden.

The publisher does not give any warranty express or implied or make any representation that the contents will becomplete or accurate or up to date. The accuracy of any instructions, formulae and drug doses should beindependently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings,demand or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with orarising out of the use of this material.

Dow

nloa

ded

By:

[Uni

vers

ity o

f Lee

ds] A

t: 13

:49

18 J

anua

ry 2

008

Coastal Management, 35:601–618, 2007Copyright © Taylor & Francis Group, LLCISSN: 0892-0753 print / 1521-0421 onlineDOI: 10.1080/08920750701593436

Performance Indicator Importance in MPAManagement Using a Multi-Criteria Approach

AMBER H. HIMES

CEMARE, University of PortsmouthH. M. Naval BasePortsmouth, United Kingdom

Much has been written about the usefulness of marine protected areas (MPAs) asa management tool. Their performance has been measured using biological andecological indicators. However, objectives of management also include economicand social responsibilities. As such, stakeholder objectives in MPA management arefrequently incompatible. This has been attributed to the variety of stakeholders withan interest in how MPAs are managed. This article considers the development of aperformance indicator hierarchy for the Egadi Islands Marine Reserve, and a multi-criteria approach implemented to define compromise positions between stakeholdersin decision-making. Data was obtained from a pairwise comparison survey usingthe analytic hierarchy process to investigate preferences of stakeholder groups forperformance indicators in evaluating marine protected area management. The analysisshowed that although there are five key stakeholder groups, none are homogenous inprioritizing performance indicators and that none are clear with respect to what theMPA means for them.

Keywords AHP, marine protected areas, Mediterranean, multi-criteria analysis,stakeholders

Introduction

The challenge facing the development of successful marine resource managementinstitutions is to determine strategies that effectively protect marine habitats, reduceoverexploitation, provide benefits for local communities, and maintain local communitiesthat depend on marine resources for their livelihood. In the measurement of performanceof marine protected areas (MPAs), socioeconomic indicators have been largely ignored(Pelletier et al., 2005). However, studies suggest that for an MPA to be accepted andsupported by local stakeholder groups (i.e., anyone who is invested in the outcome of

This is a contribution from a Ph.D. dissertation completed at the University of Portsmouth(United Kingdom). I am most grateful to the researchers at the University of Portsmouth’s Centre forthe Economics and Management of Aquatic Resources, especially Dr. David Whitmarsh, Dr. VictoriaEdwards, and Dr. Simon Mardle. I am also grateful to the researchers at the Istituto per l’AmbienteMarino Costiero in Castellammare del Golfo, Sicily and the fishers and residents of the Egadi Islands.

Address correspondence to the author’s current location: Amber H. Himes, U.S. Fish and WildlifeService, 6010 Hidden Valley Road, Carlsbad, CA, 92011, USA. E-mail: amber@tuppers.com

601

Dow

nloa

ded

By:

[Uni

vers

ity o

f Lee

ds] A

t: 13

:49

18 J

anua

ry 2

008

602 A. H. Himes

management actions or decisions related to an MPA), management must actively pursuetargets that incorporate the needs and interests of all stakeholders (Brown et al., 2001).

One way to accomplish this is to analyze stakeholder preferences for managementobjectives, interventions, and performance indicators. Each can be used to evaluate theappropriateness and efficiency of MPA management (Brown et al., 2001). Although manysuch analyses rely on semi-quantitatively evaluating stakeholder preferences in MPAmanagement, a more intense multicriteria analysis (MCA) of stakeholder preferences canbe developed to define a concise set of criteria for evaluating protected areas affectedby unique contextual factors. MCA is a decision-making tool developed for complexproblems that allows compromises between conflicting objectives to be analyzed in astructured framework. By using MCA methods, there does not have to be a consensus onthe relative importance of the criteria or the rankings of the alternatives. Through eachMCA method, respondents enter his or her own judgments about the criteria given, andmake a distinct, identifiable contribution to a jointly reached conclusion. For the purposesof this study, pairwise comparison (i.e., comparing multiple criteria by pairs and comparingpreferences between pairs) was chosen as an appropriate MCA method, as it focuses onranking preferences and is suitable for subsequent statistical analyses (Mardle & Pascoe,1999; Wattage & Mardle, 2005).

This article considers the development of a representative performance-criteriahierarchy using data obtained from a pairwise comparison survey based on the Egadi IslandsMarine Reserve (EIMR) in north-western Sicily to investigate priorities for improvingMPA management among stakeholder groups associated with the EIMR. Performanceindicators that cover management, biology, sociology, and economy are considered. Theanalytic hierarchy process (AHP) is a common tool used to evaluate preferences forand importance of a variety of criteria (Saaty, 1977). The AHP offers a methodology tocompare complex performance criteria among diverse groups. It was adopted in this studyto further analyze stakeholder preferences for performance indicators in the EIMR. For thepurposes of this study, the definition of MPA as defined in Italian law is used: “Area of themarine environment, with water, sea bottom and tracts of adjacent coastline that representimportant natural, geomorphological, physical, biochemical characteristics, with particularrelevance to the marine and coastal flora and fauna and to the scientific, ecological, cultural,educational and economic importance that they represent” (Law 979/1982).

Egadi Islands Marine Reserve

Sicily is perhaps the ideal setting for an examination of the evolution and performanceof MPAs in Italy and the Mediterranean. There is tremendous diversity in the types ofmarine resources that its MPAs protect and in the degree to which each has achieved thatprotection. The Egadi Islands Marine Reserve was chosen for the present research due to itsdramatic history, the poor level of performance of the MPA, the large variety of interestedstakeholders, and its placement in the poorest region of a highly developed country.

The Egadi Islands are located directly west of the city of Trapani at the western-mostpoint of Sicily. The EIMR was created around the Islands in 1991, one of 23 established bythe Law for the Defense of the Sea (L979/1986 Legge per la Difesa del Mare) off the coastof Italy. The MPA stretches westward off the coast of Trapani encompassing three islands,Favignana, Marettimo, and Levanzo, and two rocky outcroppings. It covers approximately53,000 ha, divided into four zones ranging from two no-take zones to zones with minimalrestrictions, and 22 kilometers of protected coastline (Figure 1).

Dow

nloa

ded

By:

[Uni

vers

ity o

f Lee

ds] A

t: 13

:49

18 J

anua

ry 2

008

Fig

ure

1.M

apof

the

Ega

diIs

land

s,Si

cily

.

603

Dow

nloa

ded

By:

[Uni

vers

ity o

f Lee

ds] A

t: 13

:49

18 J

anua

ry 2

008

604 A. H. Himes

Currently, the EIMR is managed by the local government of Favignana. The mayor ofthe local government is named President of the MPA and has responsibility of ensuring thepresence of a director and the functioning of the MPA. The Trapani Harbor Master’s Office,along with its sub-offices located on Marettimo and Favignana, has the responsibility forenforcement of the regulatory framework of the MPA and all relevant regional and nationalfishing regulations. Although the legislation that established the EIMR outlined mainobjectives of the EIMR (i.e., environmental protection, public education, archaeologicalpreservation, research, and socioeconomic development), no management plan has beenwritten and little active management is undertaken (with the exception of limited patrolsaround the no-entry zones).

With many residents dependent on fishing and marine resources (e.g., tourism, boating)for their livelihoods and survival as an isolated community, the people of the Egadi Islandshave become the unintended victims of their government’s attempts at marine conservation.Unfortunately, in most cases, the establishment of MPAs in Italy was done bureaucraticallyat the Ministry of the Environment in Rome in concert only with local governmentsand environmental organizations. Rarely have local peoples’ ideas or objections beenconsidered. In the Egadi Islands, the main proponents of the EIMR were local environmentalgroups that successfully lobbied the Ministry of Environment to create a protected area thateliminated the threat of oil drilling in local waters. Local residents and fishers were notgiven the opportunity to comment on EIMR creation and most have adamantly opposed itfrom the beginning. They feel that the EIMR, as it currently exists, is worthless and refuseto believe it could benefit them in the long run under current management. Despite this,however, most have commented that if the EIMR is managed “better” it will be a successand benefit everyone (Himes, 2003).

To date, few biological studies have examined the performance of the EIMR in termsof its ability to increase the biomass of local marine organisms. Furthermore, minimalwork has been done to determine the economic impacts and the sociocultural impacts ofthe EIMR on local stakeholders (Bertolino et al., 2001; Himes, 2003; 2005; 2007). As aresult, a study comparing the performance indicators most important to local stakeholdergroups regarding the MPA is highly relevant.

A Multicriteria Approach

The AHP has become a widely applied MCA and preference elicitation method. Introducedby Saaty (1977), it has been used in a variety of application areas to evaluate user preferencesbased on the concept of paired comparison (Saaty & Vargas, 2004). The method calls fora direct comparison between pairs of objectives in a decision-making problem, otherwiseknown as making pairwise comparisons. Pairwise comparison generally refers to anyprocess of comparing entities in pairs to judge which of each pair is preferred, or has agreater amount of some quantitative property. The method of pairwise comparison is usedin the scientific study of preferences, attitudes, voting systems, social choice, and publicchoice. In the AHP, a respondent is asked to make all pairwise comparisons so that a priorityranking can be made on a ratio scale for each objective. Four main steps have been identifiedin using the AHP framework: (1) set up a hierarchy of performance indicators; (2) collectdata through a pairwise comparison survey regarding the preferences of individuals for theindicators; (3) analyze individuals’ priority preferences; and (4) aggregate homogeneoussets of preferences to derive a set of ratings for the indicators (e.g., Leung et al., 1998;Mardle et al., 2004).

Dow

nloa

ded

By:

[Uni

vers

ity o

f Lee

ds] A

t: 13

:49

18 J

anua

ry 2

008

Performance Indicator Importance in MPA Management 605

Figure 2. Key performance indicators defined by EIMR stakeholder groups (numbers representaggregated priorities of all respondents).

Expanding the range of application, it was used in the present study to defineand analyze stakeholder group preferences for performance indicators in the EIMR.The AHP allows the decision maker to focus on developing and analyzing a formalhierarchy of all the important factors likely to differentiate good from poor decisions.It reduces the cognitive burden of prioritizing decision-making problems through the useof simple pairwise comparisons. Specifically, the methodology encourages respondents tomake subtle trade-offs between two non-quantifiable attributes at a time, thus offering amore complete understanding of the qualities of the objectives or indicators at hand. Theresulting pairwise comparison matrix provides a method to crosscheck relative values dueto built-in redundancy from the comparison of all possible pairs of objectives (Dodgsonet al., 2000). Also, the AHP requires no probabilistic assumption about the decisionalternatives. Finally, although the original methodology was developed to allow a singledecision maker to select one of many alternatives, it has been successfully extended to groupdecision-making where the single decision maker is actually a cohort of N individuals (e.g.,Duke & Aull-Hyde, 2002).

An Analysis of MPA Performance Indicators

Definition of the Performance Indicators (Step 1)

Stakeholder preferences for MPA performance indicators were investigated in 2004 usingsemi-structured interviews with a random sample of five stakeholder groups in the EgadiIslands: small-scale fishers, local residents, tourists, EIMR managers, and researchers(Himes, 2007). The main goal of the questionnaire was for stakeholders to determine whataspects of MPA management needed to be improved for the EIMR to be successful. Theresults of these interviews were then grouped and categorized to assist in defining potentialperformance indicators for the AHP analysis. Using the AHP framework, an indicator treewas then created from the most frequently cited performance indicators (Figure 2).

Dow

nloa

ded

By:

[Uni

vers

ity o

f Lee

ds] A

t: 13

:49

18 J

anua

ry 2

008

606 A. H. Himes

Table 1Description of the performance indicators used in the AHP hierarchy

Indicator Description

1 Increase management efficiency (e.g., better organized, buoys,competent managers, adequate enforcement, projects that make theMPA function well)

2 Increase the information available to tourists and locals3 Increase the amount of biological resources in local waters (e.g., there

are more fish in local waters)4 Reduce the amount of pollution (e.g., trash on beaches, boat pollution)5 Increase the number of fish caught by fishers (also indicating more fish

for sale)6 Increase the income made from local tourism7 Increase community involvement in management (e.g., fishers are

responsible for the enforcement)8 Increase the benefits that the local community obtains from the MPA

(incl. increase in work, compensation provided to the community,increase in community well-being)

At the top of the tree is the overall goal of achieving a successful MPA. Stakeholderidentified performance indicators fit into four general categories: biological/environmental,social, economic, and management. Each of these was then further defined by the eightmost frequently cited subcategories (Himes, 2007). An explanation of each subcategoryindicator is found in Table 1. Each category is represented by two subcategories thatrepresent the most frequently cited performance indicators by stakeholders previously. Thecategory “biological/environmental” was used to convey both the needs to increase fishbiomass within the EIMR through environmental protection and to decrease the level ofpollution in local waters. The category “social” is particularly related to involving the localcommunity in MPA management and allowing the community to benefit from the MPAwithout being overly penalized by its existence. The “economic” indicators were designedto represent economic improvements in the two main professions that theoretically couldbenefit from the MPA: fishing and tourism. Finally, the category “management” is intendedto include an increase in the two most cited performance indicators specifically relevant tomanagement: management efficiency and availability of information regarding the MPA tolocals and tourists.

The Survey (Step 2)

A pairwise comparison survey was designed using the structure of performance indicatorsshown in Figure 1. Standard AHP involves comparisons only within each level of thehierarchy at a time. The present study, however, takes the approach of comparing all eightobjectives at the lower level, regardless of higher-level characterization. A total of 28pairwise comparison questions were given to respondents.

The scale of comparison used was the standard AHP 9-point scale. The scale is definedas follows: (1) indifferent, (3) weak preference, (5) preference, (7) strong preference, and(9) very strong preference. Values of 2, 4, 6, and 8 are intermediate values between the two

Dow

nloa

ded

By:

[Uni

vers

ity o

f Lee

ds] A

t: 13

:49

18 J

anua

ry 2

008

Performance Indicator Importance in MPA Management 607

closest judgments (Mardle et al., 2004). An example of one comparison between two ofthe EIMR performance indicators is shown here:

Generally, a mail-based survey approach is used to distribute the questionnaire.However, based on suggestions given by Mardle et al. (2004), the questionnaire forthis study was presented in face-to-face interviews in order to prevent the loss of vitalinteraction between participants and interviewers. All interviews were conducted in thesummer of 2004. A random sample of 53 respondents completed the questionnaire. Arepresentative working population, including members of the five stakeholder groups, wasrandomly chosen to represent the general population of individuals that utilize the EIMR.Questionnaires were presented to fishers at their fishing vessels, to MPA managers at theiroffices, to researchers at local universities, and to local residents in bars and cafes in thecenter of town on each island. Due to low population numbers, a census was attemptedfor the populations of fishers, management officials and researchers. A random, stratifiedsample was taken of local residents, by attempting to interview all residents that enteredbars and cafes that the interviewers sat in, and an attempt was made to interview an equalnumber of men and women.

The Analysis of Performance Indicators Priority Preferences (Steps 3 and 4)

Priorities from the pairwise comparison scales used in the survey were derived for theindicators in terms of their importance in achieving the overall goal (i.e., a successfulMPA). For each respondent, a pairwise comparison reciprocal matrix (A) of judgments wasconstructed, a key and important feature of the AHP:

A = aij =

1 a1/a2

· · · a1/an

a2/a1

1 · · · a2/an

· · 1 ·an

/a1

an/a2

· · · 1

(1)

where ai is the relative numerical preference (from –9 to 9) of performance indicator i.Relative priorities were then derived for each of the defined alternatives from the pairwisecomparison reciprocal matrix by solving (Saaty, 1977; Wattage & Mardle, 2005):

n∑j=1

aijwj = λmaxwi,∀i(aji = 1/aij and aij > 0) (2)

where a is an individual element of the preference matrix, i and j indicate the ith and jthindicators, λ max is the largest eigenvalue, and the weights (w) are normalized appropriately,

n∑i=1

wi = 1 (3)

The positive reciprocal matrix (A) and the set of equations (2) are solved using theeigenvector method. The solution is normalized in this case as shown in equation (3).Furthermore, an indication of respondent’s consistency in providing responses to each

Dow

nloa

ded

By:

[Uni

vers

ity o

f Lee

ds] A

t: 13

:49

18 J

anua

ry 2

008

608 A. H. Himes

pairwise comparison can also be determined. A consistency index (CI) is measured for thecomparison matrix where

CI = λmax − n

n − 1(4)

The matrix A is considered to be consistent when wi = aijwj and its principal eigenvalueis equal to n (i.e., the dimension of A). The matrix A is said to be inconsistent when λmax

> n. The variance of the error inherent in estimating aij (a quantitative measure of eachrespondent’s judgment concerning the importance of objective i over objective j) may thenbe shown to equal (λmax − n)/(n − 1) (Mardle & Pascoe, 1999; Wattage & Mardle, 2005). Anindication of a respondent’s consistency can be determined and compared to an indicativeconsistency produced from randomly developed matrices. From this, a consistency ratio(CR) for an individual can be produced, calculated by

CR = (λmax − n)/(n − 1)

RI(5)

where the variance of the error is divided by an average consistency index derived fromthe RI. Perfect consistency occurs when λmax equals n (CR = 0); therefore, the closer λmax

is to n, the better the consistency. CR values of less than 10% are desired; however, manyauthors have accepted values up to 20% in post analysis (Mardle & Pascoe, 1999).

An Analysis of MPA Performance Indicators

Only 39 of the 53 individual matrices could be used in the analysis, resulting in 1,092 usablepairwise comparisons. The remaining 14 matrices were unsuitable for use in the analysisdue to the respondent’s high inconsistency (CR greater than 20%) in responses to thepairwise comparisons in the questionnaire. The analysis of the AHP results was conductedat the stakeholder group level in order to make comparisons within and between groups.Individuals were invited to partake in the survey and were asked to assign themselves to oneof five stakeholder groups: local residents, artisanal fishers (i.e., small-scale commercialfishers using low-intensity technology), researchers, EIMR managers, and tourists. Of thosesurveys used, 13.6% are researchers (biologists) from nearby universities, 18.2% engage inartisanal fishing activities, some also partaking in pescaturismo,1 40.9% are local residents(37% of which run tourism businesses), 11.4% take part in MPA management, and 15.6%are tourists. The aggregated priorities for each stakeholder group were derived by adding thepriorities obtained for the two components of each category. Table 2 and Figure 3 comparethe aggregated priorities of the four indicator categories for each of the five stakeholdergroups.

From Table 2 and Figure 3, it is apparent that stakeholders are only somewhat dividedover their preferences for the four main performance indicator categories. With the exceptionof managers who rank management twice as important as other groups and fishers who rankthe economic category almost twice as important as the other groups, an overall patternof similarity appears in Figure 3. Apart from these exceptions, the general trend ranks thesocial and biological/environmental categories relatively equal for each of the groups andconsiderably higher than the management and economic categories.

However, at the indicator level, there is much greater disparity between individualand stakeholder group preferences. In order to uncover these differences, individual and

Dow

nloa

ded

By:

[Uni

vers

ity o

f Lee

ds] A

t: 13

:49

18 J

anua

ry 2

008

Performance Indicator Importance in MPA Management 609

Table 2Aggregated priorities of the performance indicators by stakeholder group

Performance indicators Fishers Managers Researchers Residents Tourists

Management 0.126 0.404 0.238 0.166 0.096Increase management

efficiency (indicator 1)0.084 0.258 0.184 0.105 0.049

Std. Dev. 0.080 0.221 0.128 0.074 0.027Increase the information

available to tourists/locals(indicator 2)

0.042 0.146 0.054 0.060 0.047

Std. Dev. 0.028 0.105 0.021 0.043 0.021Biological/Environmental 0.273 0.206 0.298 0.343 0.343Increase biological resources in

local waters (indicator 3)0.144 0.109 0.221 0.119 0.170

Std. Dev. 0.076 0.044 0.100 0.100 0.092Reduce the amount of pollution

(indicator 4)0.129 0.097 0.076 0.224 0.174

Std. Dev. 0.101 0.033 0.049 0.152 0.126Economic 0.366 0.129 0.156 0.128 0.134Increase the number of fish

caught by fishers (indicator5)

0.139 0.060 0.092 0.063 0.069

Std. Dev. 0.068 0.030 0.035 0.061 0.047Increase the income made from

local tourism (indicator 6)0.227 0.069 0.064 0.066 0.065

Std. Dev. 0.118 0.104 0.034 0.048 0.051Social 0.235 0.260 0.308 0.363 0.427Increase community

involvement in management(indicator 7)

0.091 0.157 0.146 0.199 0.219

Std. Dev. 0.037 0.174 0.051 0.126 0.126Increase the community

benefits (indicator 8)0.143 0.103 0.162 0.164 0.208

Std. Dev. 0.073 0.081 0.043 0.099 0.102

aggregated paired comparisons for each of the indicators and the associated priorityweights were considered (Figure 4). Residents considered pollution to have a much higherranking than any other stakeholder group. However, this is closely followed by increasingcommunity benefits and community involvement, which explains the high ranking overallof the social category and shows the community’s interest in getting something out of theEIMR (Table 2).

Managers clearly rank “increasing management efficiency” with the highest priority.This should be expected because they are responsible for the overall performance of theMPA (Figure 4). Furthermore, the priority that managers assigned to increasing managementefficiency was significantly higher than the priority given to it by the other stakeholdergroups (ANOVA, F-value = 4.030, p-value = .006). This was followed by “increasing

Dow

nloa

ded

By:

[Uni

vers

ity o

f Lee

ds] A

t: 13

:49

18 J

anua

ry 2

008

610 A. H. Himes

Figure 3. Comparison of priorities of indicator categories by stakeholder group.

community involvement in management” and “increasing the availability of informationabout the MPA,” perhaps recognizing these indicators as a means to increasing managementeffectiveness. On the other hand, managers gave the individual economic and biological andenvironmental indicators a relatively low ranking (Table 2). This is interesting as it indicatesthat managers claim to be most interested in managing the MPA effectively and involvingthe local community, which are associated with the procedural aspects of management,whereas maintaining the health of the local marine environment and developing the localeconomy in conjunction with the MPA, two of the EIMR’s stated management objectives,seem to be met with low priority.

Researchers ranked an “increase the biomass of biological resources” in the localmarine environment the most important as an indicator of performance and given it apriority that is double that of any other group. The only other indicators researchers rankedvery high were “an increase in management efficiency” and “an increase in benefits that

Figure 4. Aggregated priorities for stakeholder groups.

Dow

nloa

ded

By:

[Uni

vers

ity o

f Lee

ds] A

t: 13

:49

18 J

anua

ry 2

008

Performance Indicator Importance in MPA Management 611

the community obtains” from the presence of the MPA. All this may be due to the relativeemphasis that researchers, namely biologists like those interviewed, seemed to place onenvironmental protection and conservation of marine organisms.

It was expected that fishers would rank “increasing their catch” as the highest priority;however, this ranks considerably lower than “increasing income made from local tourism”and relatively even with “increasing community benefits from the MPA” and “increasingbiological resource biomass” in the area. Although it did not score highest, the priority thatfishers give to increasing the number of fish caught is significantly higher than the othergroups (ANOVA, F-value = 2.658, p-value = .047). Interestingly, none of the prioritiesthat fishers placed on seven of the eight indicators was especially high or distinct fromthe other priorities given, with the exception of “increasing tourism income.” In fact, thepriority fishers have given to increasing income derived from tourism is significantly higherthan all other stakeholder groups (ANOVA, F-value = 8.187, p-value <.001).

Both management indicators and increasing community involvement in managementwere given the lowest priority of fishers. This is surprising because it was hypothesizedthat fishers, being the only stakeholder group whose livelihoods are directly affected by theEIMR, would take a larger interest in management procedures and interventions.

Consistency and Transitivity

Due to the complex nature and variability of the needs and interests of local stakeholdergroups, a negative effect on the degree of consistency achieved by respondents was expected.Standard AHP practice is to accept the responses of individuals where their inconsistencyis less than or equal to 10%. It is unclear in many studies if this is adhered to; however,in fisheries studies up to 20% has been accepted (Mardle & Pascoe, 1999). However,the consistency ratio traditionally measured in AHP studies resulted in a high level ofinconsistency among responses in the present study. There are two explanations for this.

First, in this study, lower level objective comparisons have been made. In the generalcase, only elements belonging to the same parent criterion are compared. As a result, in thepresent study comparisons at the lowest level were made between indicators under differentbranches of the hierarchy (Figure 2). Higher than average inconsistencies could then beexpected as respondents may have difficultly trying to make direct comparisons betweenconcepts that are not easily comparable (e.g., decreasing pollution vs. more income fromtourism).

Second, upon analysis of individual preference matrices, it was determined that themajority of responses with more than 20% inconsistency were problems of scale as opposedto inconsistency. Inconsistency is directly related to the respondent’s comprehension of theAHP scale. Respondents wanted to compare the chosen indicator with everything in general(e.g., is increasing available information important in general) instead of just with the pair athand. Furthermore, it was noted that respondents occasionally treated the scale differently.Many wanted to make strong statements of their opinions and proceeded to rank theindicators they felt the strongest about with a 9 and those that they did not feel stronglyabout with a 1, regardless of the indicator with which it was being compared.

Each respondent’s answers were examined in detail and the number and type ofcircular triads were counted for each respondent. To maintain a logical order of preferencefor objects A, B, and C follows if A is preferred to B and B is preferred to C and consequentlyA is preferred to C (Arrow, 1951). In a complex situation, it is recognized that this logicmay be broken. Kendall (1962) describes this concept as circular triads, and it is a trait inhuman decision-making to which the AHP takes account (Saaty, 1977). Using the concept

Dow

nloa

ded

By:

[Uni

vers

ity o

f Lee

ds] A

t: 13

:49

18 J

anua

ry 2

008

612 A. H. Himes

Table 3Coherence of stakeholder groups

Stakeholder group n Coherence

Managers 5 0.792Residents 14 0.866Fishers 8 0.893Tourists 7 0.906Researchers 5 0.930All respondents 39 0.862

of Kendall’s (1962,146) coefficient of consistence, a maximum of four strong cyclic triadswas determined to be an appropriate ceiling. With the number of objectives in this problem,this gives an acceptable level of significance (i.e., 5% using a χ2 distribution). Therefore,all respondents with more than four strong cyclic triads (leaving 39 of 53 respondents) wereremoved from the analysis.

Variability and Group Coherence

Overall, there is wide opinion on the priority of performance indicators. In fact, for allstakeholder groups the standard deviation is large for many of the indicators. This variabilityis clearly represented by the large error bars that accompany aggregated group preferencesin Figure 4. The main pattern noticeable in Figure 4 is that four out of the five groups tendto assign relatively similar priorities to at least three indicators, with the exception of a fewindividuals that occasionally assign excessively high or low priorities.

The points of agreement within each stakeholder group are indicators that allindividuals have given a relatively low priority to. It is therefore clear that even thoughmembers within each stakeholder group hold different viewpoints, they can find somecommon ground when it comes to prioritizing indicators that are not important to theirgroup as a whole. In an attempt to evaluate this pattern further, a measure of coherenceboth between and within groups was used (Table 3).

A measure of group coherence can be obtained through a vector-based approach bymeasuring the angles between vectors.2 This was suggested for use with AHP data throughthe analysis of group clusters by Zahir (1999). The assumption that individuals belongingto the same group will have a similar preference structure can be tested. The algorithmdescribed by Zahir (1999)< is nonparametric and based on a set tolerance level thatindicates the point at which a group is or is not considered to be part of the same group.The main advantage of this approach is that it can be used to measure the coherence ofpreferences within groups that are defined a priori. Through testing of several random setsof preferences, in this analysis, a coherence value under 0.90 is considered low, valuesbetween 0.90 and 0.93 good, and values between 0.93 and 1 to have high group coherence.

It is clear from this that the coherence of all stakeholders (0.862), irrespective of groupaffiliation, is quite poor. This is probably due to the different issues that affect each individualdepending on their occupation and affiliations. It is therefore important to evaluate the levelof coherence of self-identified stakeholder groups. Table 3 summarizes the coherence levelof each group. Both fishers and residents have very low coherence levels. This shows thatmembers of each group have diverse interests and preferences for the management of the

Dow

nloa

ded

By:

[Uni

vers

ity o

f Lee

ds] A

t: 13

:49

18 J

anua

ry 2

008

Performance Indicator Importance in MPA Management 613

EIMR. For tourists and researchers, group coherence is higher. These groups could havemore coherent opinions because of their origin outside of the EIMR and interests beingcentered on either the recreation or research value of the MPA. The most surprising result isthe coherence level of the MPA managers interviewed. Manager coherence was found to bethe lowest of all stakeholder groups. Although this should be expected, as both current andprevious local government officials were interviewed, this is concerning as it underlinesthe fact that depending on who is in charge, the management priorities could be extremelydifferent, thus affecting the consistency of management priorities significantly over time.

Understanding Variability through Cluster Analysis

The next step in understanding this lack of group coherence is to consider the differencesand similarities among individuals both within and between stakeholder groups. This wasaccomplished with a hierarchical cluster analysis measuring Euclidean distance betweenthe preferences elicited in the AHP survey. Cluster analysis is used to classify objects (orstakeholders in this case) into different groups, or more precisely, to partition a data setinto subsets (clusters), so that the data in each subset (ideally) share some common trait.Cluster analysis in this study allowed for insight into the apparent lack of coherence amongindividuals in stakeholder groups. Figure 5 summarizes the results of the cluster analysis,where each individual is identified by stakeholder group and an identification number. Sixdifferent clusters are identified, labeled A through F, in Figure 5.

A first glance at the composition of each cluster shows that the clusters are nothomogeneous with regards to the a priori identified stakeholder groups. However, ananalysis of cluster composition in Figure 5 shows a more homogeneous relationship basedon other characteristics. Information from interviews done in other studies conducted bythe author (Himes, 2003; 2005; 2007) was available that identifies the demographics ofeach respondent. This information showed an interesting pattern in the clusters. ClusterA consists of managers that have completely different priorities from the other clusters.Cluster B represents a local resident that spends half of his time on Favignana and halfin Palermo and thus perhaps views the MPA with different eyes than a full-time resident.Cluster C is comprised mainly of residents who have been highly involved and vocal inEIMR management. Cluster D represents individuals whose main interest lies in promotingtourism and fishers’ well-being, hence the presence of one manager (politically elected),a fisher cooperative representative, and fishers that have taken an active role in promotingpescaturismo. Cluster E contains individuals who are more conservation minded: thisincludes almost all researchers, more educated residents, and environmentally aware fishers.Not surprisingly, this cluster also included the recently dismissed ex-director of the EIMRwho is trained as a biologist.

Finally, cluster F represented individuals whose main interests involve the well-being ofthe tourism industry. This accounts for all but two tourists and the majority of local residentsthat run tourism businesses as well as young residents whose families earn income fromtourism activities. As seen in Table 4, group coherence greatly improves when respondentsare reorganized into these new clusters (i.e., average 0.940).

Although a priori assignment of stakeholder groups has shown to be possible in fisheriesstudies (e.g., Mardle et al., 2004), the large variation and low group coherence in responsesthat have resulted from the present study begs the question of why individual stakeholdersin the EIMR do not seem to form coherent groups based on self-identified stakeholdergroups. Three points should be made regarding the lack of group homogeneity among apriori identified groups.

Dow

nloa

ded

By:

[Uni

vers

ity o

f Lee

ds] A

t: 13

:49

18 J

anua

ry 2

008

614 A. H. Himes

Figure 5. Hierarchical cluster analysis of AHP respondents.

First, managers, specifically, tend to be extremely scattered among the clusters. Two ofthese managers, a Harbor Master officer and a representative of the EIMR advisory body,form cluster A. The other three are present in clusters E and F. This disparity can be explainedby analyzing those managers, both previously and currently responsible for management,

Dow

nloa

ded

By:

[Uni

vers

ity o

f Lee

ds] A

t: 13

:49

18 J

anua

ry 2

008

Performance Indicator Importance in MPA Management 615

Table 4Coherence of group clus-ters found by Zahir’s al-

gorithm (Zahir, 1999)

Cluster n Coherence

A 2 0.980B 1 n/aC 7 0.935D 5 0.949E 14 0.911F 10 0.925

which agreed to be interviewed. Their backgrounds are extremely diverse, ranging fromthe militarily oriented Harbor Master, to the politically oriented mayor and current director,to the ex-director who returned to a previous position as a biological researcher; all ofwhom come from different ideological backgrounds that would be expected to view MPAmanagement from disparate standpoints and sets of beliefs.

Second, due to the EIMR’s controversial past, very few local people have remainedpassive and many tend to be strongly opinionated. Some of the variation can be explainedby the respondents’ sources of income. Both fishers and residents can be divided into twoseparate groups depending on the interest the respondent has in the tourism industry: thosewho make money from tourism and those who do not. For example, three fishers in thesample have begun to take up pescaturismo in order to profit from the tourism industryas well as fishing. Equally, many residents have started businesses that cater to tourists(indicated as tourism operators in Figure 5). It therefore should not be surprising thatfishers are spread over clusters C–E and residents are spread over clusters B, C, E, and F.

Third, the extent of variability found in ranking priorities for performance indicators inthe EIMR proves just how diverse and largely heterogeneous fishery and MPA stakeholderscan be. A potential explanation is that individuals with interest in how the EIMR is managedare not clear on what the MPA is meant to accomplish and therefore have different conceptsabout the EIMR’s purpose. This could be problematic for MPA managers and the Ministry ofEnvironment as they keep trying new management interventions without having completeconsensus among stakeholders as to the purpose of the MPA. In addition, because theyform the least coherent group, there is potential that managers will always be in conflictwith one another, potentially having a detrimental effect on management efficiency andthe achievement of MPA goals and objectives. The same can be said for other stakeholdergroups. For fishers and residents specifically, until significant work is done within thelocal community to reach consensus on what is expected from the MPA, support formanagement may never be able to increase. EIMR managers can now use this informationto better understand how local interests are organized and for insight into how they canbetter collaborate with the local population.

Conclusions: Applying the AHP to the EIMR and MPA Evaluations

For the most part, the objectives and criteria used in MPA management and evaluation areincompatible (Brown et al., 2001; Unerman & Bennett, 2004). The main reason behind

Dow

nloa

ded

By:

[Uni

vers

ity o

f Lee

ds] A

t: 13

:49

18 J

anua

ry 2

008

616 A. H. Himes

such incompatibility lies in the sheer variety of stakeholder groups that have an interestin how the marine environment is managed (Brown et al., 2001). Different user groupsplace varying importance on individual criteria for successful management and, therefore,judge the performance of an MPA against their own set of priorities. Over the last decade,many MPA managers have attempted to integrate management schemes with a varied setof objectives that take into account the varied needs and interests of key stakeholders.However, rarely have managers attempted to elicit the preferences of stakeholder groups inorder to quantifiably represent stakeholder interests in management decisions.

When the MPA in the present study was instituted, the general lack of local supportfor the program had little impact due to the low level of threats to the environment at thattime. However, as the local marine environment has become more and more degraded overthe last 15 years, the need for local awareness and investment in the success of the MPAhas increased. The present study demonstrates the application of the AHP framework usingthe method in the EIMR as an aid to decision-making and performance evaluation in MPAmanagement. EIMR managers can use the results of this survey to better understand theneeds and interests of the people that use the resources they are trying to manage, andthus improve management to benefit both the environment and stakeholders. One of themost important features of this method is that both quantitative and qualitative criteriacan be integrated into the analysis of MPA performance (Mardle et al., 2004). The use ofquantitative information associated with stakeholder opinions and attitudes is innovative inthe field of MPA management.

The AHP-based survey was used here to prioritize a set of eight stakeholder-nominatedperformance indicators for five stakeholder groups present in the EIMR. The survey revealedsome important results. The analysis showed that the preferences of the key stakeholdergroups identified a priori are not homogeneous in the prioritization of performanceindicators. Instead, a cluster analysis showed virtually no similarity within common apriori defined stakeholder groups, but instead showed similarity among individuals withsimilar personal interests, for example conservation or tourism.

The results of this study can be applied to future management of the EIMR and MPAsin general. As is likely in other MPAs, stakeholder groups in the Egadi Islands do not seemat all clear with respect to what the MPA means for them. The variability found in responsesto the AHP survey shows this clearly—for example, where priorities relative to the fishingindustry, such as increasing the number of fish caught, were expected to be ranked highfor fishers, instead, fishers generally show virtually no agreement on ranking of any of theobjectives. It is also clear in the case of the EIMR that no natural clusters exist that overlapwith stakeholder-identified groups for stakeholder preferences in defining “success.” Thisinformation can also help managers understand where conflicts exist and give insight intohow to deal with that conflict. With respect to this result, the AHP framework has shown tobe particularly strong, providing quantitative information about the links and divergencesbetween attitudes regarding the MPA, both between and within established stakeholdergroups.

These conclusions leave some doubt about the robustness of characterizing stakehold-ers into predefined groups a priori. Based on this study, group incoherence could indicatethat preconceived stakeholder affiliations given by respondents in each of the methodscould lead to the wrong division of resource users. If representatives of stated stakeholdergroups (e.g., fishers) are used as a means to accomplish stakeholder participation, the groupincoherence results found in this study indicate that it would be nearly impossible to appointstakeholder representatives that could accurately represent their constituencies. In fact, ithas been acknowledged that group representatives often “speak only for themselves and

Dow

nloa

ded

By:

[Uni

vers

ity o

f Lee

ds] A

t: 13

:49

18 J

anua

ry 2

008

Performance Indicator Importance in MPA Management 617

a small fraction of their most privileged and assertive followers” (Bianchi & Kossoudji,2001, 3). This then begs the question of how representative stakeholder representativesreally are. If it is accepted that representatives may not be properly representative of theirconstituencies, as suggested by Fletcher (2003), this could pose a difficult problem for thedevelopment of successful co-management institutions. Such regimes, by definition, rely onstakeholder participation in management under the assumption that the representatives MPAmanagers work with are truly representative of their constituencies. This is one of the mostsignificant results of the present study and deserves to be recognized and studied further tounderstand its implications for successful co-management and community participation.

In the case of the EIMR, it is also possible that increased dialogue with localstakeholders to develop more of a consensus within stakeholder groups, such as fishers,that may have mutually exclusive ideas about how a protected area can be more effective.By integrating local stakeholder preferences into an evaluation of management through theAHP, EIMR managers can work with local stakeholders to value and apply the results of theevaluation to improve local management of the MPA. As a result, participatory evaluationapproaches, such as the AHP, can improve the quality of input and perspective that goes intoinvestigating MPA performance. They can also help to generate practical alternatives andoptions to improve management in addition to improving the likelihood that stakeholderswill trust and accept the results.

One thing is clear, in one of the biggest MPAs in the Mediterranean, there aremultiple and conflicting objectives and attitudes regarding management. Determining theirimportance for all stakeholder groups is key to the decision-making process and developingappropriate management strategies (Brown et al., 2001). The novelty of this research isto address this concern by quantifying stakeholder preferences for performance indicatorsto be used in MPA management evaluations and improving management interventions.Managers can use the results of studies like this to understand where stakeholder interestslie and to discover new methods for management that can improve overall effectiveness ofan MPA and local support for management.

Notes

1. Pescaturismo can be defined as any fishers whose vessel is authorized to bring tourists onboard in order to demonstrate how an artisanal fishing boat runs.

2. The dot product of two vectors will be zero for orthogonal vectors and one for equal vectors.

References

Arrow, K. J. 1951. In Social choice and individual values, New York: John Wiley and Sons.Bertolino, F., S. Lombardo, and A. Santulli. 2001. Valutazione degli effetti dell’istituzione della

riserva naturale marina delle lsole Egadi sulle attiviia di pesca. 1–41. Istituto di BiologiaMarina, Trapani, Italy.

Bianchi, R. R., and S. A. Kossoudji. 2001. Interest Groups and Organizations as Stakeholders. SocialDevelopment Paper 36, Washington, DC: World Bank.

Brown, K., W. N. Adger, E. Tompkins, P. Bacon, D. Shim, and K. Young. 2001. Trade-off analysisfor marine protected area management. Ecological Economics 37:417–434.

Dodgson, J., M. Spackman, A. Pearman, and L. Phillips. 2000. Multi-criteria analysis: A manual,London, UK: Department of the Environment, Transport and the Regions, 158 p.

Duke, J. M., and R. Aull-Hyde. 2002. Identifying public preferences for land preservation using theanalytic hierarchy process. Ecological Economics 42:131–145.

Dow

nloa

ded

By:

[Uni

vers

ity o

f Lee

ds] A

t: 13

:49

18 J

anua

ry 2

008

618 A. H. Himes

Fletcher, S. 2003. Stakeholder representation and the democratic basis of coastal partnerships in theUK. Marine Policy 27:229–240.

Himes, A. H. 2007. Performance indicators in MPA management: Using questionnaires to analyzestakeholder preferences. Ocean and Coastal Management 50:329–351.

Himes, A. H. 2005. Performance Indicators of Marine Protected Area Management: A Case Study onStakeholder Perspectives in the Egadi Islands Marine Reserve (Sicily, Italy). Ph.D. Dissertation,University of Portsmouth. Portsmouth, UK

Himes, A. H. 2003. Small-scale Sicilian fisheries: Opinions of artisanal fishers and socioculturaleffects in two MPA case studies. Coastal Management 31(4):389–408.

Kendall, M. G. 1962. Rank correlation methods, London: Griffin and Co. Ltd.Leung, P., J. Muraoka, S. T. Nakamoto, and S. Pooley. 1998. Evaluating fisheries management options

in Hawaii using analytic hierarchy process (AHP). Fisheries Research 36(2–3):171–183.Mardle, S., and S. Pascoe. 1999. A review of applications of multiple criteria decision making

techniques to fisheries. Marine Resource Economics 14:41–63.Mardle, S., S. Pascoe, and I. Herrero. 2004. Management objective importance in fisheries: An

evaluation using the analytic hierarchy process. Environmental Management 33(1):1–11.Pelletier, D., J. A. Garcia-Charton, J. Ferraris, G. David, O. Thebaud, Y. Letourneur, J. Claudet,

M. Amand, M. Kulbicki, and R. Galzin. 2005. Designing indicators for assessing the effects ofmarine protected areas on coral reef ecosystems: A multidisciplinary standpoint. Aquatic LivingResources 18(1):15–33.

Saaty, T. L. 1977. A scaling method for priorities in hierarchical structures. Journal of MathematicalPsychology 15(3):234–281.

Saaty, T. L., and L. G. Vargas. 2004. Models, methods, concepts and applications of the analytichierarchy process. In ed. et al., Boston: Kluwer Academic Publishers.

Unerman, J., and M. Bennett. 2004. Increased stakeholder dialogue and the internet: Towards greatercorporate accountability or reinforcing capitalist hegemony?. Accounting, Organizations andSociety 29:685–707.

Wattage, P., and S. Mardle. 2005. Stakeholder preferences towards conservation versus developmentfor a wetland in Sri Lanka. Journal of Environmental Management 77(2):122–132.

Zahir, S. 1999. Clusters in a group: Decision making in the vector space formulation of the analytichierarchy process. European Journal of Operational Research 112:620–634.