Ecological Informatics 4 (2009) 215–225

Contents lists available at ScienceDirect

Ecological Informatics

j ourna l homepage: www.e lsev ie r.com/ locate /eco l in f

Prevalence-adjusted optimisation of fuzzy habitat suitability models for aquaticinvertebrate and fish species in New Zealand

Ans M. Mouton a,⁎, Ian Jowett b, Peter L.M. Goethals a, Bernard De Baets c

a Laboratory of Environmental Toxicology and Aquatic Ecology, Ghent University, J. Plateaustraat 22, B-9000 Ghent, Belgiumb National Institute of Water and Atmospheric Research, Post Office Box 11115, Hamilton, New Zealandc KERMIT: Research Unit Knowledge-based Systems, Ghent University, Coupure links 653, B-9000 Ghent, Belgium

⁎ Corresponding author. Tel.: +32 9 264 39 96; fax: +E-mail address: Ans.Mouton@UGent.be (A.M. Mouto

1574-9541/$ – see front matter © 2009 Elsevier B.V. Aldoi:10.1016/j.ecoinf.2009.07.006

a b s t r a c t

a r t i c l e i n f o

Article history:Received 23 April 2009Received in revised form 12 July 2009Accepted 13 July 2009

Keywords:Performance criteriaPhysical habitatStreamBrown troutRainbow troutCaddis fly

For many years, habitat suitability models for aquatic species have been derived from ecological datasets bymodel optimisation. Previous research showed that optimisation of the predictive model performance didnot necessarily lead to ecologically relevant models due to the impact of the dataset prevalence. Therefore,the adjusted average deviation was presented as a performance criterion that allowed incorporation ofecological relevance in the model optimisation process. This paper aims to analyse the relation between theadjusted average deviation (aAD) and the training set prevalence for three species in different New Zealandriver systems: caddis flies Aoteapsyche spp., large brown trout Salmo trutta and rainbow trout Oncorhynchusmykiss. The aAD was implemented in a hill-climbing algorithm to optimise a fuzzy species distribution modelfor each species. Specifically, the hypotheses were tested that (1) similar relations between the aAD and thetraining set prevalence would be obtained, (2) training based on the aAD would lead to more accurate modelpredictions than training based on more frequently applied performance criteria such as CCI, and that (3) thefinal fuzzy model would produce a realistic model of habitat suitability. The approach in this paper mayimprove the transparency of the model training process and thus the insight into habitat suitability models.Consequently, this paper could lead to ecologically more relevant models and contribute to theimplementation of these models in ecosystem management.

© 2009 Elsevier B.V. All rights reserved.

1. Introduction

Since the nineties, habitat suitability models have increasinglybeen developed by fitting models to ecological data (Guisan andZimmermann, 2000; Elith et al., 2006; Austin, 2007; Meynard andQuinn, 2007; Peterson et al., 2007). This trend has been stimulated bythe availability of large and high quality data sets which could beapplied in the context of biogeography, conservation biology andclimate change studies (Guisan and Zimmermann, 2000; Guisan andThuiller, 2005; Araújo and Rahbek, 2006; Kerr et al., 2007). Despitethe success of data-driven ecological models, several authors agreethat the fitted model obtained after model training does not alwaysreflect the available ecological expert knowledge (Roloff and Kerno-han,1999; Pearce and Ferrier, 2000; Hirzel et al., 2001). A first possibleexplanation is that data-driven habitat suitability models effectivelymodel ecological (realised) niches rather than fundamental niches(Guisan et al., 2002; Austin, 2007). Consequently, they implicitlyincorporate biotic interactions and negative stochastic effects that canchange from one region to another (Guisan and Zimmermann, 2000;Guisan et al., 2002). Second, low predictive capability of fitted models

32 9 264 41 99.n).

l rights reserved.

may also relate to the natural variability in biological data and thecharacteristics of the model training process, such as the performancecriterion which is applied for model training.

This performance criterion is a key element in the model fittingprocedure (Fielding and Bell, 1997) and may focus on differentaspects of model performance, such as sensitivity, specificity orpredictive accuracy. The focus of the training performance criteriondefines the result of the model training and thus modellers shouldcarefully choose the criterion which corresponds the most to themodel objectives. However, it could be shown theoretically that thefocus of the performance criterion depends on the frequency ofoccurrence of a species in the training data set, referred to as theprevalence (Mouton et al., submitted for publication). This may leadto effects of species prevalence onmodel training results (Hirzel et al.,2001), despite the constant prevalence during model training.Consequently, the criterion that is used should be consistent withthe type of data and the modelling objective (Mouton et al.,submitted for publication). Previous research has shown thattraining performance criteria which focus on predictive accuracy,such as the percentages of correctly classified instances (CCI; Fieldingand Bell, 1997) are particularly sensitive to prevalence effects(Mouton et al., submitted for publication). However, most data-driven ecological modelling studies apply such criteria for modeltraining.

216 A.M. Mouton et al. / Ecological Informatics 4 (2009) 215–225

Several authors suggest that applying a training set prevalence ofapproximately 0.5 may avoid prevalence effects on the model trainingresults (Hirzel et al., 2001; Maggini et al., 2006). However, ecologicaldatasets typically show extreme prevalence values, with speciesoccurring rarely or ubiquitously. Moreover, Jiménez-Valverde and Lobo(2006) argue that unbalanced habitat suitability data are not such aproblem from a statistical point of view, and that the effects ofunbalanced prevalence should not be confused with those of low-quality data affected by false absences for example. Consequently, itmakes more sense to focus on the relation between omission andcommission errors duringmodel training than to adjust training data toa prevalence of 0.5. Therefore, adaptive training performance criteriashould enable model developers to adjust the training procedure to thetraining set prevalence. A previous study demonstrated that suchcriteriamay provide insight in the focus of the training procedure and inthe ecological relevance of the fitted models (Mouton et al., 2008). Inthis study, themodel trainingprocedure of a habitat suitabilitymodel forEuropean grayling (Thymallus thymallus L.) in a Swiss riverwas adjustedto the training set prevalence. Specifically, a fuzzy ordered classifier wasapplied and trained based on an adaptive performance criterion, theadjusted average deviation (aAD). The aAD returns the averagedeviation between the position of the predicted output class and theposition observed output class stored in the training set andincorporates the specific characteristics of fuzzy classifiers with anordered set of classes. Moreover, it can deal with the fuzzy outputs ofthese models and allows to adjust the model training procedure byvarying the parameter α.

This paper aims to analyse the relation between α and the trainingset prevalence for three other species in different New Zealand riversystems: the caddis flies Aoteapsyche spp., large brown trout Salmotrutta L. and rainbow trout Oncorhynchus mykiss (Walbaum). Like inthe previous study, aADwas implemented in a hill-climbing algorithmto optimise a fuzzy habitat suitability model for each species.Specifically, the hypotheses tested were that (1) the relationbetween α and the training set prevalence would be similar acrossthe three studied species, (2) training based on aADwould lead tomoreaccurate model predictions than training based on more frequentlyapplied performance criteria such as CCI, and that (3) the optimal fuzzymodel would produce a realistic model of habitat suitability.

2. Materials and methods

2.1. Study area and collected data

Benthic invertebrate data were collected by Surber sampling in sixNew Zealand rivers: the Mangles, Mohaka, Waingawa, Clutha (Jowettet al., 1991), Waitaki (Stark and Suren, 2003) and Tongariro (Collier,1993) rivers. The number of caddis fly larvae Aoteapsyche spp. wascounted in each of the 692 samples. Aoteopsyche is a net-spinningcaddis fly and is typically found in rivers where the substrate isrelatively stable. Larvae were found in 488 samples, which is aprevalence of 0.71. Samples were collected from as wide a range ofwater depths, velocities and substrate types as possible in each river.Mean water column velocity (at 60% of the depth) and water depthwere measured at each sampling point with a current meter on acalibrated rod. The substrate compositionwas estimated visually witha modified Wenthworth particle size scale. The substrate compositionwas converted into a single index (s) by summing weightedpercentages of each substrate type (Jowett et al., 1991). To allow fora fine-gravel (2–8 mm) category, the weighting values used were aslightly modified form of the original instream flow incrementalmethodology substrate codes (Bovee, 1982): s=0.08×bedrock+0.07×boulder+0.06×cobble+0.05×gravel+0.04×fine gravel+0.03×sand.

The habitat use by trout was measured in the Lake Wanaka outletof the Clutha River in February 2005. The Clutha River is New

Zealand's largest river and supports a very high density of large browntrout S. trutta and rainbow trout O. mykiss (Teirney and Jowett, 1990)where it flows from Lake Wanaka. At the time of the survey, the riverflow was 226 m3 s−1 (mean flow=195 m3 s−1), the average widthwas about 90m, and themaximumdepthwas 7m. Black disk visibilitywas 9.5 m and thus a diver could observe fishes in what wereconsidered undisturbed locations. Specifically, fishes that used theentire water column for feeding would move laterally or, more often,move closer to the bottom as the diver approached. Fishes wereidentified as brown trout or rainbow trout and as large-sized (N40 cm)or medium-sized (20–40 cm) fishes. This study only focuses on largebrown trout and rainbow trout. Large brown trout were observed lessfrequently than large rainbow trout, resulting in respective preva-lences of 0.03 and 0.11. These prevalences depend on the number oflocations that are defined as available habitat and describe the dataused for the analysis rather than the occurrence of trout in the river.While the diver located and counted fishes, an accompanying boatrecorded water depths and flow velocities with an acoustic Dopplercurrent profiler (ADCP; Teledyne RD Instruments, Poway, California).Although precise locations of individual fish could not be locatedaccurately, the river channel was straight and had few abruptvariations in depth, either longitudinally or laterally, so that ADCPmeasurements of depth and velocity could be assumed to apply at fishlocations. For each fish location, a measurement of depth and depth-averaged velocity was randomly selected from the relevant ADCP file.Examination of the ADCP data confirmed that therewas little variationin depth and velocity in the ADCP measurement, and only twomeasurements (1.5%) were excluded because the depth and velocityat the fish location could not be ascertained with sufficient certainty.Substrate throughout the reach was generally a mixture of boulder,cobble and gravel; cobbles were the most common substrate.Substrate was not considered further because it was relativelyconstant and appeared to have little functional relevance for thetrout. Fish locations were recorded with equal effort in all habitattypes and were supplemented by some bank observations of trout inwater near the shore in areas where the diver or boat teamwas unableto operate. Habitat availability data were collected at an average of2.9 m intervals across 15 randomly selected cross-sections at a flow ofabout 170 m3 s−1. Water surface profile modelling (RHYHABSIM;Jowett, 1996) was used to predict depths and velocities at 226 m3 s−1,the flow at which the habitat use data were collected. Additionalcross-sectional data were collected at a flow of 226 m3 s−1 with theADCP.

2.2. Fuzzy rule-based modelling and rule base optimisation

The values assigned to the input variables were described by fuzzysets (Zadeh, 1965) and not by conventional sets with crisp boundaries(hereafter called crisp sets). In contrast to a crisp set, where a givenvariable value either belongs to a set (it has a membership degree of 1to this set) or it does not, a fuzzy set is described by its membershipfunction, indicating the membership degree for each variable value tothis fuzzy set. This fuzzy approach with overlapping set boundaries isin line with the ecological gradient theory (Strayer et al., 2003),because an element can partially belong to a fuzzy set and thus have amembership degree to this fuzzy set ranging from zero to one.Consequently, the linguistic statement ‘the depth is quite low buttending to be moderate’ can for instance be translated into a depthwhich has a membership degree of 0.4 to the fuzzy set ‘low’ and of 0.6to the fuzzy set ‘moderate’. In this study, all membership functions hadtrapezoidal shapes and were defined by four parameters (a1, a2, a3,a4): the membership degree linearly increases between a1 and a2from 0 to 1, is equal to 1 between a2 and a3 and linearly decreasesfrom 1 to 0 between a3 and a4. A triangular membership function isobtained when a2 equals a3. The parameters of the membershipfunctions corresponding to the fuzzy sets used in this work are given

217A.M. Mouton et al. / Ecological Informatics 4 (2009) 215–225

in Table 1 and were optimised to create a uniform distribution of theinput variables over the fuzzy sets. The Shannon–Weaver entropy(Shannon and Weaver, 1963) quantified this uniformity and was usedas an optimisation criterion to increase the quality of the fuzzy sets.

The fuzzy rule base relates the input variables to the speciespresence and consisted of if–then rules, such as ‘IF depth IS moderateAND water velocity IS high THEN the species IS present. The if-part ofthe rule, the antecedent, describes the situation to which the ruleapplies, while the then-part, the consequent, indicates whether thehabitat in this situation is suitable or not for the considered species.Given crisp values of the four input variables, the output of the fuzzymodel is calculated as described by Van Broekhoven et al. (2006) andby Mouton et al. (2009b). First, the membership degrees of the inputvalues to the linguistic values of the input variables are determined.Next, the degree of fulfilment is calculated for each rule as theminimum of the fulfilment degrees in its antecedent. Finally, to eachlinguistic output value a fulfilment degree is assigned equal to themaximum fulfilment degree obtained for all rules containing thelinguistic output value under consideration in their consequent. Up tothis point, the procedure is the same as the one applied in Mamdani–Assilian models (Assilian, 1974; Mamdani, 1974). Whereas the outputof these models is a crisp value, it is not the purpose of a habitatsuitability model to predict a precise numerical value for theoccurrence of a given species. Specifically, it is rather the magnitudeof the frequency or probability of occurrence that is of interest.Therefore, a different kind of fuzzy model was applied, morespecifically a fuzzy ordered classifier. Whereas the output ofMamdani–Assilian models is defined by fuzzy sets, the output of thefuzzy ordered classifier in this study was defined by two crisp outputclasses: ‘absent’ and ‘present’. To each output class a fulfilment degreewas assigned equal to the maximum fulfilment degree obtained for allrules containing the output class under consideration in theirconsequent. The model output of the developed model is fuzzy andis a couple of values between zero and one and summing up to onesince the fuzzy output is obtained by normalising the fulfilmentdegrees of the output classes ‘absent’ and ‘present’. The first value isthe normalised fulfilment degree of the output class ‘absent’, whereasthe second value is the normalised fulfilment degree of the outputclass ‘present’. Consequently, the first value expresses the degree towhich spawning grayling is considered to be absent at a given riversite, whereas the second value indicates the degree to which graylingis considered to be present. Note that the output values included in thevalidation dataset are crisp values (zero or one), indicating either

Table 1Input variables recorded and the corresponding fuzzy sets of the habitat suitabilitymodels.

Species Input variable Fuzzy set Parameters Entropy

Aoteapsyche spp. Depth (m) Shallow (0.00, 0.00, 0.31, 0.79) 0.74Moderate (0.31, 0.79, 1.08, 1.32)Deep (1.08, 1.32, 2.40, 2.64)

Water velocity(m s−1)

Low (0.00, 0.00, 0.45, 1.34) 0.85High (0.45, 1.34, 1.78, 2.23)

Substrate index(−)

Low (0.00, 0.00, 1.73, 5.18) 0.30High (1.73, 5.18, 6.90, 8.63)

Large brown trout Depth (m) Low (0.00, 0.00, 0.89, 2.31) 0.93Medium (0.89, 2.31, 3.20, 3.91)High (3.20, 3.91, 7.11, 7.82)

Water velocity(m s−1)

Low (0.00, 0.00, 0.62, 1.85) 0.91High (0.62, 1.85, 2.46, 3.08)

Large rainbow trout Depth (m) Low (0.00, 0.00, 1.73, 5.18) 0.86High (1.73, 5.18, 6.91, 8.63)

Water velocity(m s−1)

Low (0.00, 0.00, 0.62, 1.85) 0.91High (0.62, 1.85, 2.46, 3.08)

The entropy, indicating the uniformity of the distribution of the values of a variable overits fuzzy sets, was calculated for all input variables.

100% absence (zero) or 100% presence (one) at a given river site.Moreover, the crisp output classes of the fuzzy ordered classifier areordered, and thus the higher the index i of the fuzzy set Ai, the higherthe degree to which a species can be considered to be present. Notethat there could be more than 2 output classes, representing differentabundance classes (Van Broekhoven et al., 2006).

To generate a reliable habitat suitability model, the consequents ofthe fuzzy rules were optimised using a nearest ascent hill-climbingalgorithm as described by Mouton et al. (2009b). Starting from fixedfuzzy sets and a rule base with randomly selected rule consequents,the consequent of one randomly selected rule was switched from‘absent’ to ‘present’ or vice versa and the impact on modelperformance was calculated. If model performance increased, thealgorithm continued with the adjusted rule base; if not, it continuedwith the original one. Each training iteration was stopped when nofurther increase of the performance criterion on the test fold wasobserved. Each training iteration was repeated and the new rule basewas compared to the rule bases obtained from each of the previousiterations. The resulting rule base similarity was the percentage of ruleconsequents that were identical for two rule bases. If the rule basewith the highest performance on the test fold was obtained 3 times,this rule base was selected as the final rule base and trainingcontinued on another fold (Mouton et al., 2009b).

2.3. Training performance criteria

Different models were trained using five performance criteria: 1)the percentage of Correctly Classified Instances – CCI (Fielding andBell, 1997), 2) Cohen's Kappa (Cohen,1960), 3) the true skill statistic –TSS (Allouche et al., 2006), 4) the average deviation – AD (VanBroekhoven et al., 2007) and 5) the adjusted average deviation – aAD(Mouton et al., 2009a). The first three criteria are based on theconfusion matrix (Fielding and Bell, 1997; Manel et al., 2001) andrange from zero (CCI and Kappa) or minus one (TSS) to one. Althoughthe TSS has been applied less frequently than CCI and Kappa, it wasincluded in this paper because Allouche et al. (2006) argue that thiscriterion is independent of prevalence.

The AD (Van Broekhoven et al., 2007) was applied because itincorporates the specific characteristics of fuzzy classifiers with anordered set of classes and can deal with the fuzzy outputs of thesemodels. Specifically, several performance criteria have been developedto evaluate and train presence–absence models, but most of thesecriteria are based on the confusion matrix which requires a thresholdto distinguish between present and absent predictions. Since thesecriteria cannot deal with the fuzzy output of a fuzzy classifier, valuableinformation may be lost by transferring this fuzzy output to the crispoutputwhich is needed to generate the confusionmatrix. Performancemeasures which are derived from the confusion matrix, for instance,are not sensitive to the position of the classes where the wrongclassification occurs (Van Broekhoven et al., 2007).

Therefore, Van Broekhoven et al. (2007) introduced the AD, whichreturns the average deviation between the position of the classobtained with the model and the position of the class stored in thetraining set. For presence/absencemodels, AD varies from 0 to 1 and iscalculated as follows:

AD =1N�XNj=1

jDj j ; ð1Þ

with N the number of data points and

Dj = A1 ymodel;j

� �− A1 ydata;j

� �: ð2Þ

Here A1(ymodel,j) is the normalised degree of fulfilment of the crispoutput class ‘absent’ corresponding to the jth data point, and A1(ydata,j)

Fig. 1. The 5 different rule base clusters obtained from rule base training based on aADwith varying α, averaged over the ten training folds at each prevalence of Aoteapsychespp. Training was performed based on 21 training sets with a prevalence rangingbetween 0 and 1 in steps of 0.05, which were derived from the original dataset. Areaswith the same shade represent the same rule base cluster, while the brightness an areareflects the number of ‘absent’ consequents in the rule base. The light area (left top)contains the rule base with no ‘present’ consequents, whereas the darkest area (rightdown) represents the rule base with no ‘absent’ consequents. The left boundary of eacharea connects themaximumvalues of α at which the rule base of this areawas obtained.The black line indicates the maximum values of α at which the first rule base wasobtained that overestimated the observations.

218 A.M. Mouton et al. / Ecological Informatics 4 (2009) 215–225

is 1 in case of absence in the dataset, 0 in case of presence, for the samedata point. The aAD is then defined as:

aAD =1N�XNj=1

jDj j + Dj

2+ α � jDj j − Dj

2

� �: ð3Þ

Consequently, if Dj is positive, the model underestimates the jthobservation and the corresponding term in aAD is simply Dj. However,if Dj is negative, the model overestimates the jth observation and thecorresponding term in aAD is given by α·|Dj|, a fraction of |Dj|, andtherefore contributing less to aAD.

2.4. Training datasets and scenarios

To analyse the dependency of α on the prevalence of the trainingdataset, present and absent instances of the original dataset wereselected randomly to create 21 new datasets with a prevalencevarying from 0 to 1 in steps of 0.05. For example, if the prevalence ofthe new dataset was lower than the prevalence of the original dataset(respectively 0.71, 0.03 and 0.11 for caddis fly, large brown trout andlarge rainbow trout), fewer present instances were selected than thenumber of present instances included in the original dataset, whereasall absent instances from the original dataset were selected.

Based on the original dataset, 21 new datasets were created with aprevalence ranging from 0 to 1 according to the aforementionedprocedure. On each of these new datasets, ten-fold cross-validationwas applied to estimate the robustness of the optimisation results. Thefolds were constructed by randomising the new data set and assigningeach data point to one fold without replacement. The speciesprevalence was constant for all ten folds and equal to the prevalenceof the new dataset. The habitat suitability models were trained basedon 25 different training scenarios. The models were trained based onaAD in the first 21 scenarios, with α varying from 0 to 1 in steps of0.05. In the last four scenarios, the models were trained based on CCI,Kappa, the TSS or AD. Consequently, 15750 (3×21×10×25) differentsimulations were performed on a Linux cluster containing 14 nodes,containing 72 Dual-Core Intel® Xeon® CPU 3.00 GHz processors intotal, 1 Gb of RAM and running a 2.6.5 kernel.

3. Results

3.1. Aoteapsyche spp.

Based on the original dataset, 21 new sets with differentprevalences were created and rule base training on these sets, basedon aAD with varying α, resulted in different final rule bases at aspecific training set prevalence (Fig. 1). Although general agreementamong these rule bases was observed, some rule bases showed slightdifferences as the prevalence values altered. This result could beexplained by the presence of ‘core rules’ and ‘ghost rules’ in thetrained rule base. Specifically, the rules in a rule base represent eachpossible combination of input variable fuzzy sets, but some rules maydescribe a combination of input variable fuzzy sets which was absentin the studied stretch. These rules are referred to as ghost rules,whereas core rules describe an environmental condition which ispresent in the studied stretch. In contrast to the core rule consequents,the ghost rule consequents could not be trained based on the availabledata and could thus randomly take any value (absent, present) of theoutput variable. However, these ghost rules did not affect the modelpredictions because they did not represent the studied stretch.Consequently, rule base training in this paper sometimes resulted inrule bases with different ghost rule consequents, but equal core ruleconsequents and thus equal model predictions. The trained rule basesin this paper could therefore be joined in 5 clusters (Fig. 1) based onthe consequents of their core rules.

Most of the values of α which delimited the clusters wereincreasing monotonically with the prevalence, but at higher pre-valence values some irregularities occurred (Fig. 1). The number of‘present’ consequents in the final rule base was negatively correlatedwith the value of α at each prevalence (Fig. 1). Consequently, at a fixedtraining prevalence, α represents the likelihood that a rule base isobtained which underestimates the observations: the lower α is, thehigher the likelihood that an overestimating rule base is found. Thefourth cluster (starting from the top left corner in Fig. 1) contains thefirst rule base which overestimates the observations. The upperboundary of this cluster (the solid black line in Fig. 1) represents themaximum values of α at which an overestimating rule base isobtained. If at a specific prevalence, α is lower than or equal to thecorresponding value of α on this line, the final rule base willoverestimate the observations. The solid black line (Fig. 1) representsthe transition from an underestimating rule base to an overestimatingone. Consequently, the rule basewhich approximates the observationsmost accurately, could be found at a value of α ranging between thesolid black line and the upper boundary of the first rule base clusterabove this line.

The results were identical across the different folds, but not all 5rule base clusters were obtained at each prevalence value. Conse-quently, plotting the cumulative predicted presence of the originaldataset (with prevalence 0.71), revealed only two possible solutions,which is illustrated for water velocity in Fig. 2. The effect of α ontraining results becomes clear by plotting the cumulative predictedpresences at a lower prevalence value (Fig. 3). The cumulative plotsprovide an indication of the ecological relevance of the optimised rulebases and show the similarity between the different rule bases. Sincethe rule bases obtained at an α value of 0.25 and 0.20, respectively,underestimate or overestimate caddis fly presence substantially, themost accurate rule basemay be found at a value of α between 0.25 and0.20 (Fig. 3).

Comparison of training based on aAD with training based on fourother performance criteria revealed that at the original prevalence, all

Fig. 2. Cumulative plots of the observations of Aoteapsyche spp. and of the rule basepredictions obtained after training based on aAD with α varying between 0 and 1 insteps of 0.05. Predictions were averaged over the 10 folds and accumulated according totheir corresponding water velocity values. Training was performed on the originaldataset with a prevalence of 0.71. Values of α resulting in the same core rule base, areindicated by the same line type. The lines show the predictions based on the rule basesobtained after training with α varying between 0 and 0.60 (thick grey line) andbetween 0.65 and 1 (thin black line). The thick black line represents the cumulativeobservations.

219A.M. Mouton et al. / Ecological Informatics 4 (2009) 215–225

criteria but Kappa produced rule bases of which all consequents were‘present’ (Fig. 4). Training based on Kappa underestimated theobservations, especially in the 0.2–0.5 m depth range and the 0.1–0.7 m s−1 water velocity range. Although false classificationsappeared to occur over the whole substrate index range, the resultsbased on this variable were similar to those based on the other inputvariables (Fig. 4a). Comparison of the five performance criteria at alower prevalence (0.20), indicated that aAD was the only criterionwhich resulted in overestimating rule bases, whereas AD and TSS ledto identical results at this lower prevalence (Fig. 4b). Moreover, thissimilarity between AD and TSS is in line with the results obtained at ahigher prevalence of the training set (Fig. 4a). To calculate aAD at bothprevalences, the maximal α value was chosen which resulted in thefirst rule base that overestimated the observations. Consequently, thismaximal α value was 0.65 at a prevalence of 0.71 and 0.2 at aprevalence of 0.2, which could also be observed in Fig. 1.

3.2. Large brown trout and large rainbow trout

For large brown trout and rainbow trout, the number of modelparameters was lower and the results were more consistent than forcaddis fly. Consequently, fewer rule bases were obtained after trainingand no clusteringwas needed (Fig. 5). For both species, 6 different rulebases were found, although for large rainbow trout, some rule baseswere only obtained within some limited α ranges. For large rainbowtrout, all values of α which delimited the rule bases were increasing

Fig. 3. Cumulative plots of the observations Aoteapsyche spp. and of the rule basepredictions obtained after training based on aAD with α varying between 0 and 1 insteps of 0.05. Predictions were averaged over the ten folds and accumulated accordingto their corresponding depth (a), water velocity (b) or substrate index (c) values.Training was performed on a modified version of the original dataset with a prevalenceof 0.20. Values of α resulting in the same core rule base, are indicated by the same linetype (thick black line: observations; thick gray line: 0≤α≤0.05; thin black line:0.10≤α≤0.15; dashed grey line: α=0.20; dotted grey line: 0.25≤α≤0.40; dashedblack line: 0.45≤α≤0.55; dotted black line: 0.60≤α≤1).

monotonously with the prevalence, whereas for brown trout someirregularities occurred (Fig. 5) at higher prevalence values. Thenumber of ‘present’ consequents in the final rule base was negatively

Fig. 4. a. The cumulative predictions of Aoteapsyche spp. for the variables depth, water velocity and substrate index obtained after training based on the percentage of correctlyclassified instances (CCI; grey line), Kappa (thin black line), the average deviation (AD; grey line), the true skill statistic (TSS; grey line), and the adjusted average deviation(aAD; grey line) with α=0.20 (observations: thick black line). Training was performed on the original dataset. b. The cumulative predictions of Aoteapsyche spp. for the variablesdepth, water velocity and substrate index obtained after training based on the percentage of correctly classified instances (CCI; solid grey line), Kappa (thin black line), the averagedeviation (AD; dashed grey line), the true skill statistic (TSS; dashed grey line), and the adjusted average deviation (aAD; dotted grey line) with α=0.20 (observations: thick blackline). Training was performed on a modified version of the original dataset with prevalence 0.20.

220 A.M. Mouton et al. / Ecological Informatics 4 (2009) 215–225

Fig. 5. The 6 different rule base clusters which were obtained from rule base trainingbased on aAD with varying α, averaged over the ten training folds at each prevalence oflarge brown trout (a) and large rainbow trout (b). Training was performed based on 21training sets with a prevalence ranging between 0 and 1 in steps of 0.05, which werederived from the original dataset. Areas with the same shade represent the same rulebase, while the brightness of an area reflects the number of ‘absent’ consequents in therule base. The light area (left top) contains the rule base with no ‘present’ consequents,whereas the darkest area (right down) represents the rule base with no ‘absent’consequents. The left boundary of each area connects the maximum values of α atwhich the rule base of this area was obtained. The black line indicates the maximumvalues of α at which the first rule base was obtained that overestimated theobservations.

Fig. 6. Cumulative plots of the observations and of the rule base predictions of largebrown trout obtained after training based on aAD with α varying between 0 and 1 insteps of 0.05. Predictions were averaged over the ten folds and accumulated accordingto their corresponding depth (a) or water velocity (b) values. Training was performedon the original dataset with a prevalence of 0.03. Values of α resulting in the same corerule base, are indicated by the same line type (solid grey line: 0; thin black line: 0.05;dashed grey line: 0.10–1; thick black line: observations).

221A.M. Mouton et al. / Ecological Informatics 4 (2009) 215–225

correlated with the value of α at each prevalence of both species(Fig. 5). For large brown trout, the second rule base (starting from thetop left corner in Fig. 5a) first overestimated the observations,whereas for rainbow trout, three different rule bases first over-estimated the observations, depending on the α value (Fig. 5b).

The plots of the cumulative predictions of large brown troutindicated that the most accurate rule base might be found at values ofα between 0.05 and 0.10 (Fig. 6), and between 0.15 and 0.20 for largerainbow trout (Fig. 7). For both species, model training based on ADand CCI led to identical underestimating rule bases, while trainingbased on TSS, aAD or Kappa resulted in rule bases that overestimatedthe observations (Figs. 8 and 9). For large brown trout, the same rulebases are obtained after training based on TSS and on aAD (Fig. 8),

whereas for rainbow trout, the rule base obtained after training basedon TSS was overestimating the observations more than the oneobtained after aAD training (Fig. 9). In contrast to the caddis flyresults, training based on Kappa led to themost accurate rule bases forboth species (Figs. 8 and 9).

3.3. Comparison of different values of α

The threshold values are defined as the first values of α at which anoverestimating rule base was obtained after model training are shownas a solid black line in Figs. 1 and 5. To analyse whether a universalrelation between the α value and the prevalence exists, thesethreshold values were compared with values from a previous studyon European grayling (T. thymallus L.) in the Aare River in Switzerland(Fig. 10). The three fish species show similar curves, whereas thevalues of α for caddis fly are significantly lower than those for the fishspecies.

222 A.M. Mouton et al. / Ecological Informatics 4 (2009) 215–225

4. Discussion

The dependency between α and the training set prevalenceillustrates the effect of the training criterion on the final model.These results agree with earlier research, which indicated thatdifferent training performance criteria could lead to different finalmodels (Mouton et al., 2009b). An optimal parameter value ortraining performance criterion could be found by applying sensitivityanalysis, but a more important problem with these flexible perfor-mance criteria could be the difficulty in deciding which models arebetter (Glas et al., 2003; Vaughan and Ormerod, 2005). The results inthis paper show that comparison of the shapes of the cumulativeprediction curves may provide an indication of the ecologicalrelevance of the different optimised models. Previous researchindicated that applying a training parameter which minimises thedifference between over- and underprediction may be a suitable ruleof thumb (Maggini et al., 2006). However, there is no straightforward

Fig. 7. Cumulative plots of the observations and of the rule base predictions of largerainbow trout obtained after training based on aAD with α varying between 0 and 1 insteps of 0.05. Predictions were averaged over the ten folds and accumulated accordingto their corresponding depth (a) or water velocity (b) values. Training was performedon the original dataset with a prevalence of 0.11. Values of αwhich resulted in the samecore rule base, are indicated by the same line type (solid grey line: 0–0.05; thin blackline: 0.10; dotted black line: 0.15; dashed grey line: 0.20–1; thick black line:observations).

Fig. 8. The cumulative predictions of large brown trout for the variables depth (a) andwater velocity (b) obtained after training based on the percentage of correctly classifiedinstances (CCI; solid grey line), Kappa (thin black line), the average deviation (AD; solidgrey line), the true skill statistic (TSS; dashed grey line), and the adjusted averagedeviation (aAD; dashed grey line) with α=0.05 (observations: thick black line).Training was performed on the original dataset.

approach to select the best model, since the quality of a modeldepends on its purpose and the context in which the model will beapplied. For instance, several authors agree that the relativeimportance of omission and commission errors may vary amongapplications (Glas et al., 2003; Loiselle et al., 2003; Vaughan andOrmerod, 2005; Wilson et al., 2005). These errors quantify aconsistent under- or overestimation of the species prevalence whichis also referred to as false–negative or false–positive error (Loiselleet al., 2003) respectively. Each performance criterion focuses ondifferent aspects of model performance. Moreover, it could be showntheoretically that the aspects on which these criteria focus vary withthe prevalence of the training dataset. Consequently, conservationistsand model developers should be aware of the different focus of thetraining performance criteria, and thus of the final model.

Another important aspect of performance criteria is the distinctionbetween omission or commission errors (Pearce and Ferrier, 2000;Rondinini et al., 2006). This issue has been addressed extensively inprevious studies (Fielding and Bell, 1997; Anderson et al., 2003;Loiselle et al., 2003; Rondinini et al., 2006; Fitzpatrick et al., 2007;

Fig. 9. The cumulative predictions of large rainbow trout for the variables depth (a) andwater velocity (b) obtained after training based on the percentage of correctly classifiedinstances (CCI; solid grey line), Kappa (thin black line), the average deviation (AD; solidgrey line), the true skill statistic (TSS; dashed grey line), and the adjusted averagedeviation (aAD; dotted grey line) with α=0.15 (observations: thick black line).Training was performed on the original dataset.

Fig. 10. The values of α at which the first rule base which overestimated the observations wasline), large rainbow trout (dotted line) and grayling (thin solid line). The results of graylingmodel training on the original data sets.

223A.M. Mouton et al. / Ecological Informatics 4 (2009) 215–225

Mouton et al., 2009b). The adjusted average deviation allowsmodellers to implement a balance between omission and commissionerrors in their models, while the value of α indicates the extent towhich a model is trained to overestimate or underestimate theobservations. However, the caddis fly results show that α may beunable to prevent a trained model from overestimating when thetraining set has high prevalence. Theoretically values of α higher than1 lead to underprediction (Mouton et al., 2009a). Further research isrequired to determine whether values of α higher than 1 lead tounderestimating models and thus improving the flexibility of aADtraining. Another aspect that may affect model results is the α stepsize which is applied in the sensitivity analysis. Our results show thatthere may be a substantial difference between two consecutive rulebases, especially between the last underestimating rule base and thefirst overestimating rule base. Previous research showed that applyingsmaller α step sizes may decrease this gap (Mouton et al., 2009a).However, this approach may only be appropriate if the number ofmodel parameters allows sufficient fine tuning ofmodel results. In thisstudy, α was not further optimised by applying smaller step sizesbecause the number of model parameters was relatively low.

For all studied species, the results suggest that the optimal α valuefor model training depends on the prevalence of the training set.However, the relation between α and the prevalence might depend onthe prevalence of the original dataset from which the differenttraining sets are derived. The prevalence of the caddis fly dataset ismuch higher than that of the two trout datasets, and thus the values ofα needed to obtain an overestimating rule base may be lower forcaddis fly than for trout. Moreover, the prevalence of the originaldataset also affects the size of the training sets with adjustedprevalences which are derived from the original data set. For thetrout species, the training sets with a high prevalence weresignificantly smaller than those with a low prevalence, whereas forcaddis fly, the training sets with higher prevalences were larger thanthose with a low prevalence. Previous studies indicated that thesample size of the training set may affect the results of the trainingprocedure (Welsh, 1996). Consequently, further research shouldreveal if the size of the training set also affects the relation betweenthe optimal α and the prevalence.

The strong selection of coarse substrata by Aoteapsyche concurswith earlier studies (Jowett and Richardson, 1990; Jowett et al., 1991)and is observed formany other benthic invertebrate species (Minshall,1984). Although many invertebrates have an upper velocity tolerancelimit, abovewhich velocity exceeds the swimming or holding ability ofthe organism or mobilises the substrate on which the organism lives,individuals may be able to tolerate high meanwater column velocitiesif friction and coarse substrata provide lower-velocity conditions nearthe bed. Despite the overestimation of the observations, the predic-tions of the models obtained after training based on aAD are similar to

obtained after model training for caddis fly (dashed line), large brown trout (thick solidwere taken from a previous study (Mouton et al., 2008). All values were obtained by

224 A.M. Mouton et al. / Ecological Informatics 4 (2009) 215–225

the observations of Aoteapsyche at the prevalence of the originaldataset. Specifically, thesemodels indicate that caddis fly larvae preferdepths between 0.2 and 0.5 m, water velocities between 0.3 and 1.1 ms−1 and a substrate index between 5 and 6. These results are similar toresults from a model study on the same data sets by Jowett and Davey(2007), who applied generalised additive models to predict theoccurrence of the same species as those in this study. The modelresults obtained from training based on aAD and Kappa at lowerprevalences (Fig. 4b) show similar trends as those at the originalprevalence (Fig. 4a). However, training based on the other threecriteria led to less ecologically relevant models at lower prevalences.For instance, training based on AD and TSS resulted in a model whichpredicted Aoteapsyche to occur only at depths higher than 1.2 m and atflow velocities higher than 1 m s−1 (Fig. 4b). Consequently, eventhough different training performance criteria may result in similarmodel predictions within the same prevalence ranges, they may leadto diverging model predictions at prevalence values outside theseranges (Fig. 4b). Jowett and Davey (2007) also suggested that therelation between depth and Aoteapsyche occurrence may vary amongdifferent rivers, but this effect was not considered in this paper sincethe number of samples available for each river was limited. Furtherresearch could reveal if this river effect is significant by applying thepresented approach to datasets of different rivers.

For large brown trout, the models trained on CCI and AD predictedbrown trout to be absent in the studied stretch, whereas the modelstrained on Kappa, TSS and aAD significantly overestimated theobservations. However, the model trained on Kappa predicted largebrown trout to occur in a narrower depth range, whereas the modelstrained on TSS and aAD predicted brown trout occurrence in a widerdepth range than the observed range (Fig. 8). Better results wereobtained for water velocity, where the models trained on Kappa, TSSand aAD all predicted brown trout to occur in the observed range, withthe models trained on TSS and aAD overestimating the observationsmore than the model trained on Kappa. Jowett and Davey (2007)argue that the 29 large brown trout observed in the Clutha River weretoo few to produce a robust habitat suitability model but wereprobably sufficient to allow a qualitative comparison with existinghabitat suitability curves. Although no habitat suitability criteria havebeen developed for brown trout in other large (N100 m3 s−1) NewZealand rivers, the results of this study reflected the range of preferredvelocities (0.35–0.6 m s−1) reported in smaller New Zealand rivers(Hayes and Jowett, 1994). The preference for water deeper than 1.5 mwas consistent with the habitat preferences reported by Hayes andJowett (1994), who considered any depths greater than 0.5 m to beideal brown trout habitat.

As with the large brown trout models, model training based on CCIand AD led to identical rule bases that predict large rainbow trout tobe absent. However, in contrast to brown trout, both Kappa and aADpredicted rainbow trout to occur in a narrower velocity range than theobserved range, while Kappa also predicted the preferred depth rangeto be narrower than the observed range. However, all three modelsindicated that the preferred water velocity of large rainbow trout inthe Clutha River was 0.7–1.3 m s−1. This range was considerablyhigher than that reported in North American studies (Bovee, 1978;Leclerc, 1983; Baltz and Moyle, 1984; Raleigh et al., 1984; Suchaneket al., 1984; Hill and Hauser, 1985; Moyle and Baltz, 1985; Cochnauerand Elms-Cockrum, 1986; Lambert and Hanson, 1989; Thomas andBovee, 1993; Pert and Erman, 1994). Most of these studies indicatedoptimal suitability at low velocities (b0.4 m s−1), probably becausethe rivers (flowsb5 m3 s−1) and fish (typically 15–30 cm) wererelatively small. Higher preferred velocities were reported in studiesof adult rainbow trout in larger rivers such as the South Platte River, inwhich fish preferred velocities of 0.5–0.6 m s−1 when actively driftfeeding through a wide range of depths and velocities at flows of 7–17 m3 s−1 (Thomas and Bovee,1993). Similarly, adult rainbow trout inthe Tongariro River (flow=30 m3 s−1) preferred velocities of 0.5–

0.7 m s−1 (Jowett et al., 1996). In large rivers in Quebec, preferredvelocities for rainbow trout ranged between 0.5 and 0.9 m s−1

(Leclerc,1983). Theminimumpreferred velocity in these three studies(0.5 m s−1) agrees with the results of the Clutha River models trainedon TSS, aAD and Kappa. The maximum preferred velocity is morevariable, however, and the upper limit in the Clutha River was thehighest reported to date, as were the preferred depths for largerainbow trout in the Clutha River (Jowett and Davey, 2007).Preferences for depth in the aforementioned studies were morevariable than for velocity, but generally were greater than 1 m(Leclerc, 1983; Thomas and Bovee, 1993; Jowett et al., 1996), and wasalso reflected in the results of the models trained on TSS, Kappa andaAD.

Our study suggests that Kappamay be an appropriate performancecriterion for robust model training. Model training based on Kappaleads to results which are relatively accurate, whereas the results ofmodel training based on TSSmay bemore variable. Due to their strongdependency on the training set prevalence, we suggest that AD andCCI may be less appropriate for model training. Despite its user-friendliness, Kappa does not allow the model developers to focus oncertain aspects of model performance and thus provides less controlon the model training process than an adjustable performancecriterion such as aAD. We hope that this paper has made the modeltraining process more transparent and increased awareness of thestrengths and weaknesses of the final model. Not only could thisimprove the reliability of ecological models in general, but alsoenhance the application of these models in ecosystem management.

Acknowledgements

Ans Mouton is a recipient of a Ph.D. grant financed by the SpecialResearch Fund (BOF) of Ghent University.

References

Allouche, O., Tsoar, A., Kadmon, R., 2006. Assessing the accuracy of species distributionmodels: prevalence, kappa and the true skill statistic (TSS). Journal of AppliedEcology 43, 1223–1232.

Anderson, R.P., Lew, D., Peterson, A.T., 2003. Evaluating predictive models of species'distributions: criteria for selecting optimal models. Ecological Modelling 162,211–232.

Araújo, M.B., Rahbek, C., 2006. How does climate change affect biodiversity? Science313, 1396–1397.

Assilian, S., 1974. Artificial intelligence in the control of real dynamical systems. Ph.D.Thesis, London University, London, Great Britain.

Austin, M., 2007. Species distribution models and ecological theory: a criticalassessment and some possible new approaches. Ecological Modelling 200, 1–19.

Baltz, D.M., Moyle, P.B., 1984. Segregation by species and size classes of rainbow-trout,Salmo-Gairdneri, and Sacramento Sucker, Catostomus-Occidentalis, in 3 Californiastreams. Environmental Biology of Fishes 10, 101–110.

Bovee, K.D., 1978. Probability of use criteria for the family Salmonidae. In: Instream FlowInformation Paper, vol. 4. U.S. Fish andWildlife Service,Washington, DC. FWS/OBS-78/07.

Bovee, K.D., 1982. A guide to stream habitat analysis using the Instream FlowIncremental Methodology. Instream Flow Information Paper 12. In: BiologicalReport, vol. 82. US Fish and Wildlife Service, Fort Collins, Colorado. 248 pp.

Cochnauer, T., Elms-Cockrum, T., 1986. Probability-of-use curves for selected Idaho fishspecies. Project performance report, Idaho Department of Fish and Game, Boise,Idaho, Project F-71-R-10.

Cohen, J., 1960. A coefficient of agreement for nominal scales. Educational andPsychological Measurement 20, 37–46.

Collier, K.J., 1993. Flow preferences of aquatic invertebrates in the Tongariro River.Department of Conservation: Science and Research Series, vol. 60. Wellington, NewZealand.

Elith, J., Graham, C.H., Anderson, R.P., Dudik, M., Ferrier, S., Guisan, A., Hijmans, R.J.,Huettmann, F., Leathwick, J.R., Lehmann, A., Li, J., Lohmann, L.G., Loiselle, B.A.,Manion, G., Moritz, C., Nakamura, M., Nakazawa, Y., Overton, J.M., Peterson, A.T.,Phillips, S.J., Richardson, K., Scachetti-Pereira, R., Schapire, R.E., Soberon, J.,Williams, S., Wisz, M.S., Zimmermann, N.E., 2006. Novel methods improveprediction of species' distributions from occurrence data. Ecography 29, 129–151.

Fielding, A.H., Bell, J.F., 1997. A review ofmethods for the assessment of prediction errorsin conservation presence/absence models. Environmental Conservation 24, 38–49.

Fitzpatrick, M.C., Weltzin, J.F., Sanders, N.J., Dunn, R.R., 2007. The biogeography ofprediction error: why does the introduced range of the fire ant over-predict itsnative range? Global Ecology and Biogeography 16, 24–33.

225A.M. Mouton et al. / Ecological Informatics 4 (2009) 215–225

Glas, A.S., Lijmer, J.G., Prins, M.H., Bonsel, G.J., Bossuyt, P.M.M., 2003. The diagnostic oddsratio: a single indicator of test performance. Journal of Clinical Epidemiology 56,1129–1135.

Guisan, A., Thuiller, W., 2005. Predicting species distribution: offering more than simplehabitat models. Ecology Letters 8, 993–1009.

Guisan, A., Zimmermann, N.E., 2000. Predictive habitat distribution models in ecology.Ecological Modelling 135, 147–186.

Guisan, A., Edwards, T.C., Hastie, T., 2002. Generalized linear and generalized additivemodels in studies of species distributions: setting the scene. Ecological Modelling157, 89–100.

Hayes, J.W., Jowett, I.G., 1994. Microhabitat models of large drift-feeding brown trout inthree New Zealand rivers. North American Journal of Fisheries Management 14,710–725.

Hill, D.M., Hauser, G.E., 1985. The effects of proposed water supply withdrawals on fishhabitat in the Piney River. Tennessee Valley Authority, Office of Natural Resourcesand Economic Development. TVA/ONRED/AWR-86/12.

Hirzel, A.H., Helfer, V., Metral, F., 2001. Assessing habitat-suitability models with avirtual species. Ecological Modelling 145, 111–121.

Jiménez-Valverde, A., Lobo, J.M., 2006. The ghost of unbalanced species distributiondata in geographical model predictions. Diversity and Distributions 12, 521–524.

Jowett, I., 1996. RHYHABSIM, river hydraulics and habitat simulation, computer manual.National Institute of Water and Atmospheric Research (NIWA), Hamilton. report,50 pp.

Jowett, I.G., Davey, A.J.H., 2007. A comparison of composite habitat suitability indicesand generalized additive models of invertebrate abundance and fish presence-habitat availability. Transactions of the American Fisheries Society 136, 428–444.

Jowett, I.G., Richardson, J., 1990. Microhabitat preferences of benthic invertebrates in aNew-Zealand river and the development of in-stream flow-habitat models forDeleatidium spp. New Zealand Journal of Marine and Freshwater Research 24,19–30.

Jowett, I.G., Richardson, J., Biggs, B.J.F., Hickey, C.W., Quinn, J.M., 1991. Microhabitatpreferences of benthic invertebrates and the development of generalized Deleati-dium spp habitat suitability curves, applied to 4 New-Zealand rivers. New ZealandJournal of Marine and Freshwater Research 25, 187–199.

Jowett, I.G., Rowe, D., West, D.W., 1996. Fishery flow requirements of the TongariroRiver. National Institute of Water and Atmospheric Research, Hamilton, NewZealand. Report ELE303.

Kerr, J.T., Kharouba, H.M., Currie, D.J., 2007. The macroecological contribution to globalchange solutions. Science 316, 1581–1584.

Lambert, T.R., Hanson, D.F., 1989. Development of habitat suitability criteria for trout insmall streams. Regulated Rivers: Research and Management 3, 291–303.

Leclerc, M., 1983. Marquage et repérage radiotélémétriques de 71 especes de poisson duterritoire central du projet archipel entre octobre 1982 et août 1983, synthèse desrésultats. Archipel de Montréal. In: Rapport Technique, Poisson, vol. 5. Québec,Canada, pp. 152–153.

Loiselle, B.A., Howell, C.A., Graham, C.H., Goerck, J.M., Brooks, T., Smith, K.G., Williams, P.H.,2003. Avoiding pitfalls of using species distribution models in conservation planning.Conservation Biology 17, 1591–1600.

Maggini, R., Lehmann, A., Zimmermann, N.E., Guisan, A., 2006. Improving generalizedregression analysis for the spatial prediction of forest communities. Journal ofBiogeography 33, 1729–1749.

Mamdani, E., 1974. Application of fuzzy algorithms for control of a simple dynamicplant. Proceedings IEE 121, 1585–1588.

Manel, S., Williams, H.C., Ormerod, S.J., 2001. Evaluating presence–absence models inecology: the need to account for prevalence. Journal of Applied Ecology 38,921–931.

Meynard, C.N., Quinn, J.F., 2007. Predicting species distributions: a critical comparisonof the most common statistical models using artificial species. Journal ofBiogeography 34, 1455–1469.

Minshall, G.W.,1984. Aquatic insect-substratum relationships. In: Resh, V.H., Rosenberg,D.M. (Eds.), The ecology of aquatic insects. Praeger, New York, pp. 358–400.

Mouton, A.M., Schneider, M., Peter, A., Holzer, G., Müller, R., Goethals, P.L.M., De Pauw,N., 2008. Optimisation of a fuzzy habitat model for spawning European grayling

(Thymallus thymallus L.) in the Aare river (Thun, Switzerland). Ecological Modelling215, 122–132.

Mouton, A., De Baets, B., Goethals, P.L.M., submitted for publication. A theoreticalanalysis of performance criteria for presence/absence species distribution models.Ecography.

Mouton, A., De Baets, B., Van Broekhoven, E., Goethals, P.L.M., 2009a. Prevalence-adjusted optimisation of fuzzymodels for species distribution. Ecological Modelling220, 1776–1786.

Mouton, A.M., De Baets, B., Goethals, P.L.M., 2009b. Knowledge-based versus data-driven fuzzy habitat suitability models for river management. EnvironmentalModelling and Software 24, 982–993.

Moyle, P.B., Baltz, D.M., 1985. Microhabitat use by an assemblage of California streamfishes – developing criteria for instream flow determinations. Transactions of theAmerican Fisheries Society 114, 695–704.

Pearce, J., Ferrier, S., 2000. Evaluating the predictive performance of habitat modelsdeveloped using logistic regression. Ecological Modelling 133, 225–245.

Pert, E.J., Erman, D.C., 1994. Habitat use by adult rainbow-trout under moderate artificialfluctuations in flow. Transactions of the American Fisheries Society 123, 913–923.

Peterson, A.T., Papes, M., Eaton, M., 2007. Transferability and model evaluation inecological niche modeling: a comparison of GARP and Maxent. Ecography 30,550–560.

Raleigh, R.F., Hickman, T., Solomon, R.C., Nelson, P.C., 1984. Habitat suitabilityinformation: rainbow trout. U.S. Fish and Wildlife Service Biological ServicesProgram. FWS/OBS/-82/10.60.

Roloff, G.J., Kernohan, B.J., 1999. Evaluating reliability of habitat suitability indexmodels.Wildlife Society Bulletin 27, 973–985.

Rondinini, C., Wilson, K.A., Boitani, L., Grantham, H., Possingham, H.P., 2006. Tradeoffs ofdifferent types of species occurrence data for use in systematic conservationplanning. Ecology Letters 9, 1136–1145.

Shannon, C.E., Weaver, W., 1963. Mathematical Theory of Communication. University ofIllinois Press, Urbana, Illinois. 144 pp.

Stark, J.D., Suren, A.M., 2003. Project aqua: environmental study – aquatic ecosystems:macroinvertebrates. National Institute of Water and Atmospheric Research,Christchurch, New Zealand. Report CHC02/59.

Strayer, D.L., Power, M.E., Fagan, W.F., Pickett, S.T.A., Belnap, J., 2003. A classification ofecological boundaries. Bioscience 53, 723–729.

Suchanek, P.M., Kuntz, K.J., McDonnell, J.P., 1984. The relative abundance, distributionand instream flow relationships of juvenile salmon in the lower Sustina River.Sustina River Aquatic Studies Program Report 7, part 2. Alaska Department of Fishand Game, Anchorage, Alaska.

Teirney, L.D., Jowett, I.G., 1990. Trout abundance in New Zealand rivers: an assessmentby drift diving, vol. 118. Ministry of Agriculture and Fisheries, Freshwater FisheriesCentre, Freshwater Fisheries, Christchurch, New Zealand. Report.

Thomas, J.A., Bovee, K.D., 1993. Application and testing of a procedure to evaluatetransferability of habitat suitability criteria. Regulated Rivers-Research & Manage-ment 8, 285–294.

Van Broekhoven, E., Adriaenssens, V., De Baets, B., Verdonschot, P.F.M., 2006. Fuzzy rule-based macroinvertebrate habitat suitability models for running waters. EcologicalModelling 198, 71–84.

Van Broekhoven, E., Adriaenssens, V., De Baets, B., 2007. Interpretability-preservinggenetic optimization of linguistic terms in fuzzy models for fuzzy orderedclassification: an ecological case study. International Journal of ApproximateReasoning 44, 65–90.

Vaughan, I.P., Ormerod, S.J., 2005. The continuing challenges of testing speciesdistribution models. Journal of Applied Ecology 42, 720–730.

Welsh, A.H., 1996. Aspects of Statistical Inference. John Wiley & Sons, New York, USA.452 pp.

Wilson, K.A., Westphal, M.I., Possingham, H.P., Elith, J., 2005. Sensitivity of conservationplanning to different approaches to using predicted species distribution data.Biological Conservation 122, 99–112.

Zadeh, L.A., 1965. Fuzzy sets. Information and Control 8, 338–353.