16
Integrative Complexity of Public Beliefs Toward Wildfire Management: Development of a Scale Joshua Carroll 1 University of New Hampshire Alan Bright Colorado State University Integrative complexity is a useful tool for examining the complex way people think about dichotomous, highly charged issues. However, the qualitative nature of this research is lengthy and requires much work from respondents and investigators alike. This paper presents the process by which a combination open-ended and fixed-item scale was developed to measure the complexity of thought that is consis- tent with integrative complexity. We used the controversial issue of wildfire man- agement in developing the scale because it has become a pervasive natural resource concern that has divided the public’s perceptions regarding its management. The resulting scale is designed for use in large surveys across any number of natural resource issues.Integrative complexity is a concept that can help in understanding how people think about dichotomous issues that are typically found in natural resource management. By understanding how people think about these issues, managers can share information and management plans with stake- holders more effectively. However, a major drawback is that the current qualitative method of measurement is a very lengthy process and the results are not generalizable. Therefore, the present paper explores an attempt to develop a combination open-ended and fixed-item scale to measure integra- tive complexity. We have done this within the context of wildfire manage- ment because it is a highly controversial topic that is a major natural resource management concern. Background of Integrative Complexity Integrative complexity is a protocol for measuring a way of thinking. It can be used to describe the structure of the thoughts people have about dichotomous issues over and above the content, and is based on the number of aspects of a problem people consider (Tetlock, 1989). Integrative complex- ity can provide additional information beyond beliefs and attitudes about 1 Correspondence concerning this article should be addressed to Joshua Carroll, 195 Hewitt Hall, University of New Hampshire, Durham, NH 03824. E-mail: [email protected] 344 Journal of Applied Social Psychology, 2010, 40, 2, pp. 344–359. © 2010 Copyright the Authors Journal compilation © 2010 Wiley Periodicals, Inc.

Integrative Complexity of Public Beliefs Toward Wildfire Management: Development of a Scale

Embed Size (px)

Citation preview

Page 1: Integrative Complexity of Public Beliefs Toward Wildfire Management: Development of a Scale

Integrative Complexity of Public Beliefs Toward WildfireManagement: Development of a Scale

Joshua Carroll1

University of New HampshireAlan Bright

Colorado State University

Integrative complexity is a useful tool for examining the complex way people thinkabout dichotomous, highly charged issues. However, the qualitative nature of thisresearch is lengthy and requires much work from respondents and investigatorsalike. This paper presents the process by which a combination open-ended andfixed-item scale was developed to measure the complexity of thought that is consis-tent with integrative complexity. We used the controversial issue of wildfire man-agement in developing the scale because it has become a pervasive natural resourceconcern that has divided the public’s perceptions regarding its management. Theresulting scale is designed for use in large surveys across any number of naturalresource issues.jasp_577 344..359

Integrative complexity is a concept that can help in understanding howpeople think about dichotomous issues that are typically found in naturalresource management. By understanding how people think about theseissues, managers can share information and management plans with stake-holders more effectively. However, a major drawback is that the currentqualitative method of measurement is a very lengthy process and the resultsare not generalizable. Therefore, the present paper explores an attempt todevelop a combination open-ended and fixed-item scale to measure integra-tive complexity. We have done this within the context of wildfire manage-ment because it is a highly controversial topic that is a major natural resourcemanagement concern.

Background of Integrative Complexity

Integrative complexity is a protocol for measuring a way of thinking. Itcan be used to describe the structure of the thoughts people have aboutdichotomous issues over and above the content, and is based on the numberof aspects of a problem people consider (Tetlock, 1989). Integrative complex-ity can provide additional information beyond beliefs and attitudes about

1Correspondence concerning this article should be addressed to Joshua Carroll, 195 HewittHall, University of New Hampshire, Durham, NH 03824. E-mail: [email protected]

344

Journal of Applied Social Psychology, 2010, 40, 2, pp. 344–359.© 2010 Copyright the AuthorsJournal compilation © 2010 Wiley Periodicals, Inc.

Page 2: Integrative Complexity of Public Beliefs Toward Wildfire Management: Development of a Scale

how people think about many controversial issues that are important intoday’s society. In fact, recent research has found that integrative complexitycan moderate the value–attitude relationship, and that individuals who thinkmore complexly about an issue rely less on held values to shape their attitudesthan previously believed, and more on their ability to think complexly(Carroll & Bright, 2009). Since values and attitudes are a precursor to beliefsand behaviors (Fulton, Manfredo, & Lipscomb, 1996), this type of informa-tion is very important to managers.

In a natural resources context, the results can help managers understandhow their actions will be received by the public. One very difficult challengethat managers face is trying to make decisions with the pressure of severalsometimes competing concerns (e.g., ecological issues, public preferences,legal mandates). Beyond this is the imminent and binding threat of litigation.Legal battles waste much of managers’ time, money, energy, and concern.Integrative complexity can provide information about the public constitu-ency that can be used to make more informed decisions.

For example, public land managers dealing with wildfire are in a particu-larly difficult position with regard to the messages of the past (e.g., firesuppression, “Fire is harmful”) and the current understanding that prescribedfire can be beneficial. Many land-management agencies now have wildland firecommunicators whose specific duties are to communicate management prac-tices to the public in hopes of garnering public acceptance and support. Theutility of integrative complexity may be found in its implications for framingmessages to the public in appropriate contexts and levels of complexity.

It is recognized that individuals will respond best to information (e.g., aforest-management campaign) that is set at the level of knowledge and com-plexity at which they function (Hunsberger, Lea, Pancer, Pratt, & McKenzie,1992). For example, it has been shown that when an individual is exposed toinformation of a higher level of complexity than that at which they typicallyfunction, they will tune out further information to simplify the input(Hunsberger et al., 1992). Appropriate measurement and use of integrativecomplexity may get beyond this stimulus blockage by assessing the levels ofcomplexity at which individuals (either collectively or individually) function,and focus information dissemination at or near these levels of complexity.

Beyond this, it has also been found that individuals can be prompted toincrease their integrative complexity thinking about an issue (Hunsbergeret al., 1992). This has been done by probing individuals to include the twomain components (i.e., differentiation, integration) in their responses,although differentiation (i.e., the ability to recognize alternate sides of anissue) was more successful. Therefore, managers may increase integrativecomplexity thinking in lower groups and then disperse information atappropriate levels of complexity in management campaigns.

INTEGRATIVE COMPLEXITY OF WILDFIRE MANAGEMENT 345

Page 3: Integrative Complexity of Public Beliefs Toward Wildfire Management: Development of a Scale

Brief History of the Use of Integrative Complexity

Integrative complexity was originally studied for pre-existing politicalspeeches. For example, Tetlock (1981, 1989) examined the integrative com-plexity of U.S. Congresspersons to see if conservatives differed in integrativecomplexity on political issues from their liberal counterparts. Tetlock (1984)also analyzed the reasoning of members of the British House of Commons,and American versus Soviet foreign policymakers (Tetlock, 1985, 1988).Wallace and Suedfeld (1988) measured the integrative complexity of 16leaders before, during, and after seven international crises. Tetlock, Armor,and Peterson (1994) examined debates over slavery in antebellum America.

There have also been integrative complexity analyses of public issuesoutside of the political arena. Kristiansen and Matheson (1990) analyzed theintegrative complexity of the public toward nuclear weapons, while Dillon(1993) compared the integrative complexity of arguments on abortionbetween statements made by “pro-choice” and “pro-life” advocates. In anatural resources context, integrative complexity has been utilized to look athow coursework in environmental education affects how college studentsview the Endangered Species Act (Bright & Wyche, 1998) and attitudestoward plant and wildlife protection (Bright & Barro, 2000). Finally, inte-grative complexity has been used to assess public attitudes toward wildfire(Burtz & Bright, 2007).

The present paper is designed to describe the concept of integrative com-plexity, discuss the drawbacks of the traditional method of measuring thisconstruct, describe the development of a scale for measuring integrativecomplexity, and discuss the implications of such a scale. This is done withinthe context of the issue of wildland fire and the related management tech-niques of prescribed burning and mechanical thinning.

Integrative Complexity

Description of the Concept

Two factors that are measured within the context of integrative complex-ity are differentiation and integration. Differentiation focuses on whether aperson acknowledges that there is more than one side or dimension to anissue or problem (Baker-Brown et al., 1992). Someone who sees an issue asblack or white (e.g., prescribed burning is dangerous and bad) is exhibitinglow differentiation on that subject, whereas one who sees two dimensions toan issue (e.g., prescribed burning is sometimes beneficial to the forest ecosys-tem but also entails some potential dangers to humans) shows somewhat

346 CARROLL AND BRIGHT

Page 4: Integrative Complexity of Public Beliefs Toward Wildfire Management: Development of a Scale

higher differentiation, and someone who sees several dimensions exhibitseven higher differentiation (Bright & Barro, 2000).

The second factor that is important for integrative complexity is integra-tion, which refers to the development of complex connections among thedifferentiated characteristics and is related to the relative importance ofperceived arguments for and against an issue (Baker-Brown et al., 1992). It isthe recognition of interrelations among different perspectives or dimensionsthat is acknowledged in the differentiation stage of measurement (Wallbaum,1993). The presence of integration regarding an issue requires that the indi-vidual has first exhibited an adequate amount of differentiation (Tetlock,1989).

Measurement of Integrative Complexity

Traditional methods of measuring the public’s integrative complexityutilize either existing text or generate new text through a paragraph comple-tion process (Schroder, Driver, & Streufert, 1967). This involves respondentswriting an essay about an issue and describing their attitudes and beliefstoward an issue. Each respondent’s essay is analyzed by several raters fordifferentiation and integration. Scoring for integrative complexity rangesfrom 1 (lowest integrative complexity score) to 7 (highest integrative com-plexity score). Specific descriptions of scores along the integrative complexitycontinuum are as follows (Baker-Brown et al., 1992; Wallbaum, 1993;Bright & Barro, 2000):

1 = No differentiation, individual sees the issue in only black orwhite terms.

3 = Individual acknowledges at least two viewpoints, and theremay be positive and negative aspects of each.

5 = Individual acknowledges not only multiple viewpoints, butthat there is a moderate level of interactions and tradeoffsamong the alternatives.

7 = Suggests that the individual also has deeply held basicvalues between the alternative issues.

Intermediate scores of 2, 4, and 6 can be assigned if raters have difficultydeciding on a score of 1, 3, 5, or 7. Table 1 provides examples of passages andtheir coding score for integrative complexity (Bright & Barro, 2000).2

2For a complete list of all of the criteria that determine integrative complexity scoring, seeBaker-Brown et al. (1992).

INTEGRATIVE COMPLEXITY OF WILDFIRE MANAGEMENT 347

Page 5: Integrative Complexity of Public Beliefs Toward Wildfire Management: Development of a Scale

Table 1

Sample Passages About Species Protection, Overall Integrative Complexity(IC) Score, and Individual Coder Scores

IC score Passage

1 (bothcoders = 1)

I feel that we should try to leave what life and animalsthat are in their own habitats alone and try our best toprotect them and the environment around them.

2 (bothcoders = 2)

Yes, but with some restraint. In my opinion the quality ofhuman life comes before the protection of diverse lifeforms. I would place preservation ahead of pure economicgain.

3 (coder1 = 4; coder2 = 3)

I feel we have to find a balance between the environmentand population. It’s important to maintain the wildlifeand plant life of the wilderness, but people also need jobs.I feel that animals will move to other habitat if maninterferes with their areas.

4 (bothcoders = 4)

I believe in protecting the diversity as long as it is donewith a reasonable budget and fiscally responsiblemanagement. The problem with programs like this is thatthey take on a life of their own. If an endeavor such asthis can be done without being over done, than I am forit. But experience tells me otherwise.

5 (coder1 = 4; coder2 = 5)

To a degree yes we must monitor the ecology. However,the government simply seems to go overboard sometimes,as it does in every area it touches. Too manyadministrative programs exist; some conflict; all areexpensive. The idea is a good one but it seems we goabout things in an all or nothing fashion. We haveevolved to the place where we can determine the degreethat a species or animal affects the other living things itinteracts with. We need to curb the human problemsworldwide—especially overpopulation or this questionwill one day be moot.

Note. There were no scores of either 6 or 7 in the study from which these data weredrawn (adapted from Bright & Barro, 2000). Grammatical errors were left in passagesto depict responses verbatim.

348 CARROLL AND BRIGHT

Page 6: Integrative Complexity of Public Beliefs Toward Wildfire Management: Development of a Scale

Drawbacks of the Traditional Measurement of Integrative Complexity

While the use of integrative complexity allows researchers to investigatean individual’s beliefs more deeply, the measurement process can be timeconsuming and requires significant effort from both researchers and respon-dents. For example, respondents are asked to complete written or typedessays and, as a result, response rates to mail surveys can suffer. Bright andBarro (2000) noted that one reason for lower than normal response ratesexperienced for their mail survey (which used a traditional method of mea-suring integrative complexity) is that many respondents found that writing anessay was either too difficult or too time consuming.

Another important factor in the measurement of the construct is how wellrespondents understand the instructions for completing the survey. In thecase of using essays to generate integrative complexity scores, instructionsmust make clear to participants that the composition should reflect opinions,valuations, or judgments and should not be merely a descriptive account,which would not be a scorable response for complexity (Baker-Brown et al.,1992).

There are also several concerns for scientists who engage in traditionalintegrative complexity research. Scoring the essays is quite time consumingand requires several researchers working cooperatively to come to an agree-ment on appropriate scores. As noted previously, the paragraph completiontest requires at least two researchers (often 3 or more are used) to score everyessay for every respondent, and then to discuss scores and their reasoning.This is a difficult and lengthy process.

Finally, the qualitative nature of integrative complexity data makes itdifficult and in many cases unfeasible for use on a large scale, restricting thegeneralizability of results from a small sample to broad populations. Thisimpedes its use in many potentially valuable studies and has limited themeasurement of integrative complexity in large survey research.

Development of the Integrative Complexity Scale

Development of an alternative and functional method for measuringintegrative complexity was applied to public perceptions of prescribedburning and mechanical thinning: two important wildland fire-managementtechniques. A scalar measure would overcome the limitations of essay com-pletion methods of integrative complexity and has additional benefits.

First, a fixed-item scale is easier for respondents to complete. The scalemeasure does away with the essay, and replaces it with fill-in spaces andcompletion of a fixed-item scale. Since it has been noted that respondents are

INTEGRATIVE COMPLEXITY OF WILDFIRE MANAGEMENT 349

Page 7: Integrative Complexity of Public Beliefs Toward Wildfire Management: Development of a Scale

reluctant to write an essay about a topic (Bright & Barro, 2000), thisapproach may result in higher response rates. Second, the increased simplic-ity of completing a scale enables the concept to be used in more broad socialscience studies, allowing a larger sample to be obtained. Third, the use of alarger sample allows integrative complexity to be used in studies in which animportant objective is to obtain results that are generalizable to a population.Fourth, the scale makes the scoring more quantifiable and overcomes thechallenges associated with translating qualitative data into quantifiable mea-surement. Fifth, the systematic and quantifiable measure of the integrativecomplexity scale allows for the concept’s use in theoretical models of atti-tudes and behavior. Finally, while ultimately a quantitative measure, thisapproach does maintain aspects of qualitative data collection in its open-ended format, allowing for deeper content analysis of comments made byrespondents.

The Issue: Wildland Fire Management,Prescribed Burning, and Mechanical Thinning

Because of its dichotomous nature, we used the highly charged issue ofwildfire management to develop and test the scale. On the one hand, fire isnatural and is needed by forest ecosystems, provides essential functions (e.g.,regeneration of vegetation), can limit the extent of the impact of disease andinsects, aids in certain wildlife habitat improvement, and leads to the reduc-tion of fuels. On the other hand, there are also costs associated with wildlandfire, such as threats to private property, natural resource harvests, air quality,and scenic beauty. It is this tradeoff between costs and benefits that compli-cates the issue and causes differences in public beliefs toward wildland firemanagement.

Wildfire management is also an important and timely issue in the West.As a result of nearly 100 years of fire suppression by federal land-management agencies, many forests in the U.S. are now loaded with fuelsand, as a result, severe wildfire conditions exist (Pyne, 2001). The major fireevents within the past decade have been particularly costly and threateningto human lives, and land-management agencies are suggesting that someburning or removal of fuels from the forest could help put these ecosystemsat a lower risk of a highly destructive conflagration, and could increasechances to attain a more sustainable fire regime (“Fire Policy and Reports,Programs, and Priorities,” 2008).

Response to the strategy of strict fire suppression has included twocommon management actions designed to reduce fuels in the forest: pre-scribed burning and mechanical thinning. Prescribed burning is defined as fire

350 CARROLL AND BRIGHT

Page 8: Integrative Complexity of Public Beliefs Toward Wildfire Management: Development of a Scale

applied to a specific land area under selected weather conditions to accom-plish predetermined management objectives. When used properly, it has beenboth effective at achieving its objectives and economically feasible. Somerisks associated with improper implementation of prescribed fire are loss ofproperty and life, liability issues, poor air quality, increased water runoff, anda temporary decrease in aesthetics.

Mechanical thinning reduces the amount of vegetation in the forest byphysically removing selected trees and plants with the intention of decreasingthe likelihood of large, uncontrollable fires. It often involves heavy equip-ment (e.g., bulldozers) or light equipment (e.g., chainsaws) entering the forestfor the cutting of trees based on a predetermined spacing or pattern tech-nique. It has become a viable option, along with prescribed fire, for removingfuel and reducing wildfire potential. Some impacts of mechanical thinning aresoil compaction, increased erosion potential, and susceptibility to competi-tive weed growth.

Identifying Public Perceptions of Wildland Fire Management

The policy of government land-management agencies to reduce wildfirerisk through the use of prescribed burning and mechanical thinning has beenmet with public concern. Some communities and groups with interests inforested areas are troubled because of the dangers associated with prescribedfires going awry and the impacts of heavy machinery and road building thataccompany mechanical thinning (Kneeshaw, Vaske, Bright, & Absher, 2004).The public has expressed concern related to these issues; and as a result,public perceptions of fire management have become an important consider-ation for agencies charged with managing areas susceptible to wildland fires.Understanding the public’s perceptions of wildfire management can helpagencies to recognize when policies might be supported by the public, alertagencies when policies may run into public opposition, and help agenciesto develop information to garner support for potentially controversialstrategies.

Limitations of Current Attitude and Belief Research RegardingFire Management

People often have many thoughts about issues such as wildfire manage-ment, many of which may not be consistent; are based on emotions, experi-ence, or neither; and may or may not be based on values that have beenshown to be linked to attitudes (Fulton et al., 1996). Knotek (2006) pointed

INTEGRATIVE COMPLEXITY OF WILDFIRE MANAGEMENT 351

Page 9: Integrative Complexity of Public Beliefs Toward Wildfire Management: Development of a Scale

out how public decisions about fire are directly influenced by their cognitivedisposition, which reflects human values (i.e., enduring beliefs), value orien-tations, attitudes, norms, and behavioral intentions.

Identifying specific beliefs and attitudes toward fire-management strate-gies provides important information for managers; however, this informationis often limited to a laundry list of what people believe about fire manage-ment. This variety of information regarding people’s thoughts about wildfiremanagement is an important aspect of the complexity with which they thinkabout the issue. This level of complexity describes how individuals thinkabout an issue; that is, the structure of their beliefs (i.e., integrative complex-ity; Tetlock, 1989).

Method

Development of the Fixed-Item Scale

Objectives in developing the scale were as follows:

Objective 1: To create a fixed-item scale that will yield integra-tive complexity scores.

Objective 2: To measure integrative complexity of thoughtregarding two wildfire-management techniques (i.e., prescribedburning, mechanical thinning) using the traditional essaycompletion method.

Objective 3: To measure integrative complexity of two wildfire-management techniques (i.e., prescribed burning, mechanicalthinning) with the newly developed scale, using the same sampleof respondents as the traditional methods test.

Objective 4: To determine the extent to which the two measuresof integrative complexity are correlated.

The scale was designed to measure the two primary components of inte-grative complexity: differentiation and integration. In this case, differentia-tion is conceptualized as the extent to which a respondent recognizes alternatesides of the issue of wildfire management. It is traditionally measured bycounting the number of positive and negative statements about an issue in anessay. High differentiation is indicated by an equal (or near equal) numberof arguments on both sides of an issue, considering the total number ofarguments.

To be consistent with the traditional method of measuring differentia-tion, the scale first required respondents to list potential arguments for and

352 CARROLL AND BRIGHT

Page 10: Integrative Complexity of Public Beliefs Toward Wildfire Management: Development of a Scale

arguments against prescribed burning or mechanical thinning. This providedan indication of the number of positive and negative aspects about eachstrategy that were salient to them, much like the traditional methods ofseeking these out from within an essay.

Integration is conceptualized as how the respondent recognizes interrela-tionships between the different sides of the issue and is linked to the relativestrengths of the perceived arguments on both sides. Traditionally, aresearcher subjectively infers the level of integration from the respondent’swriting and the respondent’s apparent level of involvement with the topic.The scale measured integration by asking respondents to indicate thestrength of each argument they had listed on a 7-point scale ranging from 1(extremely weak) to 7 (extremely strong). Integration is the relative consis-tency in the perceived strength of the for versus against arguments.

Scoring Differentiation

Differentiation was measured as a value between 0 and 1, based on theratio created by the number of arguments for versus arguments against theissue. First, the total number of arguments for was calculated. Next, the totalnumber of arguments against was calculated. The lesser of the two wasdivided by the greater to arrive at the differentiation value. A value of 0reflects no differentiation, while a value of 1 reflects the highest differentia-tion. For example, an individual who listed three arguments for and twoarguments against prescribed burns would obtain a ratio of 2 to 3, or adifferentiation score of .67. A ratio of 3 to 4 would result if an individuallisted 3 arguments for and 4 arguments against prescribed burns, for adifferentiation score of .75. Therefore, higher differentiation is also reflectedsomewhat in a higher total number of arguments (beyond the relativelybalanced number of arguments on both sides of the issue), which is consistentwith traditional integrative complexity measurement.

Scoring Integration

A ratio resulting in a score between 0 and 1 was also calculated forintegration. The means of the strengths (on a 7-point scale) of the argumentsfor and the arguments against the issue, respectively, were calculated. Thesmaller mean was divided by the larger mean to yield the integration score.

By continuing with the previous example in which an individual listedthree arguments for and two arguments against prescribed burns, we canillustrate the scoring of integration. Assume that three arguments for pre-

INTEGRATIVE COMPLEXITY OF WILDFIRE MANAGEMENT 353

Page 11: Integrative Complexity of Public Beliefs Toward Wildfire Management: Development of a Scale

scribed burns were rated 6 (moderately strong), 6, and 7 (extremely strong).The mean of the strengths of the arguments for prescribed burning would be6 + 6 + 7 divided by 3, or 6.30. Now, assume that two arguments againstprescribed burning were rated 2 (moderately weak) and 3 (slightly weak). Themean of the arguments against prescribed burning would be 2 + 3 divided by2, or 2.50. The integration score for this respondent would be the lesser valuedivided by the larger, or 2.50 divided by 6.30, yielding an integration score of0.40. An integration score of 0 indicates no integration, while an integrationscore of 1 indicates high integration.

Scoring Overall Integrative Complexity

Integrative complexity was measured as the product of the differentia-tion and the integration scores. This calculation would again yield a valuebetween 0 and 1. Completing the previous example, a differentiation score of0.67 and an integration score of 0.40 would yield an integrative complexityscore of 0.67 ¥ 0.40, or 0.27. This gives equal weight to both components (i.e.,differentiation and integration), which is consistent with the traditionalmeasurement of integrative complexity.

Testing the Integrative Complexity Scale

To test the integrative complexity scale, we used 72 undergraduate stu-dents from two different natural resources and recreation classes at ColoradoState University. First, the respondents were randomly placed into one oftwo groups—a prescribed burning group and a mechanical thinning group—representing the issue about which they would be writing. Second, for eachgroup, half of the respondents were asked to write an essay about theirassigned topic, while the other half completed the scale about that topic.After finishing this first task, those who had first written an essay thencompleted the scale, while those who had completed the scale first then wrotethe essay. There were 63 usable essay–scale pairs3 for the two issues: 33 forprescribed burning, and 30 for mechanical thinning.

There were two phases to the analysis for testing the scale integrativecomplexity score. The objective of the first phase of analysis was to identifyorder bias; that is, to see if the essay integrative complexity scores and scaleintegrative complexity scores were influenced by the order in which respon-

3An essay–scale pair consists of a completed essay and a scale for one of the strategies by oneindividual.

354 CARROLL AND BRIGHT

Page 12: Integrative Complexity of Public Beliefs Toward Wildfire Management: Development of a Scale

dents completed the tasks. Since the groups who completed the essay and thescale in a particular order were randomly composed, a significant differencein integrative complexity scores between the essays, as well as the scalebetween the two groups would suggest systematic order bias. We used anindependent-sample t test to compare the essay integrative complexity scoreof respondents who completed the essay first (Essay IC1st) with the essayintegrative complexity score of those who completed the essay second (EssayIC2nd). The same test for order bias was conducted for the scale integrativecomplexity scores (Scale IC1st vs. Scale IC2nd).

The objective of the second phase of analysis was to determine the extentto which the scale integrative complexity score of respondents correlated withthe integrative complexity score for the traditional essay method. Pearson’scorrelations were obtained between the essay score and the scale score forboth prescribed burning and mechanical thinning. The higher the correlationbetween the two methods, the more valid is the scale measure of integrativecomplexity.

Results

With regard to order bias, there were no significant differences in theintegrative complexity scores based on the order of completion for either theessay or scale scores (Table 2). Respondents who completed the essay firsthad a mean essay integrative complexity score of 3.17, while those whocompleted it second had a mean score of 2.91, t(33) = 0.73, p = .467. Respon-

Table 2

Test for Order Bias on Two Integrative Complexity (IC) Methods

Method n M t p

Essay IC1st 30 3.17 0.73 .467Essay IC2nd 33 2.91Scale IC1st 30 0.59 1.78 .079Scale IC2nd 33 0.48

Note. Essay IC1st = essay IC score for the group who completed the traditional essaymethod first; Essay IC2nd = essay IC score for the group who completed the traditionalessay method second; Scale IC1st = scale IC score for the group who completed thescale method first; Scale IC2nd = scale IC score for the group who completed the scalemethod second.

INTEGRATIVE COMPLEXITY OF WILDFIRE MANAGEMENT 355

Page 13: Integrative Complexity of Public Beliefs Toward Wildfire Management: Development of a Scale

dents who completed the scale first had a mean integrative complexity scoreof 0.59, while those who completed it second had a mean score of 0.48, t(30)1.78, p = .078. These nonsignificant differences suggest that the integrativecomplexity score was not influenced by the order in which respondentscompleted the essay and the scale.

The second phase of analysis examined the extent to which the scale-generated scores were consistent with the essay-generated scores. Pearson’scorrelation between the integrative complexity scores for the two measure-ment methods for prescribed burning was .81 ( p < .01), indicating a strongeffect size. For the mechanical thinning issue, Pearson’s correlation betweenthe integrative complexity scores for the essay and scale was .77 ( p < .01),also indicating a strong effect size. Overall, the mean number of “pro”arguments for the sample was 3.97, while the mean number of “con” argu-ments was 3.52.

Discussion

Summary of Integrative Complexity Scale Construction

The test of the integrative complexity scale yielded a strong correlationbetween the scale and essay results for both the prescribed burning andmechanical thinning issues. The scale defines the attributes of differentiationand integration and uses a straightforward approach to their measurement.Based on the high correlation with the traditional measure of integrativecomplexity, the developed scale appears to be a functional substitute for useon larger and broader applications.

Agreement of Measures

It is difficult to capture a truly qualitative component of human cognitionwith any measure, be it quantitative or qualitative. While the correlationbetween the scale and the essay method was strong, it was not 1.00. There-fore, it is concluded that the scale created was not an exact measure ofintegrative complexity, but rather a measure that seems to reflect and corre-late well with the results obtained from traditional integrative complexitymethods. This suggests that the scale captures a cognitive component ofcomplex thinking.

There are a number of benefits to the fixed-item scale for measuringintegrative complexity. First, the scale is easier to use than traditionalmethods of essay writing and essay scoring, which tax both respondents andresearchers. The scale is simple to complete and is less time consuming toscore than essays.

356 CARROLL AND BRIGHT

Page 14: Integrative Complexity of Public Beliefs Toward Wildfire Management: Development of a Scale

Second, the scale has power in terms of its open-ended format, as opposedto other closed-item scale construction methods. By being open-ended, itavoids potential shortfalls that other scales experience. Fixed-item scales maylose relevance over time as important aspects of specific issues change. Theopen-ended format overcomes this lack of salience. For example, fixed-itemscales often represent researcher-generated items. The scale always yieldsrespondent-generated items, improving their salience. With regard to theopen-ended nature of the scale, it can be used on any number of topics withlittle alteration.

Third, the qualitative component of the scale allows for deeper contentanalysis of comments provided by respondents that traditional fixed-itemscales do not. Finally, the quantifiable nature of the scale, combined with theprevious benefits, can enhance its use in the construction of broader socialpsychological models that involve values, attitudes, beliefs, and behaviors.For example, what is the relationship between values, attitudes, and integra-tive complexity?

Future studies should aim to utilize the integrative complexity scale onthis or other issues and continue to correlate findings with traditional mea-surement methods. This scalar measure will allow the concept to be utilizedmore widely and easily in larger statistical analyses, ultimately uncoveringvaluable information about complexity of thought and the implications tonatural resource management. Combined with the ability to manipulatestakeholders’ integrative complexity, a natural resource manager can ulti-mately ensure that the sometimes complex information involved in a man-agement campaign will be received more readily and ultimately will be moreeffective.

References

Baker-Brown, G., Ballard, E., Bluck, S., DeVries, B., Suedfeld, P., &Tetlock, P. (1992). The conceptual/integrated complexity scoringmanual. In C. P. Smith (Ed.), Motivation and personality: Handbook ofthematic content analysis (pp. 401–418). New York: Cambridge Univer-sity Press.

Bright, A. D., & Barro, S. C. (2000). Integrative complexity and attitudes: Acase study of plant and wildlife species protection. Human Dimensions ofWildlife, 5, 30–47.

Bright, A. D., & Wyche, S. (1998, May). The effect of environmentally basedcoursework on perceptions of a natural resource issue. Proceedings of the7th International Symposium on Society and Resource Management,Columbia, MO.

INTEGRATIVE COMPLEXITY OF WILDFIRE MANAGEMENT 357

Page 15: Integrative Complexity of Public Beliefs Toward Wildfire Management: Development of a Scale

Burtz, R. T., & Bright, A. D. (2007). Integrative complexity and attitudestoward fire management in the wildland urban interface. Journal of Parkand Recreation Administration, 25(4), 99–116.

Carroll, J. & Bright, A. (2009). The Integrative Complexity of Wildfire Man-agement Scale: Are we there yet? Human Ecology Review, 16, 211–221.

Dillon, M. (1993). Argumentative complexity of the abortion discourse.Public Opinion Quarterly, 57, 305–314.

Fire policy and reports, programs, and priorities. (2008). Retrieved February 4,2008, from www.fs.fed.us/fire/management/index.html

Fulton, D., Manfredo, M. J., & Lipscomb, J. (1996). Wildlife value orienta-tions: A conceptual and measurement approach. Human Dimensions ofWildlife, 1, 24–47.

Hunsberger, B. E., Lea, J., Pancer, S. M., Pratt, M., & McKenzie, B. (1992).Making life complicated: Prompting the use of integrative complexitythinking. Journal of Personality, 60, 95–114.

Kneeshaw, K., Vaske, J. J., Bright, A. D., & Absher, J. D. (2004, June).Acceptability norms toward wildland fire management. Proceedings fromthe 10th International Symposium on Society and Resource Management,Keystone, CO.

Knotek, K. (2006). Understanding social influences on wilderness firestewardship decisions. International Journal of Wilderness, 12, 22–25.

Kristiansen, C. M., & Matheson, K. (1990). Value conflict, value justifica-tion, and attitudes toward nuclear weapons. Journal of Social Psychology,130, 665–675.

Pyne, S. J. (2001). Year of the fires: The story of the great fires of 1910. NewYork: Viking.

Schroder, H. M., Driver, M. J., & Streufert, S. (1967). Human informationprocessing. New York: Holt, Rinehart & Winston.

Tetlock, P. (1981). Personality and isolationism: Content analysis of senato-rial speeches. Journal of Personality and Social Psychology, 41, 737–743.

Tetlock, P. (1984). Cognitive style and political belief systems in the BritishHouse of Commons. Journal of Personality and Social Psychology, 46,365–375.

Tetlock, P. (1985). Integrative complexity of American and Soviet foreignpolicy rhetoric: A time-series analysis. Journal of Personality and SocialPsychology, 49, 1565–1585.

Tetlock, P. (1988). Monitoring the integrative complexity of American andSoviet policy rhetoric: What can be learned? Journal of Social Issues, 44,101–313.

Tetlock, P. (1989). Structure and function in political belief systems. In A. R.Pratkanis, S. J. Breckler, & A. G. Greenwald (Eds.), Attitude structure andfunction (pp. 129–151). Hillsdale, NJ: Lawrence Erlbaum.

358 CARROLL AND BRIGHT

Page 16: Integrative Complexity of Public Beliefs Toward Wildfire Management: Development of a Scale

Tetlock, P., Armor, D., & Peterson, R. S. (1994). Slavery debate in antebel-lum America: Cognitive style, value conflict, and compromise. Journal ofPersonality and Social Psychology, 66, 115–126.

Wallace, M. D., & Suedfeld, P. (1988). Leadership performance in crisis: Thelongevity–complexity link. International Studies Quarterly, 32, 439–451.

Wallbaum, A. B. C. (1993). Integrative complexity, international crises, andcognitive management. In K. S. Larsen (Ed.), Conflict and social psy-chology (pp. 34–44). London: Sage.

INTEGRATIVE COMPLEXITY OF WILDFIRE MANAGEMENT 359