INFORMING CONTINGENT VALUATION METHODS

WITH INTERNET SURVEYS

Robert P. Berrens, Alok K. Bohara, Carol Silva, Hank Jenkins-Smith, Dave Weimer

Project funded by the Decision, Risk and Management Science Program, National Science Foundation

http://www.nsf.gov/cgi-bin/showaward?award=9818108

OVERVIEW• Contingent valuation surveys and passive

use benefits• Decreasing viability of telephone surveys?• Increasing viability of Internet surveys?• Study design: telephone and Internet splits

– Mental accounts (telephone and Internet) – Enhanced information (Internet)– Modified Kyoto Protocol (Internet)

• Preliminary Findings

GENERAL QUESTIONS

• Technological question: Can the Internet be used as an alternative to the telephone for administering contingent valuation (CV) surveys?

If yes, then dramatic revolution!

If no, then how far off?

GENERAL QUESTIONS, Continued

• Methodological questions: Whether or not the Internet is a survey replacement technology for the telephone, can the Internet be used as a low-cost tool for investigating methodological issues about CV?

– Role of greatly increased availability of information

– Role of mental account questions

GENERAL QUESTIONS, Continued

• Substantive questions:

– What is the willingness of the U.S. population to pay for ratification of the Kyoto Protocol?

– Would willingness to pay be substantially higher if developing countries had emission limits?

CONTINGENT VALUATION IS HERE TO STAY

• Desire to include passive-use benefits in cost-benefit analysis – Direct correspondence to option price as benefit measure

– "Structured conversation" when no "behavioral traces"

• State of Ohio v. Department of Interior (1989) – Passive-use values valid in CERCLA damage assessment

• Oil Pollution Act of 1990 – Increased use of CV in natural resource damage litigation

GROWING LEGITIMACY

• NOAA Panel (Arrow et al., 1993) – Qualified endorsement by blue ribbon panel

• Academic mainstream (environmental economists) – Since January 1990 the Journal of Environmental

Economics and Management and Land Economics have published 65 articles reporting on CV surveys (mean sample size is 730)

CV METHODOLOGICAL ISSUES

• Elicitation method – Dichotomous choice/policy referendum format preferred

• Bishop and Heberlein (1979)

• NOAA Panel recommends– Incentive compatibility (Carson, Groves, Machina, 1999)

• Consequential questions: potential effect, matters• Public good with coercive payments most likely incentive

compatible

• Payment vehicle – Non-commitment bias; reminder needed– "Top down disaggregation method" or mental accounts

MORE ISSUES

• Description of good – "Hypotheticality problems"

– Meaning and context must be understood

• Decision making – Embedding effects: responses not always scale

dependent

– "Warm glow“: broad moral attitudes rather than valuation

– Willingness-to-pay versus willingness-to-accept

DECREASING VIABILITY OF

TELEPHONE SURVEYS? • Random digit dialing as "probability sample" of households • Decreasing response rates, or increasing cost per response

– Hectic pace of life – Increased telemarketing, sometimes in guise of survey – Skepticism related to perceived biases

• Increasing complexity of telephone system – More lines per household – Lines dedicated to facsimile machines and computers – More cellular telephones (86 million subscribers in 1999) – More answering machines; more call screening – Breakdown of geographic area codes – More non-working numbers (Probability of reaching working number

fell from 21% in 1988 to 13% in 1998)

INCREASING VIABILITY OF INTERNET SURVEYS?

Spaming prohibitions prevent any analog to RDD, but ...

• Creation of proprietary panels of cooperative respondents (non-systematic recruitment)

– Harris Interactive panel of U.S. adults:• 4.4 million in January 2000

• 4.8 million in August 2000

• Over 6 million now, and growing

• Creation of proprietary panels of cooperative respondents (random recruitment)

– Knowledge Networks panel of U.S. households:• Approximately 100,000 households, growing to 250,000

• Provides fee equipment and Internet service; weekly surveys

INTERNET PENETRATION• Fraction of U.S. households with Internet access is

approaching 50%• Women and men now use at same rate

– Women: 17% in 1997; 49% in 2000

• Ethnic minorities converging– African-American 33%; Asian-American 69%;

Hispanic-Americans 47%; Caucasian-Americans 43%

• Age profile converging rapidly, but those over 55 still underrepresented

• Income converging, but …– Under $20K, 6% of Internet users, 19% of population– Over $150K, 8% of Internet users, 4% of population

POTENTIAL ADVANTAGES OF INTERNET SURVEYS

• Lower cost per respondent – Our telephone survey: about $50K for 1,700 respondents – Our Internet survey: about $40K for 13,000 respondents

• Lower cost makes larger sample sizes feasible – Within-versus across-study methodological comparisons – Greater sub-sample opportunities

• Eliminates interviewer bias

• Identification of rare respondents (medical research)

• Speed (likely application in polling – Harris did well in 2000 election forecasts)

• Opportunity to provide more and different information than possible in telephone surveys

INTERNET SKEPTICS

"... The panel used as a frame to sample respondents is not a probability sample of internet users. Even if it were, internet users are not like non-internet users. For instance, they are better educated, more affluent, and more male. The important characteristics for weighting have not yet been identified. Generalizations from a census of the growing millions in the panel do not even represent internet users. The whole panel represents nothing but itself ... I can see no valid survey purpose to the current internet enterprise. All that will happen will be the accumulation of thousands upon thousands of interviews of dubious merit that will mislead the public and destroy whatever credibility surveys and polls now have. A growing number of survey researchers are unfortunately being led to the rocks like Ulysses’ sailors following the Siren call of cheap, but worthless, data." (pp 25-26)

Warren J. Mitofsky, "Pollsters.com," Public Perspectives, June/July 1999, 24-26.

DOES WEIGHTING OFFER HOPE? • Exxon Valdez CV (Carson et al., 1993)

– Gold-plated, "litigation quality" survey: – Relatively large sample: 1,043– Cost approximately $3 million – Estimated damage from Exxon Valdez spill: $2.128 billion

• "Replication" (Harrison and Lesley, 1996) – Convenience sample: – North and South Carolina Students– Modest sample size: 449– Simplified questionnaire format– Cost: approximately $2,500 plus some beers – Estimated $4.044 billion in damages with raw sample data– Estimated $2.807 billion in damages with demographic

weighting that applies population means to coefficients estimated in willingness-to-pay function

OUR STUDY

• Telephone survey – Institute for Public Policy, University of New Mexico,

January 2000: 1,699 completed surveys

• Internet survey (planned)– Harris Interactive, Rochester, January 2000: 13,034

completed surveys

• Internet survey (subsequent)– Harris Interactive, August 2000: 11,160 completed surveys– Knowledge Networks, Stanford, November 2000: 1,500

expected

• To “take” survey– http://www.unm.edu/instpp/gcc/

EXPERIMENTAL DESIGN

TELEPHONE SURVEY

INTERNET SURVEY (Harris Interactive)

Basic Information Basic Information

Basic Kyoto

Basic Kyoto Modified

Kyoto

834 Standard Reminder 2629

(1507) 2062

(1316)

865 Mental Accounts 1386

(1496) 1011

(1265)

Enhanced Information

Standard Reminder 892

(1504) 1192

(1278)

Mental Accounts 1694

(1466) 2168

(1328)

SURVEY OVERVIEW• Demographics (education and sex)

• Environmental attitudes

• Global climate change knowledge

• Split 1

• [Basic reminder/Mental accounts]

• Split 2 (Internet only)

• [Basic information/Enhanced information]

• Kyoto Protocol background

• Split 3 (Internet only)

• [Basic Kyoto referendum/Modified Kyoto referendum]

• Attitudes toward appropriateness of willingness-to-pay

• Perceived efficacy of Kyoto Protocol

• Balance and usefulness of enhanced information (as relevant)

• Presidential candidate questions

• Respondent demographics, politics, and computer use

BASIC KYOTO PROTOCOL QUESTION

Seven preparatory questions to describe Protocol, then:

The US Senate has not yet voted on whether to ratify the Kyoto Protocol. If the US does not ratify the treaty, it is very unlikely that the Protocol can be successfully implemented.

Suppose that a national vote or referendum were held today in which US residents could vote to advise their Senators whether to support or oppose ratifying the Kyoto Protocol. If US compliance with the treaty would cost your household X dollars per year in increased energy and gasoline prices, would you vote for or against having your Senators support ratification of the Kyoto Protocol? Keep in mind that the X dollars spent on increased energy and gasoline prices could not be spent on other things, such as other household expenses, charities, groceries, or car payments.

X is randomly chosen from: 6 12 25 75 150 225 300 500 700 900

(1200 and 1800 added after telephone pretest)

MODIFIED KYOTO PROTOCOL QUESTION

(Same lead-up questions)

The US Senate has not yet voted on whether to ratify the Kyoto Protocol. If the US does not ratify the treaty, it is very unlikely that the Protocol can be successfully implemented.

An alternative to the Kyoto Protocol, which we will refer to as the Modified Kyoto Protocol, would make only one change in the agreement: It would require that developing countries, such as China, India, Mexico, Brazil, and Argentina, promise to restrict their future production of greenhouse gases to no more than 5 percent above current levels.

MENTAL ACCOUNTS TREATMENT

• Standard reminder as part of willingness-to-pay question

• Mental accounts approach – Questions to encourage budget consideration– Ask prior to valuation question– May be useful in estimating willingness-to-pay

models

MENTAL ACCOUNTS QUESTIONS First level compartment:

Now think about your average monthly income and expenses.

After you have paid all the necessary bills for such things as housing, transportation, groceries, insurance, debt, and taxes, what percent of your income is left over for optional uses on things like recreation, savings, and giving for charity and other causes?

Second level compartment:

Now think about the portion of your total income available for optional uses. On average, what percent of that amount do you use for contributions to environmental causes, such as donations for specific programs or contributions and memberships to environmental advocacy groups?

Note: Please remember that we are not asking for money for anything, and that

your answers will be kept completely confidential.

ENCHANCE INFORMATION

• Information menu– Open through referendum question and follow-

ups– Lists 27 entries– Each entry contains one page of information

• Follow-up questions– Visits and time spent– Perceived usefulness and bias

INVITATION TO MENU Thank you for your answers to the previous section. Next we would like your views on a particular environmental treaty, the Kyoto Protocol. The Kyoto Protocol is concerned with addressing the issue of possible human causes of global climate change.

Before proceeding to the questions on the Kyoto Protocol, you are invited to explore a brief information menu we have made available.

The items in the information menu are intended to provide brief and balanced summaries of research and informed opinion about global climate change, and the Kyoto Protocol. Because the volume of information on these matters is enormous and growing rapidly, we sought to make the items in the information menu broadly representative and readily understood, rather than exhaustive.

Please feel free to visit as many pages as you wish as you proceed through the survey questions.

INFORMATION MENU Global Warming Theory• Basic "Greenhouse" Mechanism • Greenhouse Gases • Carbon Dioxide Changes• Feedback Mechanisms• Feedbacks Related to Carbon Dioxide and Methane• Feedbacks Related to Water Vapor, Clouds, and Ice Cover• Feedbacks Related to Ocean Currents• Aerosol Offsets• Temperature Predictions under Current Policy

Evidence of Global Warming• Ground-Based Measurements• Atmospheric Measurements

INFORMATION MENU, Continued

Consequences of Global Warming

• Sea Level Changes

• Ecological Effects

• Agricultural Effects

• Health Effects

• Weather Extremes

Alternative Theories and Explanations for Apparent Warming

• Solar Cycles

• Measurement Issues

• Modeling Limitations

INFORMATION MENU, Continued

Kyoto Protocol on Climate Change

• Background

• Country Commitments

• Flexibility Mechanisms

• Predicted Impacts under Full Implementation

Ratification Status, Political Views, and Public Opinion

• Ratification Status

• Views in Opposition to Ratification of the Kyoto Protocol

• Views in Favor of Ratification of the Kyoto Protocol

• Selected Reports from Public Opinion Surveys

Very Preliminary Findings

• Demographics

• Knowledge about effects and causes of global climate change

• Relationships across modes

• Use and perception of bias in information

DemographicsTelephone

January 2000(Semi-Weighted)

Internet

January 2000(Weighted)

Internet

August 2000(Weighted)

Mean Age (years) 41.8 43.7 44.4

Percent Male 47.1 48.0 48.0

Percent Some College or More

66.2 48.0 49.5

Percent Hispanic 7.5 10.0 10.2

Percent African-American

7.3 12.4 11.5

Mean Annual Household Income

($)

58,404 45,064 52,189

Knowledge: Telephone Internet 1 Internet 2 

E: Temperature rises (%Y) 

89.9 

90.2 

90.7 

E: Ocean levels to fall (%N) 

51.8 

49.4 

51.0 

E: More droughts (%Y) 

76.1 

77.7 

82.5 

E: Fewer floods (%N) 

68.7 

66.6 

66.7 

E: More storms (%Y) 

85.7 

85.1 

85.7 

C: Exhaust (%Y) 

87.2 

91.4 

91.7 

C: Nuclear (%N) 

32.2 

33.9 

39.3 

C: Toxics (%N) 

31.6 

43.3 

38.4 

C: Coal (%Y) 

53.2 

63.0 

68.1 

C: Forest loss (%Y) 

83.5 

90.0 

90.1

Relationship between Ideology and Belief about Environmental Threat

(0 = no real threat; 10 = brink of collapse)

N=23,273

R2=.11

Coefficient Standard Error P-Value

Ideology

(1= SL; 7=SC)

-.34 .04 .000

Ideology* Internet 1

-.15 .06 .015

Ideology*

Internet 2

-.15 .05 .004

Internet 1 .67 .26 .010

Internet 2 .55 .24 .020

Constant 7.12 .18 .000

Relationship between Ideology and Belief about Environmental Threat Controlling with Demographic

Covariates

N=19,292

R2=.13

Coefficient Standard Error

P-Value

Ideology -.33 .05 .000Ideology* Internet 1 -.08 .06 .157Ideology*Internet 2 -.10 .06 .089

Internet 1 .37 .25 .132

Internet 2 .28 .26 .282

Five Demographic Controls:

Income (-), Age, Sex (M-), College,African-American

------ ----- -----

Constant 7.70 .24 .000

Use and Perception of Enhanced Information

• How many pages did you visit?– Mean: 6.3 (Range 0 to 27)

• How much time did you spend?– Mean: 9.1 minutes (12.3 for visitors)

• How useful did you find the information?– Mean: 5.3 (0=not at all; 10=extremely useful)

• How balanced did you find the information?– Mean: 5.3 (0=biased against GCC; 10= for)– Tri-modal distribution: 54% at 5; 10% at 0, 7%

at 10