Under Pressure to Evaluate your HHW Program?

Evaluation Concepts ina Tire Waste Program

Jennifer J. TabanicoCalifornia State University

Program Evaluation

The process of collecting, analyzing, and interpreting information about our intervention efforts that are aimed at some stated goal

Need Implementation EffectivenessEfficiency

Why We Don’t Evaluate

Not requiredComplicated statisticsLack of knowledge about research methodsExtra Cost“We’re doing something”

Why Evaluate?

Required by funding agenciesJustify program existencePreparing for budget cutsCan make changesUnintended consequencesCost efficiencyInform future efforts

Types of Evaluations

Summative EvaluationProgram goalsDid it work?

Formative EvaluationProgram processWhy worked/didn’t work

Setting Measurable Goals

Identify specific goalsWhat do you want to achieve?

• Specific behavior change• Public satisfaction• Waste reduction

Identify your measuresHow will you know if you reached your goals?

• Surveys• Observations• Waste

Stages of Evaluation

Needs assessmentWhat is happening, and what should be happening?

Program planningTheory based development of intervention

Implementation and ProcessIs program being implemented correctly?

Outcome analysisIs program causing the desired change?Is program worth the cost?

The Tire Waste Problem

31 million waste tires per year generated by California motorists (CIWMB, 2003)Proper maintenance increases life of tire and reduces wastePressure, Alignment/Balancing, Rotation, Tread

Overview of Intervention

Goal Increase frequency and accuracy of tire pressure checking behavior by motorists

ProgramMotorists at 10 gas stations 5 experimental, 5 controlMotivational messages + free pressure gauge at each pump at 5 stations

OutcomesSurvey of 381 motorists (self report & tire inspection)Air pressure machine usage

Needs AssessmentBaseline Survey Data

“Checking pressure” is the most under performed maintenance behavior

Only 27 % checked pressure monthly as recommended59% of vehicles on the roadways had at least one tire over/under inflated by 5+ psi

Motorists know they should check Don’t know correct PSI and ~50% don’t have gauge

Program PlanningApplying Theoretical Principles

Give AwaysNorm of reciprocation

Social NormsBeliefs about what other people think/do

MotivationSurvey data revealed saving money, extending tire life, and safety as motivations to act

Implementation and ProcessEvaluation of Implementation

7,565 tire gauges distributed at the pump at 5 gas stations

Dispensers refilled steadily for 4 weeks

Penetration RatesSelf report of gauge ownership at experimental vs. control stations

Do you own a tire pressure gauge?

Outcome AnalysisDid it work?

Experimental vs. Matched ControlsFollow up survey of motorists • Self reported behavior• Tire inspections

Air pressure machine usage• Unobtrusive electronic device

Self-Reported Behavior

Tire Inspections

Air Pump Usage

Conclusions

Evaluation provides information about need, implementation, effectiveness, and efficiency of program

Is it working?What is/isn’t working?

Can apply these principles to own programWorkshops, staff training on evaluation principles a good investment

References

California Integrated Waste Management Board. (2003a). Waste tire management program:2001 staff report. May 2003, Publication # 620-03-003 http://www.ciwmb.ca.gov/Publications/default.asp?pubid=1006.

California Integrated Waste Management Board (2003b). Consumers’ tire buying habits and their knowledge of tire maintenance, recycling, and disposal. November 2003,Publication # 622-03-004.

Cialdini, R. B. (2000). Influence: Science and practice (4th ed.). Boston: Allyn & Bacon.

National Highway Transportation Safety Administration. (2002). Tire pressure survey and test results. http://www.nhtsa.dot.gov/cars/rules/rulings/TirePressure

Rubber Manufacturers Association. (2002). Be tire smart: Play your PART. http://www.rma.org/tiresafety/tiremaintenanceandsafety.

Schultz, P. W. (2004). Community Based Social Market Pilot to Increase Proper Tire Maintenance. Final report submitted to the California Intergrated Management Board.

Schultz, P. W., & Oskamp, S. (2000). Social psychology: An applied perspective. Upper Saddle River, NJ: Prentice-Hall.