Measuring the ROI of Marketing Communications

A Presentation to the Corporate Communications SummitBy Katie Delahaye PaineCEO KDPaine & Partners, LLCJanuary 21, 2005

Results = ROIHard

NumbersCharts &

Graphs

Work = Reviewing

resultsLooking at

spreadsheetsDownsizing

1992 1993 19940

0.2

0.4

0.6

0.8

% of A rticles Comm Key M ess

The World According to Martians

Work = SchmoozingLunchingOpportunistic creativity

The World According to Venutians

Results =A busy trade

showAn award

A front page headline

Let Measurement be your dictionary

What we say

We got great results

What Martians hear

Blah, blah, blah

Lotus Press Coverage Analysis

Percent of impressions containing messages

by product

0% 20% 40% 60% 80% 100%

TAC

Manuscript

One Source

HAL

Positive Messages No Messages Negative Messages

Lotus Press Coverage Analysis

$0.00 $0.50 $1.00 $1.50 $2.00 $2.50

TAC

Manuscript

One Source

HAL

Cost per message communicated

Why measure ROI?

Gut feeling doesn’t cut it anymoreAccountability is criticalIf you think measurement is expensive, what’s the

cost of ignorance?Without data, how do you know what’s working?

• You can’t measure intangibles• Measurement will show that my program isn't

working• Measurement should be done either at the

start or the end of a program • Measurement is expensive

The Myths of Measurement

How to lose a budget in 10 days

1. Don’t tie results to business outcomes2. Use m7easurement to justify your existence3. Measure something that no one cares about4. Get lost in the minutiae, lose sight of the goal5. Fail to get consensus from everyone who needs the

results6. Promise a Cadillac Measurement plan on a Segway

budget7. Deliver data when you no longer need the answers

A little Measurement history

1987 1998 2000 2002 2003 2004

Automated Messagetracking

Do-it-yourselfTools

IntegratedAutomated

tools

Avg cost$30 /clip

Avg cost$20 /clip

Avg cost$10 /clip

Avg cost<$5 /clip

Avg cost$8/clip

Automated analysis

introduced

On-line analysis

24/7Access to data

The Cost of Measurement

Delahaye foundedReaders

From targetaudience

Surveys @ $20/complete

Industry standards

Standards and guidelines are available on www.instituteforpr.com Clip counts and column inches are not ROIAVEs are tabooAnalysis of messaging, positioning, issue identification are

the normCompetitive analysis is mandatoryAnalyst and quote measurement is the latest trendIntegrating media analysis with web activity, customer

outcomes is growing

7 Simple Steps to ROI Measurement

Step 1: Define the audienceStep 2: Define your measures of successStep 3: Determine the specific criteriaStep 4: Define a benchmarkStep 5: Select a measurement toolStep 6: Analyze the resultsStep 7: Take action, measure again

Step 1: Who’s behavior are you trying to influence

MVP/C

Employees

Alumni

Faculty

Potential students

Students

Community leaders

Local /State gov’t

Reduce complaints?

Increase awareness or

preference

Increase sales?

Increase loyalty?

Reduce turnover?

Lower recruitment costs?.

How does a good relationship with each audience benefit your organization?

Step 1: Define the audiencesStep 2: Define your measures of success

7 Simple Steps

Step 2: Define your measures of success

If you are celebrating complete 100% success a year from now, what is different about the organization?What about 5 years from now? If you eliminated the MarCom Department what

would be different?

7 Simple Steps to ROI Measurement

Step 1: Define the audienceStep 2: Define your measures of successStep 3: Determine the specific criteria

Step 3: What do you need to measure?

Outputs?Did you get the coverage you wanted?

Did you produce the leads you needed?

Outtakes?Did your target audience see the messages?

Did they believe the messages?

Did your relationship change?

Outcomes?Did audience behavior change?

Did the right people show up

Did sales increase?

Step 3: Defining Your Dashboard

Those numbers that most affect the

business

Increase in loyalty

Increase in purchase intent

Share of brand positioning

Most frequently used criteria

% change in awareness % change in preference% change in purchase intent% increase in prospects or new contactsCost per message communicatedCPMCost per minute spent with prospectStrength of relationshipsCost per % of target population reachedShare of recommendations (positive/negative) exposureShare of visibilityShare of quotesShare of brand benefits mentioned

0

50

100

150

200

250

300

350

400

Nov 20 - 26 Nov 27 - Dec3

Dec 4 - 10 Dec 11 - 17 Dec 18 - 24

The Pentium Flaw

Mea Culpa = Own The Problem

IBM threatens to stop using Pentium chip

Reverses decision

0

10

20

30

40

50

60

70

11/3 - 11/10 11/11- 11/18 11/19 - 11/26 11/27 - 12/4 12/5 - 12/12

Actions Speak Louder Than Words

Story broke, and the news of Levi's $200 million employee benefits package and $8million grant to local communities affected by closings showed that Levi's cares for its workers

Levi Strauss closes 11 plants and lays off 6,395

The News Was The Same, The Actions Were Different

Job cuts expected

8,000-14,000 layoffs announcedAnother

6,000 layoffs announced

Kodak’s layoffs

0

20

40

60

80

100

120

140

160

9/16 -

9/23

10/2

- 10/9

10/18

- 10/2

5

11/2

- 11/9

11/18

- 11/2

5

12/2

- 12/9

12/18

- 12/2

5

1/2 - 1

/9

7 Simple Steps to ROI Measurement

Step 1: Define the audienceStep 2: Define your measures of successStep 3: Determine the specific criteriaStep 4: Define a benchmark

Benchmark Guidelines

Stretch goalPeer companyThe underdog who's nipping-at-your-heels\Whatever keeps the Martians up at night

7 Simple Steps to ROI Measurement

Step 1: Define the audienceStep 2: Define your measures of successStep 3: Determine the specific criteriaStep 4: Define a benchmarkStep 5: Select a measurement tool

Step 5: Selecting a measurement tool

Objective Metric Tool

Increase inquiries, web traffic, recruitment

% increase in traffic#s of clickthrus or downloads

Clicktrax, Web trends

Preference % of audience preferring your brand to the competition

Survey Monkey, Zoomerang

Awareness % awareness of your productCost per Impression

Survey Monkey, Zoomerang

Communicate messages % of articles containing key messagesTotal opportunities to see key messagesCost per opportunity to see key messages

Media content analysis –

% aware of or believing in key message

Survey

Step 5: Deciding what’s best

Tool Strengths Limitations

Automated content analysis

Can analyze large volumes of articles very quickly to determine share of discussion, share of visibility and share of positioningVery fast, very efficient

Doesn’t pull out influencers and spokespeople wellDoesn’t determine toneCan’t determine subtle or complex messagesMany foreign publications are not available on line

Manual Content Analysis

Slow, cumbersomeReaders can be biased or have bad daysSeldom very timely

Excellent for complex messaging, tonality, subtle differences

On-line survey Easy to programFastInexpensiveSelf selecting audience

Most are English onlyConvenience sample (only those who have email addresses)

Paper Survey Slow More time to code and analyzeSelf selecting audience

Better sampling reaches everyone

Phone survey High response ratesFast

More expensive

Step 5: Pick a tool: The latest

Clipping servicesCustomScoop, Cyberalert, e-Watch, Factiva, Dialog, Nexis Computerized Content AnalysisBiz 360, CymfonyInternational Analysis Still mostly manualAutomated survey tools

Survey Monkey, ZoomerangWeb measurement

ClickTrax, Web trendsIntegrated Knowledge Management ToolsPerforma, Vocus

7 Simple Steps to ROI Measurement

Step 1: Define the audienceStep 2: Define your measures of successStep 3: Determine the specific criteriaStep 4: Define a benchmarkStep 5: Select a measurement toolStep 6: Analyze the results

Step 6: Analyze and glean insight

Research without insight is just triviaFigure out what works and what doesn’t workDetermine what you need to do NOWDetermine what you need to do nextMake sure your recommendations are actionable

Ask for money Get Commitment Manage Timing Influence decisions Get Outside helpJust Say No

Actionable Conclusions

What to do with the data once you get it

Share of Favorable Positioning on Key Issues

1208790

2062862

14535371

16933531

3651116

3023150

44343231

8734582

576267

1833027

1925245

3888240

10333302

875521

1489424

19519211

1800607

3197671

1569624

1925246

8767436

7122384

2595682

4284393

27822137

4026608

4658824

946118

539563

9375974

14532507

3217239

2211107

29637508

3

743974

2913607

965633

15066319

20000868

6071014

6367454

64325730

4875552

6647320

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Corporate Responsibility (+)

Employer of Choice (+)

Innovation (+)

Investment of Choice (+)

Partner of Choice (+)

Saves Money (+)

Global Strength/Reach (+)

Simplify (+)

EnvironmentallyResponsible(+)

Apple

Dell

HP

IBM

Sony

7 Simple Steps to ROI Measurement

Step 1: Define the audienceStep 2: Define your measures of successStep 3: Determine the specific criteriaStep 4: Define a benchmarkStep 5: Select a measurement toolStep 6: Analyze the resultsStep 7: Take action, measure again

Step 7: Do it again

Regular research is far more valuable than one-shotsMake sure your data is ready when you need it

7 ways to do research without a budget

1. Become someone’s research project2. Involve your board of directors and volunteers3. Research something that HAS a budget4. Take advantage of free offers5. Become a case study6. Team up with peer organizations7. Analyze data that already exists

Case Study: Rensselaer County

Goal: More favorable attitudes, better imageBenchmark: Saratoga CountyMeasurement Budget: $7500Tools used: IPR Guidelines, phone survey, Excel, Publisher

Case Study: Rensselaer County

Output Measures: 1. Total opportunities to see key messages2. Share of positives vs. negatives3. Share of visibility4. Share of positioning

Outtake measures1. More favorable attitudes2. Greater likelihood to approve

Outcome measures1. Economic growth2. Projects approved

Opinions became much more positive

How would you describe your overall view toward Rensselaer County?

47.0%

41.0%

8.0%

3.0%

1.0%

53.5%

34.7%

9.0%

3.0%

0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0%

Very Positive

Somewhat Positive

Neutral

Somewhat Negative

Very Negative

2004

2003

More people thought of the county as an excellent place to locate a business

Do you agree or disagree with the statement that Rensselaer County is an excellent place to locate a business?

44%

47%

8%

1%

0%

56%

34%

5%

1%

4%

0% 10% 20% 30% 40% 50% 60%

Strongly Agree

Moderately Disagree

Unsure

2004

2003

Strong support increased

Do you generally support or oppose economic development in Rensselaer?

52%

39%

4%

1%

4%

54%

38%

4%

1%

4%

0% 10% 20% 30% 40% 50% 60%

Strongly Support

Moderately Support

Moderately Oppose

Strongly Oppose

Unsure

2004

2003

Case Study: Central Arizona Project

Goal: Measure relationships with target audiences – media, elected officials, customers Survey instrument: IPR GuidelinesTool: Survey MonkeyBudget: $1000

Date: February 12, 2004

Case Study: Southwest Airlines

Southwest. COMVamonos release generated $38,000

in ticket sales

Southwest Airlines made four big announcements on July 15, 2004

Release about 22 new daily flights generated $1 million in ticket sales

Southwest’s PR has generated over $1.5 million in ticket sales

Service to Philadelphia

Began with 14 daily flights to 6 citiesExpanded to 28 daily flights to 14 citiesExpanding again to 41 daily flights to 17 cities

Revenue passenger miles for the eight months ended August 31, 2004, increased 11% to 36.3 billion

Source: Southwest Airlines

Predicted vs. Actual Loyalty(Based n GRP/Loyalty model)

0

10

20

30

40

50

Predicted Actual Unfavorable

Peak in unfavorable news coverage, e.g. pricing plan

Predicted vs. Actual Attitudes(Based n GRP/Loyalty model)

0

10

20

30

40

50

60

Predicted Actual Favorable

*Print and broadcast stories X circulation/viewers

Peak in favorable news coverage;President’s Conferenceon Volunteerism

Customer Sciences

Illustrative AT&T data, 1997.

Correlation between LD acquisitions and Positive Price/Value news impressions

Volume Acquisitions

Positive Price/Value Impressions

120000000

100000000

80000000

60000000

40000000

20000000

0

-20000000

600000

500000

400000

300000

200000

100000

Observed

Linear

R = .33

Customer Sciences

Changing a state’s reputation

NH’s First in the Nation Primary attracts nearly 4000 journalists to the stateNH’s image in the past has been:

“persnickety,” “flannel shirted yokels,” “worst economy in the country,” “last without Martin Luther King Day,” The campaign to “influence the influencers”

Specific endorsements increased

0

2,000,000

4,000,000

6,000,000

8,000,000

10,000,000

12,000,000

14,000,000

16,000,000

Negativerecommendation

Positiverecommendation

1995 10 - 1996 04 (Oct-Apr)

1999 11 - 2000 02 (Nov-Feb)

Negatives were reduced, positives increased

0%

10%

20%

30%

40%

50%

60%

70%

80%

Negative Neutral Positive

% of Media Stories about NH by "Tone"

1992

1996

2000

Key Messages Increased as Well

1,991,458

18,089,729

0

5,000,000

10,000,000

15,000,000

20,000,000

1996 2000

Potential readers exposed to the message that NH deserves its status as "First in the Nation"

Positioning on key issues improved as well

Top Media Impressions (000's)

0 2,000 4,000 6,000 8,000 10,000

12,000

14,000

16,000

Bitter cold

Informed Electorate

Serious Electorate

Politically Atypical

1996 2000

The financial impact of media coverage

220 million media impressions

22 million positive impressions (10%

positive impressions)

660,000 non-primary visits

(3% of 22 million)

$33 million in potential tourismrevenue

The impact of media coverage on business development

660,000 visits to NH (3% of 22 million)

13,200 corporate decision- makers and entrep. (2% of 660,000 - actual US figure is 11%)

132 new businesswith avg. 20 emp. per business= 2,640 employees

@ $10,000 subsidy/business development value per emp.= $26,400,000 132 new businesses

(1% of 13,200 or .0002 of all visits)

Thank You!

For more information on measurement, subscribe to The Measurement Standard, www.themeasurementstandard.comTo start developing your own dashboard or for a copy of

this presentation go to: http://www.measuresofsuccess.comOr call me at 1-603-868-1550Or give us your business card and we'll be happy to

send it to you