15

Prof. (Dr.) Kao Kveng Hong, PhD. , D. Litt. 15-1

Chapter Fifteen

Evaluating an IntegratedMarketing Program

Pretesting for Effectiveness

• Rocket analogy

• Decision Analyst• http://www.decisionanalyst.com

• CopyScreen

• CopyCheck

• What are the pros and cons of testing ads and marketing communication pieces at various stages of development?

15

15-2

Chapter Overview

• Matching methods with objectives• Message evaluations• Evaluation criteria• Behavioral evaluations• Evaluating public relations• Evaluating the IMC program

Evaluating an IntegratedMarketing Program15

15-3

Evaluation Categories• Message evaluation techniques

• Physical design• Cognitive elements• Affective elements

• Respondent behavior evaluations• Conative elements• Measurable with numbers• Customer actions

15-4

Evaluation and IMC Objectives• Match objectives• Pre- and posttest analysis• Levels of analyses

• Short-term• Long-term• Product-specific• Corporate level• Affective, cognitive, or conative

15-5

F I G U R E 1 5 . 1Message Evaluation Techniques and When to Use Them

• Concept testing• Copytesting• Recall tests• Recognition tests• Attitude and opinion tests• Emotional reaction tests• Physiological tests• Persuasion analysis

Prior to ad development Final stages, or finished adPrimarily after ad has been launched After ad has been launchedAnytime during or after ad development Anytime during or after ad development Anytime during or after ad development After ad has been launched

Message Evaluation Method When the Test Is Normally Used

15-6

Concept Testing• Prior to ad development• Average cost 30-second TV ad $350,000• Focus groups• Concept testing instruments

• Comprehension tests• Reaction tests

15-7

AFLACConcept Testing

Before launching the AFLAC duck advertising campaign, the agency conducted concept tests to determine which idea was the best.

15-8

Click here to play clip from AFLAC

Copytesting• Used when marketing piece is finished

or in final stages• Methods used

• Portfolio test• Theater test• Focus groups• Mall intercept

15-9

Copytesting

Copytesting can be used to determine if viewers comprehend this ad and what their reaction to it is.

15-10

Copytesting• Criticisms of copytesting

• Some agencies not using• Stifles creativity• Focus groups not good judge

• Support of copytesting• Issue of accountability• Majority support because clients want

support for ad decision

Copyright © 2010 by Pearson Education, Inc. publishing as Prentice Hall

15-11

Recall Tests

• Day-after recall (DAR)• Unaided recall• Aided recall• Incorrect answers• Use primarily after ads launched

15-12

Items Tested for Recall

• Product name or brand• Firm name• Company location• Theme music• Spokesperson• Tagline• Incentive being offered• Product attributes• Primary selling point of communication piece

F I G U R E 1 5 . 3

15-13

Recall Tests: Do Viewers Remember?

15-14

25.0%

20.0%

15.0%

10.0%

5.0%

0.0%

Brand name

Theme music

Spokesperson

Tagline

Incentive

Product Attribute

Test Ad

Competitor A

Competitor B

Sample DAR Test

15-15

30-Second TV Advertisement for Pet Food

Sample DAR Test30-Second TV Advertisement for Pet Food

9.4%

16.3%18.3%

24.6%

8.5%

14.6%

21.4%

0.0%

5.0%

10.0%

15.0%

20.0%

25.0%

Males

Females

Pet Dog Ages 18-

Ages 36-

Ages 51+

Overall Recall 12.9%

Owners

Owners

35 50

15-16

Recall DecayMagazine Ad vs. Television Ad

Source: Magazines Canada’s Research Archive

100% 100%

86%

65%75%

43%

40%

20%

0%

60%

80%

100%

DAR

Two days later

MagazineTelevision

Eight days later

15-17

Recall TestsFactors That Influence Scores

• Attitude towards advertising

• Prominence of brand name• Brand used by respondent• Institutional ads

• Respondent’s age15-18

Recognition Tests• Respondents shown marketing piece• Often used with recall tests• Good for measuring

• Reaction• Comprehension• Likeability

15-19

Recognition Tests• Expression of person’s interest

• Ad liked + 75%• Ad interesting + 50%• Brand used + 50%

• Affected by ad size, color, length• Scores do not decline over time

15-20

Recognition Tests

Can be used to measure• Reaction• Comprehension• Likeability

15-21

Attitude and Opinion Tests• Used in conjunction with other tests

• Recall tests• Recognition tests

• Closed-ended questions• Open-ended questions• Roper Start ADD + IMPACT

15-22

Emotional Reaction Tests• Affective advertisements• Used for material designed to solicit emotions• Difficult to measure emotions with questions• Warmth monitor• Emotional reaction tests are self-reported

instruments

15-23

Sample Graph from a Warmth Meter30-Second TV Advertisement

S am p le A d R atin gW arm th Meter

S ta r t 1 0 s e c o n d s 2 0 s e c o n d s 3 0 s e c o n d s

Ad section that elicited negative emotionsT a rg e t Au d ie n ce

T o ta l Au d ie n ce

15-24

Physiological Arousal Tests

• Measure fluctuations in a person’s body• Psychogalvanometer – sweat• Pupillometric test – pupils of eyes• Psychophysiology – brain waves and

currents• Cannot be faked easily

15-25

Persuasion Analysis• Appraise persuasiveness of marketing item• Requires pre- and posttests• ASI Market Research studies

15-26

Copytesting Principles of PACT

• Testing procedure should be relevant to objectives.• Researchers should agree on how the results will be used

in advance.• Multiple measures should be used.• The test should be based on some model or theory of

human response to communication.• Testing procedure should allow for more than one exposure.• In selecting alternate ads to include in the test, they should

be atthe same stage in the process as the test ad.

• The test should provide controls to avoid biases.• Sample used for the test should be representative of the

targetsample.

• Testing procedure should demonstrate reliability and validity.

Source: Based on PACT document published in the Journal of Marketing, (1982) ,Vol. 11, No. 4, pp. 4-29.

15-27

F I G U R E 1 5 . 6

Behavioral Measures

• Sales• Response rates• Redemption rates• Test markets• Purchase simulation tests

15-28

F I G U R E 1 5 . 7

Sales and Response Rates

• Measuring sales with UPC codes• Scanner data

• Retailers• Manufacturers

• Sales changes can be caused by other factors

15-29

Difficulties in EvaluatingAdvertising

•••••

Influence of other factors on behavior Delayed impact of advertisingConsumers change their mind in the storeWhether brand is in evoked setGoal of ad may be to build brand equity, not increase sales

15-30

• Changes in sales• Telephone inquiries• Response cards• Internet inquiries• Direct marketing responses• Redemption rate of sales promotion offers

• Coupons, premiums, contests, sweepstakes

F I G U R E 1 5 . 9

15-31

Responses to Marketing Messages That Can Be Tracked

F I G U R E 1 5 . 10Methods of Measuring Interactive Marketing

15-32

60.0%

50.0%

40.0%

30.0%

20.0%

10.0%

0.0%

Brand a

waren

ess

Clickth

roug

hs

Customer

know

ledge

Sales

Lead g

enerat

ion

Leng

th of

enga

gemen

tMarg

in

Impres

sions

Redempti

on ra

tes

Respons

e rate

sROI

Source: Adapted from Larry Jaffee, “Follow the Money,” Promo, Vol. 20, No. 11 (November 2007 Sourcebook), pp. 5-10.

51.0%

36.5% 44.5%41.1% 40.7%34.2%

26.6% 24.7%

12.2% 16.3%

4.6%

Online Metrics• Adknowledge

• MarketMatch Planner• Campaign Manager• Administrative Campaign Manager

• Audience demographics• MediaMetrix – basic demographics• NetRatings – GRP and other rating instruments• SRI Consulting – Psychographic information• NetGuide – Web site ratings and descriptives• BPA Interactive – Web traffic audit data

15-33

Test Markets• Used to assess:

• Advertisements• Consumer and trade promotions• Pricing tactics• New products

• Cost-effective method of evaluation prior to launch• Resembles actual situation• Design test market to model full marketing plan• Length of test market is a concern• Competitive actions must be considered

15-34

Purchase Simulation Tests

• Bias in purchase intention questions• Simulated purchase tests• Research Systems Corporation• Does not rely on opinions and attitudes

15-35

Evaluating Public Relations

• Number of clippings• Number of impressions• Advertising equivalence technique• Comparison to public relations objectives

15-36

F I G U R E 1 5 . 11

Evaluating the IMC Program

• Greater demand foraccountability

• ROI of advertising andmarketing

• Difficult to measure ROI – 70%• Difficult to define ROI – 70%

15-37

Source: Paul J. Cough, “Study: Marketers Struggle to Measure Effectiveness,” Shoot, Vol. 45, No. 29 (August 20, 2004), pp. 7-8.

T A B L E 1 5 . 2

15-38

Definitions of ROI for MarketingDefinition of ROI Percent Using

Incremental sales from marketing 66%

Changes in brand awareness 57%

Total sales revenue from marketing 55%

Changes in purchase intentions 55%

Changes in market share 49%

Ratio of advertising costs to sales 34%

Reach/frequency achieved 30%

Gross rating points delivered 25%

Post-buy analysis comparing the media plan to its delivery 21%

Measures of Overall Health of a Company

• Market share• Level of innovation• Productivity• Physical and financial resources• Profitability• Manager performance and development• Employee performance and attitudes• Social responsibility

15-39

Source: Pete Drucker, Management: Tools, Responsibilities, Practices, New York: Harper and Row, 1974.

F I G U R E 1 5 . 12

International Implications• Assessment of IMC Programs

• Domestic results• Results in other countries• Overall organization

• Individual ads and promotional programs• Local culture• Across national boundaries• Multinational – regional offices

15-40