Chapter EightChapter EightExperimentation in Marketing

Research

Chapter ObjectivesChapter Objectives

Explain the difference between descriptive and experimental research

Identify the three conditions indicating that one variable has a causal influence on another

Discuss the most appropriate applications of laboratory and field experiments

List and explain the threats to internal validity and external validity of experimental results

8 | 2

Chapter Objectives Chapter Objectives (Cont’d)(Cont’d)

Explain the difference between pre- experimental and true experimental designs

8 | 3

Eddie Bauer's Electronic WindowsEddie Bauer's Electronic Windows

Eddie Bauer, a leading tri-channel specialty retailer was looking for a way to draw more shoppers into their stores

Indiana University students conducted an in-store advertising experiment using electronic window posters (images displayed on plasma screens) on 3 selected stores

8 | 4

Electronic Windows Electronic Windows (Cont’d)(Cont’d)

RESULTS◦The number of passersby who entered the

control stores went up 7 percent◦Sales soared 56 percent compared to the

weeks before the installation of digital windows

8 | 5

ExperimentExperiment

An experiment is a procedure in which a company manipulates one (or sometimes more than one) independent or cause variable and collects data on the dependent or effect variable while controlling for other variables that may influence the dependent variable

8 | 6

8 | 7

Advertising Experiment Will replacing commercial A with commercial B lead to a marked increase in consumer preference for a company’s brand?

Pricing Experiment Can a company improve the profitability of its fashion clothing line by increasing its price by 10 percent?

Sales Productivity Experiment Will an increase in the average number of sales calls per customer from six to eight per year significantly improve sales?

Shelf Space Experiment Will decreasing the shelf space allocated to brand X detergent by 25 percent significantly lower its sales?

Direct Mail Experiment Will it be worthwhile to mail last year's donors an attractive (but expensive) brochure describing the company’s activities and soliciting higher contributions for this year?

Descriptive ResearchDescriptive Research

This research asks consumers whether they would buy more of a product if its price were lowered

Descriptive survey data will merely suggest causation

8 | 8

Experimental ResearchExperimental Research

Manipulates the independent variable or variables before measuring the effect on the dependent variable◦The effect of price changes on sales volume of

a particular product can be examined by actually varying the price of the product

The very basis of experimental research lies in the manipulation of independent variables

8 | 9

Conditions For Inferring CausalityConditions For Inferring Causality

Temporal ordering of variables ◦X Y not Y X

Evidence of association◦X and Y are related ; presence of X presence

of Y; absence of X absence of Y Control of other causal factors

◦X Y, Z Y

8 | 10

Laboratory vs. Field ExperimentsLaboratory vs. Field Experiments

A laboratory experiment is a research study conducted in a contrived setting in which the effect of all, or nearly all, influential but irrelevant independent variables is kept to a minimum

A field experiment is a research study conducted in a natural setting in which the experimenter manipulates one or more independent variables under conditions controlled as carefully as the situation will permit

8 | 11

Internal ValidityInternal Validity

Internal validity is the extent to which observed results are solely due to the experimental manipulation

Laboratory experiments are generally high on internal validity

Field experiments are generally low on internal validity

8 | 12

External ValidityExternal Validity

External validity is the extent to which observed results are likely to hold beyond the experimental setting

Laboratory experiments are generally low on external validity

Field experiments are generally high on external validity

8 | 13

Deciding Which Type of Deciding Which Type of Experiment to UseExperiment to Use

Practical Considerations ◦ Time◦ Cost◦ Exposure to competition◦ Nature of the manipulation

8 | 14

Test MarketingTest Marketing

Dunkin Donuts and Baskin-Robbins are now offered in “combo” stores

KaBloom is testing kiosk flower sales in a variety of locations

Utilities companies are experimenting with providing Internet services via existing power lines

8 | 15

McDonald's Tests McPizzaMcDonald's Tests McPizza

McDonald's test-marketed McPizza to strengthen the after-4pm adult market◦Introduced McPizza with heavy advertising,

emphasizing speedy service for pizza◦McPizza received favorable nods in some test

markets and had partial rollout nationally

8 | 16

McDonald's Tests McPizza McDonald's Tests McPizza (Cont’d)(Cont’d)

Pizza Hut, a leading competitor, reacted aggressively to McDonald's move by running a buy-one-get-one-free promotion wherever McPizza was introduced

The sales performance of McPizza did not meet management's expectations

8 | 17

Simulated Test MarketingSimulated Test Marketing

Step #1 Pre-recruitmentStep #2 Background: habits and practicesStep #3 Exposure to real advertising in a

competitive contextStep #4 Simulated store purchaseStep #5 Post: purchase inquiryStep #6 Respondents take product home

for usageStep #7 Post: usage evaluation

8 | 18

Virtual Test Markets-- Ray Burke, Virtual Test Markets-- Ray Burke, Professor of Business Administration Professor of Business Administration at Indiana Universityat Indiana University

Created a virtual store to determine how products catch a consumer's eye

Computer 3-D graphics create a feeling of being in a store, walking past shelves of grocery items just as in a real store

Consumers can pick items off the virtual shelves to examine them as in real store and can select items they would buy

8 | 19

Virtual Test MarketsVirtual Test Markets

Virtual simulated marketing tests will enable companies to examine consumers' reactions to new products, product line extensions, prices, packaging, and merchandising

8 | 20

Scanner Data AnalysisScanner Data Analysis

Electronic scanners at the checkouts capture the product sales

Marketers of packaged goods conduct sophisticated field experiments

The data from the stores are transmitted electronically to central computers for analysis and interpretation

Information Resources Inc. (IRI) and ACNielsen offer marketers a variety of services through their information system called BehaviorScan and Scantrack

8 | 21

Web-Based ExperimentsWeb-Based Experiments

Web-based experimentation will enable companies to test a wide range of possible marketing mix changes and statistically model consumer responses to these changes

8 | 22

Web-based Experiments Conducted Web-based Experiments Conducted to Test the Effectiveness of Banner to Test the Effectiveness of Banner Advertising Advertising

Random people were selected while visiting the company’s website

The questionnaire asked them to complete a short questionnaire while on the site

Participants are shown random test banner ads

Participants fill out a second survey, answering questions about the impact of the banner ads on their impressions of the brand

8 | 23

Web-based Experiments Conducted Web-based Experiments Conducted to Test the Effectiveness of Banner to Test the Effectiveness of Banner Advertising Advertising (Cont’d)(Cont’d)

What are the independent variables? What is the dependent variable? What are some validity threats?

8 | 24

Internal ValidityInternal Validity

The presence of any condition or occurrence (other than the independent variable manipulation) that can offer a compete explanation for the experimental results is a threat to internal validity

8 | 25

Threats to Internal ValidityThreats to Internal Validity

History Maturation Pretesting Instrument Variation Selection Mortality

8 | 26

Threats To External ValidityThreats To External Validity

External validity of experimental results relates to their generalizability

The various internal validity threats also indirectly affect external validity

Biases that stand in the way of generalizing experimental results: ◦ Reactive bias ◦ Pretest-manipulation interaction bias◦ Non-representative-sample bias

8 | 27

8 | 28

Pre-Experimental DesignsPre-Experimental Designs

Pre-experimental designs exert little or no control over the influence of extraneous factors

These studies are not much better than descriptive studies when it comes to making causal inferences

Pre-experimental: emphasizes the fact that these studies are more exploratory than conclusive as far as causal inferences are concerned

8 | 29

Notations for DescribingNotations for DescribingExperimental DesignsExperimental Designs

O = observation or measurementX = experimental manipulationEG = experimental groupCG = control group(R) = random assignment

8 | 30

Pre-Experimental DesignsPre-Experimental Designs

One Group, After Only◦ EG X O

One Group, Before and After◦ EG O1 x O2

Two Group, Post Hoc◦ EG X O1

◦ CG O2

8 | 31

One-Group, After-Only Design One-Group, After-Only Design (Cont’d)(Cont’d)

Casual inference from a one-group, after-only design cannot be trusted entirely

8 | 32

True Experimental DesignsTrue Experimental Designs

The presence of one or more control groups

The random assignment of units to various experimental and control groups

Random assignment distributes the sample units chosen for a study to various groups on a strictly objective basis so that the group compositions can be equivalent before an experiment is started

8 | 33