TMLE ANALYSIS

TMLE ANALYSIS

NYC PREDICTIVE ANALYTICSORI STITELMAN (ori@media6degrees.com)BRIAN DALESSANDROCLAUDIA PERLICHFOSTER PROVOSTAUGUST 11, 2011

THE CAUSAL EFFECT OF DISPLAY ADVERTISING ON CONVERSION

2

QUESTIONOF INTEREST

WHAT IS THE EFFECT OFDISPLAY ADVERTISINGON CUSTOMER CONVERSION? ?

3

WHAT ISTMLE?A SEMI-PARAMETRIC method

for estimating CAUSAL PARAMETERS

that DIRECTLY answer a

business question of interest?

4

SEMI-PARAMETRICRealistic assumptions

CAUSAL PARAMETERSEffect/Impact (NOT Coefficients)

DIRECTLYActionable

WHAT ISTMLE?A SEMI-PARAMETRIC methodfor estimating CAUSAL PARAMETERSthat DIRECTLY answer abusiness question of interest?

5

OUTLINE1. BACKGROUND: DISPLAY ADVERTISING &

2. PEEK AT RESULTS

3. A/B TESTING

4. ALTERNATIVE APPROACHES (NICE)

5. RESULTS

6. CONCLUSION

6

THEBROWSERPROCESS

2. Use observed data to build list of prospects

3. Subsequently observe same browser surfing the web the next day

4. Browser visits a site where a display ad spot exists and bid requests are made

5. Auction is held for display spot

6. If auction is won display the ad

7. Observe browsers actions after displaying the ad

1. Observe people taking actions and visiting content

7

RESULTSPREVIEW

RETARGETING

M6D

PROSPEC...

RETARGETING

M6D

PROSPEC...

RETARGETING

M6D

PROSPEC...

0%

2%

4%

6%

8%

10%

12%

14%

RELATIVE LIFT:EXPOSED VS. UNEXPOSED USERS

DID NOT SEE AD SAW ADCO

NVER

SIO

N RA

TE

1.05X

2.62X

1.11X

1.31X

0.92X2.26X

TELECOM COMPANY A

TELECOM COMPANY B

TELECOM COMPANY C

8

P

GENERAL APPROACH

?

Ψ(P)

1. State Question

3. Define Parameter

Ψ(Pn)4. Estimate Parameter

2. Define Causal Assumption/Likelihood

9

WHAT IS THE EFFECT OFDISPLAY ADVERTISING ON CUSTOMERCONVERSION?

?1. STATE QUESTION

DISPLAY ADVERTISINGShowing/Not showing a browser a display ad.

CUSTOMER CONVERSIONVisiting the advertisers website in the next 5 days.

10

P2. DEFINE CAUSAL ASSUMPTIONS/LIKELIHOOD

O = (W,A,Y) ~ P0

W – Baseline VariablesA – Binary Treatment (Ad)Y – Binary Outcome (Site Visit)

P0(O) = P(W)P(A|W)P(Y|A,W)QW

QY

gA

11

DATA STRUCTURE: OURVIEWERS

CHARACTERISTICS(W)

TREATMENT(A)

CONVERSION(Y)

Color Sex HeadShape

Ad No Ad

No Yes

12

Ψ(P)3. DEFINE PARAMETER

P0(O) = P(W)P(A|W)P(Y|A,W)

P0,a(O) = P(W)P(Y|A=a,W)

P0,a(O) = P(W)P(Ya|W)=

LIKELIHOOD

DISTRIBUTION UNDER INTERVENTION

13

Ψ(P)3. DEFINE PARAMETER

ΨAI(P)=E[YA=ad] – E[YA=no ad]

ΨRI(P)=E[YA=ad]/E[YA=no ad]

1. ADDITIVE IMPACT

2. RELATIVE IMPACT

P0,a(O) = P(W)P(Ya|W)

14

E[YA=ad] E[YA=no ad]

φn,ad/φn,no ad

1. PARAMETERS ARE COMBINATIONS OF TREATMENT SPECIFIC CONVERSION RATES

2. SO WE CAN COMBINE ESTIMATES OF THESE THESE RATES

Ψ(Pn)4. ESTIMATE PARAMETER

φn,ad-φn,no ad

15

Ψ(Pn)4. ESTIMATE PARAMETER

1. Optimal Experiment

2. A/B Testing

3. MLE Based Substitution Estimator (MLE)

4. Inverse Probability Estimators (IPTW)

5. Double Robust Estimating Equations (DR-IPW)

6. Targeted Maximum Likelihood Estimation (TMLE)

NICE

16

OPTIMALEXPERIMENT:

3.6 per 1,000

1.2 per 1,000

Compare conversion rates of seeing an ad to conversion rate without ad for SAME individuals.

17

OPTIMALEXPERIMENT:

BROWSER

SHOW AD?

OBSERVEOUTCOME

OR

OR

18

REALITY: OR

OR

OR

OR

19

OPTIMAL EXPERIMENT:Compare conversion rates of seeing an ad to conversion rate without ad for SAME individuals.

3.6 per 1,000

1.2 per 1,000

20

COMMON APPROACH:A/B TESTINGSince we can not both treat and not treat SAME individuals. Randomization is used to create “EQUIVALENT” groups to treat and not treat.

3.4 per 1,000

1.6 per 1,000

21

SIGNIFICANT COSTSASSOCIATED WITH DOING A/B TESTING WELL

1. Cost of displaying PSAs to the control(untreated group).

2. Overhead cost of implementing A/B test and ensuring that it is done CORRECTLY (Kohavi et al.)

3. Wait time necessary to evaluate the results.

22

NON INVASIVECAUSAL ESTIMATION (NICE)Estimate The Effects In The Natural Environment (Observed Data)

23

“WHAT IF”CAUSAL ANALYSIS ADJUSTING FOR CONFOUNDING

Need to adjust for the fact that the group that saw the advertisement and the group that didn’t may be very different.

24

TWO MACHINES:QY,n AND gA,n

gA

QY

Pn(A| )

Pn(Y| )Pn(Y| )

gA,n

QY,n

ESTIMATING QY AND gA

1. Many tools exist for estimating binary conditional distributions.

1. Logistic regression, SVM, GAM, Regression Trees, etc.2. Data adaptive estimation methods that use cross

validation.3. SuperLearner (R package, Eric Polley)

2. Causal analyses will benefit from the advances in parallelized routines for data adaptive estimation.

26

INVERSE PROBABILITY WEIGHTED ESTIMATORS (IPTW)

•Adjust For Confounding Through gn.

•Weight individuals that are unlikely to be shown an advertisement more than individuals that are likely to be shown an advertisement

STRATEGY

ESTIMATOR

Pn(A| )=1/2

=

1/10

Pn(A| )

QY

QY

MLE BASED SUBSTITUTION ESTIMATOR (MLE)

•Adjust For Confounding Through Q.•Predict how each observed browser will behave had they been shown an ad and had they not been shown an ad.

STRATEGY

ESTIMATOR

QY,n

QY,n27

28

DOUBLE ROBUST ESTIMATORSWhat if QY,n or gA,n are broken? The MLE based estimator and IPTW rely on consistent estimates of Q and g respectively.

gA

QY

P(A| )

P(Y| )P(Y| )

gA,n

QY,n

29

OR

STRATEGY

ESTIMATOR

AUGMENTED – IPTW (A-IPTW)

•Adjust For Confounding Through Q and g.

•Augments IPTW estimator with information from Q.

•Alternatively adjusts MLE with information from g.

Ψ(QY)

TARGETED MAXIMUM LIKELIHOOD ESTIMATOR (TMLE)

•Adjust For Confounding Through Q and g.

•Predict how each observed browser will behave had they been shown ad and had they not been shown ad.

•The new machine Q* is calibrated with concern for the parameter of interest.

•R package “tmle” at CRAN

STRATEGY

ESTIMATORQY

Ψ(QY)QY

Q*Y,n

Q*Y,n

30

CREATING Q*YN

•QYn is updated to Q*Yn using a clever covariate that is a function of g.

•Update is done through the use of a parametric submodel.

•Update is a univariate regression with the initial QYn as an offset.

QY

gA

Ψ(QY)QY

Q*Y,n

31

32

SAMPLING/ANALYSIS1. Select Prospects that we got a bid request for on day

0.

2. Observe if treated on day 1. For those treated A=1 and those not treated A=0. Collect W.

3. Create outcome window that is the next five days following treatment and observe if event occurs.

4. Estimate parameters using the methods previously described.

33

RESULTS

34

IPTW AND A-IPTW BLOW UP UNDER POSITIVITY VIOLATIONS

35

METHOD VALIDATION: A/B TEST VS TMLE

36

METHOD VALIDATION: NEGATIVE TEST

37

GROSSCONVERSIONRATESARE ONLY PARTOF THE STORY

RETARGETING

M6D

PROSPECTING

RETARGETING

M6D

PROSPECTING

RETARGETING

M6D

PROSPECTING

0.0%

0.1%

0.2%

0.3%

0.4%

0.5%

0.6%

0.7%

0.8%

0.9%

1.0%

0.5%

0.9%

0.1%

0.8%0.7%

ADDITIVE IMPACT: EXPOSED VS. UNEXPOSED USERS

ADDI

TIVE

IMPA

CT IN

CO

NVER

SIO

N RA

TE

-0.2%

TELECOM COMPANY A

TELECOM COMPANY B

TELECOM COMPANY C

38

EFFECTIVENESSVARIES BYMARKETER& CAMPAIGN

RETARGETING PROSPECTING RETARGETING PROSPECTING0%

2%

4%

6%

8%

10%

12%

14%

16%

RELATIVE IMPACTDID NOT SEE AD SAW AD

CONV

ERSI

ON

RATE

B2B COMPANY

A

B2B COMPANY

B

1.08X

1.08X4.23X

3.77X

39

RETARGETING

M6D P

ROSPECTING

RETARGETING

M6D P

ROSPECTING

RETARGETING

M6D P

ROSPECTING

RETARGETING

M6D P

ROSPECTING

0.0%

1.0%

2.0%

3.0%

4.0%

5.0%

6.0%

7.0%

8.0%

RELATIVE IMPACTDID NOT SEE AD SAW AD

CONV

ERSI

ON

RATE

1.71X1.34X

2.57X

1.17X

3.00X

2.33X

1.84X

1.11X

CAR RENTAL RESTAURANTHOTELAIRLINE

Effective creative and

targeting drive lift across

marketers and verticals

TEC

HN

OLO

GY

A

TRA

VE

L A

B2B

A

TRA

VE

L B

TELE

CO

M A

TRA

VE

L C

TELE

CO

M B

TRA

VE

L D

RE

TAIL

A

RE

TAIL

B

TRA

VE

L E

ED

UC

ATI

ON

A

TRA

VE

L F

RE

STA

UR

AN

TA

AU

TO A

RE

TAIL

C

RE

STA

UR

AN

T B

TRA

VE

L G

ED

UC

ATI

ON

B

TEC

HN

OLO

GY

B

RE

TAIL

D

TELE

CO

M C

RE

TAIL

E

AU

TO B

ED

UC

ATI

ON

C

FIN

AN

CE

A

TRA

VE

L H

B2B

B

RE

TAIL

F

TRA

VE

L I

-100%

0%

100%

200%

300%

400%

500%

600%

700%

800%

-100%

0%

100%

200%

300%

400%

500%

600%

700%

800%

M6D PROSPECTING LIFT RETARGETING LIFT

M6D

PRO

SPEC

TING

LIF

T

RETA

RGET

ING

LIF

T

For 25 out of 30 marketers,

relative lift was

higher for prospecting

candidates than for retargeting

candidates .

SUMMARY RESULTSAVERAGE RELATIVE LIFT OF 90% FOR M6D PROSPECTING

41

CONCLUSIONS1. Estimating causal effects allows one to directly estimate the

impact of the display advertisement.

2. Causal effects can be estimated in the observed data.

3. Display advertising works in varied ways. Causal analysis allows us to estimate effects on a case by case basis.

4. All methods for estimating effects are not equal.

5. These methods may be used for assessing other types of causal effects.

6. What is next?

42

VISIT OUR BLOGmedia6degrees.com/blog

43

OTHER RESOURCES FOR TMLE

Enter “Slide Show” view and click image to link to site.

44

ACKNOWLEDGEMENTS

Edward CaprioloBrian DalessandroRod HookBrian MayRyan OttobreClaudia PerlichTom Phillips

Mark van der LaanSusan GruberEric Polley

Foster Provost

45

REFERENCES1. O. Stitelman, B. Dalessandro, C. Perlich, and F. Provost. Estimating The Effect Of Online Display

Advertising On Browser Conversion. In Proceedings of KDD, Annual International Workshop on Data Mining and Audience Intelligence for Online Advertising, ADKDD ’11.

2. M. van der Laan and S. Rose. Targeted Learning: Causal Inference for Observational and Experimental Data. New York, NY: Springer Publishing Company, 2011. http://www.targetedlearningbook.com/

3. ‘tmle’ R Package http://cran.r-project.org/web/packages/tmle/index.html

4. R. Kohavi and R. Longbotham. Unexpected results in online controlled experiments. ACM SIGKDD Explorations Newsletter, 12(2):31–35, 2010.

5. R. Lewis and D. Reiley. Does retail advertising work: Measuring the effects of advertising on sales via a controlled experiment on yahoo. Technical report, Working paper, 2010.

6. D. Chan, R. Ge, O. Gershony, T. Hesterberg, and D. Lambert. Evaluating online ad campaigns in a pipeline: causal models at scale. In Proceedings of KDD, KDD ’10, pages 7–16, New York, NY, USA, 2010. ACM.