Conversion Architecture Whitepaper

8/3/2019 Conversion Architecture Whitepaper

1/12

IImmppoo

rroovv

hhii

vv YYooleeRRrrssii

uurrdduu

onnnnllii

iinngg

W

Pr

Co

AArreePPYYoo

IWhitePparedby:version

chhiirrffoo

rrSS

per

DougSchrchitectu

tteerrmma

eenn

usteExpert,

ttuunnccee

WSI

ee::


2/12

CCoonnvveerrssiioonnAArrcchhiitteeccttuurree::IImmpprroovveeYYoouurrOOnnlliinneePPeerrffoorrmmaanncceeWWhhiilleeRReedduucciinnggYYoouurrMMaarrkkeettiinnggSSppeenndd

Copyright2009byResearchandManagement.Allrightsreserved. Page2of

IInnttrroodduuccttiioonn

Researchhasshownamajordifferenceinconversionandreturnoninvestmentbetweencompanies

that

test

their

online

communications

and

those

who

dont.

Sowhydoesnteveryonlinebusinesshavearegulartestingprogram?

Thefactisthatmanybusinessesandmarketersareconfusedaboutwheretobegin,andhow.Somethat

dotestdontdoitoftenenough,orarenotsurehowtointerprettheresults.

Withsomanypotentialareastotest,includinghomepage,landingpages,payperclickads,emailsand

nowtheinfluenceofsocialmedia,marketersneedtoknow:

Whichtestsprovidethebiggestpotentialreturnoninvestment Howtostructuretestsforconsistencyandaccuracy Howtocontinueimprovingtestresultsandduplicatesuccess

Firstofallyouneedasolidbenchmarkingframeworkforbasiconlinetestingthatenablesyoutorealize

significantgainsinconversionandrevenue.

Wehopethisoverviewhelpsyourbusinessandifyouhavenotyetimplementedaformalprogramas

wellasthoselookingforhigherconversions,werecommendaWSIBenchmarkingProgramtoguideand

improveyourefforts.PleasecontactaWSIMarketingConsultanttoday!

Thereis

only

one

key

thought

process

you

need

to

act

on

to

increase

your

online

conversion

rate

and

thatissimplyTest,TestandTest.

Withinthattestingthereare4keyareasyouneedtounderstandtoincreaseyouronlineconversionand

reduceyouroverallcostperlead.

AskTheRightQuestions MeasureTheRightItems PerformTheRightTests Analyze&MakeAdjustmentsBasedOnResults

Thesetests

will

give

you

abetter

understanding

of:

your

customer

profiles,

their

online

behavior

patternsandwhatdrivesthemtotakeaction.


3/12



IIttAAllllBBeeggiinnssWWiitthhTTeessttiinnggTTooEEssttaabblliisshhBBeenncchhmmaarrkkss::

AtWSI,weapplytheconceptofbenchmarkingtoourtestingprocess.

Whatdowemeanbybenchmarking?Atestingprocessthatisuseful,practical,anddeliverspredicable

resultsthatyoucancarryforwardintoasuccessfulmarketingprogram.

Thebenchmarkresultsofouronlinetestingprocessaredirectlyrelatedtofivekeyelements.

ThisformulawascreatedbyMarketingExperimentsandisthebasisforonlineconversiontoday.

Wherein:

u=Utility

q=ResearchQuestion

t=Treatment

m=MetricSystem

v=ValidityFactor

i=Interpretation

Inotherwords,asktherightquestion,measuretherightthing,performtherighttest,andlearnfrom

theresults.

WerecommendthatbusinessesinterestedinestablishingastrongonlinepresenceusetheWSI

BenchmarkingProgramandincorporatetheseelementsintoyouronlinestrategies.

Thereare3mainareasthatneedtobetested,homepage,landingpage,emailandtheimpactofsocial

media.Usingthesetestingtechniquesyoucanseewhatinfluencesaclientorpotentialclientwhenthey

connectwithyouonline.


4/12



TTHHEEBBEENNCCHHMMAARRKKIINNGGPPRROOGGRRAAMM

11..SSttaarrttwwiitthhtthheerriigghhttqquueessttiioonn..InarecentWebcliniconthissubject,morethan60%oftheaudiencesaidthattheveryfirst

stepinsuccessfulonlinetestingaskingtherightresearchquestionistheirbiggest

challenge.

AllsinglefactorA/Bsplittestsbeginwithwhich?

Marketersmaystartwiththeinterrogativewhat?orwhy?whentheyarebrainstormingon

aninformalbasis,buttheirresearchquestionmustprogresstowhich?inorderforittobe

specificenough

to

be

used

in

aformal

test.

Forexample,Whatisthebestheadline?becomesWhichheadlinewillconvertbest?and

Whatisthebestprice?becomesWhichofthreepricepointsisbestforthisproduct?

22..DDeevveelloopptthheeTTrreeaattmmeenntt((ss))..Avariableisthegeneralelementyouintendtotest.Headline,price,andcopyareallexampleof

variables,andtherearedozensmore.Again,onekeytosuccessfulA/Btestingisknowingwhich

oneyouwanttosingleout.

Avalue

is

the

specific

option

(content,

size/amount,

color,

position)

of

the

variable

you

are

goingtotest.

ThetestControlAwillbeyourcurrentpage,ad,oremail.

ThetestTreatmentorTreatmentsBorCarethepages,ads,oremailscontainingthe

newvaluesofyourvariable.

ValuesbecomeTreatments.WhenanewTreatmentperformsbetter(betterthantheControl,

betterthananotherTreatmentifthereismorethanone),itbecomesthenewControlin

subsequenttesting.

33..DDeetteerrmmiinneetthheerriigghhttmmeettrriiccss..Yourmainobjectivewillbetodeterminethemetricsthatwillbestansweryourprimary

researchquestion(thewhich?),thoughsecondarymetricsmayalsoprovideinsight.

First,establishwhatyouaretryingtoaccomplish.


5/12



Areyoutryingtoattractmoresubscribers?

Capturemorequalifiedleads?

Obtainahigher

conversion

rate?

ProduceabetterROIonapayperclick(PPC)campaign?

Afterestablishingwhatyouwanttoachieve,choosetheelementyouwishtotestinorderto

selecttheappropriatemetric.

Thereareessentiallyfourelementsyoucanmeasure:

Theamountofactivityonyoursite Thesourceofthatactivity Thenatureofthatactivity Theresultsofthatactivity

Useorganizingquestionstohelpyouchoosetheappropriatemetric;forexample:

Question Metric

Who

visited

my

page?

Number

of

unique

visitors

(UV)

Wheredidtheycomefrom? ReferringURL,organicsearch,PPCad&socialmediainfluence

Whatdidtheydo/view? Pageviewspervisitor;clicks

Whatdidtheybuy/signupfor

ordidtheycall?Numberoforders,averageamount,totalrevenue

44..CChheecckkvvaalliiddiittyy,,ssttaarrttiinnggwwiitthhssaammpplleessiizzeessuuffffiicciieennccyy..Forconfidenceinyourfindings,testresultsmustbestatisticallymeaningful.Inotherwords,

therehastohavebeenenoughdifferenceinresultstomakethemeaningfulrightchoice.

Typically,researcherswilltoleratenomorethana5%chancethatthetestresultsaredueto

randomvariation.Thisisreferredtoas5%significancelevelor95%confidencelevel.For

example,youwouldwanttobeatleast95%confidentthattheresultsofyourtestwilldetecta

differenceinclickthroughrateof0.5%ormore.


6/12



Keypoint:assamplesize(n)increases,MarginofError(E)declines.

Checkforfouradditional,majorvaliditythreats.

Thereare

anumber

of

threats

to

the

validity

of

testing,

but

these

four

threats

are

common

in

onlinetesting:

Historyeffects Instrumentationeffects Selectioneffects Samplingdistortioneffects

Ensurethatyourtestresultshavenotbeenskewedorinvalidatedbythesethreats.

Historyeffects.Theeffectonatestvariablebyanextraneousvariableassociatedwiththe

passageoftime.Industryevents,newsevents,andholidaysareallexamplesofhistoryeffects

thatcouldskewyourtestresults.

Instrumentationeffects.Theeffectonatestvariablebyanextraneousvariableassociatedwith

achangeinthemeasuringinstrument.Havingaservergodowninthemiddleofyourtestisone

exampleofaninstrumentationeffect.

Whenreviewingyourtestprotocoltoexcludethisthreat,ask:Didanythinghappentothe

technicalenvironment,measurementtools,orinstrumentationthatcouldhavesignificantly

influencedthe

results?

Selectioneffects.Theeffectonatestvariablebyanextraneousvariableassociatedwith

differenttypesofsubjectsnotbeingevenlydistributedbetweenexperimentaltreatments.

Mixingtrafficsourcescanskewresults.Agreatlandingpagecreatedforonespecificchannel

forexample,aPPCadwithgreatcopyrelatedtoaspecificorganicsearchtermcandopoorly

whenpresentedtovisitorscomingfromadifferentPPCadforacompletelydifferentproductor

service.

Sampling

distortion

effects.

The

effect

on

the

test

outcome

caused

by

failing

to

collect

a

sufficientnumberofobservations.

Dependingontheamountoftrafficyourlandingpagenormallygets,thenumberofemail

addressesyouhavetosendyournewemailto,ortheclicksyounormallygetonyourexisting

PPCad,itmaytakedays,weeksorevenmonthstoreachthenumberofobservationsnecessary

toachievetestingvalidity.


7/12



55..CCoorrrreeccttllyyiinntteerrpprreetttthheerreessuullttss..Ifresultsarevalid,determinetheimpactanddecidewhatyournexttestshouldbe.Remember,

thewinningTreatmentbecomestheControlforyournexttest.

Ifresultsareinvalidorinconclusive,considerrerunningthetest,butkeepinmindthateven

invalidorinconclusivetestscanprovidevaluewhenproperlyanalyzed.Learnfromyourresults:

Expertiseiscumulativeknowledge.Keepcarefulrecordsoftestsandresults.Whenresultsaresurprising,lookforhiddensourcesoferrororinsight.

Lackofdifferenceismeaningful.Thatchangingthevariable(s)inthewaythatyoudidhadlittleeffectonperformanceisvaluableknowledge.Thismaybethesinglemostvaluableinsightyou

gainfromaninconclusivetest.

Try

stratification.

Try

filtering

the

results,

looking

for

patterns

(days

of

the

week,

hours

of

the

day,etc.).Youmayfindaclearstatisticalwinner.

Notealternativeorsecondaryconclusions.Lookforpatternsofperformance,eventhosethatidentifythingsthatyoudefinitelyDONTwanttodo.

BBrreeaakkiinnggDDoowwnnYYoouurrCCaammppaaiiggnnssaannddMMaarrkkeettiinnggCChhaannnneellss

HHeellppssttooDDeetteerrmmiinneetthheeFFoolllloowwiinngg::

Whoyourclientsare,whattheirneedsandwantsareandwhatdrivesthemtotakeaction.Theseare

thekeytoanyonlinemarketingprogram.Thediagrambelowillustratesastandardconversionfunnel.


8/12



Testingallowsyoutoidentifyyourclientsorpotentialclientsprofileswhichultimatelyleadstohigher

conversions. Understandingeachpotentialaudienceallowsyoutocommunicatetherightmessageat

therighttime.

Understandinghowyouraudiencemakestheirdecisionisveryimportant.Thegraphicbelowillustrates

astandardbuyingprocessinaB2Benvironmentbuyingprocess.


9/12



TTeessttPPrroottooccoollffoorrAA//BBSSpplliittTTeessttiinnggffoorrLLaannddiinnggPPaaggeess::

11..DDeevveellooppYYoouurrCCaappaabbiilliittiieessaannddSSeelleecctttthheeRRiigghhttTToooollssA/BsplittestingtoolsvaryfromsimpleCGIscriptstosophisticatedsoftwareapplications.You

willfindalistofservicesintheResourcessectionattheendofthisreport.

EvenwithoutsophisticatedA/Btestingcapability,sequentialtestingoffersyouanopportunity

tolearnmanyinsightsaboutyourpages.Formoreonsequentialtesting,seetheConcluding

Comments,below.

22..IIddeennttiiffyyYYoouurrEEssttaabblliisshheeddCCoonnttrroollPPaaggeeYourcontrolpagewillbethepageagainstwhichyoutestallsubsequentoptimizationefforts.If

youare

just

getting

started

with

A/B

testing,

your

control

page

will

be

your

current

landing

page

beforeanyoptimization.Whenanewpageperformsbetterthantheexistingcontrolpage,it

thenbecomesyourcontrolpageinsubsequenttesting.

33..EEssttaabblliisshhYYoouurrTTeessttiinnggGGooaallssaannddPPaarraammeetteerrssWhatareyoutryingtoaccomplishwithA/Bsplittesting?Areyouaftermoresubscribers,a

higherconversionrate,oragreaterreturnoninvestmentonyourPPCcampaigns?Yourgoals

willdetermineyourtestingparameters,whichwilldeterminethepotentialsuccessofyour

testingefforts.

44..DDeetteerrmmiinneeYYoouurrSSuuffffiicciieennttTTeessttIInntteerrvvaallThistimeperiodshouldallowyouenoughtimetogathersufficientdatatogaugerealinsight

aboutyourA/Btests.Identifythenumberofuniquevisitorsand/orconversionsneededto

establishgooddataandthendeterminehowlongitwilltakeyoutogeneratethistraffic.This

numberwillvaryfrombusinesstobusiness,butshouldgiveyouenoughdatatoconfidently

declarea"winner."

55..CCrreeaattee1133RRaaddiiccaallRReeddeessiiggnnssKEYPOINT:Thesepagesarenotsubtleoptimizationschangingonlyoneortwoelementsonthe

page,butarewhollydifferentpagesrepresentingaradicallydifferentapproach.

66..EEvvaalluuaatteeTThheesseeRReeddeessiiggnnssiinnAA//BBSSpplliittTTeessttssTestthesealternatelandingpagesagainstthecontrolpage.Ideally,eachpagewillbetested

againsteveryotherpage,butifthatisimpractical,testtwopagesatatimeandkeepthebestas


10/12



yourcontrolforsubsequenttesting.Weliketouseheatmapstohelptracktheimpactofallthe

keyelementsanditspotentialimpact.

77..BBaasseeddoonnRReessuullttss,,DDeetteerrmmiinneeYYoouurrTTrruueeCCoonnttrroollPPaaggeeTheradicalredesignmethodwillbemorelikelytogenerateaquantumleapinimproved

conversionrate

than

optimizing

amediocre

page

with

little

potential.

Once

you

have

identified

thebestgeneralapproach,youarenowreadytooptimizeindividualelementsonthepage.

88..OOppttiimmiizzeewwiitthhTTrraaddiittiioonnaallVVaarriiaabblleeSSppeecciiffiiccAA//BBTTeessttiinnggVVaarriiaabblleessttooTTeesstt::

Headline CalltoAction PageCopy Graphics Color ConfigurationofPageElements Etc.


11/12



CCoonncclluuddiinnggCCoommmmeennttss::

ThereareafewimportantpointstorememberaboutA/Bsplittesting:

1. Evenifyoucan'tsetupatrueA/Bsplittest(wheretwoversionsofapagearebeingdisplayedoneafteranothertoalternatevisitorstoyoursite),itisveryeasytocreateasequentialA/Bsplit

test.

KEYPOINT:Asequentialtestiswhenyoushowoneversionofapageforacertainperiod,like

twodaysoraweek,andthenshowanotherversionforthefollowingtwodaysoraweek.The

resultsmaybealittlelessreliable,butcanstillyieldvaluableinformationandtrends.

2. Testinggivesyoutheopportunitytomaximizeconversionrates,solveproblems,andchallengeassumptions.Andkeepinmindthatyouhaveopportunitiesbeyondtestingsmallchangestoa

page.

Youcanalsochallengeanexistingpagebydesigningandwritingaradicallydifferentversion,

wherealmosteverythingisdifferent.Infact,itisthroughthesedramaticallychanged

approachesthatyouaremostlikelytoachievebreakthroughimprovements.

3. Testingprovidescompanieswithaninvaluablemeanstodemonstratetocompanyheadsandmanagementthehardfiguresbehindanysuggestedchangesorimprovements.

4. Useofconsistenttestingwillincreasetheknowledgebaseofyourwebgrouporcompanysignificantly.Youwilllearnmore,andsoonbeabletodetermineasetofoptimizedpractices

thatworkbestforyourparticularbusiness.

Theabsenceofrigoroustestingleavesyouinthedark,dependingonguessworkalonewhen

creatingyourpages.

CCoonncclluussiioonn::

Mostbusinessestodaydonottakeadvantageofhowimportantitistotestregularly.Thekeyforany

businesswantingtoachieveonlinesuccessisknowinghowtocommunicatetoyourtargetaudienceand

knowingthatallpartiesdonotactonthesameprinciples.Thisalsoallowsyourtofigureoutyouronline

adspendinaneffectivemanner.Theaverageonlineconversiontodayis2%,whichmeansthat98%of

youradbudgetisnotbeingusedeffectively.

Thekeyisreallytokeepthetestingsimpleanduncomplicatedandonceyouhaveestablishedyour

benchmarksthenyoucanstarttodivertfromyourControlPage.Thiscanbeasimpleprocessandora

morecomplicatedonebutthekeyissimplytesting,testingandmoretestingoftherightthingsandyou

willincreaseyouronlineconversion.


12/12



AAbboouutttthheeAAuutthhoorr

DougSchustsbackgroundconsistsofbothonlineandtraditionalmarketing.Hismain

focusisonlinemarketingandhelpingclientsfocusnotontrafficbuttheresultsfrom

thattraffic.MostrecentlyDoughasspokenatseveralGoogleeventsonconversion

andcontinuestosharehisknowledgeandexperiencewithotherconsultantsand

clients.

RReessoouurrcceess::

MarketingExperiments MarketingSherpa Omniture Google

Documents

Conversion Architecture Whitepaper