Upload
grape
View
580
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Презентация Peter Ernst, Associate Director of Optimisation, Possible, для мастер-класса на Digital Marketing Conference 2012
Citation preview
OCTOBER 2012
TEST AND LEARN
How experimenting with content improves the user experience and drives business value
MARKETING SCIENCES ANALYTICS � MONETIZATION � OPTIMIZATION
OUR METHODOLOGY IS DESIGNED TO ANSWER QUESTIONS RATHER THAN BUILD REPORTS Where should I invest my marketing dollars? How does my digital media relate to offline purchase? What is the value of my customers? How can I increase conversion? Who should I target? Who are my most valuable customers? What is the impact of my social media strategy?
METHODOLOGY
: Goals and KPI definition
: Technical and organizational support for measurement
: Holistic view of what is working and what is not
: Create and use an ROI-driven system to get smarter
ANALYTICS
ASSIGNING MONETARY VALUES TO DIFFERENT SITE BEHAVIORS TO UNDERSTAND AND QUANTIFY THE OVERALL VALUE OF THE WEB CHANNEL.
BENEFITS
1. Measure total website value and track value over time
2. Understand how website actions relate to future profit
3. Evaluate return on marketing spend
4. Create a common language within the organization by using a common value for stakeholders to report on
5. Project prioritization
MONETIZATION
COMMON TRAITS OF OUR MOST SUCCESSFUL CLIENTS
: Digital Channel Goals are imperative
: Analyze performance based on overall business goals
: Understand data holistically, not in silos and not just digital
: Focus on opportunities and recommendations
: Monetize all key digital behaviors
: Prioritize opportunities
: Measure & quantify relationships between brands and people
: Take action with an optimization program
: Avoid average experiences
: �$+-�($34'2;
Website Optimization delivers relevancy to customers, ROI to clients Improve site performance (business objectives)
: Engage more users : Increase sales and profits : Improve lead generation and opt-ins
Validate assumptions Discover what content resonates with target audience(s)
Bounce Rate
KPIs
OPTIMIZATION
op�ti�mize [op-tuh-mahyz] Verb
1. To make as effective, or useful as possible.
Right Message
Right Audience
Right Time
Optimisation
Copy Imagery Call-to-Action �4%;
First time visitors Return visitors Paid Search traffic �4%;
Evening Weekend End of Month �4%;
Performance improvement
� �����������#��������;
A
B
Visitors randomly split into groups
Each group is shown a unique version
Results are analyzed & winner determined
Days
conversion s
� �����������#��������;
TESTING DIFFERENT DESIGN AND CONTENT VERSIONS TO DETERMINE WHICH IS MOST EFFECTIVE
EXISTING CONTENT
ALTERNATE CONTENT
Single element testing
Multiple element testing
Serve content by visitor segment
People who did this are likely to do that
Serve content unique to visitor
A/B/n Testing
Multivariate Testing (MVT)
Behavioral Targeting
Predictive Modeling
Personalization
Multi Channel Optimise messaging across fixed web, mobile, tablet, call centre, etc
Audience size Com
plex
ity
TECHNIQUES
Guessing, Before and After Testing
A/B/n and Multivariate Testing
Segmentation, Personalization, etc.
Behavioral Targeting
Multi-Channel
EVOLUTIONARY PROCESS
THE PERFORMANCE OF THE GENERAL POPULATION LIKELY DOES NOT REPRESENT THE BEHAVIOUR OF KEY ONLINE BUSINESS SEGMENTS : Making single answer decisions based on one bucket of visitors may miss
opportunities for large incremental benefits : Optimising content for key segments transforms a good website to a
great online experience
ONE SIZE DOES NOT FIT ALL
: We use data to understand customer journeys, motivations, and what messages are working and what are not
: We translate this data into insights to create personalised and targeted conversations
across touch points : We can demonstrate how establishing a culture of measurement and testing has a
revolutionary impact on marketing strategies and performance
SUCCESS COMES FROM UNDERSTANDING AND MEETING THE NEEDS OF THE AUDIENCE
WE BELIEVE
Environment Variables
Site Behavior Variables
Referrer Variables
Temporal Variables
Offline Variables
Environment Variables
IP address + Country of origin +
Time zone + Device type +
Operating system + Browser type +
Screen resolution +
Site Behavior Variables
Customer / Prospect + New/return visitor +
Previous visit patterns + Previous Product interests +
Searches + Previous online purchases +
Previous Campaign exposure + Previous Campaign responses +
Temporal Variables
Time of day + Day of week +
Recency + Frequency +
Enhance visitor engagement and
traffic flow
Improve messaging
Increase Form Completion
Provide relevant / personalized messages
HOW DOES HOMEPAGE CONTENT IMPACT BOUNCE RATE AND SITE ENGAGEMENT?
WHAT PAGE CONTENT ENGAGES VISITORS BEST? ���?����������������������������"����
HOW MANY MORE LEADS COULD WE GENERATE WITH SIMPLER FORMS?
HOW CAN WE REDUCE BOUNCE AND INCREASE VALUE OF PAID TRAFFIC? HOW CAN WE BETTER ENGAGE VISITORS FROM SOCIAL NETWORKS?
WHEN AND WHERE SHOULD CROSS-SELLS BE INTRODUCED? TO WHOM AND WHEN SHOULD DISCOUNTS AND INCENTIVES BE OFFERED?
Drive purchasing
ANSWERING BUSINESS QUESTIONS THROUGH TESTING
WEB OPTIMIZATION METHODOLOGY
Deepen engagement
Boost conversions
Increase value of media spend
Teaches what resonates/performs best with target audience
TEST RESULTS
TEST DESIGN AND VARIANT CREATION
ESTABLISH SITE GOALS AND SUCCESS METRICS (KEY PERFORMANCE INDICATORS)
IDENTIFY AND PRIORITIZE TESTING OPPORTUNITIES
FOUNDATION
1. ASSESSMENT : Define Objectives : Primary goals of site : Goals of individual sections of site 2. TEST STRATEGY : Develop hypothesis : Define Success metrics : Determine test methodology : Create metrics plan : Calculate test duration 3. EXECUTION : Create alternates : Tool set-up : Launch tests 4. RESULTS ANALYSIS : Analyze performance by variant : Analyze performance by segment : Recommendations/Next steps : Plan follow up tests
Assessment Test Strategy
Analysis Execution
WEB OPTIMISATION PROCESS
RESULTS
1 2
3 4
TESTING AND TARGETING CASE STUDIES
OBJECTIVE Increase click thru rate
ACTION TAKEN : Removed GOOD, BETTER, BEST design
elements : Reduced copy to call out only 3 features : Changed background from blue to white RESULTS : 25% CTR lift : 61% increase in revenue
Control
Winner
A/B TEST EXAMPLE: DELL iNSPIRON NOTEBOOK LANDING PAGE
Reformatted fields to make form appear shorter/consider shading Reduced the prominence of the reset button Added relevant image
Control < Existing Page Alternate
Results: 129% form completion increase
A/B TEST EXAMPLE: SIMPLIFIED TEST DRIVE FORM
1
2
3
A/B TEST EXAMPLE: FORD OF RUSSIA HOMEPAGE ROTATION
RESULTS: : 8% reduction in bounce rate : 13% increase in buying activities
OBJECTIVE: Increase engagement and buying activities
Winner Control
OBJECTIVE: Improve merchandising of accessories to increase conversion rate and RPV
Product Page Cart Page
RESULTS: : Removal of Accessories on Brand product pages delivered 60% lift in RPV : Adding Accessories to Cart page increased purchase conversion rate by 5%
TEST EXAMPLE: NOKIA ACCESSORIES CROSS SELL
1 2
1
4
3
Winner
Control
MULTIVARIATE TEST EXAMPLE: HELIO OFFER PAGE
OBJECTIVE : Reduce landing page
: abandon rate ACTION TAKEN : Redesigned call to action to
better inform customers
: 480 total variants RESULTS : 57% higher CTR
: 130% sales conversion rate
increase
A (Control)
B (Big Sexy Image)
C (Lead with Offer)
D (Name/Prices on phones)
E (Fa-La-La-La-La!)
RIGHT PANEL OPTIONS
A (Control) B (More text + price) C (more text, no price)
D (no right image or button)
RIGHT PANEL OPTIONS
A (Control)
B (Choose Device)
C (Green Choose Device)
D (Bold Choose Device)
E (Green Buy Now)
F (Bold Buy Now)
A (Control)
B (Red Button)
C (Learn More)
D (No Right Button)
��8���8��8��;����04$-��$2+$/43
LEFT PANEL RIGHT PANEL
BUTTON OPTIONS
OBJECTIVE: Increase purchase conversion rate and RPV ACTION TAKEN Targeted visitors who previously self-identified themselves as laptop shoppers with laptop content AREAS TESTED: : Content & layout in center of page : Products promoted : A/B/n test (9 variants)
RESULTS: : 24% lift in revenue per visit : 19% lift in purchase conversion rate : Significant annual monetized lift
Winner
Control
TARGETING EXAMPLE: DELL SMB RETARGETING
Popular Games Discount Games
35
OFFER TEST: Discounted games vs Popular games RESULTS: The popular games version significantly outperformed the discounted titles:
: 18% higher open rate : 91% higher click through rate : 107% more orders : 300% higher revenue
EMAIL OPTIMISATION: YAHOO GAMES
CONTENT CHANGES ARE JUST THE BEGINNING
THE QUESTION IS NOT CAN WE SERVE DIFFERENT CONTENT
Copy? Imagery?
Layout? CTA?
Offer?
New Visitors? Paid Search Traffic?
Repeat Purchasers?
PROGRAM NOT PROJECT
WEB OPTIMIZATION TEST VALUATION/PRIORITIZATION
In order to prioritize we need an estimated impact of the test. �(�7'�2'$--9�,/'7�4*'�+.1$%4�0(�4*'�4'34��7'�705-&/?4�/''&�40�4'34�+4�
WHAT WILL I GET IF I RUN THIS TEST?
What is the existing conversion rate? How epic is the scale of the test variables? How consistent is the lift relative to other tests? Would you spend the time on the test at 5%? Would you think 20% unrealistic?
5%? 10%? 20%?
HOW TO CHOOSE?
TWO GOALS: Show opportunity Show range
Upside SummaryProject Check out page redesignGoal Increase percent conversion rate on final page of purchase funnelMonthly Visits 45,000Current Conversion 5.00%Project Cost $45,000
Conversion rate
Change in Coversion
Total Units Sold
Incremental Units Sold
Monthly Incremental Profit
12-Month Incremental Profit 12-Month ROI
5.00% 2,250 - - -5.05% 0.05% 2,273 23 $2,813 $33,750 (0.25) 5.10% 0.10% 2,295 45 $5,625 $67,500 0.50 5.15% 0.15% 2,318 68 $8,438 $101,250 1.25 5.20% 0.20% 2,340 90 $11,250 $135,000 2.00 5.25% 0.25% 2,363 113 $14,063 $168,750 2.75 5.30% 0.30% 2,385 135 $16,875 $202,500 3.50 5.35% 0.35% 2,408 158 $19,688 $236,250 4.25 5.40% 0.40% 2,430 180 $22,500 $270,000 5.00 5.45% 0.45% 2,453 203 $25,313 $303,750 5.75 5.50% 0.50% 2,475 225 $28,125 $337,500 6.50 5.55% 0.55% 2,498 248 $30,937 $371,250 7.25 5.60% 0.60% 2,520 270 $33,750 $405,000 8.00 5.65% 0.65% 2,543 293 $36,562 $438,750 8.75 5.70% 0.70% 2,565 315 $39,375 $472,500 9.50 5.75% 0.75% 2,588 338 $42,188 $506,250 10.25 5.80% 0.80% 2,610 360 $45,000 $540,000 11.00 5.85% 0.85% 2,633 383 $47,813 $573,750 11.75 5.90% 0.90% 2,655 405 $50,625 $607,500 12.50
Expected Range of Values Expected Case
WHAT WILL I GET IF I RUN THIS TEST?
Follow the goals Follow the revenue
Test Plan/Roadmap
PLANNING & PRIORITIZATION
= *9�3*05-&�7'�25/�4*+3�4'34�(+234�>
In addition to ROI/lift Consider implementation risks: :Ease of technical solution : Time/production costs :Measurability of results
CONSIDER
PLANNING & PRIORITIZATION
EXECUTE
AVOID CONSIDER
RE
TUR
N O
N IN
VE
STM
EN
T (R
OI)
EASE OF IMPLEMENTATION
WEB OPTIMIZATION TEST OR REDESIGN
Test Plan Analyze
Test Plan Analyze
Test Plan Analyze
Test Plan Analyze
Test Plan Analyze
Test Plan Analyze
Test Plan Analyze
Test Plan Analyze
����������#��?����� ���� � ���������������+.1-+(+'&�
Fix Replace
OPTIMIZATION OR REDESIGN
THE THEORY OF THE LOCAL MAXIMUM:
�*'�-0%$-�.$8+.5.�+3�$�10+/4�+/�7*+%*�905?6'�*+4�4*'�-+.+4�0(�4*'�%522'/4�&'3+)/ It is as effective as its ever going to be on its current structural foundation. Even if you make 100 tweaks you can only get so much improvement.
OPTIMIZATION OR REDESIGN
������ "���������� �������?�� ASSUMING A REDESIGN FIRST IS EXPENSIVE LOOK FOR STAGNANT OR DECLINING MEASUREMENT OVER TIME. USER RESEARCH RADICAL MODULAR OPTIMIZATION FAILS TO MOVE THE NEEDLE. CIRCUMSTANTIAL EVIDENCE FROM COMPETITORS.
�� ������� ���� �?�����������������!�
WEB OPTIMIZATION MOVING BEYOND A ONE-SIZE FITS ALL EXPERIENCE PROVIDES OPPORUNITIES FOR CREATIVE
ARE YOUR BEST IDEAS BEING DISCARDED?
�������!����������=����">������
EXISTING CONTENT
ALTERNATE CONTENT
90% 10%
IS IT REALISTIC TO ANTICIPATE THE PREFERENCES OF HUNDREDS OF THOUSANDS OF INDIVIDUALS?
START BY ANALYZING RESULTS BY SEGMENT
: GOALS FIRST, TESTS SECOND
: FOSTER AN OBSESSION WITH OUTCOMES
: PREPARE TO BE SURPRISED WHEN YOUR CUSTOMERS CHOOSE THEIR FAVOURITES
: EXPERIMENTATION HAS A REVOLUTIONARY IMPACT ON MARKETING STRATEGIES
: SEGMENTED TESTS ARE SMARTER TESTS
: MAKE IT A PROGRAM, NOT A PROJECT
: DEMONSTRATE HOW INSIGHTS ARE BEING INCORPORATED THROUGHOUT ������?�������������������
BUILDING A CULTURE OF TESTING AND MEASUREMENT
WEB OPTIMIZATION OPEN QUESTION/DISCUSSIONS?
PETER ERNST Associate Director of Optimization Possible [email protected]
THANK YOU