EVALUATING YOUR EMAIL BLASTS WITH A/B TESTING
Jason Zanon, Senior Support Specialist
By the conclusion of this training, you will be able to:
• Understand the purpose and primary uses of A/B split testing
• Set up A/B split tests in Salsa• Evaluate the metrics A/B split tests
generate
Learning Objectives
Photo (cc) Yair-haklai at http://tinyurl.com/ccov4aq
Will more people respond to my call to action if I send it
earlier in the morning?
Would a new donate
button perform better?
Which subject line will get recipients to open the message?
A/B TestingSplit Testing
Bucket Testing
How does A/B testing work?
Compare opens, clicks, conversions
The strongest test goes to the full list
A/B Testing: The Salsa Ingredients
Creative Commons image from jordanmit09 http://tinyurl.com/prc3ty9
A/B Testing: The Salsa Ingredients
(This slide fulfills my quota of dubiously overstretched Salsa metaphors.)
A/B Testing: The Salsa Ingredients
Creative Commons image from jordanmit09 http://tinyurl.com/prc3ty9
Steps to A/B Testing in Salsa
• Create and target a main blast just like any email, as the framework for your A/B test
• Split off two test panels targeted to percentages of your main blast’s target pool
• Send those test panels, edited to reflect the desired variation(s) in content
• Send the main blast, with content of the “winning” test version
• Create and target a main blast just like any email, as the framework for your A/B test
• Split off two test panels targeted to percentages of your main blast’s target pool
• Send those test panels, edited to reflect the desired variation(s) in content
• Send the main blast, with content of the “winning” test version
Steps to A/B Testing in Salsa
Begin with a normal, everyday email blast
Steps to A/B Testing in Salsa
• Create and target a main blast just like any email, as the framework for your A/B test
• Split off two test panels targeted to percentages of your main blast’s target pool
• Send those test panels, edited to reflect the desired variation(s) in content
• Send the main blast, with content of the “winning” test version
Use the “A/B Split Test Blasts” button in the very last step …
… then set the percentage sizes of the recipient pool who’ll get test blasts …
… and proceed to introduce the test variable by editing the two test copies.
blast
Steps to A/B Testing in Salsa
• Create and target a main blast just like any email, as the framework for your A/B test
• Split off two test panels targeted to percentages of your main blast’s target pool
• Send those test panels, edited to reflect the desired variation(s) in content
• Send the main blast, with content of the “winning” test version
Submit each test blast
Compare!
Compare!
Steps to A/B Testing in Salsa
• Create and target a main blast just like any email, as the framework for your A/B test
• Split off two test panels targeted to percentages of your main blast’s target pool
• Send those test panels, edited to reflect the desired variation(s) in content
• Send the main blast, with content of the “winning” test version
Swap into the main blast the content of the top test
• Isolate one variable per test• Subject line• Personalization• Salutation• From Name• Day of week, time of day• Images and design elements• Wording/presentation of action buttons and links
• Consider statistical significance• Re-test things you “know”: it’s an ever-changing
environment
A few tips …
CONTACT INFO
Read, learn, discuss – help.salsalabs.com
Training & Learning Team – [email protected]
Salsa Support – [email protected]
THANK YOU!
Recommended
