COMPANY NAME www.nicecompany.com

Best practices for building a testing culture

Presented by : Mari Ju and John Oppenheimer Optimizely User Group - August 2015

Product Marketing Manager Mari Ju

About US

mari.ju@autodesk.com linkedin.com/in/mariju @mari4ju

Marketing Operations Analyst John oppenheimer

john.oppenheimer@autodesk.com linkedin.com/in/oppenheimerjohn

about Autodesk

Mission: Help people imagine, design, and create a better world

Est. 1982

12+ million users

Autodesk

Mission: Unlock the creativity in everyone

Est. 2011

170+ million users

Autodesk Consumer and 3d printing

Testing at Autodesk

Spring ‘14 Fall ‘14 Spring ‘15 Fall ‘15

Start small

Build trust

GENERATE Excitement

Advance & REPEAT

Training

•  Onsite demo

•  Team sessions

•  Dedicated champions

Pick a starter property

•  Simple first test as proof of

concept

Phase 1: Start Small

vs.

Original Variation

Results: Variation showed 24% more subscriptions compared to the Original

Address potential concerns:

•  Test proposals to larger team

•  Allow for plenty of QA

•  Connect developers with Optimizely

support team

•  Have developers launch test

•  One test at a time, simple tests only

Phase 2: Build trust

Spread testing throughout the organization: •  Add more products/domains •  Get more team members involved Improve testing program: •  Develop testing workflow:

Data à User Testing à Hypothesis à A/B Test

•  Advanced tests (multi-page, redirects, cross-platform) that leverage integrations

•  Establish testing as essential when updating the site

Share results (both wins and losses): •  Distribute reports via email and internal blog •  Celebrate team wins (Optie finalist)

PhaSE 3: Generate excitement

Continuously advance our testing program: •  Standardized idea backlog and prioritization rubric

•  Create an archive of previous tests that’s easy to share and search

•  Work closely with designers and developers to launch more technical tests

•  Start experimenting with personalization •  Launch mobile app A/B testing program Evangelize benefits of testing:

•  Internally – other products, departments, office locations, and more

•  Externally – with partners and user groups

Phase 4: advance

Original Variation

1 2

3 4

Demonstrate potential impact of testing with a simple test

1. Start small

Increase complexity and frequency of tests for greater impact, and share results

3. Generate excitement

Bring all stakeholders into the process to gain trust, and eventually autonomy

2.Build trust

Continue to evaluate and improve testing program

4. ADVANCE

Steps for building a testing culture

COMPANY NAME www.nicecompany.com

Questions?

Mari Ju – mari.ju@autodesk.com

John Oppenheimer – john.oppenheimer@autodesk.com