[500DISTRO] The Scientific Method: How to Design & Track Viral Growth Experiments

BUILDING A GROWTH MACHINE THE METHOD BEHIND THE MACHINE

!

Brian Balfour :: @bbalfour :: http://www.coelevate.com

http://www.Coelevate.com

WANT TO KNOW THE #1 QUESTION I’M ASKED ABOUT GROWTH?



WHAT TACTIC WOULD YOU USE TO GROW MY COMPANY?



4Brian Balfour :: @bbalfour :: http://www.coelevate.com


ONE…TIP…

SECRET…HACK…

TACTIC…Brian Balfour :: @bbalfour :: http://www.coelevate.com




THE ANSWER TO…

IS NOT WHAT MOST THINK IT IS

WHAT TACTIC WOULD YOU USE TO GROW MY COMPANY?



IT HAS NOTHING TO DO WITH TACTICS



IT HAS EVERYTHING TO DO WITH PROCESS



PROCESS?!?!?!WHAT CAN BE SO SPECIAL ABOUT PROCESS?



4 REASONS TO FOCUS ONPROCESS FIRST, TACTICS SECOND



12

1. WHAT WORKS FOR OTHERS, WON’T WORK FOR YOU



YOUR AUDIENCE IS DIFFERENT



YOUR PRODUCT IS DIFFERENT



YOUR BUSINESS MODEL IS DIFFERENT



YOUR CUSTOMER JOURNEY IS DIFFERENT



YOUR BUSINESS IS DIFFERENT PLAIN AND SIMPLE



18

2. GROWTH = SUM(1, 2, 3, 4….X)Brian Balfour :: @bbalfour :: http://www.coelevate.com


What happened here?



20

SILVER BULLETS DON’T EXIST



21

3. RATE OF CHANGE IS ACCELERATINGBrian Balfour :: @bbalfour :: http://www.coelevate.com


SEO No Keyword Data, Author Rank, Content Freshness, Social Shares, Search Result Format

DISPLAY RTB, No Cookies, Mobile

EMAIL Tabbed Inboxes, Inbox Actions, Unsubscribe, Mobile

FACEBOOKNewsfeed Ads (4 formats), Video Ads,

Optimized CPM, CTA Format, Custom Audiences, etc, etc

CHART CREDIT: JAMES CURRIER



24

4. YOU NEED A MACHINEBrian Balfour :: @bbalfour :: http://www.coelevate.com


A MACHINE IS…

1

2 PREDICTABLE

SCALABLE

3 REPEATABLEBrian Balfour :: @bbalfour :: http://www.coelevate.com


THE MACHINE PRODUCES THE TACTICS THE PROCESS MAKES THE MACHINE



SO LET’S SEE THE PROCESS…



THE GOALS OF THE PROCESS

1 RHYTHM Momentum is powerful. Establish a cadence to fight through failures, get to successes, and find momentum

2 LEARNING Constant learning of our customer, product, channels and feeding that learning into the process to improve.

3 AUTONOMY Individuals decide what they work on to achieve the team goals.

4 ACCOUNTABILITY With autonomy, comes accountability. You don’t have to be right all the time, but there is an expectation to improve.



THE FIRST STEP: KNOW WHERE WE ARE GOING



HOW WE SET GOALS

OBJECTIVE: Qualitative statement !

TIMEFRAME: 30 - 90 Days !

KR1: Measurable Goal 1 (Hit 90% Of Time) KR2: Measurable Goal 2 (Hit 50% Of Time) KR3: Measurable Goal 3 (Hit 10% Of Time) !



ONCE WE KNOW WHERE WE ARE GOING WE HIT THE ROAD AS FAST AS POSSIBLE



6. Systemize (Playbooks)

1. Brainstorm (Backlog)

2. Prioritize (Experiment Doc + Backlog)

3. Test (Experiment Doc)

4. Implement

5. Analyze (Experiment Doc)



FOUR KEY DOCUMENTS

1 BACKLOG The backlog of experiment ideas.

2 PIPELINE Experiments that were ran or are currently running and the highlights of their results.

3 EXPERIMENT DOC Objective, Hypothesis, Experiment Design, Results, Analysis, and Learnings

4 PLAYBOOKS Step by step guides for successful experiments that should be repeated.



STEP ONE: BRAINSTORM

BRAINSTORM ON THE INPUTS NOT THE OUTPUTS…



STEP ONE: BRAINSTORM

ACTIVATION RATE = STEP 1 CONV * STEP 2 CONV * STEP 3 CONV

IF WE WANT TO IMPROVE THIS

WE BRAINSTORM ON THESE



4 WAYS TO GENERATE GROWTH IDEAS (Read The Innovators Solution)

1 OBSERVE How are others doing it? Look outside of your immediate product space. Walk through together.

2 QUESTION Question brainstorming. Why? What is… What if… What about… How do we do more of…

3 ASSOCIATE Connect the dots between unrelated things. i.e What if our activation process was like closing a deal?

4 NETWORK Growth mastermind groups. Build your network of growth people. I’m on the phone every day for ideas

STEP ONE: BRAINSTORM CREATE A BACKLOG



STEP TWO: PRIORITIZE

1

2 IMPACT

PROBABILITY

3 RESOURCES

LOW - 20%, MEDIUM - 50%, HIGH - 80%

Comes from your hypothesis. Take into account long lasting effects vs one hit wonders.

Marketing, Design, Engineering 1 Hour, 1/2 Day, 1 Day, 1 Week



STEP TWO: PRIORITIZE CREATING A HYPOTHESIS

IF SUCCESSFUL, [VARIABLE] WILL INCREASE BY [IMPACT], BECAUSE

[ASSUMPTIONS].



STEP TWO: PRIORITIZE JUSTIFYING YOUR ASSUMPTIONS


1

2 QUALITATIVE

QUANTITATIVE

3 SECONDARY

Previous Experiments, Surrounding Data, Funnel Data

Surveys, Support Emails, User Testing Recordings

Networking, Blogs, Competitor Observation, Case Studies


STEP TWO: PRIORITIZE CREATE EXPERIMENT DOC



STEP THREE: TEST

WHAT IS THE MVT (MINIMUM VIABLE TEST)

TO UNDERSTAND OUR HYPOTHESIS?



STEP THREE: TEST WRITE THE EXPERIMENT DESIGN IN EXPERIMENT DOC



STEP FOUR: IMPLEMENT

GET SHIT DONE



STEP FIVE: ANALYZE

1

2 ACCURACY

IMPACT

3 WHY?

How close to your hypothesis?

The most important question you can ask. Why did you see the result that you did?

What were the results of the experiment?



STEP FIVE: ANALYZE LEARNINGS + ACTION ITEMS IN EXPERIMENT DOC



STEP SIX: SYSTEMIZE

1 PRODUCTIZE Productize as much as you can with technology and engineering.

2 PLAYBOOKS For the things you can’t productize, build into step by step playbooks to make them repeatable.



6. Systemize (Playbooks)

1. Brainstorm (Backlog)

2. Prioritize (Experiment Doc + Backlog)

3. Test (Experiment Doc)

4. Implement

5. Analyze (Experiment Doc)

REPEAT



WHAT A TYPICAL WEEK LOOKS LIKE

Monday Tuesday Wednesday Thursday Friday

Growth Meeting - Personal Learnings - Goal Review - Experiment Review - This Week’s

Experiments

Predicting, Prioritizing, Implementing, Analyzing, Systemizing



ZOOM OUT (set OKR’s) ZOOM IN (4 - 12 weeks)

ZOOM OUT ZOOM IN

ZOOM OUT….



MACRO OPTIMIZATION QUARTERLY REVIEW

1

2 ACCURACY

BATTING AVG

3 THROUGHPUT

How many successes to failures? Is it improving over time?

Are your hypotheses getting more accurate?

How many experiments are you running in a given time period? How do you do more?



THANKS!

MORE ON MY SITE: HTTP://WWW.COELEVATE.COM


BRIAN BALFOUR VP Growth @ HubSpot Previously EIR @ Trinity Ventures, Co-Founder @ Boundless, Co-Founder @ Viximo (acq by Tapjoy)

http://WWW.COELEVATE.COM


[500DISTRO] The Scientific Method: How to Design & Track Viral Growth Experiments

machine brian

process brian

brian balfour

experiment

experiment

machine

process

tactics

Text of [500DISTRO] The Scientific Method: How to Design & Track Viral Growth Experiments

[#500Distro] Heating Up the Open Rate: New School Rules for Email Marketing

[500DISTRO] Making Business Go BOOM: The 5 W's of B2B Content Marketing

[500DISTRO] Sharing Lab Notes: 27 Referral Program Hack-tics in 20 Minutes

[#500Distro] The Billion Dollar Question

[#500Distro] Measuring for Impact: Knowing When, What & How to A/B Test

Viral Viral

[500DISTRO] The Only Metric That Matters

RANDOM KNOTTING AND VIRAL DNA PACKING: THEORY AND EXPERIMENTS De Witt Sumners Department of Mathematics Florida State University Tallahassee, FL 32306

[500DISTRO] The Scientific Method: How to Design & Track Viral Growth Experiments

Viral mortality of the marine alga Emiliania huxleyi ...Bratbak et al.: Viral mortality of Emiliania huxleyi 41 Table 2. Summary of results from the mesocosm experiments showing the

Bronquiolíte viral viral aguda

Morfologia Viral e Replicação viral

[500DISTRO] Do You Speak Growth? Examining the Language Behind What Users Want

[500DISTRO] Cracking the SEO Code: Tricks & Tactics To Magnify Search Visibility

[500DISTRO] Dissecting UX: Funnel Fixes & Other CRO Tips for Growth That Pays

[#500Distro] Designing a Killer First-Run Experience: Exploring the Pros & Cons of Wizards, Guided Tours & More

[500DISTRO] Shake, Mix, Stir, KABAM! How One Startup Grew from $0 to $360MM in 4 Years

[500DISTRO] Going for Global: 5 Guerrilla Tactics When the Slick Stuff Fails

Viral - cl.cam.ac.ukpv273/slides/VIRAL-Presentation.pdf · Uvod Teorija Viral arhitektura Zaključci Viral Realnesimulacijekonkurentnihprocesanadmultipleksmrežama PetarVeličković,AndrejIvašković

[500DISTRO] Measuring for Engagement: Understanding User Gains, Losses & Levels of Interaction

[500DISTRO] Under the Microscope: How Airbnb Thinks About Product/Market Fit, Team & More

[500DISTRO] Converting with Facebook: How to Optimize Ads, Target Audiences & Boost Your ROI

[500DISTRO] Video Virology 101: How to Produce, Test & Distribute Content that Connects

* FIEL EXPERIMENTS AND MODELING OF D VIRAL TRANSPOR ITN GROUNDWATER · 2012-05-22 · * FIEL EXPERIMENTS AND MODELING OF D VIRAL TRANSPOR ITN GROUNDWATER J ... 1994 Mold; z et al.,

[500DISTRO] Catalyzing with Content: What B2Cs Should Be Doing in the First 6 Months

[500DISTRO] Measuring on Mobile: Making the Most of the Signup Funnel & Other Analytic Hot Spots

Presentations tips viral viral

[500DISTRO] The Early Stage Growth Playbook: Testing Features, Debunking "Best Practices," & Rethinking the Org Chart

A Viral Metagenomics Study of Honey Bee Colony Collapse ... slides_Colony Collapse Disorder... · Pathology (agent distribution) Challenge experiments (reproduce disease) 454 Genome

[500DISTRO] Mix It Up: How to Leverage Product to Discover New Growth Channels

Internet

[500DISTRO] The Scientific Method: How to Design & Track Viral Growth Experiments

Text of [500DISTRO] The Scientific Method: How to Design & Track Viral Growth Experiments