© 2003 – 2013 Bigcommerce Pty. Ltd.

Building Products Quantitatively in SaaS

Businesses

Soren Harner, Bigcommerce

Wednesday, 6 November 13

Stage 1 StartupBookstrapped

Stage 2 StartupStage 3 AcquisitionVenture-backed > $80MM

2000 Search & Analytics, Director Engineering

Stage 2 Startup2007 VP Engineering

2010 Founder & Datavore

Stage 2 StartupVenture-backed $35MM

VP Engineering

About me

2012

Wednesday, 6 November 13

Hill Climbing

Wednesday, 6 November 13

SaaS Business Levers• Cost to acquire customers (CAC)

• Average Revenue per User (ARPU)

• Churn

• Goal: Negative revenue retention (upgrades)

• Conversion (CVN) by channel

• Payback

• Life time value (LTV)

• LTV = ARPU/Churn

• LTV / CAC > 1 is success!

• Monthly Recurring Revenue (MRR)

• Magic Number http://blog.asmartbear.com/coc-cancellation-saas-metric.html

Wednesday, 6 November 13

LTV / CAC

0

0.3

0.6

0.9

1.2

Q1-10 Q2-10 Q3-10 Q4-10 Q1-11 Q2-11 Q3-11 Q4-11

Model“successful”

To IPO range is maybe 4 to 6 LTV / CAC today

Fine print: Like all metrics use to track, not as strategy

Wednesday, 6 November 13

Tested hundreds of versions

Wednesday, 6 November 13

Signup, upgrade, onboarding

• Invest in a great experience

• Test with usertesting.com

• Track the funnel:

• Number of trials

• Leads to Trials to Paid by weekly cohort

• Onboarding success

Wednesday, 6 November 13

Product UX Testing

Workshops(personas)

Concepting(sketches)

Prototypes Qualitative(Axure)

Quantitative(Loop11)

You can use an iPad at a Cafe and show keynote.Key point is to test mockups on people first.

Numbers don’t tell you how to improve.

Wednesday, 6 November 13

Feedback

Wednesday, 6 November 13

Measure Product Success

• Customer success

• Time to first sale

• Time to launch

• Trend monthly or weekly cohorts

• Engagement score during trial

• Churn and conversion predictive models

Wednesday, 6 November 13

Feature KPI• Before you build a

feature, define a success KPI

• e.g. Number of product reviews completed

• e.g. 30% fewer abandoned carts

• e.g. completing task more quickly

• Add instrumentation before and after

• Deploy to subset of customers to test

• Consider Mixpanel

Wednesday, 6 November 13

Split Test Important Things

• Persona analysis and customer interviews on new brand

• Route traffic p new, 1-p old (control)

• Measure effect on conversion with 95% confidence interval of a difference and ZScore

Wednesday, 6 November 13

https://developer.amazon.com/sdk/ab-testing/reference/ab-math.html

A/B Testing Report and significance

p +/- (1.65 x SE)

Wednesday, 6 November 13

Network Diffusion

Atlassian JIRA to OS communities.

SEO / SEM limitations — budget and effectiveness.

Wednesday, 6 November 13

Power law

http://www.hpl.hp.com/research/idl/papers/ranking/ranking.html

Think 80/20 rule.

Store revenue, traffic, disk usage, number products, etc.

Wednesday, 6 November 13

Continuous Delivery

• Feature flag everything and gently deploy

• Deploy daily — huge difference!

• Read 12factor.net (dev, staging, production)

• Monitor (i.e. NewRelic)

• Practice BDD & continuous integration

• Always have a recovery plan

Wednesday, 6 November 13

Ask permission, get feedback

Wednesday, 6 November 13

User Stories

Wednesday, 6 November 13

Commutative flow

Wednesday, 6 November 13

Startup Friendly Stack• Use Ruby Sinatra (or Clojure) +

CoffeeScript + Backbone + Handlebars on Heroku

• Nesta for CMS and learn textmate, Haml & Sass, see https://github.com/sharner/Workshop-Site

• HTML5 mobile first, responsive design

• Use Mailchimp, Wufoo, SurveyMonkey

• R for data (http://r-project.org) — treat as code, put in Github

• Log events to understand customer success (see Heroku Plugins)

Wednesday, 6 November 13

Peopleware

• Do learn to code simple website changes and growth hacking

• Learn some stats

• Do find a technical co-founder

• Don’t offshore

• Don’t churn people

• Do find designer who codes frontend and does user testing

Wednesday, 6 November 13