#mobileproduct

Make Impactful Changes, Every Time

Using Data to Hone Your Mobile Product Focus

#mobileproduct

Poll• What’s your most important KPI(s) on mobile right

now?A) DownloadsB) DAUs/MAUsC) LTV of UsersD) VisitsE) RatingsF) Other: Type in Question Box

#mobileproduct

Who Are We?• Kendrick Wang • Emily Carrion

Product Marketing @ Apptimize Head of Marketing @Apptentive

#mobileproduct

Fast, intelligent mobile iterations, without disrupting the user experience.

• Mobile A/B Testing• Instant Update• Feature Flagging• Staged Rollouts

Apptimize

#mobileproduct

ApptentiveMobile customer engagement software to help enterprise brands listen to, engage with and retain their mobile customers.

• Make your brand accessible• Proactively engage at smart times• Make data driven decisions

#mobileproduct

Agenda• You’ll Learn How To:– Make smart product decisions– Get qualitative and quantitative data before

deploying to all your users– Validate their effects– Make the most of limited resources

#mobileproduct

You’re Always Juggling Priorities

• Only have so many:– Engineers– Sprints

• Not everything can be on the roadmap

#mobileproduct

Everything Has a Cost• There are always opportunity costs• Lots of pressure on product decisions

#mobileproduct

You Have to Learn From All Teams

#mobileproduct

And Everyone Has An Input“I have this idea…”

“What if we…”“Our competitors have it...”

“Loads of users want to...”“The next big thing...”

“It’ll only take a few minutes...”

#mobileproduct

Have you every had an executive tell you to build a feature that you didn’t think your users actually wanted?

#mobileproduct

Customer Feedback Justifies Product Roadmap

PROBLEM• Unjustified feature request led

to unhappy customers & 1-star ratings

SOLUTION• Implemented Apptentive in-app

surveys

#mobileproduct

Customer Feedback Justifies Product Roadmap

RESULTS• Within a week, collected 10,000

responses• Used customer data to back up

product roadmap, and build better app

• Rewarded with high volume of 5-star ratings, and reduced churn

#mobileproduct

Question:• What other indicators or KPIs do you

use to determine what gets on the product roadmap?

(use the question box)

#mobileproduct

The Key Is In Data• How can we be more deliberate

about it?– How do we :• Determine the right data to look at?• Prioritize the right data?• Make sure the changes actually work?

#mobileproduct

What Do Most PMs Care About?

#mobileproduct

Bonus: 16 Mobile Metrics that Matter http://bit.ly/AdvMobilePM

#mobileproduct

Understand How Users Behave

Qualitative Quantitative

Observation User FeedbackUser Testing

Analytics

Testing User Testing A/B Testing

#mobileproduct

Conversion Funnels• Where are the drop-

off points?• When are users not

behaving how you expect/want?

#mobileproduct

Conversion Funnels

#mobileproduct

User Testing• Hire testers to observe behavior• Identify:– Confusion– Drop off points– Unexpected behaviors

• Debrief + follow up

#mobileproduct

What if you could ask your actual customers for feedback directly in-app?

#mobileproduct

Customer Feedback – Your Secret Weapon

#mobileproduct

Use Customer Feedback to Release a New Product

#mobileproduct

Using these techniques helps you narrow your focus

#mobileproduct

But you want to know a change’s effects before you deploy to your

entire userbase

(There are no take backs in mobile)

#mobileproduct

Why can’t I browse titles before I sign up?

#mobileproduct

#mobileproduct

So They Created a Hypothesis

#mobileproduct

Determine New User Behavior

#mobileproduct

Netflix Findings• You need to validate your ideas• Back up your claims with data

#mobileproduct

‘s

Q: “What types of things does Netflix A/B Test?”

“Short answer – almost everything” –Neil Hunt, Chief Product Officer @ Netflix

#mobileproduct

Deploy Without Disrupting• Push winners out instantly• Roll back changes instantly with feature

flagging

#mobileproduct

Most Effective Way to Test• Be like Netflix• Use A/B Testing– Draws direct cause-effect relationship– Gives you hard numbers– Reveals assumptions

#mobileproduct

Understand How Users Behave

Qualitative Quantitative

Observation User FeedbackUser Testing

Analytics

Testing User Testing A/B Testing

#mobileproduct

SummaryCustomer Feedback + A|B Testing

= Better Product

#mobileproduct

#CustomerLove

#mobileproduct

Questions?