SPEAKER

What we have learnt @Vinted

Experiments

Tautvydas Gylys

Experiments - scientific approach

Observe > Hypothesis > Test > Prove / Reject

photo:

2 types of experimentation

Conversion optimisation Hypothesis driven development

To improve smth To create smth new

Will be talking about today

How do we experiment @Vinted?

case study: Conversion to lister optimisation

case study: Conversion to lister optimisation

case study: Conversion to lister optimisation

2. Ideas and assumptionsData analysis

2. Ideas and assumptionsInterviews

Data analysis

2. Ideas and assumptionsInterviews

Data analysisBrainstorm

case study: Conversion to lister optimisation

case study: Conversion to lister optimisation

4. Experiment design

4. Experiment design

case study: Conversion to lister optimisation

7. Analysis and documentationDashboard v1

Dashboard v2

7. Analysis and documentation

results: Listers flow optimisation~30 ideas

7 experiments

2 implemented

Learning!

Required registration example

Conversion to lister: -

Enter product > Register > Listing form

Conversion to lister:

Enter product > Listing form > Register on upload

Conversion to lister:

Register > Enter product > Listing form

Conversion to lister:

Required registration V2

Required registration V2

Conversion to lister:

Take a way: in apps members are willing to invest time upfront to register. That way they feel invested and explore the product with more attention.

Lessons learned

Idea: change to “Buy now”

Goal: to improve conversion to buyer

Is it a good idea?

Idea: change to “Buy now”

Goal: improve conversion to buyer

Goal to increase conversion by 3%

Goal to increase conversion by 2x

Yes!

No!

Is it a good idea?

Idea: change to “Buy now”

Goal: improve conversion to buyer

Goal to increase conversion by 3%

Goal to increase conversion by 2x

Yes!

No!

Is it a good idea?

Always have measurable goals before designing an experiment.

How many variables?

Control: Not required Test: Required

How many variables?

Control: Not required Test: Required

Picture

Copy

CTA

Screen size

Screen number

ControlModal screen

/ Not required

Test AModal screen

/ Required

Test BFull screen

/ Not required

Test CFull screen

/ Required

ControlModal screen

/ Not required

Test AModal screen

/ Required

Test BFull screen

/ Not required

Test CFull screen

/ Required

Be aware of how many variables you are testing.

Experiment culture changes• Before: Dedicated experiment team (I have an idea, let’s test it!) • Now: Each team runs experiments (I have a problem which potential solution I test)

Experiment culture changes• Before: Dedicated experiment team (I have an idea, lets test it!) • Now: Each team runs experiments (I have a problem which potential solution I test)

• Before: Test everything • Now: Test only big things, trust gut feeling for small

Experiment culture changes• Before: Dedicated experiment team (I have an idea, lets test it!) • Now: Each team runs experiments (I have a problem which potential solution I test)

• Before: Test everything • Now: Test only big things, trust gut feeling for small

• Before: Data only + Gut feeling • Now: Data + Interviews + Gut feeling

Experiment culture changes• Before: Dedicated experiment team (I have an idea, lets test it!) • Now: Each team runs experiments (I have a problem which potential solution I test)

• Before: Test everything • Now: Test only big things, trust gut feeling for small

• Before: Data only + Gut feeling • Now: Data + Interviews + Gut feeling

• Before: Dig deep to learn the impact • Now: Fail fast

Thanks!