Deep Learning-based Recommendations forGermany’s Biggest Online Vehicle Marketplace

Bigdata.AI Summit, Hanau, March 1, 2018Florian Wilhelm, Arnab Dutta

2

Introduction

Marcel KurovskiData Scientistinovex GmbH

� @FlorianWilhelm

� FlorianWilhelm

florianwilhelm.info

Dr. Arnab DuttaData Scientistmobile.de GmbH� @kopfhohen

� kraktoso

Dr. Florian WilhelmData Scientistinovex GmbH

� squall-1002

3

Outline

§Introduction§Use-cases§Traditional§Deep Learning§Results

4

MOBILE.DEGERMAN MARKET LEADER

13.5 MIO UNIQUE USER PER MONTH

1.6 MIO VEHICLES

290EMPLOYEES

DREILINDEN / FRIEDRICHSHAIN BERLINHEADQUARTERS

Part ofebay Tech

5

IT-project house for digital transformation:‣ Agile Development & Management‣ Web · UI/UX · Replatforming · Microservices‣ Mobile · Apps · Smart Devices · Robotics‣ Big Data & Business Intelligence Platforms‣ Data Science · Data Products · Search · Deep Learning‣ Data Center Automation · DevOps · Cloud · Hosting‣ Trainings & Coachings

Using technology to inspire our clients. And ourselves.

inovex offices inKarlsruhe · Pforzheim · Köln München · Hamburg · Stuttgart.

www.inovex.de

6

Outline

§Introduction§Use-cases§Traditional§Deep Learning§Results

7

Why Recommendations?Why Recommendations?

Show width of offering

Inspiration

Engagement

8

- - engagement- - inspiration- - relevance

Why Recommendations?

- - high click-through-rate - - small exit- & bounce-rates

User Benefits

Business Benefits

9

X X XX

Mobile.de Conversion Funnel

WishlistHome Search Result Page

View Contact Buy

X

10

Recommendations on Home

Home

Recommendations based on preferences of visiting users as an alternative entry point.

WishlistHome SRP View Contact Buy

11

Recommendations on View Item Page

VIP

Recommendations based on the specific make and model a user is viewing to present alternatives

WishlistHome SRP View Contact Buy

12

Recommendations on your Wishlist

Recommendations based on the specific make and model of a deleted ad to provide almost identical recommendations

Recommendations based on theusers car preferences and the parking lot items.

WishlistHome SRP View Contact Buy

13

§Introduction§Use-cases§Traditional§Deep Learning§Results

14

Collaborative Filtering

favourited or viewed

rated highly

also favourited or viewedrecommend

items similar

15

Summary of Collaborative FilteringüCollective behaviour of users

üStandard-Method (it works, it’s reliable etc.)

x Cold Start Problem: New listings need a certain number of clicks to be recommended.

x Sparsity problems: lot fewer interactiondata points than total items and users.

x Content agnostic

xOnly “batch-based” learning

16

Looking For: Used Car (100%)

Prefers (Make): BMW (50%), Audi (50%)Prefers (Model): Audi A3 (25%), Audi A4 (25%),

BMW 318 (50%)Searching In: lat 52.5206, lon 13.409Search Radius: 300kmPreferred Price: 20 000€ ± 1500€ Preferred Mileage: 10 000km ± 5000km

User Preferences

Anonymous

Content-based Filtering: User Preferences

17

Content-based Filtering

interacted

<Price: 10K, Category: small>

<Price: 6K, Category: small>

<Price: 90K, Category: sports>

less similar

<Price: 10K, Category: small>

similar

recommend

18

Summary of Content-basedüWorks even if there are no otherusers

ücontent-based preferences ofusers based on a weighted vectorof item features

xHard to do recommendations fornew users (cold start problem)

xNon-applicable for heterogenouscontent types

x Low diversity, i.e. more of the same

19

Traditional Hybrid Recommender

Collaborative Filtering

Hybrid Recommender

Contentbased

PPP P

P

Looking For: Used Car (100%)

Prefers (Make): BMW (50%), Audi (50%)Prefers (Model): Audi A3 (25%), Audi A4 (25%),

BMW 318 (50%)

Searching In: lat 52.5206, lon 13.409Search Radius: 300kmPreferred Price: 20 000€ ± 1500€Preferred Mileage: 10 000km ± 5000km

User Profile

BuyerLast Action: YesterdayFrequent User

User 12345

Likelihood to buy: 88 %

Elastic Search Query

ü based on ES and Mahoutü comprehensible and debuggableü robust and reliable conceptsü easy to tune for different use-casesx incapable of capturing inherent non-

linear feature dependenciesx lots of manual feature engineering

20

§Introduction§Use-cases§Traditional§Deep Learning§Results

21

Deep Learning

„[...] reported a 29% sales increase to$12.83 billion [...]“

Deep Learning Success StoriesReasons for Deep Learning

• captures nonlinear relations

• holistic approach

• less feature engineering

• improved quality

Search

Recommendations

22

Find the car that perfectly fits your life

User’s Car Preferences Car Pool + Attributes(make, model, color, price, …)

Flexible(cold-start, uncertainty, real-time, ...)

Interactions of other users(views, parkings, contacts)

23

Recommendation Task

Deep Black Box

Mobile.de Listings

1:

2:

4:

3:

5:

PersonalizedRanking

„Which car would the user most likely consider next?“

24

Deep Learning Network

mileagepricecolor

history

mileagepricecolor

views

...

0.38

0.250.79

...0

10

...0

0.20.8

...

...

scal

ing

enco

desc

alin

gen

code

...

0.35

-0.152.03

cont.

cat.

cont.

cat.

Outp

ut Probability that user likes vehicle

user

embe

ddin

gsite

m e

mbe

ddin

gsdeep component

RankNet

ItemNet

UserNet

25

Deep Learning Recommender - Architecture

ad storage

embeddings

RankNet

UserNet

ItemNet

CandidateGeneration

ANN Index

Candidate ServiceRanking Service

Web Service

User Preference APIRecommendation Service

26

Technology Stack

Annoy ANN by Spotify

Hardware� GPU-Server

� NVIDIA Tesla K80� 4x Intel Xeon 3.5 GHz� 64GB RAM,

850GB Disk

LightFMby Lyst

27

§Introduction§Motivation & Use-cases§Theory§Deep Learning§Results

28

Improvements by Deep Learning

0,25%

0,35%

0,45%

0,55%

0,65%

0,75%

0,85%

0,95%

1,05%

1,15%

k = 1 k = 5 k = 10 k = 30 k = 100

MAP@

k

Collaborative FilteringTraditional HybridDeep Recommender

+73%+143%

29

Conclusion

Deep Learning

Flexibility

HybridAbstraction

30

Recommendations support users to find the perfect vehicle based on their preferences and similarities to other users

31

Any questions?lusion