Buzz Words Dunning Multi Modal Recommendations

1©MapR Technologies - Confidential

Multi-Modal Recommendations

Multiple Kinds of Behavior for Recommending

Multiple Kinds of Things

What’s Up

What is this multi-modal stuff? A simple recommendation architecture Some scary math Putting it into a deployable architecture Final thoughts

Contact:– tdunning@maprtech.com– @ted_dunning– @apachemahout– @user-subscribe@mahout.apache.org

Slides and such (available late tonight):– http://www.slideshare.net/tdunning

Hash tags: #bbuzz #mapr #recommendations

Recommendations

Often known (inaccurately) as collaborative filtering Actors interact with items– observe successful interaction

We want to suggest additional successful interactions Observations inherently very sparse

Examples of Recommendations

Customers buying books (Linden et al) Web visitors rating music (Shardanand and Maes) or movies (Riedl,

et al), (Netflix) Internet radio listeners not skipping songs (Musicmatch) Internet video watchers watching >30 s (Veoh)

What is this multi-modal stuff?

But people don’t just do one thing

One kind of behavior is useful for predicting other kinds

Having a complete picture is important for accuracy

What has the user said, viewed, clicked, closed, bought lately?

A simple recommendation architecture

Look at the history of interactions

Find significant item cooccurrence in user histories

Use these cooccurring items as “indicators”

For all indicators in user history, add up scores

Recommendation Basics

History:

User Thing1 3

History as matrix:

(t1, t3) cooccur 2 times, (t1, t4) once, (t2, t4) once, (t3, t4) once

t1 t2 t3 t4

u1 1 0 1 0

u2 1 0 1 1

u3 0 1 0 1

A Quick Simplification

Users who do h

Also do r

User-centric recommendations

Item-centric recommendations

Coocurrence

t1 t2 t3 t4

t1 2 0 2 1

t2 0 1 0 1

t3 2 0 1 1

t4 1 1 1 2

Problems with Raw Cooccurrence

Very popular items co-occur with everything– Welcome document– Elevator music

That isn’t interesting– We want anomalous cooccurrence

Coocurrence

t1 t2 t3 t4

t1 2 0 2 1

t2 0 1 0 1

t3 2 0 1 1

t4 1 1 1 2t3 not t3

t1 2 1

not t1 1 1

Spot the Anomaly

Root LLR is roughly like standard deviations

A not A

B 13 1000

not B 1000 100,000

A not A

not B 0 2

A not A

not B 0 10,000

A not A

B 10 0

not B 0 100,000

0.44 0.98

2.26 7.15

Root LLR Details

In Rentropy = function(k) { -sum(k*log((k==0)+(k/sum(k))))}rootLLr = function(k) { sign = … sign * sqrt( (entropy(rowSums(k))+entropy(colSums(k)) - entropy(k))/2)}

Like sqrt(mutual information * N/2)See http://bit.ly/16DvLVK

Threshold by Score

Coocurrence

t1 t2 t3 t4

t1 2 0 2 1

t2 0 1 0 1

t3 2 0 1 1

t4 1 1 1 2

Threshold by Score

Significant cooccurrence => Indicators

t1 t2 t3 t4

t1 1 0 0 1t2 0 1 0 1t3 0 0 1 1t4 1 0 0 1

So Far, So Good

Classic recommendation systems based on these approaches– Musicmatch (ca 2000)– Veoh Networks (ca 2005)

Currently available in Mahout– See RowSimilarityJob

Very simple to deploy– Compute indicators– Store in search engine– Works very well with enough data

What’s right about this?

Virtues of Current State of the Art

Lots of well publicized history– Musicmatch, Veoh, Netflix, Amazon, Overstock

Lots of support– Mahout, commercial offerings like Myrrix

Lots of existing code– Mahout, commercial codes

Proven track record Well socialized solution

What’s wrong about this?

Too Limited

People do more than one kind of thing Different kinds of behaviors give different quality, quantity and

kind of information

We don’t have to do co-occurrence We can do cross-occurrence

Result is cross-recommendation

Symmetry Gives Cross Recommentations

Why just dyadic learning?

Why not triadic learning?Why not cross learning?

For example

Users enter queries (A)– (actor = user, item=query)

Users view videos (B)– (actor = user, item=video)

A’A gives query recommendation– “did you mean to ask for”

B’B gives video recommendation– “you might like these videos”

The punch-line

B’A recommends videos in response to a query– (isn’t that a search engine?)– (not quite, it doesn’t look at content or meta-data)

Real-life example

Query: “Paco de Lucia” Conventional meta-data search results:– “hombres del paco” times 400– not much else

Recommendation based search:– Flamenco guitar and dancers– Spanish and classical guitar– Van Halen doing a classical/flamenco riff

Real-life example

Hypothetical Example

Want a navigational ontology? Just put labels on a web page with traffic– This gives A = users x label clicks

Remember viewing history– This gives B = users x items

Cross recommend– B’A = label to item mapping

After several users click, results are whatever users think they should be

Nice. But we can do better?

things

thingtype 1

thingtype 2

action1item type1

action2item type2

Summary

Input: Multiple kinds of behavior on one set of things

Output: Recommendations for one kind of behavior with a different set of things

Cross recommendation is a special case

Now again, without the scary math

Input Data

User transactions– user id, merchant id– SIC code, amount– Descriptions, cuisine, …

Offer transactions– user id, offer id– vendor id, merchant id’s, – offers, views, accepts

Input Data

User transactions– user id, merchant id– SIC code, amount– Descriptions, cuisine, …

Offer transactions– user id, offer id– vendor id, merchant id’s, – offers, views, accepts

Derived user data– merchant id’s– anomalous descriptor terms– offer & vendor id’s

Derived merchant data– local top40– SIC code– vendor code– amount distribution

Cross-recommendation

Per merchant indicators– merchant id’s– chain id’s– SIC codes– indicator terms from text– offer vendor id’s

Computed by finding anomalous (indicator => merchant) rates

Search-based Recommendations

Sample document– Merchant Id– Field for text description– Phone– Address– Location

– Indicator merchant id’s– Indicator industry (SIC) id’s– Indicator offers– Indicator text– Local top40

Sample query– Current location– Recent merchant descriptions– Recent merchant id’s– Recent SIC codes– Recent accepted offers– Local top40

SolRIndexerSolR

IndexerSolrindexing

Cooccurrence(Mahout)

Item meta-data

Indexshards

Complete history

SolRIndexerSolR

IndexerSolrsearchWeb tier

Item meta-data

Indexshards

User history

Contact:– tdunning@maprtech.com– @ted_dunning– @apachemahout– @user-subscribe@mahout.apache.org

Slides and such (available late tonight):– http://www.slideshare.net/tdunning

Hash tags: #bbuzz #mapr #recommendations

We are hiring!

Objective Results

At a very large credit card company

History is all transactions, all web interaction

Processing time cut from 20 hours per day to 3

Recommendation engine load time decreased from 8 hours to 3 minutes

Recommendation quality increased visibly

Thank You

Buzz Words Dunning Multi Modal Recommendations

Technology

Dunning 95

Dunning time-series-2015

Billing Dunning Paying

Oscon data-2011-ted-dunning

Dunning ml-conf-2014

Sian Dunning Portfolio

SAP IS-Utilities: Dunning Configuration and … IS-Utilities: Dunning Configuration and Process Steps SAP COMMUNITY NETWORK scn.sap.com © 2012 SAP AG 3 Introduction Dunning is …

Kruger Dunning

Dunning Voir Dire

10 Slide PowerPoint MaryBeth Dunning

Oman's Ability to Attract FDI: Dunning Instrument Survey ...Dunning (2014), Cantemir (2012), Forstenlechner and Rutledge (2011), Faeth (2009), James (2009), Lim (2008), Dunning (2008)

Graphlab Ted Dunning Clustering

Berlin Buzz Words - Apache Drill by Ted Dunning & Michael Hausenblas

Strathmuir, Dunning, PH2 0QQStrathmuir, Dunning, PH2 0QQ 01738 443456 | Situated within the picturesque Strathearn Valley on the outskirts of Dunning, is …

Defcon 18-dunning-breaking-bluetooth

Oscon Data 2011 Ted Dunning

Dunning welsh-newspapers-130314110640-phpapp01

Eclectic paradigm Dunning

Overconfidence Among Beginners: Is a Little Learning a ... · Dunning-Kruger effect (Dunning, 2011; Kruger & Dunning, 1999), a situation in which people are so unskilled they lack

Dunning, Thad - Crude Democracy