Download pdf - From hits to niches? ...or how popular artists can bias music recommendation and discovery

2nd netflix workshop // ACM KDD / Las Vegas, US // August, 24th 2008

From Hits to Niches?...or how popular artists can bias

music recommendation and discovery

Òscar Celma, Pedro Cano(Music Technology Group ~ UPF)

(Barcelona Music and Audio Technologies, BMAT)

ACM KDD // 2nd netflix workshop // 2008 // òscar celma & pedro cano

The problem


music overload

• Today(August, 2007)

iTunes: 6M tracks / 3B Sales P2P: 15B tracks 53% buy music on line

• Tomorrow All music will be on line Billions of tracks Millions more arriving every week

• Finding new, relevant music is hard!


Can recommender systems help us?


the Long Tail of popularity

• Help me find it!


If you like The Beatles you might like ...


the musical Turing Test

• Which recommendation is from a human, which is from a machine?

List #2

• Chuck Berry

• Harry Nilsson

• XTC

• Marshall Crenshaw

• Super Furry Animals

• Badfinger

• The Raspberries

• The Flaming Lips

• Jason Faulkner

• Michael Penn

List #1

• Bob Dylan

• Beach Boys

• Billy Joel

• Rolling Stones

• Animals

• Aerosmith

• The Doors

• Simon & Garfunkel

• Crosby, Stills Nash & Young

• Paul Simon


the musical Turing Test

• Which recommendation is from a human, which is from a machine?

List #2

• Chuck Berry

• Harry Nilsson

• XTC

• Marshall Crenshaw

• Super Furry Animals

• Badfinger

• The Raspberries

• The Flaming Lips

• Jason Faulkner

• Michael Penn

List #1

• Bob Dylan

• Beach Boys

• Billy Joel

• Rolling Stones

• Animals

• Aerosmith

• The Doors

• Simon & Garfunkel

• Crosby, Stills Nash & Young

• Paul Simon

Machine:

Up to 11

Human




• popularity bias

• low novelty


Our proposal


Long Tail popularity + item similarity network




Item-to-item Complex Network Analysis


complex network analysis

• Datasets: Artist recommendation networks CF*: Social-based, incl. item-based CF (Last.fm)

“people who listen to X also listen to Y”

CB: Content-based Audio similarity “X and Y sound similar”

EX: Human expert-based (AllMusicGuide) “X similar to (or influenced by) Y”



• Small-world networks

Network traverse in a few clicks



• Last.fm is a scale-free network power law exponent for the cumulative indegree

distribution

A few artists (hubs) control the network



• Indegree – avg. neighbor indegree correlation







Kin(Robert Palmer)=430=>avg(Kin(sim(Robert Palmer)))=342




Kin(Robert Palmer)=430=>avg(Kin(sim(Robert Palmer)))=342

Kin(Ed Alton)=11=>avg(Kin(sim(Ed Alton)))=18



• Last.fm presents assortative mixing Artists with high indegree are connected together,

similarly for low indegree artists r = Pearson correlation




|------------|---------|-----|-----------|

| | Last.fm | CB | Exp (AMG) |

|------------|---------|-----|-----------|

|Small World | yes | yes | yes |

| | | | |

| Scale-free | yes | No | No |

| | | | |

|Ass. mixing | yes | No | No |

|------------|---------|-----|-----------|

• Summary



• But, still some remaining questions...

Are the hubs the most popular artists?

Can we navigate through the Long Tail, via artist similarity?






Item popularity analysis


the Long Tail in music

• last.fm dataset ~260K artists


the Long Tail in music

• last.fm dataset ~260K artists

radiohead (40,762,895)

red hot chili peppers (37,564,100)

muse (30,548,064)

the beatles (50,422,827)

death cab for cutie (29,335,085)pink floyd (28,081,366)coldplay (27,120,352)

metallica (25,749,442)


the Long Tail model [Kilkki, 2007]

• F(x) = Cumulative distribution up to x



• Top-8 artists: F(8)~ 3.5% of total plays

50,422,827 the beatles40,762,895 radiohead37,564,100 red hot chili peppers30,548,064 muse29,335,085 death cab for cutie28,081,366 pink floyd27,120,352 coldplay25,749,442 metallica



• Split the curve in three parts

(82 artists) (6,573 artists) (~254K artists)


Combining


(item-item similarity)

with the Long Tail

(item popularity)




artist indegree vs. artist popularity

• Are the network hubs the most popular items?

???


artist indegree vs. artist popularity Last.fm: correlation between Kin and playcounts


artist indegree vs. artist popularity Audio CB similarity: no correlation


artist indegree vs. artist popularity Expert: correlation between Kin and playcounts


artist similarity vs. artist popularity

• “From Hits to Niches” navigation in the Long Tail using artist similarity

how many clicks?



• “From Hits to Niches” Audio CB similarity example (VIDEO)



• “From Hits to Niches” Audio CB similarity example

Bruce Springsteen (14,433,411 plays)




Bruce Springsteen (14,433,411 plays) The Rolling Stones (27,720,169 plays)




Bruce Springsteen (14,433,411 plays) The Rolling Stones (27,720,169 plays) Mike Shupp (577 plays)



• navigation in the Long Tail Similar artists, given an artist in the HEAD part:

Also, it can be seen as a Markovian Stochastic process...

54,68%

(0%)

45,32%64,74%

28,80%6,46%

60,92%33,26%

5,82%

CF CB EXP

Head Mid Tail Head Mid Tail Head Mid Tail



• navigation in the Long Tail Markov transition matrix



• navigation in the Long Tail Markov transition matrix



• navigation in the Long Tail Last.fm Markov transition matrix



• navigation in the Long Tail From Head to Tail, with P(T|H) > 0.4 Number of clicks needed

CF : 5 CB : 2 EXP: 2 HEAD

TAIL#clicks?


artist popularity

|-----------------------|---------|-----|-----------|


|-----------------------|---------|-----|-----------|

| Indegree / popularity| yes | no | yes |

| | | | |

|Similarity / popularity| yes | no | no |

|-----------------------|---------|-----|-----------|

Summary


conclusions|-----------------------|---------|-----|-----------|


|-----------------------|---------|-----|-----------|

| Small World | yes | yes | yes |

| | | | |

| Scale-free | yes | no | no |

| | | | |

| Ass. mixing | yes | no | no |

|-----------------------|---------|-----|-----------|

| Indegree / popularity| yes | no | yes |

| | | | |

|Similarity / popularity| yes | no | no |

|-----------------------|---------|-----|-----------|

| POPULARITY BIAS | YES | NO | FAIRLY |

|-----------------------|---------|-----|-----------|


future work


future work

• 3D Long Tail item popularity recommendation

network (item-to-item)

user profile


future work



user profile

relevant items


future work



user profile

relevant items

relevant and novel?

...paper @ ACM RecSys'08

2nd netflix workshop // ACM KDD / Las Vegas, US // August, 24th 2008

From Hits to Niches?...or how popular artists can bias music recommendation

and discovery

THANKS!!!

Òscar Celma, Pedro Cano(Music Technology Group ~ UPF)

(Barcelona Music and Audio Technologies, BMAT)