Annotating Music Collections: How Content-Based Similarity Helps to Propagate Tags

SWC Announcement

Annotating Music Collections: How Content-Based Similarity Helps to Propagate Labels

Mohamed Sordo, Cyril Laurier and scar Celma

(Music Technology Group, UPF)

Outline

Introduction

Motivation and goals

Evaluation

Demonstration

Conclusions

introduction

Massive Increase in volume of online music

Long Tail Economics

Make everything available

Help me find it

introduction:: last year's work...

Searchsounds (ISMIR, 2006)

Exploits MP3-blogs info

Natural Lang. Processing

Audio similarity (more like this)

motivation:: annotating music collections

Tag sparsity

Postal Service - Such Great Heights

2.4 million num. plays

100s of tags

Mike Shupps's - All Over Town

3 num. plays

0 tags

Cold start problem for new songs

goals

Ease the process of annotating large music collections

No learning from audio to tags, but...

Content-based similarity to propose tags among songs

Timbre, rhythm, tonality, etc.

our proposal

Annotated

our proposal

wisdom of (song) crowds

FOR FREE!!!Annotated

our proposal

evaluation...? wisdom of crowds & feedback

FOR FREE!!!Annotated

our proposal

Relevance feedback from the users

Spencer Tunick's wisdom of crowds

http://www.flickr.com/photos/guacamoleproject/487376758/

our proposal

new song, no tags

rockIndieCuteguitarDrums90sFastWeirdtweeQuirky

noise popCuteplayfulDrums80sFastWeirdSweetQuirky

Cuterockpop90sFun

metalrockEdgyguitarthrash90sFierceWeirdconcertLoud

??????????????????

thrashloudmetaldeath

thrashrockEdgygothicheavy metal90sWeirdconcertLoud

our proposal

propose tags (wisdom of song crowds)



Cuterockpop90sFun




thrashrock90sLoudmetalWeird

our proposal

users validate/add tags



Cuterockpop90sFun




thrashrock90sLoudmetalWeird

experiments

Ground Truth

5481 annotated songs from Magnatune (John Buckman)

29 Different labels (rock, instrumental, classical, relaxing, etc.)

Avg. 3 tags per song

Only CB similarity, no user feedback

2 experiments

All tags

Only mood tags (smaller collection)

experiments:: results

Some basic comments...

Increasing number of similar songs (10, 20, 30,...)

Bad recall, precision

Best results: ~10 similars

Using album, artist filtering

Need to increase the number of similar songs

Best results: ~20 similars

Metrics

Precision / Recall (F-measure, alpha = 2)

Spearman Rho

(Details on the paper!)

demo

Searchsounds, with users

~250,000 full songs

~48% songs annotated, avg. 2 tags per song

from CDBaby, Magnatune, etc.

conclusions

Extent annotations in a music collection

General experiment

10 similar, recall >0.4

40%+38% = 78%

Mood experiment

P@1 = 0.5

30%+35% = 65%

Avoid cold start problem (new song)

Limitations

General experiment: Low precision, due to (valid) proposed tags, that were not in the GT

Mood experiment: Mysterious label

questions?

Preguntes?

Preguntas?

Questions ?

Fragen ?

Annotating Music Collections: How Content-Based Similarity Helps to Propagate Labels

Mohamed Sordo, Cyril Laurier and scar Celma

(Music Technology Group, UPF)

experiments:: metrics

SongId usana31605

Manually annotated (ground truth)

Classical, Instrumental, Baroque, Piano

Proposed labels (and frequency)

Instrumental (0.55), Classical (0.40), Baroque (0.26), Harpsichord (0.24)

Precision = TP / (TP + FN)

3 / 4 (added Harpsichord)

Recall = TP / (TP + FP)

3 / 4 (Piano is missing)

F-measure (alpha = 2 , more weight to Recall)

Spearman-Rho


How does CB performs...?


With album, artist filtering...


Recall the recall

ismir2007 // mohamed sordo & cyril laurier & scar celma

ISMIR 2007 / Vienna, Austria / Sept. 27th 2007

Economy & Finance

Annotating Music Collections: How Content-Based Similarity Helps to Propagate Tags