Designing Software for Personal Music Management

and Access

Frank Shipman & Konstantinos Meintanis

Department of Computer Science

Texas A&M University

Why Music Management?

Lots of people have large personal collections.

Common software supports access via context-independent metadata– Artist, title, album, year, …

But selection happens based on personal interpretation– Memories, emotional effect, …

Room for improvement

Supporting Personal Interpretation

How to do this?– Could create metadata fields … but experience

shows that those will not be used.– Could use a Web 2.0 strategy (few contributors,

many consumers) but that only works if there is agreement.

Instead, we draw from our experience in supporting personal interpretation during information analysis

Spatial Hypertext

Provide a visual workspace for arranging and annotating content that facilitates rapid expression.

Prior uses:

– Intelligence analysis

– Meeting management

– Authoring papers or presentations

“Float the Idea” Study

To explore the potential, we decided to see how people would use an existing spatial hypertext to work with music.

Visual Knowledge Builder (VKB)– Users organize information objects in a

hierarchy of collections– Users modify visual attributes of

information objects to indicate characteristics of the content

The Study

Participants– 12 graduate students age 24-38– Most had substantial personal music collections– Only 1 was satisfied with playlist creation features

in existing software

Music collection– participants chose between existing collections of

100 songs in rock or classical music

Task– organize the collection for later use in 60 minutes – create three playlists for events in 30 minutes– Time is an issue for this activity

Example Organizations

Single-level organization using

collections, color and border width

Organization using categories,

subcategories and labels

What worked and what did not

The Good– Expressed lots of personal opinions,

memories, activities, and effects.– Preview feature good for reminding

The Bad– Wanted to search by traditional metadata– Needed to hear more for unknown songs

An Augmented Design

Design goals:– Combine access via context-free metadata

with personal expression– Support rapid access and playlist creation

through suggestions– Provide audio summaries of songs

The result: MusicWiz

MusicWiz

Spatial Hypertext Workspace

FileSystemTreeView

Related Songs Tree View& Search Results

PlaybackControls

PlaylistPane

Spatial Hypertext Workspace

MusicWiz

Generating Music Suggestions

Suggestions are meant – to provide more rapid access to music

users want– to identify music that the user might like but

not know they like– to be appropriate based on the user’s

current mood and activity

MusicWiz uses a multifaceted approach to generating suggestions

Facets of Similarity

Metadata– How many metadata fields are the same or similar

Lyrics– How similar are the topics in the songs

Audio characteristics– How similar is the beat, brightness, pitch, and the key

User expression– How similar are the user’s placement and visualization

Web-based information– How similar do others think of the songs

Generating Music Summaries

Most previews are either the introduction or the refrain of the song.

Research focus has been on detecting the refrain.

We focus on creating multiphrase music summaries – summaries that piece together selections from the song.

How to select the sections?

Our Approach(es)

Three phrase summaries with – the most salient phrase (8 seconds)– two supplemental phrases (6 seconds each) – 1 second of silence between the phrases

Developed three algorithms– All share the approach to selecting most salient

phrase– Differ in selection of supplemental phrases based on

whether to place greater value on• the sonic uniqueness of the phrases or• the frequency of the phrases in the song.

Study of Summaries

Purpose – Evaluate and compare algorithms’ power in

the tasks of remembering known songs and getting familiar with new ones

Participants– Fifteen, over 18 years old, mainly students,

including 12 men and 3 women. – Majority (67%) had some kind of music

education and more than half of them (67%) a personal music collection of at least 50 songs (8 participants had more than 200 songs)

Study Design

• Participants were asked to choose and rate the summary that best represents each of twenty popular rock and pop songs

• Four 22-second summaries per song, three generated with our algorithms and one representing the first 22 seconds of the song

• No time limit for the completion of the task

Self-Reported Results

Parts / features of songs fundamental for becoming familiar with and recalling music – Introduction most

important for remembering

– Refrain best for becoming acquainted with the music

Important parts for getting familiar with music

25% 25%

42%

17%8%

0%

10%

20%

30%

40%

50%

60%

Vocals /Chorus /

Parts withlyrics

Intro Refrain Rhythm -Changesin beat

VerseS

ub

ject

s (%

)

Important parts for recalling music

39%

54%46%

15%

0%0%

10%

20%

30%

40%

50%

60%

Vocals /Chorus /

Parts withlyrics

Intro Refrain Rhythm -Changesin beat

Verse

Su

bje

cts

(%)

Best Summary ResultsDistribution of participants’

selections for their favored summary:– The introduction was chosen

in only 13% of the cases while the REA in 35% of the cases

– The difference between the introduction-based algorithm and the REA and IA was statistically significant (Tukey HSD test, P=0.001 and P=0.041 respectively, α=0.05)

Algorithm Use

13%

35%

28%24%

0%

5%

10%

15%

20%

25%

30%

35%

40%

Introduction REA IA SDEA

Algorithm

Ch

oic

es (

%)

Discussion

Study showed that– the multi-phrase summaries represent

music better than just its introduction– the selection of the best summary

was independent of its effectiveness and the music knowledge and recency of access

– there was a likely preference for algorithms that emphasize the selection of repeated phrases

Summary

Examining personal music management

Explored use of spatial hypertext to organize music collections

Designed MusicWiz to combine context-free metadata and personal expression

Developed and compared algorithms for generating multi-phrase music summaries.