Embed Size (px)
Citation preview
AnIntelligentOnlineMusicPlayer
Semester Project,
March 2009 – April 2009
Student: Mihai Calin
ETH – ITET Master Studies
Advisors: Olga Goussevskaia / Michael Kuhn
Supervisor: Prof. Dr. Roger Wattenhofer
Task Formulation
Media usage is changing rapidly these days. This process has been ignited by several technological advances, in particular, the availability of broadband internet, the World Wide Web, affordable mass storage, and high-quality media formats, such as mp3. Many music lovers have now accumulated collections of music that have reached sizes that make it hard to maintain an overview of the data by just browsing hierarchies of folders and searching by song title or album. Search methods based on song similarity offer an alternative, allowing users to abstract from manually assigned metadata, such as, frequently imprecise or incorrect, genre information. In a context where music collections grow and change rapidly, the similarity-based organization has also the advantage of providing easy navigation and retrieval of new items, even without knowing songs by name. This opens possibilities, such as sophisticated recommendations, context-aware retrieval, and discovery of new genres and tendencies.
In previous projects we have created a Euclidean map of the world of music, which contains more than 400K songs. This map places similar songs close to each other, whereas the distance between unrelated songs becomes large. Such a map exhibits several advantages in terms of applications. It allows to quickly find songs similar to each other, to define regions of interest, or to create smooth playlists by following paths, such as straight lines between start- and end-songs, for example. Based on this map, we have developed the music-explorer website (www.musicexplorer.org) that provides a similarity based view on music collections.
The current website, however, provides only limited benefits to the user. In particular, it is not able to directly play music. The goal of this thesis is to improve in this point by connecting the music-explorer website to existing web-content. In a first step, Mihai will study different APIs, such as the last.fm, youtube, google, or facebook API, and investigate how the music-explorer project could benefit from each of these data sources.
Dependent on the outcome of the API evaluation, Mihai will then design one or several new use-cases for the music-explorer website that involve some of the evaluated data sources. In a last step, Mihai will then implement at least one of the proposed use cases and make it available to the user on www.musicexplorer.org.
Abstract
The continuous growing of people’s music library requires more advanced ways of computing playlists through algorithms that match tracks to the user’s preferences. Several approaches have been made to enhance the user’s listening experience; while most of them rely on the music content provided by the user, this project presents an online application that sources the audio content from publicly available resources (YouTube). A playlist generation algorithm is developed that uses only one seed track to compute a playlist of arbitrary length. For sourcing the audio content, YouTube’s track coverage is analyzed and statistics show that, in a real-life usage scenario, almost 80% of the tracks are available while the rest have rather lower popularity. The resulting application is a fully functional but feature limited online music player that can also serve as a framework for future playlist generating algorithms or other content sources.
Overview 1 Introduction
Introducing the current report and placing it aside of similar ETH internal and other works. Some introductory words about the music space, the goal and the approach used in this report.
2 Problem Statement
This chapter debates the necessity of a new solution to make the world ofintelligenttrackcomparisonevenmoreaccessibletotheenduser, followedbyanapproachsketch.
3 Related Work
Whenanew,revolutionary,productcomestomarketitusuallyhasamixedimpactontheaudience.Thischapteranalyzessomeoftheachievementsbutalsoflawsofexisting solutionsand therefore representsabase for thediscussion in theVisionChapter.
4 Vision
Thischaptersetsthegroundrulesfordevelopingtheapplication,mostlybasedonexperience gathered from already existing implementations. Important decisionsabout audio content sourcing and application promotion are discussed andreasonedinthisandalsothenextchapter.
5 Audio and Video Content Provider
This chapter analyzes advantages and disadvantages of the different contentprovidersandpresentsacoverageanalysisforYouTube.
6 Architecture
Afterdiscussingtheapplication’sapproachtoservingitsgoal,it’stimetodescribethe technicaldetailsof the implementation.What functions should theclientandserversideimplement?Howmodularizedcanandshouldtheapplicationbebuild?What’sthebestapproachindealingwiththehugedataamount?WhatadvantagesdothedifferentAPIsbringandhowcantheybeused?
7 Conclusions
Theconclusionssectionsummarizestheprojects finalstateaswellasreconsidersthemainideasandresults.
8 Discussion and Future Work
The Section dealswith the ideas that didn’tmake it to the development processmainlybecauseofthetighttimescheduler,butthatareimportanttomentionandatsomepointoranotherinafuturedevelopmenttheyhavetobeaddressed.
TableofContents
1 Introduction ................................................................................................................. 3
2 ProblemStatement .................................................................................................... 42.1 Motivation.......................................................................................................................... 42.2 Approach............................................................................................................................ 5
3 RelatedWork ............................................................................................................... 63.1 ETHProjects...................................................................................................................... 63.2 ExternalProjects ............................................................................................................. 7
4 Vision.............................................................................................................................. 84.1 RequirementSpecifications ........................................................................................ 84.2 UserPerspective.............................................................................................................. 94.3 ImplementationasFacebookApp...........................................................................10
5 AudioandVideoContentProvider.....................................................................115.1 ContentProviderAnalysis .........................................................................................115.2 YouTubeAPI ...................................................................................................................125.2.1 TechnicalPossibilities .........................................................................................................125.2.2 CoverageAnalysis..................................................................................................................12
5.3 QueryDesign...................................................................................................................155.4 PostprocessingYouTubeResults ...........................................................................16
6 Architecture ...............................................................................................................186.1 ClientandServersideFunctions .............................................................................186.1.1 TheClientSide ........................................................................................................................186.1.2 TheServerSide.......................................................................................................................196.1.3 TheCommunication .............................................................................................................20
6.2 MusicSpace(KDTree) .................................................................................................206.3 PlaylistGenerationAlgorithm..................................................................................216.3.1 ExistingSolutions ..................................................................................................................216.3.2 FirstApproach ........................................................................................................................216.3.3 Improvement...........................................................................................................................23
6.4 UserFeedback ................................................................................................................246.4.1 DirectandIndirectUserFeedback.................................................................................246.4.2 ActivityandFeedbackLogStructure ............................................................................25
7 Conclusions.................................................................................................................268 DiscussionandFutureWork ................................................................................278.1 Achievements .................................................................................................................288.2 FutureWork....................................................................................................................288.2.1 NicetoHave .............................................................................................................................288.2.2 FeedbackAnalysisandIntegration................................................................................288.2.3 UserAwareness......................................................................................................................29
9 Bibliography...............................................................................................................29
3
1 IntroductionMusic has always been ameans of entertaining people even from the earliestages of the civilization. Historically it was produced by musicians and onlyavailable during life concerts. The technological evolutionmade it possible tosave themusiconvinylplates, laterelectromagnetic chargedstripes,CDsuntilthe technologybroughtus tosaving tracksdigitally.Whendealingwithahugecollectionoftracks,peopleencountermanagementproblemstheydidnothavebefore.Sotheyhavetodevelopnewwaysofusingthemusiccollectionfortheirentertainment.Playlistsareagoodapproachforsavingsuccessionsoftracksthatone likes. The most dominant problem of existing playlist generationmechanisms is, however, their lack of flexibility: new tracks are notautomaticallyadded,theydon’tadapttotheuser’scurrentmoodetc.
A new approach in dynamically organizing tracks into playlists is on its way:companies like last.fm already suggest an algorithm of mapping songs one toeachotherbasedontheir“similarities”;buthowtocomputethesesimilarities?Oneway,thatdidnotprovetobeveryproductive,istoanalyzetheaudiocontentof the track– itsaudio frequencies.Thisway, tracksaresplit incategories like“HeavyMetal”and“Blues”,butpeopledonot likeall tracksofacertaingenderand thesegendersmightbe inaccurate.Anotherway,which is givenmoreandmore attention by researchers and companies worldwide, is computingsimilaritiesbetweentracksbasedonuserinput.Asanexample:iftwousersaddthe same two tracks to their playlists, one can deduce that these tracks aresimilar and so, also other people that pick one of them are likely to enjoy theotheroneaswell.
Lorenzi(Lorenzi,2007)proposesawayofrepresentingthesimilaritybetweentracks ina10‐dimensionalEuclidianspace (furthercalledmusicspace),wherethe closeness of tracks is approximately proportional to their similarity. 7Msongscurrentlyappearinthedatabase,butonly500Kofthemhaveenoughuserstatistics tobemapped in thegraph.Using this simplifiedandcomputationallyefficientwayoffindingsimilartracks,severalapplicationscanexplorenewwaysofcomputingplaylists.Mostofthemoffersupportinplaylistgenerationbutnonealso provides the tracks to be played. This could be seen as a disadvantagebecausenotallpeoplepossessalltracksthataresuggestedbythespace.
Thisreportsetsitsgoalindevelopinganapplicationthatusesthespaceofmusicfor computing intelligent playlists and, more important, trying to deliver therequired songs to the user through publicly available tracks on YouTube.com.Forpromotionpurposesandalsouserdatagathering,itwaschosentomaketheapplicationavailableonFacebook1,whichensuresaneasyreferraltofriends.
Thedevelopmentprocessisnotaneasyoneasmanytechnologiesarerequiredtoworktogethertoprovidetheuserwithagoodexperience.Thehugeamountofdatathathastobeprocessedisdemandingwithrespecttobothalgorithmsand
1www.facebook.com‐communitynetworkingsite
4
underlying hardware. In the end of this report, a fully functional, but feature‐limited,versionofanonlinemusicplayerispresented.Thisapplicationcouldbethe step stone of future, more detail‐oriented applications because it sets thebasisofanewplayground:anonlinemusicplayer.
2 ProblemStatementThis chapter debates the necessity of a new solution to make the world ofintelligenttrackcomparisonevenmoreaccessibletotheenduser, followedbyanapproachsketch.
2.1 MotivationSeveral solutions already use intelligent playlists embedded in music playersinstalled on computers. There are also online solutions, the most popular ofwhichis last.fm,whichactsasapersonalizedradiostationthatplayspreferredmusic.Ontheotherhanditdoesnotallowplaybackofacertaintrack.Therearealso other solutions, like the genius function of iTunes or theMusic Explorer;both use the user’s music collection to generate playlists. The biggestdisadvantage of the latter solution is that the user can use only tracks thathe/shealreadyhasonhis/herPCtogenerateplaylists.Ofcoursethislimitsthepowerorthealgorithmverymuch.
There are already services that provide the music content (like last.fm orYouTubetonameafew)soit’sanaturalconclusiontotrytousetheseservicesinconnectionwiththeplaylist‐generatingalgorithm.
In order to understand the utility of such an application, just imagine thefollowing scenario: one enjoys listening to music while working. It is notcommontostoremusiconthecompany’scomputersooneratherhasapersonalmp3playerwithhimselfduringofficetime.Ifonetakesenoughtimetoprepareonesplaylistsinordertofitonescurrentmood,itisaprettydecentsolution.Butwhat if new tracks appear that one might like? One first has to do seriousresearch inorder to find themandthengo throughbuying them,downloadingthem to his/her mp3 player, updating the playlists… it already sounds verydifficult, right? Now the suggested scenario is the following: one opens awebsite,typesinatrackthatreflectsonescurrentmoodandhits“play”.That’sit!Theplayerchoosestracksthatonelikes,alsoplaysnewtracksthatonedidnothearbefore,andcangolikethisforhoursandhourswithoutrepetition.Onecangoonwithone’sworkandinordertostopthemusic,oneonlyhastohitstoporclosethebrowser.Thesimplicityofthesolutionspeaksforitself.
Thegoalofthisthesisistoanalyzeandimplementanapproachofbuildingsucha web‐basedmusic player. The questions it has to find answers for are: Howshould the user interaction be designed to maximize the user satisfaction?Where to source the audio (and video) data from,while ensuring amaximumcoverage?Andfinally,howtopromotetheapplicationinordertoattractasmanyusersaspossible?
Thedifferentimplementationpossibilitiesareevaluatedandthebestsolutionisimplemented. The logic behind the web‐based music player computes a
5
sequenceof tracksbasedon their similarity.At thesame time,userbehavioraldata is gathered that helps further releases to be even more user friendly.Another important aspectof theapplication is its extensibility.Modularity andcode reusing are very important parameters of this application, as it acts as aversion1.0forfuturereleases.ThesefuturereleaseswillbeabletointeractwiththeuserforfindingthebesttrackvideoonYouTubeortodeterminethemusicpreferencesofusersandevenadaptthespacetothenewusagestatistics.
2.2 ApproachTheanalysisofthecurrentlyavailabletoolstoaccomplishthetaskisoneofthemost important steps because the ground concepts of the application shouldneverchange,regardlessofitsfuturecomplexity.
The several possible implementations of the web service together with thebalancingofcomputingtasksbetweenserverandclientarethefirstparametersthathavetobedefinedforasolidbase.Alsotheprogramminglanguageplaysacrucial part in the development process, as it is shown later. The amount ofcallbacks to the database in favor of less memory usage is also an importantaspect that is difficult to estimate from the start. In order to allow a highflexibility while still maintaining a small dataflow, the implementation of thelogicismainlyontheserver.TheUIresponsibilityisfullyretainedbytheclientsideaswellasservicingUIrequestsandonlynotifytheserverofsuchactivity.
Inordertoachievethehighgoalsthatwereset,thestructureoftheapplicationisimportant to be highly modularized to allow interchanging the modules withbetter, more complex implementations. It is important to determine whichcomponents arepossible andalso easy tomodularize,without introducing toomuch communication overhead in the interfaces. It turns out that the musiccontentrelatedjobscaneasilybemodularized,aswellastheDBrelatedjobsandthe playlist computing tasks. The core of the application only needs to handlethesemodules and the logging task.Also the communicationwith the client ismodularized,making it particularly easy to implement new clients running onthesameserviceornewservicestoservethesameclient.
Oneofthehardesttasksisdetermininghowtoprovidetheaudiocontenttotheuser. The playlist computing is based on a music space implementation,describedlater.Fortheaudiocontentthereisanywaynoguaranteebecauseitisan externalized task. Several providers of audio content are analyzed and thebestoneimplementedinsuchawaythatitisreliablefortheusers.
Fortheclientside, JavaScriptandFlasharetheonlytwosolutionstoprovideagood user experience because there is no need to install any software.WhenworkingwithJavaScript,AJAX(Wikipedia,AJAX(programming),2008)isneededfor the seamless communication between client and server – thus enabling aricher user experience – and also for avoiding communication overhead. GWT(Google,2008)dramaticallyeasestheinteractionandobjectexchangebetweentheclientsideandthewebservicebecausethistoolenablestheprogrammingoftheclientsideinjavaandconvertingittoJavaScript.
Finally, one of the most important steps in the future extension of thisapplication is the logging of user activity. The activity of the users has to be
6
loggedinsuchamannerthatconstructiveinformationcanbeextractedfromthelogs.It’simportanttobuildthelogsrightsuchthatfutureideascanfindthedatatobasetheirresearchon.Forthisstepnoresourceistooexpensive.Onehastodifferentiate between user activity and user feedback. User activity can berepresented through actions like: jumping to a certain track, rearranging thegeneratedplaylist; user feedback is the action of notifying the application of abadchosentrackorofatrackthattheuserdoesn’tlike.Intheendthelogsaresavedinthedatabaseandtheirexactstructurediscussed.
Intheendofthereport,animplementationispresentedshowingawebbased,personalized,musicplayer.Itisaninteractivewebsitewheretheuserselectsafavorite track and generates a playlist based on this. The playlist has thefollowingproperties:
• Eachtrackintheplaylistissimilartotheonebefore;• A track is only allowed to be repeated if it has not been in the last 24
tracks;thisshouldguaranteeabouttwohoursofnon‐repeatingtracks;• Tracks from the same artist do not appearmore than at least 4 tracks
apart;• Theuserhas thepossibility toreorder the tracks, toremove themor to
playthemdirectly,withoutwaitingfortheirturntocome;• Themusicvideoofthecurrentlyplayingtrackappearsontheright;• Theuserhas thepossibility toopen theplaying track inYouTube (with
possibilitytoswitchtofullscreen).• The user has the possibility to give feedback about the quality of the
recordingorweatherhe/shelikesthetrackornot;• All user activity like reordering playlist, searching for tracks, track
removal–butalsohis/herdirectfeedback–isloggedforfutureanalysis.
Afterdescribingsomeofthemostimportantfeaturestheapplicationhastocover,ananalysisofalreadyexistingsolutionsismadeintheRelatedWorkChapter.
3 RelatedWorkWhenanew,revolutionary,productcomestomarketitusuallyhasamixedimpactontheaudience.Thischapteranalyzessomeoftheachievementsbutalsoflawsofexisting solutionsand therefore representsabase for thediscussion in theVisionChapter.
3.1 ETHProjectsSeveral projects belonging to the Distributed Computing Group, part of theElectrical Engineering Institute of the ETH Zürich, have presented differentapproachesasaplaylistgeneratingtoolbasedonaspeciallycreatedmusicspace.
Thefirstandmostimportantwasthecreationofthemusicspaceitself(Lorenzi,2007).BasedonuserstatisticsprovidedbyLast.fm,Lorenzipresentedawayofrepresenting the data such that the similarity between two tracks can bequantified.Bymappingtrackstopointsina10‐dimensinoalspace,thesimilarityof two tracks canbe easily computed as theEuclidiandistancebetween them.Throughthis, computationallyhardproblemsbecomeeasyaccessible forother
7
applications, such as: comparing similarities between several track pairs,computingthenearestneighbor,findingtracksalongacertainpathor“negating”atrack–meaningfindinganothertrackthatisfarawayfromthistrack.Afurtherdiscussionof thecorrectnessof thismapping isoutof thescopeof thisreport;thereforethisreportconsiderstheprovidedmusicspaceasefficientandusesitasitis.
Based on the previously described music space, several applications weredeveloped to bring the advantages of the music space to the end user. Themusicexplorerwebapplication(Gonukula,Kuederli,&Pasquier,2008)presentsanapproachoforganizingtheuser’smusiccollectionbasedonanonlineservice.Theuserhasthepossibilitytouploadthetitlesofitsmusiccollection,selecttwoof themandthealgorithmcomputesaplaylist that isasmoothcrossover fromonetracktotheother.Both,themusicexplorerandtheapplicationpresentedinthisreport,allowtheusertogenerateintelligentplaylists.Thedownsideofthemusicexplorer is therequirement that theuseralreadypossesses the tracks tobeplayed.Ifhis/hercollectionislimited,soistheplaylistsattheoutput.Usingpubliclyavailabletracksnotonlymakeiteasierfortheuserbutalsotheresultsarebebetter.
Bossard presents a new way of exploring the Space of Music with a visualapproach that should allow the user to visually select tracks from the musicspace (BossardL. , 2008).The ideasare implementedon theandroidplatformthroughacustomizablemusicplayer.
3.2 ExternalProjectsSeveralcompanieshaverecentlydiscoveredtheadvantagesofuser‐personalizedplaylists and are currently building their business model based on providinghighqualitybutsimpletousesolutionstotheircustomers.Last.fmisoneofthefirst ones on themarket to offer personalizedmusic playback; it achieves thisthrough an online “radio station” that is supposed to playmusic that the userlikes.TheLast.fmapproachisverysimilartotheoneproposedbythisreport:itis an onlinemusic player that usually plays the user’s favorite tracks and alsointroduce new tracks if available.However, because the personalization of theplayer isdoneonaper‐userbasis(opposedto theper‐sessionproposalof thisreport) it could happen that it does not react so fast tomood changes in theuser’sbehavior.Thebiggestdrawback is still the inabilityof theuser to selectcertaintrackstoplay;evenifhe/shehasacertainfavoritetrackinmind,he/shehastosticktotheautomaticallygeneratedplaylistoftheplayer.Thislimitationis probably more license‐based than technological, but it is a field whereimprovementisdesired.
AnotherapproachtodeliveringsimilartrackstoauseristheGeniusfunctionoftheAppleiTunesmusicplayer.Theprincipleisfairlyeasy:iTunesanalyzesyourmusic library,submitsthetrackstoan iTunesserver,whichreturnspopularitymeasurements personalized for your tracks.During playback, the user has thepossibilitytoselectatrackandlettheapplicationgenerateaplaylistcontainingtrackssimilartotheselectedone.Thebiggestdrawbackofthisapproachisthelimitationtotheuser’spersonalcomputer,wherehe/sheholdshis/herprivatemusiccollection.
8
During the development process, two new companies launched animplementation that is very similar to thepresented approach.Oneof them isSongza(www.songza.com),anonlinemusicplayerwhereuserscansearchforatrackandplayit.Theaudio(andsometimesalsovideo)contentissourcedfromYouTube or Last.fm. The user has the possibility tomanually create a playlist.However no mechanism of automatically generating playlists is yet available.Thesecond,andmuchmoresimilartothisreport’simplementation,isDropPlay(www.dropplay.com). It allows the user to search for songs, generate playlistsbasedon similarity andalso share these songswith friendsonFacebook.Onlytimewillshowwhichofthetwoimplementationswillbebetterfortheusers.
The discussion of external projects is continued in the Vision chapter,where theprosandconsofthementionedapplicationsarefurtheranalyzedandweighted.
4 VisionThischaptersetsthegroundrulesfordevelopingtheapplication,mostlybasedonexperience gathered from already existing implementations. Important decisionsabout audio content sourcing and application promotion are discussed andreasonedinthisandalsothenextchapter.
4.1 RequirementSpecificationsAs presented in the related work section, there already are severalimplementations that are more or less similar to this report’s solution. All ofthemhavetheirstrengthsandweaknessessotheyaregoingtobeanalyzedandcombined towork in the targeted scenario. The goal is to create aweb basedapplicationthatissimpletouseandthatcanfunctionindependentoftheusersresources.Whatcanbelearnedfromotherapproachesandwhatcanbeusedtoincreasethevalueofthisimplementation?
Compared to the current musicexplorer web application, the designedapplication is also aweb‐based service and targets the same audience: peoplewhowant to have theirmusic experience enhancedwithout a lot of effort ontheir side. The users interact with the existing musicexplorer application bysubmitting all the titles of theirmusic library and then selecting two of thosetracks that are used as endpoints of a playlist. All the tracks in betweenrepresentasmoothprogress fromone track to theother.This implementationfaces theproblem that it completelydependson theuser toprovide theaudiocontent. The quality of the offered result is only as good as the number andvarietyofsubmittedtracks;eventhebestimplementationperformspoorlyifitisfedwithlowqualityresources.Alsothemappingoftracksfromtheuser’stags2totheapplicationstagsisaproblemtotakeintoaccount.Thelackofownmusiccontentisalsoaproblemforpeoplethatdonotownabigmusiccollectionordonot have it at hand. Hence, the current application sets a high priority to
2Trackmeta‐data,namelyartistandtitle
9
providingtheintelligentplayliststogetherwiththeaudiocontent.ThisensuresbothagreaterQoS3andeaseofuse.
GeneratinganintelligentplaylistandthenplayingthetracksissimilartoLast.fm.InthecaseofLast.fmhowever,licenselimitationspreventtheusertosearchforspecific tracks and play them. As long as the designed application has norestriction in this matter, it is a high priority to implement such a popularfeature.A searchbar allows theuser to browse through the library and selectindividual tracks to play. Last.fm also avoids displaying an actual playlist,althoughmostusersliketoatleastseethetracksthatfollow,andpossiblyeveneditthislist.Tohonortheplaylistgeneratingfeatureoftheapplication,aplaylistis displayed to the user together with the possibility to add, remove andrearrange tracks from the list. This makes the application very similar to themusicplayerstheuserisalreadyusedto.
Asdiscussedinthelastparagraph,theUI4imitatesacommonmusicplayersuchthattheusersdonothavetogetusedtosomethingcompletelynew.Thisbringsthequestionwheatertoimplementtheapplicationasaninstallable–standaloneapplicationorasawebservicerunninginabrowser.Moderntechnologies likeJavaScriptandAJAXallowasimpledevelopmentofasolutionrunninginthewebbrowser.Alsoencouragingforusingthisapproachisthefactthattheapplicationmost likely provides online‐based streamed audio content. Online applicationsprovidedecent results if theydon’thave resource‐consuming client side tasks;alsotheyareeasytoupdateandmaintain.Inrespecttoallthementionedpros,itmakes sense to develop the application as a JavaScript based implementationthatrunsdirectly in thewebbrowser,without theneed todownloador installanything.
4.2 UserPerspectiveAs one of the important goals is to have a user‐friendly application, it isnecessary to provide good results with a minimum of information requestedfromtheuser.Asasecond,butalsoveryimportantfeature,theuseractionshaveto be logged for understanding and improving the application’s behavior. Theloggingshouldbemadeoptional,toavoidprivacyissues.
Understanding the resources that are at disposal is necessary to find theminimuminformationrequiredtoreturnsatisfactoryresults.Inordertoprovidetracksthattheuserlikestolistento,theapplicationhastoknowwhicharethepreferencesoftheuser.Last.fmdoesthatbytrackingtheuser’spreferredtracksover several sessions and probably computes a user profile based on thisbehavior.Thisapproach isanelaborateonebut introducesahugeoverhead intheuser‐managementalgorithms.Andiftheuserdecidesatsomepointtolistentosomethingelse,theinertiaofthealgorithmmakesthisverydifficult.Thebestapproachthatalsoensuresaminimumuserinteractionisnousertrackingatall.Theapplicationshouldworkwellforeachuser,independentonhis/herhistorywiththeapplication.
3QualityofService4Userinterface
10
Choosing not to implement any usermanagement requires obtaining asmuchinformation as possible at the start of each session. In order to make it veryflexible,theapplicationcanaskfor“guidelines”(favoritetracksoftheuser)eachtime it’s requested to compute a new playlist. The approach of another ETHprojectwastoasktheuserforseveralseed5tracksandcomputingavirtualpathinthemusicspacethatconnectedallthepoints.Ofcoursetheseedlistwasalsorearrangedsuchthattheresultingpathisminimal.Usingseveralseedsisagoodapproach because it gives more details about the user’s current musicpreference. But on the other side, the user might feel overwhelmed by theinformationhe/shehastoprovidetocreateasimpleplaylist.Inrespecttothat,thefirstapproachistoasktheuserforonlyoneseedtrack.Thistrackisusedtocreateanewplaylist,and if theuserdoesn’t like theoffered trackshe/shecaninsert a new seed or just reshuffle. Testswill show if thisminimal user inputapproachisefficientenoughinprovidinggoodtracks.
User feedback is very important for improving this application but it is wellknown that the average user does not usually bother to give feedback; and ifhe/shedoes,he/sheusuallyreports features thatarenotworking.Toadapt tothisattitude,itisgoodtoimplementawayofmonitoringtheuser’sactions,likewhen he/she changed the track, when he/she removed a track from theproposedones,etc.Alsotheuserhastobegiventhepossibilitytoreportfaultyfeatures.Becausethemusiccontentprovider isYouTubeandnoguaranteecanbegivenfortheplayedcontent,theuserhasthepossibilitytoreporttracksthatdon’tfulfillhis/herexpectations.Forexample,ifatrackisalowquality,concertrecorded version, the user can notify the application and other users do notreceiveitanymore.Anothercomplaintpossibilityisabouttheplaylist‐generatingalgorithm itself. The user is able to report a track that he/she doesn’t like somuch, indicating an inconsistency in the playlist generating algorithm or theunderlyingmusicspace.
4.3 ImplementationasFacebookAppPromoting the application tomany users is important – among other reasonsalso because user’s behavior is recorded and eventually leads to a betterapplication. Applications usually get promoted through commercials, referralprogramsetc.Astheapplicationisoneofakind,wordofmouthisthebestwayto promote it. Community networks like Facebook and MySpace6 are perfectenvironments for deploying and promoting such community‐targetedapplications. This networking platforms already have thousands of dailyreturning users signed up and also facilitate the integration of applicationsthroughspeciallydesignedAPIs.FacebookAPIsalsoofferthepossibilitytoreaduser data; this could come in handy at a later development stage.Because thepossibilityofrecommendingapplicationsbetweentheusers isprovidedby theFacebook platform, the application is basically promoting itself – provided itprovesit’susefulness.AlsoappearingonaFacebookofficialsite,theapplication
5 Seedsor seed tracks refer to the trackprovidedby theuserandusedby theplaylistgeneratingalgorithmtofindsimilartracks.6www.myspace.com‐communitynetworkingsite
11
is granted more trust from the users, making it possible to integrate it evenbetterwiththecommunity.
Facebook offers twopossibilities of designing an application: either the app iswritten in a Facebook‐specific markup language or any other web site. If thelatterischosen,theapplicationrunsinaniFrame,ofmaximum760visiblepixelsinwidth.Because the application is intended to runalso as a stand‐alonewebsite,thedeploymentinaniFrameischosen.
Choosing thepromotion throughFacebookdoesn’t limit the application in anyway.ItstillrunsinawebbrowserwithoutanyrestrictionsandindependentofFacebook.Onecanseethisdeploymentasanadd‐ontothefunctionality.
Also the DropPlay application, presented in the Related Works Chapter,integrates with Facebook and uses Facebook to recommend tracks to friends(andthroughthisindirectlyreferthesitetothefriends).However,theDropPlayapplicationisnotdeployedasaFacebookapp;notdoingsoneglectsthetrustyenvironment for the users. To speculate upon the reasons, the width of theiFramemightbetoorestrictiveforDropPlay’sUI.
Havingdiscussedthegroundrulesoffunctionalityandalsoreasonedthedecisionsthrough arguments mainly from the users perspective, the spotlight continuestowardsthesourcingoftheaudiocontent.
5 AudioandVideoContentProviderThis chapter analyzes advantages and disadvantages of the different contentprovidersandpresentsacoverageanalysisforYouTube.
5.1 ContentProviderAnalysisThe necessity of providing public music content was analyzed earlier in thisdocument, concluding that it is important for a good user experience. To betaken into consideration, a music provider has to fulfill the condition that itoffersacomplete(orhigh)coverageofthetrackdatabase.Withrespecttothisrequirement, only YouTube and Last.fm can be considered good candidates atthismoment.
YouTube’sprosare:aneasytouseAPIforimplementingthevideocontentinthewebbrowser;a java library formaking thesearch for trackseasyand fast; thetracksareuploadedbyusers,whichmeansthedataisalwaysuptodatewiththelatesttracks.Onthedownside,becausethecontentisuser‐generated,thequalityofsomecontentissobad,aresultspost‐processingalgorithmisnecessary.AlsonotalltracksarefoundonYouTube,butpopulartracksareusuallyavailable.
Last.fmhasabettercoverageofthetracksinthemusicspaceandthequalityofthecontentisalsoveryhigh.ThebiggestdownsideofLast.fmisthatthemajorityof tracks are only 30 seconds snippets of the actual tracks. This is somethingunacceptableforamusicplayer,thusturningthedecisioninfavorofYouTube.
The developers of Songza couldn’t apparently decide for one of the providersandimplementedboth.ItisagoodapproachbecausethesongsonLast.fmhavea
12
guaranteedqualityand theapplicationonly switches toYouTube if there isnounclipped track on Last.fm. Implementing both alternatives however, bringsproblemsespeciallyintheUIpartoftheapplication.Alsoindirectuserfeedbackismuchhardertotrace.ThisiswhytheapplicationisrelyingonYouTubeasit’scontentprovider.
5.2 YouTubeAPI
5.2.1 TechnicalPossibilitiesAs discussed in the previous section, YouTube is the chosen music contentprovider.Itshouldbenotedatthispointthatnomusiccontentisbeingheldoreventransferredthroughtheserverhostingtheservice.OnlytheYouTubemovieidispassedtotheclientapplicationthatembedsthevideousingYouTube’sownplayer.
The YouTube API offers the possibility to domost of the actions performablemanuallythroughthewebsiteinaprogrammaticmanner.Theseactionsincludereceivingalistofresultstoasearchqueryandforeachofthesetracks:itsid,thelink to its implementation, link to its thumbnail etc. Moreover, the API offersmethodstosetthequeryparameters,like:thecountryrestriction,thelanguage,theuploaderandalsotheorderoftheresults.
The API also offers possibilities of logging in with a user account, uploadingvideosfromthataccount,generatingplaylistsandmanyothersfeatures,buttheyare(currently)ofnousetothisapplication.
5.2.2 CoverageAnalysisYouTubewaschosenoverLast.fmtobe thesolemusiccontentproviderof theprojectbothbecauseof its simple touseAPIbutalso for coveringmostof thepopulartracks.CoverageisoneofthestrongpointsofLast.fm,butthefactthatfor most tracks only a 30 seconds sample is provided, biased the decision inYouTube’sfavor.
ToanalyzetowhatextendthelibraryiscoveredbytracksfromYouTubeatestwas run on samples taken from theDB. In order to have an accurate result, acertainnumberofrandomsampleswerechosenfromtheDBinordertomakeanassumptionaboutthecoverage.Theresultsshowthat from137samples,morethanhalfhadnocorrespondingtrackinYouTube(Figure5‐1).
13
Figure51YouTubeTrackCoverage
The coverage test was run over 7M+ tracks available in the DB, randomlychoosing 137 tracks. But around 60% of the tracks have a popularity of 1,meaning that only one user ever listen to them. As a comparison, the averagepopularityoftherestof40%ofthetracksis24withamaximumofaround62K.Thestatisticdoesn’tnecessaryreflecttherealcoverageoftheDB,becausemostusers only listen to the most popular tracks – also enforced by the playlistgenerationalgorithmthatselectmostlyratherpopulartracks.Hence,inordertoreflecttheeffectivecoverage,anewstatisticapproachischosen,detailednext.
found;47%
notfound;53%
14
Figure52DBTracksPopularityDistribution
Figure5‐2 illustratesthepopularitydistribution inthedatabase.Fromtheplotonecanforexampleseethatmorethan100Ktrackshavepopularityabove10.Inorder to create a weighted analysis of the coverage, tracks for analysis werechoseninthefollowingmanner:thelistoftrackswassortedbypopularityandthe first 10 tracks were added to the statistic. Next, 10 tracks were chosenrandomlyfromthenextpowerof10tracks(forthesecondstep100)andsoonuntilthelimitof7Mtrackswasreachedandthesamplecontained191tracks.
Found;81%
NotFound;19%
15
Figure53YouTubeWeightedTrackCoverage
Figure5‐3showstheYouTubecoveragetheweightedstatisticdiscussedearlier.Only19%ofthetracksarenotfoundatallonYouTube,andmostofthemareina lower popularity range, as shown in Figure 5‐4 ‐ YouTube Popularity vsCoverage.
Figure54YouTubePopularityvsCoverage
5.3 QueryDesignAs a first approach, the query sent to YouTube contains only the track’s artistandtitle,concatenatedinastringandseparatedbyspace.ThiswasthoughtofasafirsttrybuttheresultsreturnedbyYouTubearegoodenoughtocontinuewiththissolutioninthedevelopmentenvironment.
Otherparameterswerealsosuppliedtothequery,liketoreturnonlyvideosthatare visible in the client’s country. To achieve this, the clients IP address isprovidedtogetherwiththequery.Alsotheresult’ssortingisrequestedtobeby‘relevance’. With this measures, the first result returned by YouTube is goodenoughtobeusedintheimplementation.
It is not certain whether the video contains the correct track at satisfactoryquality.User feedbackhelps a future versionof this application to identify thebestmatchingvideotoagiventrack.Alsoat thistimethequerydoesnotneedanyoptimizationbutinfuturethismightchange,asYouTubeischangingaswell.Inthenextsubsections,thereisalsoadiscussionaboutthequeryresults’post‐processing.
0
0.2
0.4
0.6
0.8
1
1.2
1to10 10to100 100to1K 1Kto10K 10K‐100K
Coverage
Popularity
PopularityvsCoverage
16
5.4 Post‐processingYouTubeResultsItwasmentionedinthequerydesignsectionthatitisnotnecessarytooptimizetheYouTubequeryforthetimebeing.Buthowaboutpost‐processingofresults.Doesthatbringbettercontentforthetracks?
Theweightedstatisticspresentedintheprevioussectionisalsousedtoanalyzethe effect of post‐processing the results returned by YouTube. Based on thisstatistics,thecurrentlyimplementedalgorithm(thatjustreceivestheresultthatYouTubesetsasmost‘relevant’)returnsthemanuallycomparedresultsonlyfor61%ofthetracks(Figure5‐5).ThisshowsthatYouTubefindsanadditional20%ofthetracksbutthealgorithmfailstoprovidethembecauseofthelackofpost‐processing.
Figure55YouTube'sRelevanceSortAnalysis
The following approach shows how a few basic rules could help theimplementationincreasetheresultsqualitythroughresultspost‐processing.Theproposed rules are created based on observations when analyzing the trackswhereanimplementationbetterthanthefirstonewasamongthefirst3results.
The post‐processing algorithm should process only the first three resultsproposed by YouTube when sorted by relevance. In the following a ratingmechanism is presented that, after applied to all the three results, determinesthebest onebasedon theobtained score.Note that the rating values areonlyfirst guesses that provide decent results; they have to be optimized and theirperformanceanalyzedtobeforebeingusedintheapplication.
Allthreetrackshaveascoreof0atbegin.Tothisscorethealgorithmcanaddorsubtractpoints.
BestResult;61%
Abetterimplementationinthenexttworesults;39%
17
Table51PostProcessingRatingRules
Rule ScoreIsfirsttrackinresultset +3Issecondtrackinresultset +1
Musicvideo +4Official/original +4Video +2Lyrics +3HQ/HD +2HighQuality +2Mtv/bbc +1Live ‐2Tour ‐2Instrumental ‐4
Either titleordescriptioncontainstheword:
Cover ‐4
In order to see the rules in action, the tracks from the statisticwere analyzedagain,showingtheresultsinFigure5‐6.Thepercentageofnotdeliveredtrackshas lowered by 12% (from 39% to 27%) leaving only 8% of the results non‐optimized.Thisanalysisshowsthat thepost‐processingcanhavean importantimpactontheresultandshouldbeimplementedinfuture.
Figure56YouTube'sTrackCoverageAfterPostProcessing
HavingdecideduponYouTubeascontentprovider,therestoftheimplementationdetailsarediscussedinthenextchapter.
BestResult;73%
Abetterimplementationinthetirstthreeresults;27%
18
6 ArchitectureAfterdiscussingtheapplication’sapproachtoservingitsgoal,it’stimetodescribethe technicaldetailsof the implementation.What functions should theclientandserversideimplement?Howmodularizedcanandshouldtheapplicationbebuilt?What is the best approach in dealing with the huge data amount? WhatadvantagesdothedifferentAPIsbringandhowcantheybeused?
6.1 ClientandServersideFunctionsThe goal of the application is set and also the ground rules – also forimplementation – have also been set in the Vision Chapter. The application isdesignedasatwo‐sidedapplication:clientandserverside.
6.1.1 TheClientSideTheclientsidemustimplementtheUIoftheapplication.Itmustbeastand‐aloneprogram that only requests and delivers parameterized data from and to theserver. It is important tohavea strict and simple communicationbetween thetwoapplications(clientandserver)suchthatanextensionorevenreplacementof each one of them is easy to understand by the developer. The clientapplicationisresponsiblefordeliveringanintuitiveUIandhandlingallrequestsoftheuserthatarerelatedtotheUI.BasicallytheclientimplementsthewholemusicplayerUI,withplay,pause,repeatfunctionsaswellasplaylistgenerationandmodification.Thelistoftracksactuallypresentinaplaylistisgeneratedbythe server and returned to the clientwith enough parameters (video id) suchthat the client itself candownload themusic content fromYouTube.Nomusiccontentisavailablefromorroutedthroughtheapplication’sservice.
Theseedtobeusedforgeneratinganewplaylistisnotselectedinatraditional‘search and select candidate’ manner. A type sensitive field responds to eachkeystrokewithasuggestionlistfortracksthatcontainthewordseitherasartistortitle.Theuserhastoselecta track fromthis listanduse itasaseedfortheplaylist he/she wants to generate. The client does not fetch results from theserverforeachlettertheusertypes.Adelayissetsuchthatasuggestionrequestissenttotheuseronlyiftheuserhasstoppedwritingformorethan‘delay’timeago.
Theplaylistisfullycustomizableandofferstheuserthepossibilitytorearrangethe tracks in order to influence the playing order, to remove and to jump totracks.Norestrictionsaremadeontherearrangingoftheplaylist;forexampleifauserwhishestoreplayatrackthathe/shealreadyheard,he/sheisabletodothat.Alsotheplaylistisresponsibleofalwayshavingaminimalnumberoftracksloaded after the currently playing one. If this number decreases because ofplaying track advancement, track removal or reordering, the playlistautomaticallyloadsnewtracksuntiltheminimumsizeisreachedagain.
TheUIhasanareausedtodisplaytrackspecificinformationliketitle,artistandother. The user uses this area also to directly give feedback about the currenttrack (lowqualityorbad track choice).Adirect link to the implementationonYouTubeisalsoavailable.
19
6.1.2 TheServerSideThe servermust provide the following two services for the client application:playlistgenerationandsuggestiongeneration.Thecoreoftheserveristheonlynon‐interchangeablepartanditactsasabridgebetweenthemodulestoprovidethe services requested by the client. The server side application consists ofmodules that handle: the database, the YouTube related data, themusic spacedata and the client connection socket. The client connection socket is distinctfromthecoremodule,suchthatitcanalsobeinterchangeableifadifferenttypeofclientisusedtoconnecttothesameserver.Inthiscasehowever,thecorehasto bemodified aswell to provide the new functionalities. Also the socket hasresponsibilities likemaintaining the session and converting the data from thestructures(Objects)usedintheservertotheonesusedinthecommunicationtotheclient.Figure6‐1‐ModularStructureoftheServerSideApplicationoffersanoverviewofthemodularstructure.
Figure61ModularStructureoftheServerSideApplication
Thesuggestionservice7,usesanalgorithmthatexecutesanindexedcrawlinthedatabase for the words currently entered by the user. Words shorter than 3letters are not included to avoid a high number of results. Also the resultsreturned by the suggestion algorithm are sorted by popularity, such that themostpopular tracks comeup in the limitednumberof suggestions at an earlystageoftyping.
The server also offers the possibility tomonitor and change its state by othermeansthantheclientapplicationortheconsole.Aservletconnectstotheserviceand enquires status information like: state of themusic space, state of the dbconnectionorerrorsthatwerethrownuponclientrequests;thisinformationisprinted out in a web site so it can be accessed via a web browser. The
7Servicethatproposestracksasresultofuserinsertedwords.Describedlater.
20
administrationtoolcanbeextendedtoimplementfunctionalitytoreinitializetheserver,toreadandoutputuserstatisticsetc.
6.1.3 TheCommunicationUsing only standardized communication between the client and server helpsfuturedevelopersto interchangethecomponentsseamlessly. It’s important forcalls to be serviceddirectly, to return the requested values and to finishuponreturning. Otherwise, if calls depend on local variables and states, thecommunicationbecomesverycomplex.Theonlytwostatestheservicecanbeinare: initializing and running. The initializing state is only at startup so it isassumed no request ever finds the server in this state (except for theadministrationinterface).
Well‐defined interfaces are at the connection of the client application to theclientsocketoftheserver;betweentheclientsocketandthecoreoftheserverandbetweenthecoreandeachofthemodulesoftheserverapplication.Severalobjects have been specially designed for – and are used only in – thecommunication between server and client. Each request is executedimmediately, sequentially without (many) threads running concurrent tocompute theresults.Responses torequestsaresynchronous ‐alwaysreturnedtothecallandnotsentbackatalatertime(whendone).
6.2 MusicSpace(KDTree)Themusicspaceisbuiltoftracksthataremappedtopointsina10dimensionalfield. To be able to find nodes in this space and get their neighbors, it isnecessarytofindamethodofstoringthedatatobeaccessibleinafastway.AKDTree (Wikipedia,KDTrees, 2009) is a right choice for this, as it is very fast atcomputing thenearest neighbor –O(logn) – and alsonearest neighborhood –basically the two functions needed for the proposed playlist generationalgorithm.
There are only few libraries that offer solutions forKDTree implementations.OneofthemisPerst8,adatabasethatcanstoredatainaKDTree.AsPerstisstillunder development, the KD Tree algorithms are not complete and there is noefficient implementation for retrieving the nearest neighbor or nearestneighborhood.TheadvantageofPerstwasthatthewholemusicspacewouldn’thavebeenstoredintothememory,butratherinadatabase.
ThesecondoptionthatisalsousedbecauseoftheshortcomingsofPerst,isaKDTreefullyloadedintothememory.KDTree.jarisaKDTreeimplementinglibraryforjava.ThedisadvantageofusingKDTree.jaristhehugeamountofusedsystemmemory,occupiedbythemusicspace,butalsothereloadingprocessthathastoruneverytimetheserverisrestarted.Howeverthereisalsoanadvantagewhensavingthetreeinmemory:highaccessspeed.Onlythecoordinatesofatrackaresaved in the tree, together with the trackID and the track’s popularity. Thetrack’s title and artist together with other information is crawled from thedatabaseatruntimeasthisoccupiestoomuchmemory.
8Opensource,object‐orienteddatabase–www.mcobject.com/perst/
21
6.3 PlaylistGenerationAlgorithm
6.3.1 ExistingSolutionsThebaseoftheplaylistgenerationalgorithmisthemusicspacecontainingdataabout similarity between tracks. The music space basically states that thesmallerthedistancebetweentwotracks,themoresimilartheyare.Thetopicofcreating algorithms that use this property of the music space to generateplaylists hasbeen addressedby severalETH internal projectsdiscussed in theRelatedWorkSection.
Theexistingmusicexplorerwebapplicationallowstheuser,inavisual2Dway,toselect twotracksrepresenting thebeginningandtheendofhis/herplaylist.The algorithm thendraws a line in themusic space connecting the two tracksandaddstracks found inthevicinityof this lineto theplaylistsuchthat in theend the user has a playlist that represents a smooth transition between thechosen tracks. Some problems arise from using this approach, because thealgorithmneverknowswhattwotrackstheuserselects.Ifhe/shejuststatestwoofhis/herfavoritetracks,thedistancebetweenthesetwoisveryshortinmusicspacemetric,notallowingthealgorithmtofindoptimaltransitiontracks.Averyhigh distance might make the transition not smooth enough. Optimizing inrespecttothisvariable–distance–isadifficulttaskandit’sevenmoredifficulttoproveitscorrectness.Alsoanotherdownsideisthatthegeneratedplaylisthasafixedsizeandcannotbeextended.
Also the Amarok2 plug‐in (Känel, 2008) uses a similar way of computingplaylists.Thedifferenceisthatitdoesnottakeonlytwoinputsbutanindefinitenumber, such that the user can state several of his/her favorite tracks. First,thesearereorderedsuchthataminimalpathisusedtoreachallofthem.Next,thealgorithmwalksalong thepathswithapredefinedstepsizeandaftereachstepitchoosesthenearesttracktothatpointinthespace.Afterreachingtheenddestination,itrandomlychoosesapointinthegraphandwalkstothatonesuchthattheusercanexploreothertypesofmusic.Thealgorithmisintuitiveandhasseveral advantages; the user can insert several favorite tracks and implicitlyhelps the application to understand his/her music taste better. The randomchosenpointat theendof thewalkencourages theuser toexplorenewtracksbutevenmore,itallowsthegenerationofan‘infinitelylong’playlist.
6.3.2 FirstApproachBoth the approaches presented in the previous section tend to exceed thesimplicity that’s set as a goal of this application. Also the user might beoverwhelmedwhenasked to insert several tracks;he/shemight justhaveonetrackinmindthathe/shelikestolistento.
Onefirstapproachwhendealingwithonlyoneseedtrackistosimplytakethekclosesttrackstotheseed.Throughthis,theuserislikelytolikethegeneratedlistof tracks.An immediateproblem iswhen trying togeneratean ‘infinitely long’playlist – the transition between the tracks is not very smooth after a while.Figure 6‐2 shows succession of tracks in a playlist after a certain number oftrackshavebeenalreadyplayed.Youcanseehowthealgorithmmakesverybigjumps toget the track that’snearest to theseed.Note that red tracksareones
22
thathavealreadybeenpickedbythealgorithmpreviouslyandcannotbechosenanymore.
Figure62Jumpingbehaviourwhenusingstaticseed
A modification of this algorithm might bring the solution to this problem:reassigning the seed for each iteration (Figure 6‐3). The playlist generatingalgorithmisasuccessiveiterationofanalgorithmthatchoosesthenearesttrackto the seed, followed by setting the newly found track as seed for the nextiteration.Inthiswaytheapplicationavoidsasmuchaspossiblejumpsoverlongdistances inmusic spacemetric. In the same scenario as the last example, themodifiedalgorithmperformswithoutbigjumps.
23
Figure63Smootherjumpingwhendynamicallyassigningseeds
The modified algorithm also has other advantages: the distance between thechosen tracks is alwaysvery short, guaranteeinga smooth transition.Notonlycan the algorithm provide an infinite number of tracks, but also each track isguaranteedtobeverysimilartothepreviousone.Therandomwalkingbehaviorof the Amarok algorithm (Känel, 2008) is also simulated, allowing the user torandomlyexploreunknownregionsofthemusicspace.
6.3.3 ImprovementChoosing the nearest track to the current seed provides results equallydistributed regarding the popularity of the tracks. But the probability of usersenjoyingthereturnedtracksisgreaterifthepopularityofthereturnedtrackismaximized. An improvement suggestion is therefore not to return the tracknearesttotheseed,butthetrackinthevicinityofthenodethathasthehighestpopularity.Tofindit,thealgorithmqueriesforthefirst10tracksinitsvicinityand takes the onewith themaximum popularity. Figure 6‐4 shows a playlist‐computingexamplewithandwithoutthisenhancement.
24
Figure64Algorithmperformancewith(right)andwithout(left)vicinityprioritysearch
The example in Figure 6‐4 shows the algorithm performing with or withoutvicinitypopularity search (thevicinity consistsof three tracks).On the left thealgorithm selectsmuch less popular tracks in the first 10 than the one on theright;butallthiscomesatacost:thestepsizeoftherightalgorithmisnomorethanthreetimesgreater(onaverage)thantheoneontheleft.
Because songs of the same artist are usually very close together in themusicspace,theapplicationtendstoselectseveralsongsofthesameartistandplacethemsuccessively in theplaylist.This isanundesiredbehavior.Toavoid it thealgorithmcanbetaughttoalsoavoidtrackswhoseartisthasappearedinthelast4 tracks. By applying this additional constraint, the algorithm avoids placingtracksofthesameartistlessthen4tracksapart.
Whensearchingfortracksinthevicinityofaseed,tracksthathavealreadybeenaddedtothisplaylistinthepastareomitted–asarule.Doingso,ifthealgorithmwasnotable to ‘move’ toadifferentregion in themusicspace itstartspickingthe lower popularity tracks. So after 24 tracks the algorithm can start addingtracksthatwerepresent24tracksago.Why24?24trackswithanaverageof5min/trackarealmosttwohoursofmusic–enoughtimetore‐enjoythetracks.
6.4 UserFeedbackThe importanceofuser feedback(both indirectanddirect)hasbeendiscussedthroughout several sectionsof this report.Most importantwhendesigning thefeedbackalgorithmwasthedecisionwhatdatatostoresothatitisofusetoanyfutureevaluation.
6.4.1 DirectandIndirectUserFeedbackDirectuserfeedbackisrepresentedbythefeedbacktheuserchoosestosubmittotheapplicationinordertohelpthecommunity.Currentlytwotypesofdirectfeedbackare implemented: ‘Quality is bad’ and ‘I don’t like the track’.Because
25
theapplicationdoesn’tdirectlyreacttofeedback(yet)thisfeedbackisstoredforalaterprocessing.Thisdirectfeedbackshowsforexamplelearnhowmanyusersactuallyusedthefeedbackalgorithm.
Theusersshouldhit‘Qualityisbad’tosignalthataYouTubevideoofatrackhasbadqualityordoesn’tshowthetrackatall. Ina futureversion, theapplicationimmediatelyreactstothisfeedbackandproposeadifferenttrackor,evenbetter,lettheuserproposeabettertrackbypresentingalistofalternatives.
The‘Idon’tlikethistrack’isintendedforusersnotifyingthealgorithmthatthechosentrackwasnotchosenrightbytheplaylistgenerationalgorithm,becausethisalgorithmshouldonlygeneratetracksthattheuserlikes.
Indirect user feedback is feedback recorded and reported by the clientapplicationandwithouttheknowledgeoftheuser.It’simportanttologalsothebehavior and commands of the users to be able to figure out algorithm andapplicationflaws.Forexample,ifauserlistenstothefirst5secondsofthefirstthree tracks, but listens the whole fourth track, it means that the first threetrackswerenotagoodchoiceforthisuser.
6.4.2 ActivityandFeedbackLogStructureBecausemost computations of the application are doneper sessionbasis, it isgoodtokeeptrackoftheuser’ssessionsandtheiractivityinthesesessions.Thatis why the application uses a table called tlbFeedbackSessions to saveinformation about the session. This information includes the sessionID of thesession – which at the same time is the date and time the session was firststartedstatedinmillisecondssinceJan1st,1970–andtheIPaddressoftheuserto compute a user‐country statistic and to help user‐tracking over severalsessions.
All commands issued by the users are saved in a table calledtblFeedbackCommands.Mostof thecommands issuedby theuser(asdirectorindirect feedback) but also commands issued automatically by the application(likeatrackfinishingplayback)arerecordedtogetherwiththeirtimeofissue–at the same time being a commandID, the associated sessionID, the operationcode and the trackID upon which the command was executed. The list ofoperationcodeswiththeirexplanationisfoundinTable5‐1.
Table61OperationCodesLogs
OperationCode Command TrackID‐1* Reportbadvideo ‐2* Reportbadchoice 0 Atrackreachedtheendandthe
nextisplayedThenexttrackID
1 Manuallyselectatracktoplay 2 Useoneofthetracksasseed 3 Manuallyremoveatrack 4 Trackliftedtobemoved** 5 Trackputdownaftermove** trackID beforewhich the
26
dragged track was putdown, or ‐1 if it was putdown as last track in theplaylist
6 Togglerepeatoff 7 Togglerepeaton 101*** GeneratePlaylistCommand Theseed102*** GetmoreTracks Theseed103*** RefreshPlaylist Theseed
*Negativeoperationcodesshowdirectuserfeedback,whilepositivecodesshowindirectuserfeedback.
**These twocommandsaresupportedbutnot implementedon theclientsideapplication. Hence, if a track is moved no command trigger is issued to theserver.
*** Commands having codes of the form 10* are not issued by the client sideapplication, but by the service itself, when the user requests new tracks forexample
ThetabletblFeedbackTracksholdssomespecialcasedata.Whentheuserissuesa generate playlist command, the service logs that command intblFeedbackCommandsbutalsosavesthetracksreturnedtotheuserasresultofhis/her request in the tblFeedbackTracks table. Each track is saved with thefollowingfields:
• commandsID–representingthecommandIDwhoseresultgeneratedthistrackID;
• thetrackID;• theposition,representingthepositionofthetrackinthereturnedarray+
videoID.
The latter is important forexampleforevaluatingcommands like1–Manuallyselectatracktoplay.Iftheuserdidmanuallyselectatracktoplay,itcouldmeanthistrackisoneoftheusersfavoritessothealgorithmcanseewhereinthelistitwasandhowthealgorithmcouldbeimprovedtohavethistrackappearearlierintheplaylist.
7 ConclusionsThe conclusions section summarizes the projects final state and reconsiders themainideasandresults.
The goal of this thesis was to implement a user friendly, very simple to use,onlinemusicplayer.Theapproachsetoffbydecidingwhichthegroundrulesoftheimplementationare.Topursuethemainruleofcreatinganeasytousemusicplayer, an in‐browser implementation has been chosen but this couldn’t livewithout a server implementing some services the client application couldn’t
27
handle by itself. Among these services, the most important was to generateplaylistsforthemusicplayerbasedontracksimilarityanalysis.
TheUIwas thenext thing todebateandhowtodesignanUI that isbothverysimpleandintuitive,butalsopowerfulenoughtoprovideenoughinformationtothe service when asking for playlists and also to be able to record the user’sactionforafutureevaluation.
YouTube has been chosen as main music content provider and the mainarguments that brought it a step in front of Last.fm were the good enoughcoverage of the music space and an availability of full‐length tracks, whereasLast.fm usually provides only a 30 seconds snippet for a track. Analysis ofYouTubecoverageshownthatinareallifeexample,YouTubeofferscontentforabout 80% of the tracks. Unfortunately, the current implementation can findonly 60% of the tracks in a satisfactory quality range. However it has beenshownthatwithpost‐processingoftheYouTuberesults,anadditional17%canbemappedtogoodqualitytrackimplementationsonYouTube,makingitabletoprovidegoodresultstoabout77%oftheuser’srequests.Alsoithasbeenshownthat through a good feedback system, these numbers can be boosted evenfurther.Notethatthesepercentagesreflecttherealworldusagestatisticsoftheapplication(i.e.highlypopulartracks);theycannotbegeneralizedforthewholemusicspace.
The playlist generating algorithm is based on analysis of the provided musicspace and its possibilities as similarity measurement tool. Several algorithmspreviously developed by ETHmembers were presented and analyzed both inrespecttotheirownadvantagesandlimitationsbutalsotheiruseinthecontextof this project. An algorithm has been developed that uses only one seed tocreate an ‘infinite’ series of succeeding tracks – meaning tracks that are eachverysimilartothepreviousandnextone.Thesimplicityofthealgorithmmakesitpossible to requestanynumberof tracks fromthesystemand thesimilaritymetric between two successive tracks to be constant over the whole list ofreturnedtracks.Also,thelistattemptstotaketheuseronrandomwalksinthemusic space to explore new regions of music. Enhancements brought to thealgorithm,likeusingthepopularityvalueofeachtracktomakeamoreadvancedselection of neighboring tracks, raised the averagepopularity of the outputtedtracksmakingthealgorithmprovidetracksthatareevenmoreattractivetotheusers.
Thegoalofbuildingasimpleskeletonapplicationforprovidinganonlinemusicplayer has been achieved. The application is a good playground for furtherdevelopmentsorinnovationsespeciallyinthealgorithmicpart.
8 DiscussionandFutureWorkThe Section dealswith the ideas that didn’tmake it to the development processmainlybecauseofthetighttimeschedule,butthatareimportanttomentionandatsomepointoranotherhavetobeaddressedinfuturedevelopment.
28
8.1 AchievementsThe goal of the thesis was to create a simple application to be used as aplaygroundforfutureideasofprovidingvaluablemusiccontenttousers.Alsoasimplealgorithmwasdevelopedthatdeliversgoodplaylistswithminimalinputfromtheuserside.
8.2 FutureWork
8.2.1 NicetoHaveEspeciallyonthesideofplaylist‐generatingalgorithmsthereisspaceforalotofnewideas.PastETHinternalreportshaveshownthattherearealotofideasthatcanmakethebestoutofthegivenmusicspace.Someoftheideaswerediscussedinthisreporttofindanewalgorithmthatcombinestheadvantagesoftheotheralgorithmswhilerepressingthedownsides.Ofcoursenoalgorithmisperfectineverysense,butespeciallythisreportshowsthatonecanfindabetteralgorithmas other approaches, as long as it’s customized on the targeted solution.Algorithms that requiremore elaborate user input canbe easily deployed andtestedonthedevelopedapplicationbecauseofitsmodule‐basedstructure.SomeexamplesofsuchalgorithmsarefoundintheotherETHinternalreportslikethemusicexplorerwebapplication, theAmarokpluginanda (not furtherdetailed)like/dislike track generation algorithm (developed for the Android platform,(BossardL.,2008)currentlyundersubmission).
When discussing the KDTree and its all‐in‐memory implementation, thedisadvantagesofthissolutionhavebeenpresented.AfuturestepistointegratetheKDTreemoreintoadatabase,ratherthanstoringthedatainmemory.
InretrievingtheresultsfromYouTubetheapplicationdoesperformwellonlyon60%ofthetracks.Apost‐processingalgorithmhasbeenpresentedthathelpstherising thismargin to77%; thisalgorithmhasnotbeenyet implemented in theapplication.Alsotheintegrationofuserfeedbackisveryhelpfulifconsidered.
Dependingon theconcurrentuseraccesson theservice,partsof,or thewholeconceptofsharedresourceswouldhavetobegeneratedallover.Especiallythesuggestion retrieval algorithm, currently running in theServer’s core, couldbeexternalized to amodule and enhanced not to crawl the database each time auserhitsakeystroke.
8.2.2 FeedbackAnalysisandIntegrationThe feedback recorded by the application is very important for it’s futuredevelopment.Itgivesalotofinformationabouttheusersbehaviorandallowsagoodanalysisof theiractionswhenusingtheapplication.Basedontheresults,someofthepartsoftheapplicationcanbechanged,mostprobablytheplaylistgenerationalgorithmifitturnsoutthatthepeoplearenotverysatisfiedwiththeresults.
Even more important is to analyze and react to direct user feedback. This isespeciallyessentialinmanagingthequalityoftheYouTubevideosofthetracks.Because this quality warranty is out of the hands of this application it’simportanttoreacttotheuser’sfeedbackonthismatter.
29
Implementing a mechanism that allows users to suggest a better YouTubecontent based on several proposals is an important tool for the currentapplication,becauseitwouldallowtheuserstodirectlyhelpatprovidinggoodmusiccontent.Through implementinganadvanceduser feedback function, theapplicationcouldbedesignedtoanalyzethereceivedfeedbackandtoupdateitsstateautomaticallyandinrealtime,forthebenefitofthewholecommunity.
8.2.3 UserAwarenessAnother point only touched by the current implementation is the Facebookintegration.TheapplicationisavailableonFacebookanduserscanrecommendit to their friends,but theapplication isnotusing theaccess to theuser’sdatayet. This data could prove very important, especially for generating theapplication’sown–unbiased–tracksimilarityspace.Trackslistenedtobyuserscould be analyzed over several sessions, not only session‐based as it happensnow,moreoveritcouldbecomparedtothehistoryoffriendstodeterminewaysofrecommendingtracksbasedonthefriends’preferences.
Havingsuchaknowledgeabouttheuserscouldleadtocompletelynewwaysofgenerating playlists, maybe without any user interaction at all… similar toLast.fm’sapproach,butwithpossiblybetter resultsbecause theplaylists couldbebasednotonlyontheprivatebutalsoonthefriends’trackhistory.
9 BibliographyBossard, L. (2008). Pancho The Mobile Music Explorer. Zürich:DCG.TIK.EE.ETZH.CH.
Bossard, L. (2008). Visually and Acoustically Exploring the HighDimensionalSpaceofMusic.Zürich:DCG.TIK.EE.ETHZ.CH.
Gonukula, A., Kuederli, P., & Pasquier, S. (2008).MusicExplorer: Exploring theSpaceofSongsonyourPC.Zurich:DCG.TIK.EE.ETHZ.CH.
Google. (2008, 0418).GoogleWebToolkit. Retrieved0418, 2008, fromGoogeCode:http://code.google.com/webtoolkit/
Lorenzi, M. (2007). Similarity Measures in the World of Music. Zurich:DCG.TIK.EE.ETHZ.
Unknown.(2008).AmarokPlugin.Zürich:DCG.TIK.EE.ETHZ.CH.
Wikipedia. (2008, 04 18). AJAX (programming). Retrieved 04 18, 2008, fromWikipedia:http://en.wikipedia.org/wiki/Ajax_(programming)
Wikipedia. (2009, 04 22). KD Trees. Retrieved 04 22, 2009, from Wikipedia:http://en.wikipedia.org/wiki/Kd_tree
