A Subsequence Matching with Gaps-Range-Tolerances Framework: A Query-By-Humming Application

  • View
    15

  • Download
    0

Embed Size (px)

DESCRIPTION

A Subsequence Matching with Gaps-Range-Tolerances Framework: A Query-By-Humming Application. Alexios Kotsifakos University of Texas at Arlington, University of Athens (Greece). Joint work with Panagiotis Papapetrou (Aalto University, Finland) Jaakko Hollmen (Aalto University, Finland) - PowerPoint PPT Presentation

Transcript

  • A Subsequence Matching withGaps-Range-Tolerances Framework:A Query-By-Humming ApplicationAlexios KotsifakosUniversity of Texas at Arlington, University of Athens (Greece)Joint work withPanagiotis Papapetrou (Aalto University, Finland)Jaakko Hollmen (Aalto University, Finland)Dimitrios Gunopulos (University of Athens, Greece)

    VLDB 2011

  • *OutlineMotivationRelated WorkProblem SettingSMBGT: The proposed methodExperimental EvaluationSummary and Conclusions

  • *Introduction MotivationSuppose you hear a song but cannot recall its nameOne solution: Hum a short part of it and perform a search on a music repository to find it or songs similar to it.

    Query-By-Humming (QBH): Given a hummed query song search a music db for the K most similar songs.

    Subsequence Matching: Given a query sequence and a db of sequences, identify the subsequence in db that best matches the query.

  • *Subsequence Matching in a DatabasedatabasequeryWhat subsequence of any database sequence is the best match for Q ?

  • *Converting music pieces to time seriesEach music piece is a sequence of notes.

    Notes consist of pitch (frequency) and duration.

    Part of the Happy Birthday music score.The 2D time series representation of the above music score.Pitch interval : distance between two pitches.IOIR : duration ratio of two consecutive notes

  • QBH: A real exampleHummed query: Yesterday (Beatles)Target: Yesterday

    VLDB 2011

  • QBH: A real exampleHummed query: Yesterday (Beatles)Matched target subsequence

    VLDB 2011

  • *QBH: A noisy application domainSlight or more serious errors due to instant or temporary key or tempo loss error-tolerant method.

    Allow skipping a bounded number of elements in both query and target sequences due to serious humming errors.

    Constrain the length of the matching subsequence.

    Optional constraint to the min. number of matching elements.

  • *Our contributionsA subsequence matching framework that allows fora constrained number of gaps on both query and target sequences,variable tolerance levels in the matching (no probabilistic model),a matching range that constrains the max. match length,bounding the min. # of matched elements.

    SMBGT: Given a query Q and a target sequence X (with |Q|

  • *OutlineMotivationRelated WorkProblem SettingSMBGT: The proposed methodExperimental EvaluationSummary and Conclusions

  • *Related WorkDP-based methodsWhole sequence matching: DTW and variants (cDTW, EDR, ERP) robust to misalignments and warps, LCSS allows for gaps, edit distance, Iliopoulos et al., etc.Subsequence matching: SPRING finds the subsequences of evolving numerical streams closest to a query, DTWc and DTWs account for local adjustments of tempo, Han et al. (2007) performed uniform segmentation and sliding windows for near exact matching where user selects segments length.n-gram methods for near exact matchingAll account for 1D + fail to handle noise imposed by users.Probabilistic-based methods (HMMs)

  • *OutlineMotivationRelated WorkProblem SettingSMBGT: The proposed methodExperimental EvaluationSummary and Conclusions

  • *Problem Setting: Representing Musical PiecesExpressing pitchAbsolute pitch: frequency of the note.Pitch interval: frequency difference between two adjacent notes.

    Encoding durationInter-Onset-Interval (IOI): difference in time onsets of two adjacent notes.IOI Ratio (IOIR): ratio of IOIs of two adjacent notes.Log IOI Ratio (LogIOIR): logarithm of IOIR.

    To save computational time we deal with note transitions:,

    Performed quantizations on pitch interval with modulo 12, and on LogIOIR to the closest integer or closest value in [-2, 2].

  • *Problem Setting: Definitions

    X = {x1, , xn} representing music piece of length |X|,pair of real values for pitch and duration, DB={X1, , XN }music db,X[ts : te]={xts , , xte} subsequence of X (not always continuous), Q = {q1, , qn }.

    Definition 1 (Variable error-tolerant match):

  • *Problem Setting: Definitions Definition 2 (Common bounded-gapped subsequence): SMBGT: error-tolerant matching denoted as -match.

  • *Problem Setting:Example and Problem FormulationExample = 2, = 1, r = 6, = 1 (absolute tol.)Q = {6, 3, 10, 5, 3, 2, 9}, X = {1, 1, 3, 4, 6, 9, 2, 3, 1}Q[2:6] with GQ = {2, 4, 5, 6} and X[3:8] with GX = {3, 4, 7, 8} Q = {6, 3, 10, 5, 3, 2, 9}X = {1, 1, 3, 4, 6, 9, 2, 3, 1}

    Problem: Given a database DB with N sequences of arbitrary lengths, a query sequence Q, and positive integers and r, find set of the top-K subsequences withSMBGT(Q, X) = {Q[2:6], X[3:8]}

  • *OutlineMotivationRelated WorkProblem SettingSMBGT: The proposed methodExperimental EvaluationSummary and Conclusions

  • *SMGTSMGT: = = inf.Array a (|Q+1|)*(|X|+1). For all i in {1,,|Q|}, j in {1,,|X|}

    Matrix s keeps in si,j for each ai,j the start point of its best alignment.Online computation: two 1D arrays prev, cur, track scores and start points (prev.value, cur.value, prev.start, cur.start) of j-1 and j columns of a.Best solution (SMGT) in best = (bestvalue, beststart, bestend) accounting for .Check subsequences with length r (Reset()). Update values:Match ai,j = 1Mismatch left cell can be inherited if length < r-1, top cell always.

  • *SMBGT

    Mismatch (propagation()): store the largest # of matched elements that can be propagated vertically or horizontally, not violating and .

    Astart , Bstart : store the latest match positions in X and QAdditional propagation check in Reset().

  • *SMBGT: Example - InitializationQ = {0, -4, 1, 2, -2}, X = {0, 0, -4, 3, 0, 2, -3, 1} = 2, = 1, = 3, r = |Q| = 5, = 0Alignment array

    X00-4302-31Q00000000000-401020-20

  • *SMBGT: Example - Computation

    aXQ00-4302-3101-411020-20

    AstartXQ00-4302-3101-411020-20

    sXQ00-4302-3101-411020-20

    BstartXQ00-4302-3101-411020-20

  • *SMBGT: Example - Computation

    aXQ00-4302-31011-411100200-200

    AstartXQ00-4302-31012-412100200-200

    sXQ00-4302-31012-412100200-200

    BstartXQ00-4302-31011-411100200-200

  • *SMBGT: Example - Computation

    aXQ00-4302-310111-411210022000-2000

    AstartXQ00-4302-310122-412310032000-2000

    sXQ00-4302-310122-412210022000-2000

    BstartXQ00-4302-310111-411210022000-2000

  • *SMBGT: Example - Computation

    aXQ00-4302-3101111-411221002220000-20000

    AstartXQ00-4302-3101222-412331003320000-20000

    sXQ00-4302-3101222-412221002220000-20000

    BstartXQ00-4302-3101111-411221002220000-20000

  • *SMBGT: Example - Computation

    aXQ00-4302-31011111-411222100222200000-200000

    AstartXQ00-4302-31012225-412333100333200000-200000

    sXQ00-4302-31012225-412222100222200000-200000

    BstartXQ00-4302-31011111-411222100222200000-200000

  • *SMBGT: Example - Computation

    aXQ00-4302-310111111-411222110022202000003-2000003

    AstartXQ00-4302-310122255-412333510033302000006-2000006

    sXQ00-4302-310122255-412222510022202000002-2000002

    BstartXQ00-4302-310111111-411222110022202000004-2000004

  • *SMBGT: Example - Computation

    aXQ00-4302-310111111-411222110022202000003-2000003

    AstartXQ00-4302-310122255-412333510033302000006-2000006

    sXQ00-4302-310122255-412222510022202000002-2000002

    BstartXQ00-4302-310111111-411222110022202000004-2000004

  • *SMBGT: Example - Computation

    aXQ00-4302-310111111-411222110022202000001-2000003

    AstartXQ00-4302-310122255-412333510033302000006-2000006

    sXQ00-4302-310122255-412222510022202000006-2000002

    BstartXQ00-4302-310111111-411222110022202000004-2000004

  • *SMBGT: Example - Computation

    aXQ00-4302-310111111-411222110022202000001-2000001

    AstartXQ00-4302-310122255-412333510033302000006-2000006

    sXQ00-4302-310122255-412222510022202000006-2000006

    BstartXQ00-4302-310111111-411222110022202000004-2000004

  • *SMBGT: Example - Computation

    aXQ00-4302-3101111111-411222111002220020000011-20000011

    AstartXQ00-4302-3101222555-412333551003330020000066-20000066

    sXQ00-4302-3101222555-412222551002220020000066-20000066

    BstartXQ00-4302-3101111111-411222111002220020000044-20000044

  • *SMBGT: Example - Computation

    aXQ00-4302-31011111110-411222110100222002200000112-200000111

    AstartXQ00-4302-31012225550-412333550100333008200000668-200000666

    sXQ00-4302-31012225550-412222550100222005200000665-200000666

    BstartXQ00-4302-31011111110-411222110100222003200000443-200000444

  • *SMBGT: Example - SolutionQ = 0-412-2bestvalue = 3, beststart = 2, bestend = 6X = 00-4302-31skipped

    aXQ00-4302-31011111110-411222110100222002200000112-200000111

  • *OutlineMotivationRelated WorkProblem SettingSMBGT: The proposed methodExperimental EvaluationSummary and Conclusions

  • *Experimental Setup: Data & EvaluationMusic DB: 5643 freely available MIDI files.

    6 synthetic query sets, 100 queries/set of lengths 13-137Q0: exact segments of the db.Q.10 Q.50: randomly modified 10-50% elements (both dimensions) of each query in Q0, with at most 3 consecutive elements changed.

    100 hummed queries of lengths 14-76.

    DP and Model-based competitor methodsDTW variants: DTWs, DTWc, SPRING modified to allow r.Edit: Version for music modified for LogIOIR + quantizationsIliopoulos et al.: Elastic version for subsequence matching supporting constant and variable tolerance, modified to allow r.HMMs: Each db sequence modeled by an HMM.

  • *Experimental Setup: Evaluation measures

    Recall: % of queries for which correct answer is in top-K.MRR: mean inverse rank of queries in their top-K results.AR: average rank of a