Jiaping Zhao and Laurent Itti

shapeDTW: shape Dynamic Time Warping

Time series alignment

Time series alignment✔ Use univariate time series alignment as examples✔ But straightforward to extend to multivariate time

series alignment (see speech signal alignment in our paper)

Time series alignment Align a pair of temporal sequences with:

✔ Local nonlinear distortion✔ Local phase shifting

Sequence A

Sequence B

Time series alignment Dynamic time warping (DTW)

Sequence A

Sequence B

a sequence of scalars !

Time series alignment Limitations of DTW

✔ disregard local structural similarities

Similarity between two points ismeasured by their y-value difference !

A peak is matched to a valley, because of their similar y-values !

Two circled-out points have similar y-values.

Time series alignment Existing remedies

✔ Enforce band constraints (Batista et al. 2011; Candan et al. 2012)

✔ DTW variants (dDTW, Keogh & Pazzani 2001; wDTW, Jeong et al. 2011)

✔ Distance metric learning (Lajugie et al. 2014)

Point-wise distance metric learningEuclidean → Mahalanobis !Although using a different distance metric,distance between points is still derived from their y-value difference !

Band constraintsApplication dependent !

DTW variants: dDTW & wDTWThey still compute distance between points by their y-value difference !

shapeDTW Our solution

✔ Shape Dynamic Time Warping (shapeDTW)● Essentially a DTW algorithm● But further consider local structural similarities

shapeDTW Motivation

✔ Image matching in computer vision● SIFT (Lowe 2004)

● SURF (Bay et al. 2006)

✔ Two pixel similarity is measured based on their local patch similarity, instead of their RGB-value similarity

(a) image matching

(b) sequence alignment

i

jTherefore similarity between point i and j is better to be measured by the similarity between their local subsequences !

shapeDTW Algorithm: two steps

✔ Transform raw sequences into local descriptor sequences

✔ Run DTW

Transform raw sequences intodescriptor sequences

Run DTW to aligndescriptor sequences

shapeDTW Algorithm

✔ transformation

Sequence A

Sequence B

scalar vector !→

Local subsequence around each point !

Raw sequences: a sequence of scalars

Descriptor sequences: a sequence of vectors

Step 1: Transform raw sequences intodescriptor sequences

shapeDTW Algorithm

✔ DTW alignment

Sequence A

Sequence B

scalar vector !→

Local subsequence around each point !

Raw sequences: a sequence of scalars

Descriptor sequences: a sequence of vectors

Step 2: run DTW to aligndescriptor sequences A and B

shapeDTW Algorithm: pipeline

shapeDTW Difference with DTW

Transform raw sequences intodescriptor sequences

Run DTW to aligndescriptor sequences

Run DTW to alignraw sequences

DTW:

ShapeDTW:

shapeDTW Difference with DTW

Sequence A

Sequence B

Key difference: point-wise distance

DTW:

ShapeDTW:

scalar → vector !

Not two scalar difference, but similarity between two local subsequences !

shapeDTW Experiments – alignments

✔ Qualitative alignments