Upload
jack-white
View
221
Download
0
Tags:
Embed Size (px)
Citation preview
Feature Point Detection and Curve Approximation for Early
Processing of Free-Hand Sketches
Tevfik Metin Sezgin and Randall Davis
MIT AI Laboratory
Metin Sezgin
Outline
• Motivation and problem statement• Feature point detection
- Curvature and speed as information sources- Average based filtering; Scale space filtering- Hybrid fit generation- Handling strokes with curves
• Evaluation• Related work• Future work
Metin Sezgin
Terminology
•Sketch: Informal,messy diagramsconsisting of several strokes
•Stroke: Array of timestampedpoints
•Feature Point: Cornersof a stroke (vertices)
Metin Sezgin
Task• Given a freehand stroke, generate a
description of the stroke in terms of geometric primitives such as circles, lines, and curves
Metin Sezgin
Lifecycle of a stroke
Data Collection
Data Collection
Vertex DetectionVertex
Detection
PolylineFit
PolylineFit
Oval FitOval Fit
Curve Detection
Curve Detection
Line Fit
Line Fit
Complex Fit(lines and
curves)
Complex Fit(lines and
curves)Best Fit
SelectionBest Fit
Selection OutputOutput
Metin Sezgin
Lifecycle of a stroke
Data Collection
Data Collection
Vertex DetectionVertex
Detection
PolylineFit
PolylineFit
Oval FitOval Fit
Curve Detection
Curve Detection
Line Fit
Line Fit
Complex Fit(lines and
curves)
Complex Fit(lines and
curves)Best Fit
SelectionBest Fit
Selection OutputOutput
Metin Sezgin
Collecting Strokes
Metin Sezgin
Vertex detection
Metin Sezgin
Vertex detection
Curvature
Metin Sezgin
Vertex detection
Curvature Speed
Metin Sezgin
Problems with local extrema
• Input is noisy- Digitization noise- Hand jittering (tremor)
• Picking all local extrema introduces many false positives
• Picking the global extremum gives us a single point
• Picking the n-best extrema doesn’t help either
Metin Sezgin
Solutions
• Average-based filtering- Compute threshold based on data average- Partition using this threshold- Select global extremum in each partition
Metin Sezgin
Average-based filtering
Curvature Speed
Metin Sezgin
Average-based filtering
• Pros- Reliable (when data isn’t too noisy)- Fast: O(n)- False positives easy to filter out
• Cons- Still false positives- Need pre-set constant for scaling
Metin Sezgin
Scale-space filtering
• Real world data has features at many scales
• Represent the data multiple different scales
• Selecting scale(s) at which data is meaningful
Metin Sezgin
Deriving the scale space
Convolve the original signal with Gaussian signals of increasing width
Feature points in the curvature scale space
Metin Sezgin
Selecting a scale
Metin Sezgin
Selecting a scale
Metin Sezgin
Scale selected by the algorithm
The original strokeOutput generated at the
scale chosen by our algorithm
Metin Sezgin
Handling extremely noisy strokes (cont.)
Curvature fit using SSF, 9 fps Speed fit using SSF, 7 fps
Metin Sezgin
Combining information sources
• Individual detection methods may miss points
Speed fit missing a point
Curvature fit missing a point
• Need a way to combine information provided by each method
• Generate a series of hybrid fits and pick the best
Metin Sezgin
Hybrid fit generation
• Given N fits generated using N different methods- Within each fit, compute vertex certainties and sort by
certainty
• Initial fit contains points common to all methods• Generate N new fits by adding best candidate
from each method. • Select the one of these with the least error• Terminate when error smaller than preset error
bound
Metin Sezgin
Data Collection
Data Collection
Vertex DetectionVertex
Detection
PolylineFit
PolylineFit
Oval FitOval Fit
Curve Detection
Curve Detection
Line Fit
Line Fit
Complex Fit(lines and
curves)
Complex Fit(lines and
curves) Best FitSelectionBest Fit
Selection OutputOutput
Metin Sezgin
Curve approximation
c2
c1
uv
Metin Sezgin
Results
Metin Sezgin
Results
Metin Sezgin
User study results
• All users except one perceived the system as more natural.
• Overall it took users much less time to specify geometries.
• All subjects except one indicated they would prefer an interface that feels like our system for specifying shapes in a sketchy fashion.
• Strokes classified correctly 96% of the time
Metin Sezgin
Related work
• Existing systems lack- Support for arbitrary shape creation with a
single stroke- Automatic feature point detection- Modeless drawing- Natural feel
Metin Sezgin
Future work
• Integration with the new sketch recognition architecture
• Support for over-tracing
• Further investigation of sketching behavior to enhance/aid feature point detection and stroke classification