Upload
eyal
View
41
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Wrapping Snakes For Improved Lip Segmentation. Matthew Ramage Dr Euan Lindsay (Supervisor) Department of Mechanical Engineering. Presentation Overview. Brief overview of Visual Speech Recognition Requirements of a good Lip Segmenter One approach: Traditional Snakes - PowerPoint PPT Presentation
Citation preview
Wrapping SnakesFor Improved Lip Segmentation
Matthew Ramage
Dr Euan Lindsay (Supervisor)
Department of Mechanical Engineering
Presentation Overview
Brief overview of Visual Speech Recognition
Requirements of a good Lip SegmenterOne approach: Traditional SnakesAn improved approach: “Wrapping
Snakes”
What is Visual Speech Recognition?
Uses video footage to identify spoken wordsDoes not use any audio information
Audio-only speech recognition accuracy falls dramatically as audio signal degraded
Basic Principle:Mouth shape is based on the underlying
structure of the word being spoken
What makes a good Lip Segmenter?
Accurately represents lip shapeRobust against poor initialisation
Can handle multiple enclosed regionsReliably handles noisy images
Can handle weak lip features near strong noise
Quickly finds the lip shape
One Approach: Traditional Snakes
A form of Active Contour Model
Series of connected points Internal forces:
Tension Rigidity
External forces: Constraints
Image Forces: Based on image features Using snakes to track the outer
lip boundary (Kass et al. 1988)
Kass, M, Witkin, A & Terzopoulos, D, 1988, 'Snakes: Active contour models', International Journal of Computer Vision, vol. 1, no. 4, pp. 321-31.
Traditional Snakes
Problems with Traditional Snakes
Not robust against poor initialisationCannot handle multiple enclosed regions
Cannot reliably handle noisy imagesCannot handle weak target features with
strong noise
Multiple Enclosed Regions
(Traditional Snake)
Weak Features & Strong Noise
(Traditional Snake)
The Problem?
Traditional Snakes don’t continue along features
The Problem?
Traditional Snakes don’t continue along features
The Solution: A Wrapping Force
A substitute for the Image ForceBased on the Image Force
Modified by the snake’s shape and position
When a snake curves away from a feature, apply a force along the featurePushes snake along the feature
Calculating the Wrapping Force
Component of Image Force in the direction of the Snake’s Normal
Wrapping Behaviour
Cutting the Snake
Cut the snake when separate sections come into contact
Benefits:Distinguishes between multiple fully
enclosed regions Ignores areas of noise near the lipsAllows the lip shape to be accurately found Increased robustness of lip segmentation
Pinching Forces
A subset of points a made “Pinch Points”
Non-adjacent pinch points attract when close enough
Pulls sections of the snake together when close enoughHelps with cutting
Multiple Enclosed Regions
(Wrapping Snake)
Weak Features & Strong Noise
(Wrapping Snake)
Properties of Wrapping Snakes
Accurately represents lip shapeRobust against poor initialisation
Can handle multiple enclosed regionsReliably handles noisy images
Can handle weak lip features near strong noise
Quickly finds the lip shape
Wrapping SnakesFor Improved Lip Segmentation
Matthew Ramage
Dr Euan Lindsay (Supervisor)
Department of Mechanical Engineering
Handles Very Poor Initial Position
(Wrapping Snake)
Multiple Enclosed Regions
(Traditional Snake)
Multiple Enclosed Regions
(Wrapping Snake)