Video Object Extraction Using Feature Matching Based on Nonlocal MattingMeidya Koeshardianto, Eko Mulyanto and Mochamad Hariadi | ITS Surabaya, Indonesia

What is video object extraction and how to make it easy implementation?

Question :

Matting Equations

• Video object extraction is used for extracting foreground and background

object from still image or video sequences.

𝐼 = 𝛼𝐹 + 1 − 𝛼 𝐵

Constraints (Scribbles and Trimap)

Trimap Interface :

Bayesian Matting (Chuang et al CVPR 01)

Poisson Matting (Sun et al SIGGRAPH 04)

Random walk (Grady et al 05)

Etc..

Scribbles Interface :

Wang&Cohen ICCV 05

Closed Form Matting TPAMI 08

Nonlocal Matting CVPR 11

Etc..

𝛼 = 1

𝛼 = 1

𝛼 = 0

𝛼 = 0

𝛼 ∈ [1,0]

Problems

1. Every frames need constraints to determine Foreground and Background

2. Accurate object extraction on each frames.

• Automatic constraints (stroke/scribbles)

• Feature matching

• Nonlocal matting for Video Object Extracting

SIFT Algorithm is used

Laplacian

TransformVideo / image sequence

Image Template

Nonlocal MattingPoint Matching Alpha Matte

Point Extraction

Procedure

Key Point Extraction

Step 1

Feature Matching

Step 2

Laplacian Transform (Nonlocal Matting)

Step 3

Why nonlocal matting is employed?

Intepretation Laplacian as a graph 𝐺 = 𝑉, 𝐸 where 𝐴 is weight of edge 𝐸(𝑖, 𝑗)

With derivation of 𝑫𝜶 = 𝑨𝜶 then 𝑫− 𝑨 𝜶 ≈ 𝟎 or 𝜶𝑻𝑳𝜶 ≈ 𝟎 where 𝑳 = 𝑫 − 𝑨 𝑻(𝑫 − 𝑨)

The quadratic form 𝜶𝑻𝑳𝜶Measure of smoothness along the

edge of 𝐺

Why nonlocal matting is employed?

If there is a subset of pixels that exactly cluster in the graph implied by Laplacian 𝐿 then the

value of the objective function 𝑞(𝛼) is minimized if the rest of the pixels in the cluster are labeled

Automatic Scribbles - SIFT

Key Detection

Keypoint Descriptor

Automatic Scribbles - SIFT

Feature Matching

𝜃 = 0,2 – 0,4 𝜃 = 0,5

𝜃 = 0,6 𝜃 = 0,7

𝜃 = 0,8 𝜃 = 0,9

Image template for automatic video object extraction

Answer :

Experiments – Samples Video

Conclusion and Road a Head

• Feature matching could be used as scribbles or stoke for object extraction,

nonlocal matting method.

• According from our sample video, it show that objects can be extracted by

most satisfactory parameter using 𝜃 = 0,05

• Future work include investigating updated or replaced image template

periodically. So that, the image template feature will be still recognized on the

frame.