Upload
others
View
6
Download
0
Embed Size (px)
Citation preview
OutlineObject – Scene Analysis
EE 576 - Object - Scene Representation
H.I. Bozma
Electric Electronic Engineering
Bogazici University
March 28, 2012
H.I. Bozma EE 576 - Object - Scene Representation
OutlineObject – Scene Analysis
Object – Scene AnalysisIntroductionModel-Based Object RepresentationGraph Representation
H.I. Bozma EE 576 - Object - Scene Representation
OutlineObject – Scene Analysis
IntroductionModel-Based Object RepresentationGraph Representation
Scene Understanding
Scene analyis (scene understanding) is the ability to make usefuldescriptions of the given scene.
◮ Identify objects and relationships between them
◮ Locate a certain object
◮ Make measurements for inspection
H.I. Bozma EE 576 - Object - Scene Representation
OutlineObject – Scene Analysis
IntroductionModel-Based Object RepresentationGraph Representation
Problems in Scene Analysis
◮ Representation
◮ Occlusion
◮ Image complexity
◮ Scene complexity
◮ A priori knowledge is essential
H.I. Bozma EE 576 - Object - Scene Representation
OutlineObject – Scene Analysis
IntroductionModel-Based Object RepresentationGraph Representation
Representation
◮ Levels of abstraction◮ Specific objects◮ Generic objects◮ Functional representations
◮ Properties of objects: Shape, color, texture, function
◮ Shape representation: Collection of points, boundaries,surfaces, volumes, collection of parts
H.I. Bozma EE 576 - Object - Scene Representation
OutlineObject – Scene Analysis
IntroductionModel-Based Object RepresentationGraph Representation
Projection and Occlusion
Projection and occlusion – create difficulties.Scenes 3-D – images 2-D.Projected shapes – Depend on the viewpoint.Objects occlude each other → Only parts of objects visible →
Problems in segmentation.
H.I. Bozma EE 576 - Object - Scene Representation
OutlineObject – Scene Analysis
IntroductionModel-Based Object RepresentationGraph Representation
Complexity
◮ Images are complex◮ Nature – Monocular, stereo, range◮ Quality – nature
◮ Scenes are complex◮ Large number of objects◮ Occlusion◮ Similar objects◮ Complex shapes◮ Complex patterns◮ Illumination
H.I. Bozma EE 576 - Object - Scene Representation
OutlineObject – Scene Analysis
IntroductionModel-Based Object RepresentationGraph Representation
General Approach
◮ Choose a representation model
◮ Find instances of known models in the scene
◮ Estimate their pose
H.I. Bozma EE 576 - Object - Scene Representation
OutlineObject – Scene Analysis
IntroductionModel-Based Object RepresentationGraph Representation
Complexity
◮ The number of models in the database
◮ Complexity of the image
◮ Complexity of the scene
H.I. Bozma EE 576 - Object - Scene Representation
OutlineObject – Scene Analysis
IntroductionModel-Based Object RepresentationGraph Representation
Matching Models – General Approaches
◮ Template matching
◮ Correlate a template with the image◮ Invariant to translation, sensitive to rotation, scaling and
viewpoint◮ Useful for 2D binary patterns (character recognition)
◮ Feature vectors
◮ Describe by features – Area, moments, Fourier coefficients◮ Solve segmentation◮ Viewpoint invariance is hard
◮ Features and the relationships between them
H.I. Bozma EE 576 - Object - Scene Representation
OutlineObject – Scene Analysis
IntroductionModel-Based Object RepresentationGraph Representation
Representation and Recognition Paradigm
◮ Could the image be a projection of the model?
◮ Match part of a scene with a model → Search for best fit◮ Estimate object pose◮ Project model and verify
◮ What types of features to match and how?◮ Feature matching◮ Structured description matching
H.I. Bozma EE 576 - Object - Scene Representation
OutlineObject – Scene Analysis
IntroductionModel-Based Object RepresentationGraph Representation
Recognition Paradigm
Figure: Recognition paradigm
H.I. Bozma EE 576 - Object - Scene Representation
OutlineObject – Scene Analysis
IntroductionModel-Based Object RepresentationGraph Representation
Feature Matching
◮ Match image and model features → Search for best fit
◮ Check for consistency using constraints that depend onrepresentatin
H.I. Bozma EE 576 - Object - Scene Representation
OutlineObject – Scene Analysis
IntroductionModel-Based Object RepresentationGraph Representation
Feature Matching
Figure: Feature matching
H.I. Bozma EE 576 - Object - Scene Representation
OutlineObject – Scene Analysis
IntroductionModel-Based Object RepresentationGraph Representation
Search Tree: Breadth-First
Figure: Breath-first search of a match tree.
H.I. Bozma EE 576 - Object - Scene Representation
OutlineObject – Scene Analysis
IntroductionModel-Based Object RepresentationGraph Representation
Perceptual Grouping
Perceptual grouping: Organization of image features into higherlevel feature
◮ Descriptive – Make geometrical structure explicit
◮ Hierarchical
◮ Invariant – if possible
H.I. Bozma EE 576 - Object - Scene Representation
OutlineObject – Scene Analysis
IntroductionModel-Based Object RepresentationGraph Representation
Perceptual grouping
Figure: Perceptual organization
H.I. Bozma EE 576 - Object - Scene Representation
OutlineObject – Scene Analysis
IntroductionModel-Based Object RepresentationGraph Representation
Perceptual Organization Criteria
◮ Similarity – Features (Lines – Bending energy)
◮ Proximity
◮ Continuation
◮ Symmetry
◮ Closure
◮ Familiarity
H.I. Bozma EE 576 - Object - Scene Representation
OutlineObject – Scene Analysis
IntroductionModel-Based Object RepresentationGraph Representation
Bending Energy
Minimum bending energy - as measured by curvature.
Figure: Perceptual organization
H.I. Bozma EE 576 - Object - Scene Representation
OutlineObject – Scene Analysis
IntroductionModel-Based Object RepresentationGraph Representation
Perceptual grouping – Bending energy
→
Figure: Perceptual organization
H.I. Bozma EE 576 - Object - Scene Representation
OutlineObject – Scene Analysis
IntroductionModel-Based Object RepresentationGraph Representation
Representation By Parts
◮ One approach to structural shape representation is graphrepresentation
◮ A graph consists of two entities: nodes P and edges E (P)between nodes.
◮ Properties of nodes and edges in the model database.
Some examples are:
◮ Line drawings consisting of vertices (nodes) and lines (edges)
◮ Surfaces (nodes) and their relationships (edges)
◮ Complex shape represented by parts (nodes) and therelationships (edges)
H.I. Bozma EE 576 - Object - Scene Representation
OutlineObject – Scene Analysis
IntroductionModel-Based Object RepresentationGraph Representation
Recognition by Structure Matching
◮ Here, the model is based on a graph that represents theobjects’ parts and their relationships.
◮ Given an incoming image, a similar model is constructed andmatched with those in the model database.
H.I. Bozma EE 576 - Object - Scene Representation
OutlineObject – Scene Analysis
IntroductionModel-Based Object RepresentationGraph Representation
Graph Matching
Given G = (P ,E (P),R(P)) and G′ = (P ′
,E′(P ′),R ′(P ′)
◮ Two nodes n ∈ P and n′∈ P
′ form an assignment ifR(n) = R
′(n′)
◮ Two assignments (n1, n′
1) and (n2, n′
2) are compatible ifE (n1, n2) = E (n′1, n
′
2)
◮ Isomorphism: Compatibility for all nodes
◮ Isomorphism: Compabilility for a subset of nodes
H.I. Bozma EE 576 - Object - Scene Representation
OutlineObject – Scene Analysis
IntroductionModel-Based Object RepresentationGraph Representation
Graph Isomorphism
Finding an isomorphism is NP-hard:
H.I. Bozma EE 576 - Object - Scene Representation
OutlineObject – Scene Analysis
IntroductionModel-Based Object RepresentationGraph Representation
Graph Isomorphism – Ex 2
Figure: Graph Matching - IIH.I. Bozma EE 576 - Object - Scene Representation
OutlineObject – Scene Analysis
IntroductionModel-Based Object RepresentationGraph Representation
Aspect Graphs
Aspect graph – A series of two-dimensional views (nodes) of aknown object that are related to each other (edges) via 3Dtransformations.
◮ Generate the aspect graph
◮ Maintain them in some convenient representation in storage,
◮ Match one or more two-dimensional views of an unknownobject against the stored views of the known object,
Recognition problem → Transformed to a series of two-dimensionalmatching problems.
H.I. Bozma EE 576 - Object - Scene Representation
OutlineObject – Scene Analysis
IntroductionModel-Based Object RepresentationGraph Representation
Varying viewpoints
Figure: Different views of an object.H.I. Bozma EE 576 - Object - Scene Representation
OutlineObject – Scene Analysis
IntroductionModel-Based Object RepresentationGraph Representation
Varying Viewpoints
Figure: Different views of a L-shaped object.H.I. Bozma EE 576 - Object - Scene Representation
OutlineObject – Scene Analysis
IntroductionModel-Based Object RepresentationGraph Representation
Aspect Graph
Figure: Aspect graph of an a L-shaped shaped object.H.I. Bozma EE 576 - Object - Scene Representation