Computer Vision at IPMComputer Vision at IPM

Mehrdad ShahshahaniMehrdad Shahshahani

Institute for Studies in Theoretical Physics Institute for Studies in Theoretical Physics and Mathematicsand Mathematics

International Workshop on Computer VisionInternational Workshop on Computer VisionApril 26-30, Tehran,IranApril 26-30, Tehran,Iran

Computer Vision GroupComputer Vision Group

Masoud AlipourMasoud Alipour Somayeh DanafarSomayeh Danafar Ali FarhadiAli Farhadi Hanif MohammadiHanif Mohammadi Nima RazaviNima Razavi Azad ShadmanAzad Shadman Lila TaghaviLila Taghavi Ali-Reza TavakoliAli-Reza Tavakoli

Scope of EffortScope of Effort

Limited to the Analysis of A Single ImageLimited to the Analysis of A Single Image

Object DifferentiationObject Differentiation

SegmentationSegmentation

Conspicuously Absent: Use of a Data BankConspicuously Absent: Use of a Data Bank

MethodologiesMethodologies

Emphasis on Experimental MethodsEmphasis on Experimental Methods

Statistical Analysis Statistical Analysis – Higher Order StatisticsHigher Order Statistics– SVD TransformsSVD Transforms– Application of Methods of Computational GeometryApplication of Methods of Computational Geometry

Memory/PriorsMemory/Priors

Variation of Correlations (cont.)Variation of Correlations (cont.)

Variation of Correlations (cont.)Variation of Correlations (cont.)

Variation of Correlations (cont.)Variation of Correlations (cont.)

Rough Classification of ImagesRough Classification of Images

Rough Classification of Images Rough Classification of Images (cont.)(cont.)

DetectionDetection

Detection (cont.)Detection (cont.)

Detection (cont.)Detection (cont.)

General ConclusionGeneral Conclusion

Analysis of local correlations in a single image Analysis of local correlations in a single image allows the detection of an extraneous object in a allows the detection of an extraneous object in a texture environment.texture environment.

SegmentationSegmentation

Application of analysis of correlations to segmentation of Application of analysis of correlations to segmentation of imagesimages

Requires more elaborate analysisRequires more elaborate analysis

Roughly Speaking, two step process:Roughly Speaking, two step process:

– 1. Identification of regions (windows) containing object.1. Identification of regions (windows) containing object.– 2. Determination of the boundary of the object.2. Determination of the boundary of the object.

Segmentation (cont.)Segmentation (cont.)

Segmentation (cont.)Segmentation (cont.)

Segmentation (cont.)Segmentation (cont.)

Segmentation (cont.)Segmentation (cont.)

General ConclusionGeneral Conclusion By analysis of local correlations segmentation can By analysis of local correlations segmentation can

be achieved on the basis of local structure of be achieved on the basis of local structure of textures.textures.

Not necessary to make use of memory.Not necessary to make use of memory.

Analysis is based on a single image.Analysis is based on a single image.

Complexity of algorithm is O(N).Complexity of algorithm is O(N).

A Test CaseA Test Case

How can one tell the difference How can one tell the difference between a cat and a dog?between a cat and a dog?

The question can be viewed from a The question can be viewed from a neurophysiologic or image processing point of neurophysiologic or image processing point of

viewview..

Can measures of statistical variability be used in Can measures of statistical variability be used in distinguishing between dogs and cats?distinguishing between dogs and cats?

LPC Surfaces LPC Surfaces

One canonically constructs a surface (LPC surface)One canonically constructs a surface (LPC surface)

from the analysis of local correlations of an image.from the analysis of local correlations of an image.

LPC Surfaces (cont.)LPC Surfaces (cont.)

LPC Surfaces (cont.)LPC Surfaces (cont.)

LPC Surfaces (cont.)LPC Surfaces (cont.)

LPC surfaces are highly non-differentiable.LPC surfaces are highly non-differentiable.

Discrete geometry of LPC surfaces.Discrete geometry of LPC surfaces.

Curvature of a triangulated surface.Curvature of a triangulated surface.

Triangulation of a SurfaceTriangulation of a Surface

Curvature of a triangulationCurvature of a triangulation

Curvature at a vertex v isCurvature at a vertex v is

6 – number of edges incident on v6 – number of edges incident on v

General Conclusion: Count the number of triangles General Conclusion: Count the number of triangles to obtain measure of statistical variability of the to obtain measure of statistical variability of the surface.surface.

Counting trianglesCounting triangles

Counting TrianglesCounting Triangles

Statistical Variability of textures of cats and dogs Statistical Variability of textures of cats and dogs reflected in discrete curvature LPC surfaces.reflected in discrete curvature LPC surfaces.

It can be achieved more simply by a judicious It can be achieved more simply by a judicious method for counting triangles per unit area.method for counting triangles per unit area.

Can tell the difference between a Can tell the difference between a REALREAL dog and dog and

a a REALREAL cat! cat!

Singular Value DecompositionSingular Value Decomposition

SVD decomposition of sliding windowsSVD decomposition of sliding windows

S=UDVS=UDV

Diagonal entries positive and in decreasing order.Diagonal entries positive and in decreasing order.

Do the diagonal matrices D contain significant Do the diagonal matrices D contain significant

information about structural content of an image?information about structural content of an image?

SVD (continued)SVD (continued)

SVD (continued)SVD (continued)

SVD (continued)SVD (continued)

SVD (continued)SVD (continued)

SVD TransformsSVD Transforms

From Diagonal entries of SVD decomposition of From Diagonal entries of SVD decomposition of sliding windows on an image we construct the SVD sliding windows on an image we construct the SVD transform or SVD surface.transform or SVD surface.

SVD Transform (cont.)SVD Transform (cont.)

SVD TransformSVD Transform

Application of SVD TransformsApplication of SVD Transforms

1. Detection of objects in a texture background.1. Detection of objects in a texture background.

2. Detection of fractures or defects.2. Detection of fractures or defects.

3. Segmentation of Images.3. Segmentation of Images.

4. Determination of location of eyes.4. Determination of location of eyes.

DetectionDetection

Detection (continued)Detection (continued)

Detection (continued)Detection (continued)

Detection of FracturesDetection of Fractures

SegmentationSegmentation

Segmentation (continued)Segmentation (continued)

Effect of change in lighting and blurring on Effect of change in lighting and blurring on segmentationsegmentation

Segmentation (continued)Segmentation (continued)

Segmentation (continued)Segmentation (continued)

Segmentation (continued)Segmentation (continued)

Conclusion: Conclusion:

Segmentation via SVD transforms isolates objects Segmentation via SVD transforms isolates objects on the basis of their local texture structures.on the basis of their local texture structures.

Is not sensitive to changes in lighting, orientation, Is not sensitive to changes in lighting, orientation,

or similar distortions.or similar distortions.

Locating the Eyes

- =

SVD Transform

Edge detection

Edge detection with noise removal

RobustRobust

Analysis of SVDAnalysis of SVD

Understanding the meaning and implications of Understanding the meaning and implications of the SVD decompositionthe SVD decomposition

Substituting the diagonal part D from one image Substituting the diagonal part D from one image into another.into another.

Analysis of SVD (cont.)Analysis of SVD (cont.)

D woman in U,V LenaD woman in U,V Lena

ws=4ws=4

Analysis of SVD (cont.)Analysis of SVD (cont.)

ws=4ws=4

D Lena in U,V womanD Lena in U,V woman

Analysis of SVD (cont.)Analysis of SVD (cont.)

ws=32ws=32

D Lena in U,V womanD Lena in U,V woman

Analysis of SVDAnalysis of SVD

ws=4ws=4

D Lena in U,V randomD Lena in U,V random

Analysis of SVD (cont.)Analysis of SVD (cont.)

ws=32ws=32

D random in U,V LenaD random in U,V Lena

ConclusionConclusion

Diagonal SVD contains significant information.Diagonal SVD contains significant information.

Relative importance of D relative to U,V decreases as Relative importance of D relative to U,V decreases as window size increases.window size increases.

U, V contain information about correlational structure of U, V contain information about correlational structure of image.image.

For small window sizes U and V behave like high frequencies.For small window sizes U and V behave like high frequencies.

Other ResearchOther Research

1. Eye/Iris Printing1. Eye/Iris Printing

2. Shape Matching2. Shape Matching

3. Robotic Motion3. Robotic Motion