CSE 140: Computer VisionCSE 140: Computer Vision

Camillo J. TaylorCamillo J. Taylor

Assistant ProfessorAssistant Professor

CIS Dept, UPennCIS Dept, UPenn

What is Computer VisionWhat is Computer Vision

Recover information about the Recover information about the scene from one or more imagesscene from one or more images

Relevant questionsRelevant questions• What’s out thereWhat’s out there• Where is itWhere is it• Where is it goingWhere is it going

Computer Vision as an Computer Vision as an Inverse ProblemInverse Problem

Given information about the geometry Given information about the geometry of the scene, surface reflectance of the scene, surface reflectance properties, lighting configuration and properties, lighting configuration and camera model we can generate an camera model we can generate an imageimage

The problem of computer vision is to The problem of computer vision is to infer these properties of the scene infer these properties of the scene from the image data (a 2D array of from the image data (a 2D array of numbers]numbers]

OutlineOutline

ReconstructionReconstruction RecognitionRecognition

StereoStereo

Given two images of a scene taken Given two images of a scene taken from known positions with from known positions with calibrated cameras recover the 3D calibrated cameras recover the 3D structure of the environment.structure of the environment.

Correspondence ProblemCorrespondence Problem

Stereo algorithms attempt to Stereo algorithms attempt to automatically detect correspondences automatically detect correspondences between points in the two images.between points in the two images.

Once these correspondences have Once these correspondences have been determined, the locations of been determined, the locations of these points in space can be these points in space can be determined from a simple determined from a simple triangulationtriangulation

SSD matchingSSD matching

The sum of squared differences is The sum of squared differences is a commonly used matching a commonly used matching criterion in stereo algorithmscriterion in stereo algorithms

SSD i d I i j d I i jl rj

( , ) ( ( ) ( ))

2

Constraints that can be Constraints that can be employed in matchingemployed in matching

UniquenessUniqueness OrderingOrdering SmoothnessSmoothness

Recovering 3D models Recovering 3D models from 2D imagefrom 2D image

For single images we can only For single images we can only recover the geometry if we make recover the geometry if we make some assumptions about the some assumptions about the shape of objects that we’re looking shape of objects that we’re looking atat

ReconstructionsReconstructions

More ReconstructionsMore Reconstructions

RecognitionRecognition

One statement of the recognition problemOne statement of the recognition problem• Given a 3D model of a target object determine Given a 3D model of a target object determine

whether that object appears in a given imagewhether that object appears in a given image In the alignment approach to recognition In the alignment approach to recognition

the computer hypothesizes matches the computer hypothesizes matches between features in the model and between features in the model and features in the image and then tests features in the image and then tests whether any other features support this whether any other features support this matchmatch

Other approaches to Other approaches to recognitionrecognition

Image basedImage based• Directly compare stored image to Directly compare stored image to

acquired image to determine whether acquired image to determine whether or not they are similaror not they are similar

• Useful in face recognitionUseful in face recognition

Recognizing articulated Recognizing articulated objectsobjects

David Forsyth’s Horse finderDavid Forsyth’s Horse finder