Introduction to Computer Vision CS / ECE 181B Thursday, April 13, 2004 Image Formation

Introduction to Computer Vision

CS / ECE 181B

Thursday, April 13, 2004

Image Formation

Course web site

• http://www.ece.ucsb.edu/~manj/cs181b

• http://www.ece.ucsb.edu/~manj/ece181b

• Not a substitute for attending the class– Note that not all lecture are in PPT. I will be using the board for

most “math” related stuff.

Prereqs and background knowledge

• E.g., I assume you know:– Basic linear algebra

– Basic probability

– Basic calculus

– Programming languages (C, C++) or MATLAB You had two sessions on MATLAB

Your job

• You are expected to:– Attend the lectures and discussion sessions

You're responsible for everything that transpires in class and discussion session (not just what’s on the slides)

– Ask questions in class – participate!

– Do the homework assignments on time and with integrity “Honest effort” will get you credit

– Give us feedback during the quarter

First part of course: Image Formation

• Geometry of image formation(Camera models and calibration)– Where?

• Radiometry of image formation– How bright?

Digital images

• We’re interested in digital images, which may come from– An image originally recorded on film

Digitized from negative or from print– Analog video camera

Digitized by frame grabber– Digital still camera or video camera– Sonar, radar, ladar (laser radar)– Various kinds of spectral or multispectral sensors

Infrared, X-ray, Landsat…

• Normally, we’ll assume a digital camera (or digitized analog camera) to be our source, and most generally a video camera (spatial and temporal sampling)

What is a Camera?

• A camera has many components– Optics: lens, filters, prisms,

mirrors, aperture

– Imager: array of sensing elements (1D or 2D)

– Scanning electronics

– Signal processing

– ADC: sampling, quantizing, encoding, compression

May be done by external frame grabber (“digitizer”)

• And many descriptive features– Imager type: CCD or CMOS

– Imager number

– SNR

– Lens mount

– Color or B/W

– Analog or digital (output)

– Frame rate

– Manual/automatic controls

– Shutter speeds

– Size, weight

– Cost

Camera output: A raster image

• Raster scan – A series of horizontal scan lines, top to bottom – Progressive scan – Line 1, then line 2, then line 3, …

– Interlaced scan – Odd lines then even lines

Raster patternProgressive scan

Interlaced scan

Pixels

• Each line of the image comprises many picture elements, or pixels– Typically 8-12 bits (grayscale) or 24 bits (color)

• A 640x480 image:– 480 rows and 640 columns

– 480 lines each with 640 pixels

– 640x480 = 307,200 pixels

• At 8 bits per pixel, 30 images per second– 640x480x8x30 = 73.7 Mbps or 9.2 MBs

• At 24 bits per pixel (color)– 640x480x24x30 = 221 Mbps or 27.6 MBs

Aspect ratio

• Image aspect ratio – width to height ratio of the raster– 4:3 for TV, 16:9 for HDTV, 1.85:1 to 2.35:1 for movies

– We also care about pixel aspect ratio (not the same thing) Square or non-square pixels

Sensor, Imager, Pixel

• An imager (sensor array) typically comprises n x m sensors– 320x240 to 7000x9000 or more (high end astronomy)– Sensor sizes range from 15x15m down to 3x3 m or smaller

• Each sensor contains a photodetector and devices for readout

• Technically: – Imager – a rectangular array of sensors upon which the scene is

focused (photosensor array)– Sensor (photosensor) – a single photosensitive element that generates

and stores an electric charge when illuminated. Usually includes the circuitry that stores and transfers it charge to a shift register

– Pixel (picture element) – atomic component of the image (technically not the sensor, but…)

• However, these are often intermingled

Color sensors

• CCD and CMOS chips do not have any inherent ability to discriminate color (i.e., photon wavelength/energy)– They sense “number of photons”, not wavelengths

– Essentially grayscale sensors – need filters to discriminate colors!

• Approaches to sensing color– 3-chip color: Split the incident light into its primary colors

(usually red, green and blue) by filters and prisms Three separate imagers

– Single-chip color: Use filters on the imager, then reconstruct color in the camera electronics

Filters absorb light (2/3 or more), so sensitivity is low

3-chip color

Incidentlight

Lens

Neutral densityfilter

Infraredfilter

Low-passfilter

To R imager

To G imager

To B imager

Prisms

How much light energy reaches each sensor?

Single-chip color

)),((),(

)),((),(

)),((),(

dyydxxIfyxB

dyydxxIfyxG

dyydxxIfyxR

B

G

R

±±=±±=±±=

Incidentlight To imager

• Uses a mosaic color filter– Each photosensor is covered by a single filter

– Must reconstruct (R, G, B) values via interpolation

Eye is not a (digital) camera! (or, is it?)

http://www.discoveryfund.com/anatomyoftheeye.html

Image Formation

• Projection Geometry

• Radiometry (Image Brightness) - to be discussed later in SFS.

Pinhole Camera

(source: A Guided tour of computer vision/Vic Nalwa)

Perspective Projection

(source: A Guided tour of computer vision/Vic Nalwa)

Perspective Projection

Some Observations/questions

• Note that under perspective projection, straight-lines in 3-D project as straight lines in the 2-D image plane. Can you prove this analytically?– What is the shape of the image of a sphere?

– What is the shape of the image of a circular disk? Assume that the disk lies in a plane that is tilted with respect to the image plane.

• What would be the image of a set of parallel lines– Do they remain parallel in the image plane?

Note: Equation for a line in 3-D (and in 2-D)

Line in 3-D:

Line in 2-D

By using the projective geometry equations, it is easy to show that a line in 3-D projects as a line in 2-D.

Vanishing Point

• Vanishing point of a straight line under perspective projection is that point in the image beyond which the projection of the straight line can not extend.– I.e., if the straight line were infinitely long in space, the line would

appear to vanish at its vanishing point in the image.

– The vanishing point of a line depends ONLY on its orientation is space, and not on its position.

– Thus, parallel lines in space appear to meet at their vanishing point in image.

Vanishing Point

(sou

rce:

A G

u id e

d to

ur o

f co

mpu

ter

visi

on/V

ic N

a lw

a)

The Vanishing Point

(source: A Guided tour of computer vision/Vic Nalwa)

Vanishing point (last slide!)

• For any given spatial orientation, the vanishing point is located at that point on the projection surface where a straight line passing through the center of projection with the given orientation would intersect the projection surface.

Planar vs Spherical Perspective Projection

(source: A Guided tour of computer vision/Vic Nalwa)

Spherical Perspective Projection

• Under parallel perspective projection, straight line map onto straight line.

• Question: What do straight lines map onto under spherical perspective projection?

Orthographic Projection

• Projection onto a plane by a set of parallel rays orthogonal to this plane.

X x

Y yi

i

=

=0

0

(source: A Guided tour of computer vision/Vic Nalwa)

Approximation of Perspective Projection

A. object dimensions are small compared to the distance of the object from the center of projection.B. Compared to this distance, the object is close to the straight line that passes through COP and is orthogonal to the IP.

Approximation by Parallel Projection

(source: A Guided tour of computer vision/Vic Nalwa)

Parallel Projection

• Parallel Projection is a generalization of orthographic projection in which the object is projected onto the image plane by a set of parallel rays that are not necessarily orthogonal to this plane.

• Perspective projection can be approximated by parallel projection up to a uniform scale factor whenever the object’s dimensions are small compared to the average distance of the object from the center of projection.

Note: Imaging with a lens

Misfocus Blur

Brightness

• Irradiance, as a measure of image brightness– Irradiance is the power per unit area (Watts per square

meter) of radiant energy falling on a surface.

EP

A=dd

Irradiance

Brightness

• Scene Brightness -- Radiance

– Radiance is the power emitted per unit area into a cone of directions having unit solid angle (Watts per square meter per steridian.)

LP

A=dd dw

2

Image Formation: Summary

– Projection Geometry What determines the position of a 3D point

in the image?– Image Brightness

What determines the brightness of the image of some surface?

This we will discuss later when we talk about shape from shading.

Summary

Projection Geometry - determines the position of a 3D point in the image.– Perspective projection– approximations using

orthographic projection parallel projection

– terminology center of projection vanishing point optic axis focal point, focal length