Upload
monica-black
View
227
Download
2
Embed Size (px)
Citation preview
Lecture 2Imaging Geometry and OpenCV
Slides by:David A. Forsyth
Clark F. OlsonJean Ponce
Linda G. Shapiro
• Abstract camera model - box with a small hole in it
• Pinhole cameras work in practice
Pinhole Cameras
Distant objects are smaller
Common to draw film planein front of the focal point.Moving the film plane merelyscales the image.
Parallel lines meet
• Cartesian coordinates:– We have, by similar triangles, that:
– (X, Y, Z) ~ (f X/Z, f Y/Z, f)
– f is called the focal length.),(),,(
Z
Yf
Z
XfZYX
[X, Y, Z]
[fX/Z, fY/Z]
The equation of projection
We won’t worry much about lenses in this class.
The reason for lenses
Lens distortion
“Barrel distortion” of rectangular grid is common for inexpensive lenses.
Precision lenses can be expensive.
Zoom lenses often show severe distortion.
Fish-eye lenses also have severe distortion.
• Images are not continuous• Typically captured with a CCD camera (charge-coupled-device)• The amount of light striking each location on a grid is integrated over
some time period• Rows are read out one at a time• For color images, successive
pixels usually correspond to different colors
• High quality color cameras usea beam splitter and 3 separateCCD chips
• APS (active pixel sensor) is acheaper (lower quality) technology
Image capture
Resolution
Resolution often (but not always) refers to the number of pixels in the image.
Lower resolution has fewer pixels.
Interestingly, faces of people you know can usually be recognized at 64x64 (or less) pixels.
Squint and look at the lowest resolution image.
Some of my research
Time permitting, the following slides give a very brief overview of some of my previous and current research.
10
Visual Terrain Mapping for Mars Exploration
11
Pose Sampling for Efficient Model-Based Recognition
12
Careful sampling of the viewpoints reducesthe complexity from cubic to linear in thenumber of craters.
Robust Registration of Aerial Image Sequences
13
Goal: Provide registration between images and maps for persistent aerial surveillance
Registration results
Simple and Efficient Projective Clustering
14
Projective clustering problems have the following properties:• Many dimensions - d• Many points – n• Clusters do not form in
the full d-dimensional space
• No labeled exemplars
Keypoint recognition
15
A popular object recognition technique uses descriptive “keypoints” that have been extracted from images.
We are investigating getting better use out of the color information in the image when creating keypoint descriptors.
16
Image classificationCurrent techniques for image classification cluster image keypoints into “visual words” similar to text retrieval methods.
We are studying the use of projective clustering to improve performance.
http://www.sccs.swarthmore.edu/users/09/btomasi1/tagging-products.html
Programming in OpenCV
In OpenCV, images are represented as matrices (as in linear algebra).
Mat image = imread("photo.jpg"); // Most generic declaration
The image could have a number of underlying data types for each pixel:
uchar – unsigned byte (greyscale image)
Vec3b – vector of 3 bytes (color image)
Point2f – point in two dimensions, float
many others…
Creating images
Images can be created using a number of methods:
using namespace cv; // all my code assumes thisMat image; // creates 0x0 imageMat image = … // uses copy constructorMat image(rows, cols, type); // type is CV_8U, for exampleMat image(rows, cols, type, scalarValue);
Example: Mat allBlue(360, 480, CV_8UC3, Scalar(255, 0, 0));
Mat_<Vec3b> colorImage = imread(“color.jpg”);// Can be convenient, but now limited to Vec3b images (matrices)// Also, must declare as a similar parameter type when passed
Copying images
Be careful to remember that most image copy and pass by value methods do NOT perform a deep copy.
image2 = image1; // shallow copy
void someMethod(Mat imageParam); // shallow copy
If you want a deep copy, then use clone (or copyTo):
image2 = image1.clone(); // deep copyimage1.copyTo(image2); //
deep copy
Memory management
Memory management is handled by the Mat class.• This is different from the IplImage class in OpenCV 1• This works correctly even if multiple images share the same data• A reference count is kept for each image• The data is deallocated only when the reference count goes to
zero• However, this can allow privacy leaks unless you are careful
Backwards compatibility
OpenCV 2 is backwards compatible with OpenCV 1.
IplImage *iplIm = cvLoadImage("photo.jpg");// Do some work with image herecvReleaseImage(&iplIm); // necessary to prevent
memory leak
Can convert to Mat simply:
Mat converted(iplIm); // do not release image until finished
Image manipulation
OpenCV provides many methods to manipulate entire images:
Filtering: blur, smooth, median, gradient, laplacian
Transformations: resize, affine, perspective, color space, threshold, flood fill
Segmentation: grabCut, watershed
Feature detection: edges, corners, lines, circles, template matching, SIFT