賴珮雯 P76994369 2010/1/10 Keywords: panorama, SURF, stitching, multi-band blending, LM, bundle...
Preview:
Citation preview
- Slide 1
- P76994369 2010/1/10 Keywords: panorama, SURF, stitching,
multi-band blending, LM, bundle adjustment
- Slide 2
- Introduction Modified SURF Comparison of SIFT and modified SURF
Experiments and results Time cost test Conclusion and future works
Reference
- Slide 3
- Stitching multiple images is a popular method of effectively
increasing the field of view of a camera, by allowing several views
of a scene to be combined into a single view. Stitching includes
two main parts: image matching and image blending. find the motion
relationship between two images or several images
- Slide 4
- For matching, there are two ways: direct method and feature
detection method Direct method: is inconvenient because it always
needs a high quality image. feature detection method: such as SURF
and SIFT. For blending: weight averaged linear blending, multi-band
blendingetc multi-band blending: good performance between quality
of result and time cost
- Slide 5
- This paper a panorama image stitching process which combines an
image matching system: Modified SURF + multi-band blending. 1. find
the SURF features from the image(use KNN and RANSAC) 2. estimate
the homography matrix according to LM ( Levenberg-Marquardt )
method, 3. adjust the coordinate of images 4. blend the images by
multi-blending to remove the stitch seam and illumination
discrepancy
- Slide 6
- SURF method: relying on integral images for image convolutions
building on the strengths of the leading existing detectors and
descriptors (using a Hessian matrix- based measure for the
detector, and a distribution-based descriptor) simplifying these
methods to the essential
- Slide 7
- Integral images allow the computation of rectangular box
filters in near constant time This paper used KNN to find the
nearest neighbor with setting k to 2 RANSAC is used to estimate a
model for finding the minimize error matches set, which can
maintain the correct matches by comparing the distance of the
closest neighbor to that of second-closest neighbor
- Slide 8
- After extracting invariant scale features, we got potential
feature matches by using k- nearest neighbor method, and then
remove the mismatches with RANSAC algorithm. modified SURF Though
SIFT get more matches than Modified SURF, Modified SURF is much
faster than SIFT
- Slide 9
- Figure 1 and Fig. 2 give the match results of Modified SURF and
SIFT in scale and illumination change respectively: win
- Slide 10
- modified SURF was not better than SIFT in rotation, Modified
SURF is as robust as SIFT in other performance.
- Slide 11
- Summary: SURF describes image faster than SIFT by 3 times SURF
is not as well as SIFT on invariance to illumination change and
viewpoint change
- Slide 12
- bundle adjustment: to transform the images into a same
coordinate or computing surface 1. Choose one of the images to be
reference surface 2. transform each of other images to the
reference surface usual choice for compositing larger panoramas is
to use a cylindrical or spherical projection.
- Slide 13
- Transformation: compute the homography and optimize the
parameters of the matrix 1. find out the best neighbor image for
each image 2. calculate the distance between the two neighbor
images 3. minimize the distance value to adjust the matrix between
the neighbor images (use LM)
- Slide 14
- Levenberg-Marquardt (LM): nonlinear function least square ,
using nonlinear minimum square evaluation to minimize the transfer
error, which is calculated as equation: Is correspondent with
points X Homography matrix Euclidian distance
- Slide 15
- goal : is to produce a resulting image where no transition can
be seen between the original source images. Linear method: may
results in ghosting artifacts, blurring. But Linear blending method
is fast and can be a good compromise between quality and speed if
you are not too demanding on quality
- Slide 16
- multi-band blending (or called by pyramid blending): effective
for image stitching without blurring and ghosting artifacts. It
will produce much better results than the "Linear" mode. Multiband
blending scheme ensures smooth transitions between images despite
illumination differences
- Slide 17
- compare two methods: Refernce:
http://www.cs.ubc.ca/~lowe/425/slides/11-PanoramasAR.pdf
- Slide 18
- Multi-band Blending: compare to linear blending multi-band
blending can make image more clear in detail. In the paper, author
use 2-band. band weighting( Gaussian function) weighting ( Gaussian
function) band blending
- Slide 19
- The Laplacian pyramid of the final image is formed as equation:
Pyramid blending gradually blends the lower frequencies of the
images while maintaining a sharper transition for the higher
frequencies where X1,k and X2,k are the kth level of Laplacian
pyramid decomposition Yk is the kth level of Laplacian pyramid
decomposition for the final combination result Mk is the kth level
of Gaussian pyramid decomposition of the image mas Mk is the kth
level of Gaussian pyramid decomposition of the image mas
- Slide 20
- Environment: studio 2008 C++ with OpenCV library Flow diagram
has tow parts: matching and blending The connection of the two
parts is the correspondence pairs.
- Slide 21
- Flow of matching: the goal of which is to find the largest
feature points good to transformation Detect feature points square
Euclidean distance ratio between neighbors is calculated estimate a
model of consensus set that minimizes matching error Output for
blending as itsinput
- Slide 22
- Flow of blending: H can be estimated based on those
correspondence pairs Correct some stitching error of color and
illumination. the images has been transformed into the
corresponding image in a same coordinate system by the H matrix the
images has been transformed into the corresponding image in a same
coordinate system by the H matrix
- Slide 23
- Experiments consist of two parts: panorama quality (stitching)
test and time cost test. A good stitching program should make
panorama seamless and clear and be fast for using in various
application such as real time processing.
- Slide 24
- Stitching test: there are three seams in Fig.4 Fig. 4. Panorama
with obvious seam before blending processing Fig. 5. Panorama with
seamless after blending processing of Fig. 4 1 2 3
- Slide 25
- Next, we did an experiment with large data set. In this
experiment, we use 16 images of Camp dataset We will see that the
present stitching process can show its good performance for large
image dataset
- Slide 26
- Fig. 6. Panorama stitched 16 images based on SIFT. Fig.7.
Panorama stitched 16 images based on modified SURF
- Slide 27
- The present system is faster than SIFT demo as shown in Fig 8
and Fig 9, due to using fast modified SURF. Fig. 8. Time cost
number of images The present system is almost 4 times (3.56~4.46)
faster than SIFT demo
- Slide 28
- Each experiment the time will change a little because of the
CPU and memory, the reasonable experiment time is needed Fig 9
shows the time cost when the size of images that used are
different
- Slide 29
- The present panorama image stitching process has a good
performance. Due to Modified SURF, high-resolution panorama can be
created in case there are some changes of illumination, color, blur
and et cetera, and processed fast.
- Slide 30
- Reasons for good performance: K-NN and RANSAC improves the
repeatability of matching. Bundle adjustment and multi-band
blending make the panorama seamless. LM is used to estimate the
homography, which makes the transformation more accurate
- Slide 31
- SURF is poor at handling viewpoint change handling illumination
change Plus, the present system shows its defects when there are
some noise images that are not neighbored removing the noise before
stitching processing
- Slide 32
- Herbert Bay, Andreas Ess, Tinne Tuytelaars, Luc Van Gool. SURF:
Speeded Up Robust Features. Computer Vision and Image Understanding
(CVIU), Vol. 110, No. 3, pp. 346--359, 2008. Luo Juan and Oubong
Gwun. A Comparison of SIFT, PCA-SIFT and SURF. International
Journal of Image Processing (IJIP), Volume (3): Issue (4)
2009.