Image Warping (Szeliski 3.6.1)

  • View
    24

  • Download
    2

Embed Size (px)

DESCRIPTION

Image Warping (Szeliski 3.6.1). http://www.jeffrey-martin.com. cs129: Computational Photography James Hays, Brown, Fall 2012. Slides from Alexei Efros and Steve Seitz. f. f. f. T. T. x. x. x. f. x. Image Transformations. image filtering: change range of image g(x) = T(f(x)). - PowerPoint PPT Presentation

Text of Image Warping (Szeliski 3.6.1)

  • Image Warping (Szeliski 3.6.1)cs129: Computational PhotographyJames Hays, Brown, Fall 2012Slides from Alexei Efros and Steve Seitzhttp://www.jeffrey-martin.com

  • Image Transformationsimage filtering: change range of imageg(x) = T(f(x))image warping: change domain of imageg(x) = f(T(x))

  • Image Transformationsffggimage filtering: change range of imageg(x) = T(f(x))image warping: change domain of imageg(x) = f(T(x))

  • Parametric (global) warpingExamples of parametric warps:translationrotationaspectaffineperspectivecylindrical

  • Parametric (global) warpingTransformation T is a coordinate-changing machine:p = T(p)What does it mean that T is global and parametric?Is the same for any point pcan be described by just a few numbers (parameters)Lets represent T as a matrix: p = Mp

    p = (x,y)p = (x,y)

  • ScalingScaling a coordinate means multiplying each of its components by a scalarUniform scaling means this scalar is the same for all components: 2

  • Non-uniform scaling: different scalars per component:

    Scaling

  • ScalingScaling operation:

    Or, in matrix form:scaling matrix SWhats inverse of S?

  • 2-D Rotationx = x cos() - y sin()y = x sin() + y cos()

  • 2-D Rotationx = r cos (f)y = r sin (f)x = r cos (f + )y = r sin (f + )

    Trig Identityx = r cos(f) cos() r sin(f) sin()y = r sin(f) cos() + r cos(f) sin()

    Substitutex = x cos() - y sin()y = x sin() + y cos()f

  • 2-D RotationThis is easy to capture in matrix form:

    Even though sin(q) and cos(q) are nonlinear functions of q,x is a linear combination of x and yy is a linear combination of x and y

    What is the inverse transformation?Rotation by qFor rotation matricesR

  • 2x2 MatricesWhat types of transformations can be represented with a 2x2 matrix?2D Identity?2D Scale around (0,0)?

  • 2x2 MatricesWhat types of transformations can be represented with a 2x2 matrix?2D Rotate around (0,0)?2D Shear?

  • 2x2 MatricesWhat types of transformations can be represented with a 2x2 matrix?2D Mirror about Y axis?2D Mirror over (0,0)?

  • 2x2 MatricesWhat types of transformations can be represented with a 2x2 matrix?2D Translation?Only linear 2D transformations can be represented with a 2x2 matrixNO!

  • All 2D Linear TransformationsLinear transformations are combinations of Scale,Rotation,Shear, andMirrorProperties of linear transformations:Origin maps to originLines map to linesParallel lines remain parallelRatios are preservedClosed under composition

  • Consider a different Basisj =(0,1)i =(1,0)qq=4i+3j = (4,3)p=4u+3v

  • Linear Transformations as Change of BasisAny linear transformation is a basis!!!j =(0,1)i =(1,0)pij = 4u+3vpx=4ux+3vxpy=4uy+3vyv =(vx,vy)u=(ux,uy)puvpijpuv = (4,3)

  • Whats the inverse transform?How can we change from any basis to any basis?What if the basis are orthogonal?v =(vx,vy)u=(ux,uy)puvj =(0,1)i =(1,0)pij = (5,4)pijpuv = (px,py) = ?= pxu + pyv

  • Projection onto orthogonal basisv =(vx,vy)u=(ux,uy)puvj =(0,1)i =(1,0)pij = (5,4)pijpuv = (upij, vpij)pijpuv==yxyxyyxxvvuuvvuu45

  • Homogeneous CoordinatesQ: How can we represent translation as a 3x3 matrix?

  • Homogeneous CoordinatesHomogeneous coordinatesrepresent coordinates in 2 dimensions with a 3-vector

    homogenous coords

  • Homogeneous CoordinatesAdd a 3rd coordinate to every 2D point(x, y, w) represents a point at location (x/w, y/w)(x, y, 0) represents a point at infinity(0, 0, 0) is not allowedConvenient coordinate system to represent many useful transformations

  • Homogeneous CoordinatesQ: How can we represent translation as a 3x3 matrix?

    A: Using the rightmost column:

  • TranslationExample of translation

    tx = 2 ty = 1Homogeneous Coordinates

  • Basic 2D TransformationsBasic 2D transformations as 3x3 matricesTranslateRotateShearScale

  • Matrix CompositionTransformations can be combined by matrix multiplicationp = T(tx,ty) R(Q) S(sx,sy) p

  • Affine TransformationsAffine transformations are combinations of Linear transformations, andTranslationsProperties of affine transformations:Origin does not necessarily map to originLines map to linesParallel lines remain parallelRatios are preservedClosed under compositionModels change of basis

    Will the last coordinate w always be 1?

  • Projective TransformationsProjective transformations Affine transformations, andProjective warpsProperties of projective transformations:Origin does not necessarily map to originLines map to linesParallel lines do not necessarily remain parallelRatios are not preservedClosed under compositionModels change of basis

  • 2D image transformationsThese transformations are a nested set of groups Closed under composition and inverse is a member

  • Matlab Demohelp imtransform

  • Recovering TransformationsWhat if we know f and g and want to recover the transform T?e.g. better align images from Project 1willing to let user provide correspondencesHow many do we need?xxT(x,y)yyf(x,y)g(x,y)?

  • Translation: # correspondences?How many correspondences needed for translation?How many Degrees of Freedom?What is the transformation matrix?xxT(x,y)yy?

  • Euclidian: # correspondences?How many correspondences needed for translation+rotation?How many DOF?

    xxT(x,y)yy?

  • Affine: # correspondences?How many correspondences needed for affine?How many DOF?

    xxT(x,y)yy?

  • Projective: # correspondences?How many correspondences needed for projective?How many DOF?

    xxT(x,y)yy?

  • Example: warping trianglesGiven two triangles: ABC and ABC in 2D (12 numbers) Need to find transform T to transfer all pixels from one to the other.What kind of transformation is T?How can we compute the transformation matrix:

    T(x,y)?ABCACBSourceDestination

  • warping triangles (Barycentric Coordinaes)Very useful for Project 3 (hint,hint,nudge,nudge)ABCACBSourceDestination(0,0)(1,0)(0,1)changeof basisInverse changeof basisDont forget to move the origin too!

  • Image warpingGiven a coordinate transform (x,y) = T(x,y) and a source image f(x,y), how do we compute a transformed image g(x,y) = f(T(x,y))?xxT(x,y)f(x,y)g(x,y)yy

  • f(x,y)g(x,y)Forward warpingSend each pixel f(x,y) to its corresponding location (x,y) = T(x,y) in the second imagexxT(x,y)Q: what if pixel lands between two pixels?yy

  • f(x,y)g(x,y)Forward warpingSend each pixel f(x,y) to its corresponding location (x,y) = T(x,y) in the second imagexxT(x,y)Q: what if pixel lands between two pixels?yyA: distribute color among neighboring pixels (x,y)Known as splattingCheck out griddata in Matlab

  • f(x,y)g(x,y)xyInverse warpingGet each pixel g(x,y) from its corresponding location (x,y) = T-1(x,y) in the first imagexxQ: what if pixel comes from between two pixels?y

  • f(x,y)g(x,y)xyInverse warpingGet each pixel g(x,y) from its corresponding location (x,y) = T-1(x,y) in the first imagexxQ: what if pixel comes from between two pixels?yA: Interpolate color value from neighborsnearest neighbor, bilinear, Gaussian, bicubicCheck out interp2 in Matlab

  • Forward vs. inverse warpingQ: which is better?

    A: usually inverseeliminates holeshowever, it requires an invertible warp functionnot always possible...

    *