Jigsaw Puzzle Solver

Puzzle SolverSravan BhagavatulaEE 638 ProjectStanford ECE

OverviewPurpose of ProjectHigh Level ImplementationScale Invariant Feature TransformExplanation of AlgorithmResultsFuture Work

Purpose of ProjectSolving a jigsawFinding placementsBased on locations in original picture

Representative used interchangeably with informative3High-level ImplementationNeeds two inputsPiecesOriginal ImageOutputsNumbered piecesOriginal with placements

Scale Invariant Feature TransformObject Recognition technique (David Lowe)Rotation / orientation change was a problemFeatures obtained similar to neuron responses in inferior temporal cortex (for primate vision)

Object Recognition from Local Scale-Invariant Features, D. G. Lowe, International Conference on Computer Vision, Corfu, Greece, Sept. 1999.

Scale Invariant Feature TransformKeypoint LocationsDefined as extrema of a difference-of-Gaussian function applied in scale spaceLocal Image DescriptionRobust descriptor to local affine distortion

Scale Invariant Feature TransformComputationally efficient one second/image order of 1000 featuresOcclusionsTested very well for rotation / scale changesChosen for invariance

Explanation of AlgorithmP Image of piecesS Image of complete picture

Find the keypoints in P and S with vl_siftOutput a modified P, with piece labelsUse kmeans() to cluster the keypoints in each pieceTake a small number of points per clusterAround 20 30.Compare these keypoints with ones in S2-norm comparison of the SIFT keypoint descriptors

Explanation of Algorithm Cont.Find locations in S of matchesThese basically count as the location of each pieceClassify each region of matches into clustersI.E., choose a central point to designate as the label of the regionOutput a modified version of S using these cluster labelsOne that has the same labels as the one in P, such that similar pieces are in the right locations

Results

Future WorkBackground of pieces needs to be uniform Additional step to make the background uniform?Try out orientation, lighting changesClustering without numPiecesTest it on much larger puzzles (~1000 piece, perhaps)Computation timeSolve without the solution imageMuch harder, more than just feature matching