Semantic feature analysis in raster mapsTrevor Linton, University of Utah1AcknowledgementsThomas HendersonRoss WhitakerTolga TasdizenThe support of IAVO Research, Inc. through contract FA9550-08-C-005.Objectives for instruction and expected results and/or skills developed from learning. 2Field of StudyGeographical Information SystemsPart of Document Recognition and Registration.What are USGS Maps?A set of 55,000 1:24,000 scale images of the U.S. with a wealth of data.Why study it?To extract new information (features) from USGS maps and register information with existing G.I.S and satellite/aerial imagery.

Relative vocabulary list. 3Register information not just with existing GIS, and register it with aerial imageryProblemsDegradation and scanning produces noise.Overlapping features cause gaps.Metadata has the same texture as features.Closely grouped features makes discerning between features difficult.

Relative vocabulary list. 4Problems Noisy Data

Scanning artifact which introduces noiseRelative vocabulary list. 5Problems Overlapping FeaturesMetadata and Features overlap with similar textures. Gaps in data.

Relative vocabulary list. 6Problems Closely Grouped FeaturesClosely grouped features make discerning features difficult.

Relative vocabulary list. 7Thesis & GoalsUsing Gestalt principles to extract features and overcome some of the problems described.Quantitatively extract 95% recall and 95% precision for intersections.Quantitatively extract 99% recall and 90% precision for intersections.Current best method produces 75% recall and 84% precision for intersections.Relative vocabulary list. 8What the current best method isApproachGestalt PrinciplesOrganizes perception, useful for extracting features.Law of SimilarityLaw of ProximityLaw of ContinuityRelative vocabulary list. 9Approach Gestalt PrinciplesLaw of SimilarityGrouping of similar elements into whole features.Reinforced withhistogram models.

Relative vocabulary list. 10Approach Gestalt PrinciplesLaw of ProximitySpatial proximity of elementsgroups them together.Reinforced through TensorVoting System

Relative vocabulary list. 11Approach Gestalt PrinciplesLaw of ContinuityFeatures with small gaps should be viewed as continuous.Idea of multiple layers offeatures that overlap.Reinforced by Tensor VotingSystem.

Relative vocabulary list. 12Draw differences between closure and continuity -- more 1d for continuity where as closure closes a figure in a 2d senseApproach Framework Overview

Relative vocabulary list. 13change "Curve and Junction Map" to "Curve and Junction Maps" -- change Graph Search Algorithm to Knowledge Based Approach

Change Unit Width Features -> Roads & IntersectionsPre-ProcessingClass Conditional Density ClassifierUses statistical meansand histogrammodels. = Histogram modelvector.Find class k with thesmallest is the classof x.

Relative vocabulary list. 14Is it Class Conditional Classifier or Class Condition Density ClassifierPre-Processingk-Nearest NeighborsUses the class that is found most often out of k closest neighbors in the histogram model.Closeness is defined by Euclidian distance of the histogram models.

Relative vocabulary list. 15Pre-ProcessingKnowledge Based ClassifierUses logic that is based on our knowledge of the problem to determine classes.Based on information on the textures each class has.

Relative vocabulary list. 16Pre-ProcessingOriginal Image with Features Estimated

Relative vocabulary list. 17Pre-ProcessingOriginal Image with Roads Extracted

Class condition classifier k-Nearest Neighbors Knowledge BasedRelative vocabulary list. 18Tensor Voting SystemOverview

Relative vocabulary list. 19Tensor Voting SystemUses an idea of VotingEach point in the image is a tensor.Each point votes how other points should be oriented. Uses tensors as mathematical representations of points.Tensors describe the direction of the curve.Tensors represent confidence that the point is a curve or junction.Tensors describe a saliency of whether the feature (whether curve or junction) actually exists.

Relative vocabulary list. 20Tensor Voting SystemWhat is a tensor?Two vectors that are orthogonal to one another packed into a 2x2 matrix.

Relative vocabulary list. 21Talk to tolga if he believes thisTensor Voting SystemCreating estimates of tensors from input tokens.

Principal Component Analysis

Canny edge detection

Ball Voting

Relative vocabulary list. 22Tensor Voting SystemVotingFor each tensor in the sparse fieldCreate a voting field based on the sigma parameter.Align the voting field to the direction of the tensor.Add the voting field to the sparse field.Produces a dense voting field.

Relative vocabulary list. 23Tensor Voting SystemVoting FieldsA window size is calculated from

Direction of each tensor in the field is calculated from

Attenuation derived from

Relative vocabulary list. 24Has a direction and falls off for the attenuation, possibly put note on what c doesTensor Voting SystemVoting Fields (Attenuation)Red and yellow are higher votes, blue and turquoise lower.Shape related to continuation vs. proximity.

Relative vocabulary list. 25Talk about why the shape is important, talk about why the shape is more round then straight and its relationship to continuationTensor Voting SystemExtracting features from dense voting field. determines the likelihood of being on a curve. determines the likelihood of being a junction.If both 1 and 2 are small then the curve or junction has a small amount of confidence in existing or being relevant.

Relative vocabulary list. 26Tensor Voting SystemExtracting features from dense voting field.

Original Image Curve Map Junction MapRelative vocabulary list. 27Post-processingExtracting features from curve map and junction map.

Global Threshold and Thinning

Local Threshold and Thinning

Local Normal Maximum

Knowledge Based Approach

Relative vocabulary list. 28Post-processingGlobal threshold on curve map.

Applied Threshold Thinned ImageRelative vocabulary list. 29Post-processingLocal threshold on curve map.

Applied Threshold Thinned Image

Relative vocabulary list. 30Post-processingLocal Normal MaximumLooks for maximum over the normal of the tensor at each point.

Applied Threshold Thinned Image

Relative vocabulary list. 31Post-processingKnowledge Based ApproachUses knowledge of types of artifacts of the local threshold to clean and prep the image.

Original Image Knowledge Based ApproachRelative vocabulary list. 32ExperimentsDetermine adequate parameters.Identify weaknesses and strengths of each method.Determine best performing methods.Quantify the contributions of tensor voting.Characterize distortion of methods on perfect inputs.Determine the impact of misclassification of text on roads.

Relative vocabulary list. 33ExperimentsQuantitative analysis done with recall and precision measurements.Relevant is the set of all features that are in the ground truth.Retrieved is the set of is all features found by the system.tp = True Positive, fn = False Negative, fp = False PositiveRecall measures the systems capability to find features.Precision characterizes whether it was able to find only those features. For both recall and precision, 100% is best, 0% is worst.

Relative vocabulary list. 34ExperimentsData Selection

Data set must be large enough to adequately represent features (above or equal to 100 samples).

One sub-image of the data must not be biased by the selector.

One sub-image may not overlap another.

A sub-image may not be a portion of the map which contains borders, margins or the legend.

Relative vocabulary list. 35Add confidence interval to why 100 samples is good/bad?ExperimentsGround TruthManually generated from samples.Roads and intersections manually identified.Ground Truth is generated twice, those with more than 5% of a difference are re-examined for accuracy.

Ground truth Original ImageRelative vocabulary list. 36ExperimentsBest Pre-Processing MethodAll pre-processing methods examined without tensor voting or post processing for effectiveness.Best window size parameter for k-Nearest Neighbors was qualitatively found to be 3x3.The best k parameter for k-Nearest Neighbors was quantitatively found to be 10.The best pre-processing method found was the Knowledge Based Classifier

Relative vocabulary list. 37ExperimentsTensor Voting SystemResults from test show the best value for is between 10 and 16 with little difference in performance.

Relative vocabulary list. 38ExperimentsTensor Voting SystemContributions from tensor voting were mixed.Thresholding methods performed worse.Knowledge based method improved 10% road recall, road precision dropped by 2%, intersection recall increased by 22% and intersection precision increased by 20%.

Relative vocabulary list. 39ExperimentsBest Post-ProcessingFinding the best window size for local thresholding.Best parameter was found between 10 and 14.

Relative vocabulary list. 40ExperimentsBest Post-ProcessingThe best post-processing method was found by using a nave pre-processing technique and tensor voting.Knowledge Based Approach performed the best.

Relative vocabulary list. 41ExperimentsRunning the system on perfect data (ground truth as inputs) produced higher results then any other method (as expected).Thesholding had a considerably low intersection precision due to artifacts produced in the process.

Relative vocabulary list. 42ExperimentsBest combination found was k-Nearest Neighbors with a Knowledge Based Approach.Note the best pre-processing method Knowledge Based Classifier was not the best pre-processing method when used in combinations due to the type of noise it produces.With Text: 92% Road Recall, 95% Road Precision82% Intersection Recall, 80% Intersection PrecisionWithout Text: 94% Road Recall, 95% Road Precision 83% Intersection Recall, 80% Intersection Precision

Relative vocabulary list. 43ExperimentsConfidence Intervals (95% CI, 100 samples)Road Recall:Mean: 93.61% CI [ 92.47% , 94.75% ] 0.14%Road Precision:Mean: 95.23% CI [ 94.13% , 96.33% ] 0.10%Intersection Recall:Mean: 82.22% CI [ 78.91% , 85.51% ] 3.29%Intersection Precision:Mean: 80.1% CI [ 76.31% , 82.99% ] 2.89%

Relative vocabulary list. 44ExperimentsAdjusting parameters dynamically

Dynamically adjusting the between 4 and 10 by looking at the amount of features in a window did not produce much difference in the recall and precision (less than 1%).

Dynamically adjusting the c parameter in tensor voting actually produced worse results because of exaggerations in the curve map due to slight variations in the tangents for each tensor.

Relative vocabulary list. 45Future Work & IssuesTensor Voting and thinning tend to bring together intersections too soon when the road intersection angle was too low or the roads were too thick.The Hough transform may possibly overcome this issue.

Relative vocabulary list. 46Future Work & IssuesScanning noise will need to be removed in order to produce high intersection recall and precision results.

Relative vocabulary list. 47Future Work & IssuesClosely grouped and overlapping features.

Relative vocabulary list. 48Show the results from tensor voting and why its a problem.Future Work & IssuesDeveloping other pre-processing and post-processing techniques.Learning algorithmsVarious local threshold algorithmsRoad following algorithms

Relative vocabulary list. 49