Spatial Data MiningSpatial Data Mining

Practical Approaches for Analyzing Relationships Practical Approaches for Analyzing Relationships

Within and Among MapsWithin and Among Maps

Berry & Associates // Spatial Information Systems2000 S. College Ave, Suite 300, Fort Collins, CO 80525Phone: (970) 215-0825 Email: jberry@innovativegis.com

…visit our Website at www.innovativegis.com/basis

““GIS technology is rapidly moving beyond GIS technology is rapidly moving beyond mapping and spatial database management mapping and spatial database management to analytical capabilities that assess spatial to analytical capabilities that assess spatial

relationships within decision-making relationships within decision-making contexts”contexts” (JKB)(JKB)

Presented byPresented by Joseph K. BerryJoseph K. BerryW.M. Keck Scholar in Geosciences, University of DenverW.M. Keck Scholar in Geosciences, University of Denver

Visually Comparing MapsVisually Comparing Maps

But just how similar are the maps?But just how similar are the maps?

……what proportion has the same what proportion has the same classification?classification?

……where are they different?where are they different?

……where are they the same?where are they the same?

and of course, and of course, your response should be your response should be objective and repeatableobjective and repeatable

I bet you've seen and heard it a thousand times I bet you've seen and heard it a thousand times before before a speaker waves a laser pointer at a a speaker waves a laser pointer at a couple of maps and says something like "see how couple of maps and says something like "see how similar the patterns are."similar the patterns are."

(Berry)(Berry)

Approach for Comparing Discrete MapsApproach for Comparing Discrete Maps

(Berry)(Berry)

Coincidence Summary Results Coincidence Summary Results (Table 1)(Table 1)

(Berry)(Berry)

(Table 1)(Table 1)

Proximal Alignment Results Proximal Alignment Results (Table 2)(Table 2)

(Berry)(Berry)

(Table 2)(Table 2)

Approach for Comparing Map SurfacesApproach for Comparing Map Surfaces

(Berry)(Berry)

Statistical Test Results Statistical Test Results (Table 3)(Table 3)

……Statistical Tests of entire surface or Statistical Tests of entire surface or partitioned areaspartitioned areas

(Berry)(Berry)

(Table 3)(Table 3)

Percent Difference Results Percent Difference Results (Table 4)(Table 4)

(Berry)(Berry)

……Percent Difference Percent Difference between two map between two map

surfacessurfaces

(Table 4)(Table 4)

Surface Configuration Results Surface Configuration Results (Table 5)(Table 5)

(Berry)(Berry)

The two superimposed maps at The two superimposed maps at the left side of figure show the the left side of figure show the normalized differences in the normalized differences in the slope and aspect angles (dark slope and aspect angles (dark red being very different). The red being very different). The map of the overall differences map of the overall differences in surface configuration in surface configuration (Sur_Fig Index) is the average (Sur_Fig Index) is the average of the two maps. of the two maps.

Note that over half of the Note that over half of the map area is classified as map area is classified as low difference (0-20) low difference (0-20) suggesting that the two suggesting that the two surface maps align fairly surface maps align fairly well overall. well overall.

(Table 4)(Table 4)

Visualizing Spatial RelationshipsVisualizing Spatial Relationships

(Berry)(Berry)

What spatial What spatial relationships do you relationships do you see?see?

Interpolated Spatial DistributionInterpolated Spatial Distribution

Phosphorous (P)

……do relatively high levels do relatively high levels of P often occur with high of P often occur with high levels of K and N?levels of K and N?

……how often?how often?

……where?where?

Calculating Data DistanceCalculating Data Distance……an n-dimensional plot depicts the multivariate distribution; the distance an n-dimensional plot depicts the multivariate distribution; the distance

between points determines the relative similarity in data patterns between points determines the relative similarity in data patterns

……the closest floating ball is the least similar (largest data distance) from the comparison pointthe closest floating ball is the least similar (largest data distance) from the comparison point(Berry)(Berry)

Identifying Map SimilarityIdentifying Map Similarity

(Berry)(Berry)

The green tones indicate field locations with fairly similar P, K and N levels; red tones indicate dissimilar The green tones indicate field locations with fairly similar P, K and N levels; red tones indicate dissimilar areas. areas.

……the relative data distance between the comparison point’s data pattern the relative data distance between the comparison point’s data pattern and those of all other map locations form a and those of all other map locations form a Similarity IndexSimilarity Index

(See Map Analysis, “(See Map Analysis, “Topic 16, Calculating Map Similarity” for more information)

Clustering Maps for Data ZonesClustering Maps for Data Zones

(Berry)(Berry)

(Cyber-Farmer, Circa 1990)(Cyber-Farmer, Circa 1990)

Variable Rate ApplicationVariable Rate Application……fertilization rates vary for the different fertilization rates vary for the different clusters “on-the-fly”clusters “on-the-fly”

……groups of “floating balls” in data space groups of “floating balls” in data space identify locations in the field with similar data identify locations in the field with similar data patterns– patterns– data zonesdata zones

…a map stack is a spatially organized set of numbers

Evaluating Clustering ResultsEvaluating Clustering Results

(Berry)(Berry)

Map Surface Correlation/RegressionMap Surface Correlation/RegressionHistogram/Map View--Histogram/Map View-- Data SpaceData Space (joint magnitude of values)(joint magnitude of values)

are linked toare linked to Geographic SpaceGeographic Space ((position of values)position of values)

(Berry)(Berry)

Creating Prediction Models Creating Prediction Models (Scatter Plot)(Scatter Plot)

……aa Scatter Scatter Plot Plot identifies the “joint condition” at each map identifies the “joint condition” at each map location; the trend in the plot forms a prediction equationlocation; the trend in the plot forms a prediction equation

(Berry)(Berry)

Deriving a Predictive Index Deriving a Predictive Index (NDVI)(NDVI)

……an index combining the Red and NIR maps can be used to generate an index combining the Red and NIR maps can be used to generate a better predictive model a better predictive model

Normalized Difference Vegetation IndexNormalized Difference Vegetation Index NDVI= ((NIR – Red) / (NIR + Red))NDVI= ((NIR – Red) / (NIR + Red))

for the sample grid locationfor the sample grid location NDVI= ((121-14.7) / (121 + 14.7))= 106.3 / 135.7= .783NDVI= ((121-14.7) / (121 + 14.7))= 106.3 / 135.7= .783

(Berry)(Berry)

Evaluating Prediction Maps Evaluating Prediction Maps (Spatial error analysis)(Spatial error analysis)

……the regression equation is evaluated and the predicted map is compared the regression equation is evaluated and the predicted map is compared to the actual measurements to generate an error mapto the actual measurements to generate an error map

Error = Predicted - ActualError = Predicted - Actual

for the sample grid locationfor the sample grid location YYestest = 55 + (180 * .783) = 196 …error is 196 – 218 = 22 bu/ac = 55 + (180 * .783) = 196 …error is 196 – 218 = 22 bu/ac

Note that the average error is 2.62 and 67% of the predictions are within +/- 20 bu/acNote that the average error is 2.62 and 67% of the predictions are within +/- 20 bu/acAlso, most of the error is concentrated along the edge of the fieldAlso, most of the error is concentrated along the edge of the field

(Berry)(Berry)

Exercise #8c, page 30 – Create a regression model relating Yield and NDVI

(See Map Analysis, “(See Map Analysis, “Topic 16, Predicting Maps” for more information)

Stratifying Maps for Better PredictionsStratifying Maps for Better Predictions

(Berry)(Berry)

Stratifying by Error ZonesStratifying by Error Zones

Other ways to stratify mapped data—Other ways to stratify mapped data—1) Geographic Zones, such as proximity to the field 1) Geographic Zones, such as proximity to the field edge; 2) Dependent Map Zones, such as areas of low, edge; 2) Dependent Map Zones, such as areas of low, medium and high yield; 3) Data Zones, such as areas medium and high yield; 3) Data Zones, such as areas of similar soil nutrient levels; and 4) Correlated Map of similar soil nutrient levels; and 4) Correlated Map Zones, such as micro terrain features identifying Zones, such as micro terrain features identifying small ridges and depressionssmall ridges and depressions. .

The The Error ZonesError Zones map is used map is used as a template to identify the as a template to identify the NDVI and Yield values used to NDVI and Yield values used to calculate three separate calculate three separate prediction equations. prediction equations. A A Composite PredictionComposite Prediction map is map is created by applying the created by applying the equations to the NDVI data equations to the NDVI data respecting the template map respecting the template map zones.zones.

(See Map Analysis, “(See Map Analysis, “Topic 16, Stratifying Maps for Better Predictions” for more information)

Assessing Prediction ResultsAssessing Prediction Results

(Berry)(Berry)

Whole FieldPrediction

StratifiedPrediction

ActualYield

none

none

Error Map for Stratified Prediction

80%

Error Map

The Precision Ag ProcessThe Precision Ag Process (Fertility example)(Fertility example)

As a combine moves through a field 1) it uses GPS to check its location then 2) checks As a combine moves through a field 1) it uses GPS to check its location then 2) checks the yield at that location to 3) create a continuous map of the yield variation every few the yield at that location to 3) create a continuous map of the yield variation every few feet. This map 4) is combined with soil, terrain and otherfeet. This map 4) is combined with soil, terrain and other maps to derive a 5) “Prescription Map” that is used to maps to derive a 5) “Prescription Map” that is used to 6) adjust fertilization levels every few feet in the field.6) adjust fertilization levels every few feet in the field.

Variable Rate ApplicationVariable Rate Application

Step 6)Step 6)

(Berry)(Berry)

Cyber-Farmer, Circa 1992Cyber-Farmer, Circa 1992……come a long ways babycome a long ways baby

5.00 10.00 15.00 20.00 25.00 30.00

5.00

10.00

15.00

20.00

25.00

30.00

35.00

40.00

45.00

Prescription MapPrescription Map Step 5)Step 5)

Zone 1

Zone 3

Zone 2

Farm dBFarm dBStep 4)Step 4)

Map AnalysisMap Analysis

On-the-Fly On-the-Fly Yield MapYield Map

Steps 1)–3)Steps 1)–3)

Spatial Data MiningSpatial Data Mining

……making sense out of a map stackmaking sense out of a map stack

(Berry)(Berry)

Mapped data that Mapped data that exhibits high exhibits high spatial spatial dependencydependency create create strong prediction strong prediction functions. As in functions. As in traditional statistical traditional statistical analysis, spatial analysis, spatial relationships can be relationships can be used to predict used to predict outcomesoutcomes

……the difference is the difference is that spatial statisticsthat spatial statisticspredicts wherepredicts where responses will be responses will be high or lowhigh or low

Spatial Data MiningSpatial Data Mining Practical Approaches for Analyzing Relationships Within and Among MapsPractical Approaches for Analyzing Relationships Within and Among Maps

Berry & Associates // Spatial Information Systems2000 S. College Ave, Suite 300, Fort Collins, CO 80525Phone: (970) 215-0825 Email: jberry@innovativegis.com

…visit our Website at www.innovativegis.com/basis

Presented byPresented by Joseph K. BerryJoseph K. BerryW.M. Keck Scholar in Geosciences, University of DenverW.M. Keck Scholar in Geosciences, University of Denver