……a mini-workshop on instructional materials for moving a mini-workshop on instructional materials for moving precision agriculture beyond mappingprecision agriculture beyond mapping

Analyzing Precision Ag DataAnalyzing Precision Ag Data

Joseph K. BerryJoseph K. Berry

Berry & AssociatesBerry & Associates2000 South College, Suite 3002000 South College, Suite 300

Fort Collins, CO 80525Fort Collins, CO 80525

Email: Email: jberry@innovativegis.comjberry@innovativegis.comWeb Site: Web Site: www.innovativegis.com/basiswww.innovativegis.com/basis

AgKnowledge GIS Faculty Development WorkshopAgKnowledge GIS Faculty Development Workshop

August 12-15, 2002August 12-15, 2002Kirkwood Community College, Cedar Rapids, IowaKirkwood Community College, Cedar Rapids, Iowa

……about doing the right thing at the right about doing the right thing at the right place and timeplace and time

……identifies and responds appropriately identifies and responds appropriately to the variability within a fieldto the variability within a field

……augments (not replaces) indigenous augments (not replaces) indigenous knowledge knowledge

(Circa 1992)

What Is Precision Agriculture? What Is Precision Agriculture? (new technology)(new technology)

……stackable livestockstackable livestock

((BerryBerry))

What Is Precision Agriculture? What Is Precision Agriculture? (Whole-field)(Whole-field)

Whole-Field ManagementWhole-Field Management is based on broad averages of field is based on broad averages of field data with management actions directed by “typical” conditions. data with management actions directed by “typical” conditions.

Weigh-wagon, or grain elevator measurements, established a Weigh-wagon, or grain elevator measurements, established a field’s yield performance. Soil sampling determined the typical field’s yield performance. Soil sampling determined the typical nutrient levels within a field. From these and other data the best nutrient levels within a field. From these and other data the best overall seed variety was chosen and a constant rate of fertilizer overall seed variety was chosen and a constant rate of fertilizer applied, as well as a bushel of other decisions— applied, as well as a bushel of other decisions— treating the treating the entire field as uniform within its boundariesentire field as uniform within its boundaries..

Whole-fieldWhole-field assumes the “average” assumes the “average” conditions are the same everywhereconditions are the same everywhere

((BerryBerry))

What Is Precision Agriculture? What Is Precision Agriculture? (Site-specific)(Site-specific)

It involves assessing and reacting to field variability by tailoring It involves assessing and reacting to field variability by tailoring management actions, such as fertilization levels, seeding rates management actions, such as fertilization levels, seeding rates and variety selection, to match and variety selection, to match changing field conditionschanging field conditions. It . It assumes that managing field variability leads to both cost savings assumes that managing field variability leads to both cost savings and production increases, as well as improved stewardship and and production increases, as well as improved stewardship and environmental benefits.environmental benefits.

Z1

Z3

Z2

Z1

Z2

Management-zonesManagement-zones breaks the field breaks the field into areas of similar conditionsinto areas of similar conditions

Surface-mapsSurface-maps breaks the field into breaks the field into consistent pieces that track the consistent pieces that track the

specific conditions at each location specific conditions at each location

Site-Specific ManagementSite-Specific Management recognizes the variability within a recognizes the variability within a field and changes management actions throughout a field. field and changes management actions throughout a field.

((BerryBerry))

• Global Positioning System (GPS) – Global Positioning System (GPS) – Establishes Establishes position in a fieldposition in a field

• Data Collection Devices (IDIData Collection Devices (IDI- monitors- monitors) – ) – Collects Collects data “on-the-fly”data “on-the-fly”

• Geographic Information Systems (GIS) – Geographic Information Systems (GIS) – Used for Used for data visualization and analysisdata visualization and analysis

• Intelligent Implements (IDIIntelligent Implements (IDI- controls- controls) – ) – Provides Provides variable rate control “on-the-fly”variable rate control “on-the-fly”

A new application of existing A new application of existing technologies to assist intechnologies to assist in managing field managing field variability…variability…

What Is Precision Agriculture? What Is Precision Agriculture? (Technologies)(Technologies)

((BerryBerry))

• Continuous Data Logging Continuous Data Logging 1) Yield Map1) Yield Map

““What you see is what you get”What you see is what you get”

• Discrete Point Sampling Discrete Point Sampling 2) Condition Maps2) Condition Maps

““Guessing over the whole field”Guessing over the whole field”

• Mapped Data Analysis Mapped Data Analysis 3) Relationships3) Relationships

““Now what could’ve caused that?”Now what could’ve caused that?”

• Spatial Modeling Spatial Modeling 4) Prescription Map4) Prescription Map

““Do this here, but not over there”Do this here, but not over there”

Where is WhatWhere is What

Why and So WhatWhy and So What

Do WhatDo Whatand Whereand Where

Utilizes Utilizes spatial relationshipsspatial relationships in a field for site-specific in a field for site-specific management…management…

((BerryBerry))

What Is Precision Agriculture? What Is Precision Agriculture? (Processing Steps)(Processing Steps)

((BerryBerry))

A new application of the A new application of the Spatial TechnologiesSpatial Technologies……

……that utilizes that utilizes spatial spatial relationshipsrelationships in a field for site- in a field for site-specific management of fieldsspecific management of fields

Precision Agriculture’s Big PicturePrecision Agriculture’s Big PictureEnabling technologiesEnabling technologies

Data processing approachData processing approach

www.innovativegis.com/basis/pfprimer/

Where is WhatWhere is WhatWhy and So WhatWhy and So What

Do What and WhereDo What and Where

Overview of the Case StudyOverview of the Case Study

Ex#1Ex#1

((BerryBerry))

Example Applications–Example Applications–

several annotated examples of several annotated examples of grid-based map analysisgrid-based map analysis

Install MapCalcInstall MapCalc–– software software system for hands-on experiencesystem for hands-on experience

Workshop objectives are to ) Workshop objectives are to ) introduce the introduce the concepts, procedures and issuesconcepts, procedures and issues surrounding precision agriculture data and surrounding precision agriculture data and 2) provide 2) provide hands-on experiencehands-on experience in analysis of these datain analysis of these data

Fall 2002Fall 2002

Map Data Visualization and SummaryMap Data Visualization and Summary

((BerryBerry))

Table 1. Workbook Topics

Overview of the Case Study

Mapped Data Visualization and Summary

Comparing Mapped Data

Spatial Interpolation

Characterizing Data Groups

Developing Predictive Models

Analyzing Spatial Context

Numerical statisticsNumerical statistics …min, max, range, …min, max, range, mean, median standard deviation, variancemean, median standard deviation, variance

Geographic statisticsGeographic statistics …total area, area …total area, area by class, drill-downby class, drill-down

Descriptive statisticsDescriptive statistics

Drill-DownDrill-Down

Map Data Visualization and SummaryMap Data Visualization and Summary

Ex#2Ex#2

((BerryBerry))

Comparing Map DataComparing Map Data

((BerryBerry))

Table 1. Workbook Topics

Overview of the Case Study

Mapped Data Visualization and Summary

Comparing Mapped Data

Spatial Interpolation

Characterizing Data Groups

Developing Predictive Models

Analyzing Spatial Context

Comparing Discrete Maps Comparing Discrete Maps (Joint coincidence)(Joint coincidence)

What differences do you see?What differences do you see? – –““How different are the maps?”How different are the maps?”

““How are they different?” How are they different?” ““Where are they different?”Where are they different?”

……but map patterns can be but map patterns can be quantitatively compared quantitatively compared

A A Coincidence TableCoincidence Table reports the reports the number of cells for each joint number of cells for each joint condition with diagonal cells condition with diagonal cells

identifying agreement. identifying agreement.

((BerryBerry))

Comparing Map SurfacesComparing Map Surfaces (Difference map)(Difference map)

……greengreen indicates indicates areas of increased areas of increased

productionproduction

……yellowyellow indicates indicates minimal changeminimal change

……redred indicates indicates decreased productiondecreased production

1997_Yield_Volume1997_Yield_Volume- 1998_Yield_Volume- 1998_Yield_Volume

Yield_DiffYield_Diff

Map VariablesMap Variables… map values within an analysis grid … map values within an analysis grid can be mathematically and statistically analyzed can be mathematically and statistically analyzed

Ex#3Ex#3

((BerryBerry))

Spatial InterpolationSpatial Interpolation

((BerryBerry))

Table 1. Workbook Topics

Overview of the Case Study

Mapped Data Visualization and Summary

Comparing Mapped Data

Spatial Interpolation

Characterizing Data Groups

Developing Predictive Models

Analyzing Spatial Context

““Mapping the Variance”Mapping the Variance”

……Soil SamplesSoil Samples are collected with GPS coordinates are collected with GPS coordinates

……the location and nutrient levels of the sample points the location and nutrient levels of the sample points ((Discrete DataDiscrete Data) are used to estimate the nutrient ) are used to estimate the nutrient pattern throughout the field (pattern throughout the field (Continuous DataContinuous Data))

Spatial Expression of Arithmetic AverageSpatial Expression of Arithmetic Average

((BerryBerry))

Spatial InterpolationSpatial Interpolation (Mapping spatial variability)(Mapping spatial variability)

……the geo-registered the geo-registered soil samples form a pattern soil samples form a pattern of “spikes” throughout the of “spikes” throughout the field. field. Spatial InterpolationSpatial Interpolation is similar to throwing a is similar to throwing a blanket over the spikes that blanket over the spikes that conforms to the pattern.conforms to the pattern.

……all interpolation algorithms assume that all interpolation algorithms assume that 1) “1) “nearby things are more alike than nearby things are more alike than distant thingsdistant things” (spatial autocorrelation), ” (spatial autocorrelation), 2) 2) appropriate samplingappropriate sampling intensity, and intensity, and 3) suitable sampling pattern.3) suitable sampling pattern.

……the continuous surfaces produced “map the continuous surfaces produced “map the spatial variation in the data samples.the spatial variation in the data samples.

((BerryBerry))

Ex#4Ex#4

Spatial InterpolationSpatial Interpolation (Average vs. IDW)(Average vs. IDW)

Comparison of the interpolated surface to the whole field average Comparison of the interpolated surface to the whole field average shows shows large differenceslarge differences in localized estimates in localized estimates

Difference Map

((BerryBerry))

Spatial InterpolationSpatial Interpolation (Compare maps)(Compare maps)

Comparison of the IDW and Krig interpolated surfaces shows Comparison of the IDW and Krig interpolated surfaces shows small small differencesdifferences in in localized estimates in in localized estimates

Difference Map

((BerryBerry))

Spatial Interpolation TechniquesSpatial Interpolation TechniquesCharacterizes the spatial distribution by fitting a mathematical Characterizes the spatial distribution by fitting a mathematical

equation to localized portions of the data (roving window)equation to localized portions of the data (roving window)

((BerryBerry))

Spatial InterpolationSpatial InterpolationAssessing Interpolation Results (Residual Analysis)Assessing Interpolation Results (Residual Analysis)

……the best map is the the best map is the one that has the bestone that has the best““guesses”guesses”

((BerryBerry))

Spatial InterpolationSpatial InterpolationA Map of Error (Residual Map)A Map of Error (Residual Map)

……shows you where your estimates are likely good/badshows you where your estimates are likely good/bad

((BerryBerry))

Characterizing Data GroupsCharacterizing Data Groups

((BerryBerry))

Table 1. Workbook Topics

Overview of the Case Study

Mapped Data Visualization and Summary

Comparing Mapped Data

Spatial Interpolation

Characterizing Data Groups

Developing Predictive Models

Analyzing Spatial Context

Visualizing Spatial Relationships

(Berry)(Berry)

What spatial What spatial relationships do you relationships do you see?see?

Interpolated Spatial DistributionInterpolated Spatial Distribution

Phosphorous (P)

……do relatively high levels do relatively high levels of P often occur with high of P often occur with high levels of K and N?levels of K and N?

……how often?how often?

……where?where?

Calculating Data DistanceCalculating Data Distance……an n-dimensional plot depicts the multivariate distribution; the distance an n-dimensional plot depicts the multivariate distribution; the distance

between points determines the relative similarity in data patterns between points determines the relative similarity in data patterns

(Berry)(Berry)……the closest floating ball is the least similar (largest data distance) from the comparison pointthe closest floating ball is the least similar (largest data distance) from the comparison point

Identifying Map SimilarityIdentifying Map Similarity

(Berry)(Berry)

The green tones indicate field locations with fairly similar P, K and N levels; red tones indicate dissimilar The green tones indicate field locations with fairly similar P, K and N levels; red tones indicate dissimilar areas. areas.

……the relative data distance between the comparison point’s data pattern the relative data distance between the comparison point’s data pattern and those of all other map locations form a and those of all other map locations form a Similarity IndexSimilarity Index

Clustering Maps for Data ZonesClustering Maps for Data Zones

(Berry)(Berry)

(Cyber-Farmer, Circa 1992)(Cyber-Farmer, Circa 1992)

Variable Rate ApplicationVariable Rate Application……fertilization rates vary for the different fertilization rates vary for the different clusters “clusters “on-the-flyon-the-fly””

……groups of “floating balls” in data space groups of “floating balls” in data space identify locations in the field with similar data identify locations in the field with similar data patterns– patterns– data zonesdata zones

…a map stack is a spatially organized set of numbers Ex#5Ex#5

Evaluating Clustering ResultsEvaluating Clustering Results

(Berry)(Berry)

……if the boxes do not overlap (much), the clusters are distinct.if the boxes do not overlap (much), the clusters are distinct.

Developing Predictive ModelsDeveloping Predictive Models

((BerryBerry))

Table 1. Workbook Topics

Overview of the Case Study

Mapped Data Visualization and Summary

Comparing Mapped Data

Spatial Interpolation

Characterizing Data Groups

Developing Predictive Models

Analyzing Spatial Context

NIR (R)

Red (G)

Green (B)(Beyond our sight)(Beyond our sight) Color Infrared Color Infrared

P

K

ph

etc.etc.

RS Imagery as GIS Data LayersRS Imagery as GIS Data Layers

Remote sensing Remote sensing images are composed images are composed of numbersof numbers, just like , just like any other map in a any other map in a grid-based GIS…grid-based GIS…

““Map-ematical Map-ematical Processing”Processing”

1481485252

2626

4444

4343

257257

7.27.2

4646

3434

5757

312312

7.57.5

A RS image is just a “A RS image is just a “shishkebab shishkebab of numbersof numbers” like any other ” like any other grid map (raster)grid map (raster)

ImageImage

((BerryBerry))

Creating Prediction Models Creating Prediction Models (Scatter Plot)(Scatter Plot)

……aa Scatter Plot Scatter Plot identifies the “joint condition” at each map identifies the “joint condition” at each map location; the trend in the plot forms a prediction equationlocation; the trend in the plot forms a prediction equation

(Berry)(Berry)

Map SetMap Set New Graph New Graph Scatter Plot Scatter Plot

Deriving a Predictive Index Deriving a Predictive Index (NDVI)(NDVI)

……an index combining the Red and NIR maps can be used to generate an index combining the Red and NIR maps can be used to generate a better predictive model a better predictive model

Normalized Difference Vegetation IndexNormalized Difference Vegetation Index NDVI= ((NIR – Red) / (NIR + Red))NDVI= ((NIR – Red) / (NIR + Red))

for the sample grid locationfor the sample grid location NDVI= ((121-14.7) / (121 + 14.7))= 106.3 / 135.7= .783NDVI= ((121-14.7) / (121 + 14.7))= 106.3 / 135.7= .783

(Berry)(Berry)

Evaluating Prediction Maps Evaluating Prediction Maps (Spatial error analysis)(Spatial error analysis)

……the regression equation is evaluated and the predicted map is compared the regression equation is evaluated and the predicted map is compared to the actual measurements to generate an error mapto the actual measurements to generate an error map

Error = Predicted - ActualError = Predicted - Actual

for the sample grid locationfor the sample grid location YYestest = 55 + (180 * .783) = 196 …error is 196 – 218 = 22 bu/ac = 55 + (180 * .783) = 196 …error is 196 – 218 = 22 bu/ac

Note that the average error is 2.62 and 67% of the predictions are within +/- 20 bu/acNote that the average error is 2.62 and 67% of the predictions are within +/- 20 bu/acAlso, most of the error is concentrated along the edge of the fieldAlso, most of the error is concentrated along the edge of the field

(Berry)(Berry)

Error = Predicted - Actual

Stratifying Maps for Better PredictionsStratifying Maps for Better Predictions

(Berry)(Berry)

Stratifying by Error ZonesStratifying by Error Zones

Other ways to stratify mapped data—Other ways to stratify mapped data—1) 1) Geographic ZonesGeographic Zones, such as proximity to the field , such as proximity to the field edge; 2) edge; 2) Dependent Map ZonesDependent Map Zones, such as areas of low, , such as areas of low, medium and high yield; 3) medium and high yield; 3) Data ZonesData Zones, such as areas , such as areas of similar soil nutrient levels; and 4) of similar soil nutrient levels; and 4) Correlated Map Correlated Map ZonesZones, such as micro terrain features identifying , such as micro terrain features identifying small ridges and depressionssmall ridges and depressions. .

The The Error ZonesError Zones map is used map is used as a template to identify the as a template to identify the NDVI and Yield values used to NDVI and Yield values used to calculate three separate calculate three separate prediction equations. prediction equations. A A Composite PredictionComposite Prediction map is map is created by applying the created by applying the equations to the NDVI data equations to the NDVI data respecting the template map respecting the template map zones.zones.

Assessing Prediction ResultsAssessing Prediction Results

(Berry)(Berry)

none

none

Error Map for Stratified Prediction

80%

Error Map

Whole FieldPrediction

StratifiedPrediction

ActualYield

The Precision Ag Process (Fertility example)

As a combine moves through a field As a combine moves through a field 1)1) it uses GPS to check its location then it uses GPS to check its location then 2)2) checks checks the yield at that location to the yield at that location to 3)3) create a continuous map of the yield variation every few create a continuous map of the yield variation every few feet. This map feet. This map 4)4) is combined with soil, terrain and other is combined with soil, terrain and other maps to derive a maps to derive a 5)5) “Prescription Map” that is used to “Prescription Map” that is used to 6)6) adjust fertilization levels every few feet in the field. adjust fertilization levels every few feet in the field.

Variable Rate ApplicationVariable Rate Application

Step 6)Step 6)

(Berry)(Berry)

Cyber-Farmer, Circa 1992Cyber-Farmer, Circa 1992……you’ve come a long ways babyyou’ve come a long ways baby

5.00 10.00 15.00 20.00 25.00 30.00

5.00

10.00

15.00

20.00

25.00

30.00

35.00

40.00

45.00

Prescription MapPrescription Map Step 5)Step 5)

Zone 1

Zone 3

Zone 2

Farm dBFarm dBStep 4)Step 4)

Map AnalysisMap Analysis

On-the-Fly On-the-Fly Yield MapYield Map

Steps 1)–3)Steps 1)–3)

Spatial Data Mining

……making sense out of a map stackmaking sense out of a map stack

(Berry)(Berry)

Mapped data that Mapped data that exhibits high exhibits high spatial spatial dependencydependency create create strong prediction strong prediction functions. As in functions. As in traditional statistical traditional statistical analysis, spatial analysis, spatial relationships can be relationships can be used to predict used to predict outcomesoutcomes

……the difference is the difference is that spatial statisticsthat spatial statisticspredicts wherepredicts where responses will be responses will be high or lowhigh or low

Analyzing Spatial ContextAnalyzing Spatial Context

((BerryBerry))

Table 1. Workbook Topics

Overview of the Case Study

Mapped Data Visualization and Summary

Comparing Mapped Data

Spatial Interpolation

Characterizing Data Groups

Developing Predictive Models

Analyzing Spatial Context

Characterizing SlopeCharacterizing SlopeA digital terrain surface is formed by A digital terrain surface is formed by assigning an elevation value to each assigning an elevation value to each cell in an analysis grid. The “slant” of cell in an analysis grid. The “slant” of the terrain at any location can be the terrain at any location can be calculated—calculated— inclination of a plane inclination of a plane fitted to the elevation values of the fitted to the elevation values of the immediate vicinity immediate vicinity

Micro Terrain Analysis Micro Terrain Analysis (Deriving Slope and Flow)(Deriving Slope and Flow)

Characterizing Surface FlowCharacterizing Surface FlowA map of surface flow is simulated by A map of surface flow is simulated by aggregating the “steepest downhill aggregating the “steepest downhill paths” from each cell— confluence paths” from each cell— confluence

Slope and Flow maps draped over Slope and Flow maps draped over vertically exaggerated terrain surfacevertically exaggerated terrain surface

((BerryBerry))

Calibrating Slope and Flow Classes: Calibrating Slope and Flow Classes: Areas of Gentle, Moderate, and Areas of Gentle, Moderate, and Steep slopes are identified; areas of light, moderate and heavy flows are identifiedSteep slopes are identified; areas of light, moderate and heavy flows are identified

Micro Terrain Analysis Micro Terrain Analysis (Slope and Flow Classes)(Slope and Flow Classes)

((BerryBerry))

Determining Erosion Potential: Determining Erosion Potential: The slope and flow classes are The slope and flow classes are combined into a single map identifying erosion potentialcombined into a single map identifying erosion potential

Micro Terrain Analysis Micro Terrain Analysis (a simple erosion model)(a simple erosion model)

((BerryBerry))

Map Analysis Macros Map Analysis Macros (Fat buttons)(Fat buttons)

Assembly languageAssembly languageProgramming Languages (Visual C++)Programming Languages (Visual C++)

Programming Objects (ActiveX Controls)Programming Objects (ActiveX Controls)

General Software SystemGeneral Software System Application Languages (MapCalc scripting)Application Languages (MapCalc scripting) Fat Buttons (Embedded Macros)Fat Buttons (Embedded Macros)

Application-Specific SystemApplication-Specific SystemExecution Environment (Visual Basic) Execution Environment (Visual Basic) Fat Buttons (Embedded Controls)Fat Buttons (Embedded Controls)

InputInput

Parameter SpecificationParameter Specification

OutputOutput

Ex#6Ex#6

((BerryBerry))

• Limited ApproachLimited Approach – Mapping vs. Data Mapping vs. Data

Analysis; Tools vs. ScienceAnalysis; Tools vs. Science

• Science LinkScience Link – “Scientific Method” Doctrine, “Scientific Method” Doctrine,

The “Random” Thing, Appropriate Driving The “Random” Thing, Appropriate Driving Variables, Correlation vs. CausationVariables, Correlation vs. Causation

• Market ConfusionMarket Confusion – Empirical Verification, Empirical Verification,

Economic Validation, Rationalization (Productivity Economic Validation, Rationalization (Productivity vs. Stewardship)vs. Stewardship)

Gaps in Our ThinkingGaps in Our Thinking

……Environmental Trump CardEnvironmental Trump Card

Education, Enlightenment,Education, Enlightenment, Economics, EnvironmentEconomics, Environment((BerryBerry))

……Education/Training is KeyEducation/Training is Key

……a mini-workshop on instructional materials for moving precision agriculture beyond mappinga mini-workshop on instructional materials for moving precision agriculture beyond mapping

Analyzing Precision Ag DataAnalyzing Precision Ag Data

See See http://www.innovativegis.com/basis/pfprimer/Default.htmlhttp://www.innovativegis.com/basis/pfprimer/Default.html

……to access the online to access the online Precision Farming PrimerPrecision Farming Primer

See See http://www.innovativegis.com/basis/MapAnalysis/Default.html http://www.innovativegis.com/basis/MapAnalysis/Default.html

……to access the online to access the online Map Analysis Map Analysis bookbook

"Precision farming isn't "Precision farming isn't just a bunch of pretty just a bunch of pretty

maps, but mapped data maps, but mapped data and a set of procedures and a set of procedures

linking these data to linking these data to appropriate management appropriate management

actions."  actions."  

Berry & Associates // Spatial Information SystemsBerry & Associates // Spatial Information Systems2000 South College, Suite 300, Fort Collins, CO 805252000 South College, Suite 300, Fort Collins, CO 80525

Email: Email: jberry@innovativegis.comjberry@innovativegis.comWeb Site: Web Site: www.innovativegis.com/basiswww.innovativegis.com/basis

Presented by Joseph K. BerryPresented by Joseph K. Berry