Embed Size (px)
Citation preview
EXPLORING SPATIAL CORRELATION IN RIVERS
by Joshua French
Introduction
A city is required to extends its sewage pipelines farther in its bay to meet EPA requirements.
How far should the pipelines be extended?
The city doesn’t want to spend any more money than it needs to extend the pipelines. It needs to find a way to make predictions for the waste levels at different sites in the bay.
With the passage of the Clean Water Act in the 1970’s, spatial analysis of aquatic data has become even more important.
Section 305 b) requires state governments to make, “a description of the water quality of all navigable waters in such State. . .”
It is not physically or financially possible to make measurements at all sites. Some sort of spatial interpolation will need to be used.
Usually we might try to fit some sort of linear model to the data to make predictions. Usually we assume observations are independent.
For spatial data however, we intuitively know that two sampling sites close together will probably be similar.
We would expect that two sites in close proximity would be more similar than two sites separated by a great distance.
We can use the correlation between sampling sites to make better predictions with our model.
The Ohio River
The Road Ahead
- Methods- Introduction to the Variogram- Exploratory Analysis- Sample Variogram- Modeling the Variogram
- Analysis- 3 types of results
- Conclusions- Future Work
Introduction to the Variogram
Spatial data is often viewed as a stochastic process.
For each point x, a specific property Z(x) is viewed as a random variable with mean µ, variance σ2, higher-order moments, and a cumulative distribution function.
Each individual Z(xi) is assumed to have its own distribution, and the set {Z(x1),Z(x2),…} is a stochastic process.
The data values in a given data set are simply a realization of the stochastic process.
We want to measure the relationship between different points. Define the covariance for Z(xj) and Z(xk) to be:
Cov(Z(xj),Z(xk))=E[{Z(xj)-µ(xj)} {Z(xk)-µ(xk)}]
where µ(xj) and µ(xk) is the mean of Z at each respective location.
However, we have a problem. We don’t know the means at each point because we only have one realization.
To solve this, we must assume sort of stationarity – certain features of the distribution are identical everywhere.
We will work with data that satisfies second-order stationarity.
Second-order stationarity means that the mean is the same everywhere: i.e. E[Z(xj)]=µ for all points xj.
It also implies that Cov(Z(xj),Z(xk)) becomes a function of the distance xj to xk.
Thus,
Cov(Z(xj),Z(xk)) = Cov(Z(x),Z(x+h))
= Cov(h)
where h measures the distance between two points.
We can then derive that
Cov(Z(x),Z(x+h)) =E[(Z(x)-µ)(Z(x+h)- µ)]
= E[(Z(x)(Z(x+h))-µ2]
Sometimes it is clear that our data is not second-order stationary.
Georges Matheron solved this problem in 1965 by establishing his “intrinisic hypothesis”.
For small distances h, Matheron held that
E[Z(x)-Z(x+h)]=0
Looking at the variance of differences, this
leads to
Var[Z(x)-Z(x+h)] =E[ (Z(x)-Z(x+h))2 ]
= 2 γ(h)
Intrinsic stationarity is good because analysis may be conducted even if second-order stationarity is violated. Unfortunately, the covariance equation is not defined for intrinsic stationarity.
For this reason, we will work with data that is second-order stationarity. If second-order stationarity is violated by the original data, then we will perform additional procedures to work with data that is second-order stationary.
Note that second-order stationarity implies intrinsic stationarity, so the variogram equation is still defined.
Under second-order stationarity, γ(h)=Cov(0)-Cov(h).
γ(h) is known as the semi-variogram. In practice however, it is usually referred to as the variogram.
Things to know about variograms:
1. γ(h)= γ(-h). Because it is an even function, usually only positive lag distances are shown.
2. Nugget effect - by definition, γ(0)= 0. In practice however, sample variograms often have a positive value at lag 0. This is called the “nugget effect”.
3. Tend to increase monotonically
4. Sill – the maximum variance of the variogram
5. Range – the lag distance at which the sill is reached
The following figure shows these features
Variogram Example
Lag Distance
Var
ianc
e
0 1 2 3 4 5
0.0
0.5
1.0
1.5
sill
nugget
range
Exploratory Analysis
Before we model variograms, we should explore the data.
We need to make sure that the data analyzed satisfies second-order stationarity
We need to check for outliers
We need to make sure that the data is not too badly skewed (G1>1)
We can look at the river data as a one-dimensional linear system. It is fairly easy to check for stationarity using a scatter plot.
RMI
Squ
are
Roo
t of P
erce
nt In
vert
ivor
e
0 200 400 600 800 1000
02
46
810
If there is an obvious trend in the data, we should remove it and analyze the residuals.
If the variance increases or decreases with lag distance, then we should transform the variable to correct this.
To check for outliers, we may use a typical boxplot.
If the data contains outliers, we should do analysis both with and without outliers present.
020
4060
80
If G1>1, then we should transform the data to approximate normality if possible. To check approximate normality, the standard qqplot can be used.
Quantiles of Standard Normal
Obs
erve
d
-2 0 2
23
45
67
8
3.3 The Sample Variogram
One of the previous definitions of semivariance is:
The logical estimator is:
where N(h) is the number of pairs of observations associated with that lag.
].) )Z()Z( ( [ E2
1)γ( 2hxxh
N(h)
1j
2jj ] )z(x)z(x [
)2N(
1)(γ h
hhˆ
Sample Variogram Example
Lag Distance
Va
ria
nce
0 20 40 60 80
20
00
04
00
00
60
00
08
00
00
10
00
00
Modeling the Variogram
Our goal is to estimate the true variogram of the data.
There were four variogram models used to model the sample variogram: the spherical, Gaussian, exponential, and Matern models.
Variogram Models
Lag Distance
Va
ria
nce
0 1 2 3 4 5 6
0.0
0.2
0.4
0.6
0.8
1.0
ExponentialSphericalGaussianMatern
The algorithm used to fit the spherical model uses least squares.
The algorithm used to fit the exponential, Gaussian, and Matern models is maximum likelihood.
The spherical model is fit to get an estimate of the sill, nugget, and range.
These estimates will be used to fit the other three models.
The “best model” will be the model that minimizes the AICC statistic.
Analysis
The data analyzed is a set of particle size and biological variables for the Ohio River.
The data was collected by The Ohio River Valley Sanitation Commission. This is better known as ORSANCO.
ORANSCO data collection
There were between 190 and 235 unique sampling sites, depending on the variable.
Some sites had more than one observation. In these situations, the average value for the site was used for analysis.
Ohio River Sampling Sites
Longitude (NAD27)
La
titu
de
(N
AD
27
)
-88 -86 -84 -82 -80
37
38
39
40
Pittsburgh, PA
Cairo, IL
Cincinnati, OH
Louisville, KY
There were two main types of data: particle size data and biological levels.
The particle size data measured percent gravel, percent sand, percent fines, percent hardpan, percent boulder, and percent cobble.
The biological data measured- Number of individuals at a site- Number of species at a site- Percent tolerant fish- Percent simple lithophilic fish (fish that lay eggs
on rocks)- Percent non-native fish- Percent detritivore fish (fish that eat mostly
decomposed plants or animals)- Percent invertivore (fish that eat mostly
invertebrate animals)- Percent Piscivore (fish that eat mostly other fish)
The results of the analysis fell into three main groups:
- Sample variogram fit well
- Sample variogram did not fit well
- Analysis not reasonable
Good Results: Number of Individuals at a site
Skewness coefficient of data is 8.16. This is much too high.
The data is transformed using the natural logarithm
New skewness coefficient is reduced to .56. Not perfect, but much less skewed.
Check Normality of log(Num Individuals)
Quantiles of Standard Normal
log
(Nu
mb
er
of
Ind
ivid
ua
ls)
-3 -2 -1 0 1 2 3
45
67
8
Check Second-Order Stationarity of log(Num Individuals)
RMI
log
(Nu
mb
er
of
Ind
ivid
ua
ls)
0 200 400 600 800 1000
45
67
8
Check for outliers of log(Num Individuals)4
56
78
There are a number of outliers for the transformed variable
We should do analysis with and without the outliers present
log(Num Individuals) Sample Variogramwith outliers
Lag Distance (Mi)
Va
ria
nce
0 50 100 150 200 250
0.3
00
.35
0.4
00
.45
0.5
00
.55
Check normality of log(Num Individuals) without outliers
Quantiles of Standard Normal
log
(Num
In
div
idu
als
) w
/o o
utli
ers
-3 -2 -1 0 1 2 3
4.0
4.5
5.0
5.5
6.0
6.5
log(Num Individuals) Sample Variogramwithout outliers
Lag Distance (Mi)
Va
ria
nce
0 50 100 150 200 250
0.2
00
.25
0.3
0
We were not able to model the sample variogram perfectly, but we were able to detect some amount of spatial correlation in the data, especially when the outliers were removed.
For the transformed variable without outliers, the exponential model estimated the nugget to be .20, the sill to be .2709, and the range to be 37.7 miles.
Poor Results: Percent Sand
Skewness coefficient only .18, so skewness not a major factor.
Check second-order stationarity using scatter plot.
Check Stationarity of Percent Sand
RMI
Pe
rce
nt
Sa
nd
0 200 400 600 800 1000
02
04
06
08
01
00
There appears to be a trend in the data.
After removing the trend, the data appears to be second-order stationary.
The residuals are also approximately normal.
Check stationarity of percent sand residuals
RMI
Pe
rce
nt
Sa
nd
Re
sid
ua
ls
0 200 400 600 800 1000
-60
-40
-20
02
04
0
Check normality of percent sand residuals
Quantiles of Standard Normal
san
d$
resi
d
-3 -2 -1 0 1 2 3
-60
-40
-20
02
04
0
Sample Variogram of percent sand residuals
Lag Distance (Mi)
Va
ria
nce
0 50 100 150 200 250
40
04
50
50
05
50
60
06
50
70
0
The sample variogram does not really increase monotonically with distance.
Our variogram models cannot fit this very well.
Though we can obtain estimates of the nugget, sill, and range, the estimates cannot be trusted.
No results: Percent Hardpan
This variable was so badly skewed that analysis was not reasonable.
The skewness coefficient is 12.38. This is extremely high.
QQplot of Percent Hardpan
Quantiles of Standard Normal
Pe
rce
nt
Ha
rd P
an
-3 -2 -1 0 1 2 3
05
01
00
15
02
00
25
0
Scatter plot of Percent Hardpan
RMI
Pe
rce
nt
Ha
rd P
an
0 200 400 600 800 1000
05
01
00
15
02
00
25
0
The data is nearly all zeros!
There is also an erroneous data value. A percentage cannot be greater than 100%.
Data analysis does not seem reasonable. Our data does not meet the conditions necessary to use the spatial methods discussed.
Conclusions
Able to fit sample variogram reasonably well – percent gravel, number of individuals, number of species
Not able to fit sample variogram well – percent sand, percent detritivore, percent simple lithophilic individuals, percent invertivore
No results – remaining variables
Summary of ResultsResponse Transformation Trend Removed Model Nugget Sill Range
Percent Gravel Exponential 286.09 335.53 72.9 milesPercent Sand 38.1082+.0330x Gaussian 520.88 658.32 71.67 miles
Percent CobblePercent Hardpan
Percent FinesPercent Boulder
Number of Individuals Natural Log Gaussian 0.29 0.39 44.19 milesNumber of Individuals Natural Log (no outliers) Exponential 0.2 0.27 37.69 miles
Number of Native Species 17.7849-.0042x Gaussian 10.1 11.87 39.93 milesPercent Tolerant Individuals
Percent Lithophilic Individuals Square Root 15.5364-.0023x Matern 0.92 2.76 44.02 milesPercent Nonnative Individuals
Percent Detritivore Square Root Exponential 1.09 1.57 24.08 milesPercent Detritivore Square Root (no outliers) Exponential 0.94 1.4 19.17 milesPercent Invertivore Square Root 6.5207-.0039x Exponential 1.4 2.97 13.43 milesPercent Piscivore
Future Work
Data set involving three streams in Norfolk, Virginia. Each stream has 25 observations. Collected by researchers at Old Dominion University.
Difficulties to overcome - What is the best way to measure distance between points? - Few observations - Overlapping points after coordinate conversion
Problem: What is the best way to measure distance between points?
There is some aspect of two-dimensionality to the data, but it is still really a one-dimensional problem.
Paradise Creek Region of Interest
Paradise Creek Sampling Sites
UTMX
UT
MY
0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4
0.0
0.2
0.4
0.6
Problem: 25 observations per stream is considered the minimum number of points to create a variogram
- the sample variogram will be very rough
- our variogram model estimates will probably be bad
To correct this, we will explore the possibility of combining the data from the three streams
Problem: Overlapping points after conversion
- Original data in longitude/latitude coordinates
- Convert to UTM coordinates so that Euclidian distance makes sense
- Converted UTM coordinates often result in overlapping sites (and even fewer unique sampling sites)
Stream Sampling Sites (Lat/Long)
Longitude (NAD27)
La
titu
de
(N
AD
27
)
-76.290 -76.288 -76.286 -76.284 -76.282
36
.80
03
6.8
02
36
.80
43
6.8
06
36
.80
8
Stream Sampling Sites (UTM)
UTM X
UT
M Y
920200 920400 920600 920800 921000
40
82
80
04
08
32
00
40
83
60
0
Stream Sampling Sites (Lat/Long)
Longitude (NAD27)
La
titu
de
(N
AD
27
)
-76.290 -76.288 -76.286 -76.284 -76.282
36
.80
03
6.8
02
36
.80
43
6.8
06
36
.80
8
Stream Sampling Sites (UTM)
UTM X
UT
M Y
920200 920400 920600 920800 921000
40
82
80
04
08
32
00
40
83
60
0
Acknowledgments
- My committee: Dr. Urquhart, Dr. Wang, and Dr. Theobald
- Dr. Davis and Dr. Reich for answering my spatial questions and letting me use their S-Plus spatial library
Concluding Thought
Before you criticize someone, you should walk a mile in their shoes. That way, when you criticize them, you’re a mile away and you have their shoes.
- Jack Handey
