Embed Size (px)
Citation preview
Charles University
FSV UK
STAKAN III
Institute of Economic Studies
Faculty of Social Sciences Institute of Economic Studies
Faculty of Social Sciences
Jan Ámos VíšekJan Ámos Víšek
Econometrics Econometrics
Tuesday, 14.00 – 15.20
Charles University
Second Lecture
Schedule of today talk
A brief repetition of the “results” of the first lecture.
The Ordinary Least Squares
What it is, does it exist at all, formula and properties ( in the form of a theorem).
An alternative method
Galton, F. (1886): Regression towards mediocrity in hereditary stature. (Návrat k průměru ve zděděné postavě.)
Journal of the Anthropological Institute vol.~15, pp. 246-263.
0XY
How to estimate from data?0
REGRESSION MODEL
At the end of previous lecture we arrived at:
Response
variable
Explanatory variable
Tiii
p XY)(rR
1
tan2
iX
iY )(r)(Sn
1i
2i
Find minimum of over all !!)(S pR
-th residual i
The method is called : The ( ordinary ) least squares
Adrien Marie Legendre (1805) Carl Friedriech Gauss (1809)
n,,2,1i,XYp
1ji
0jiji
2n
1ij
p
1jijiR
)n,OLS( XYminargˆp
2n
1i
TiiR
XYminarg p
The Ordinary Least Squares Odhad metodou nejmenší čtverců
)n,OLS(̂
2n
1i
TiiR
XYminarg p
The Ordinary Least Squares Odhad metodou nejmenší čtverců
Does it exist at all?
)XXXX(Y)(r pp131321211111
)XXXX(Y)(r pp232322212122
)XXXX(Y)(r pnp33n22n11nnn
22n
1i
Tii )(rXY)(S
)(ri
)n,OLS(̂ pRminarg
22n
1i
Tii )(rXY)(S
nj
j2
j1
X
X
X
)j(X
p
1jj
)j(XY)(r
2p
1jj
)j(XY
2p
1jj
)j(XY
Estimate by OLS (odhad MNČ)
-th explanatory variable ( -tá vysvětlující veličina)
jj
)n,OLS(̂ pRminarg
The Ordinary Least Squares Odhad metodou nejmenší čtverců
2p
1jj
)j(XY
}XZ:RZ{)X(
p
1jj
)j(n
M
Linear envelope of ( lineární obal )
XX
)X(Zminarg M2
ZY
)X(Zminarg M ZY
)n,OLS(ˆX
)X(Rn M
.Z
ZY Y
)2(X
)1(X
)X( M
.
)n,SLO(ˆX
The first explanatory variable
)ˆ(r )n,SLO(
Y
)X(Rn M
.
The second explanatory variable
..
)n,OLS(ˆX
The first explanatory variable
)ˆ(r )n,SLO(
)X(Rn M
.
The second explanatory variable
X
)(r Y
The solution exists and is unique.
2n
1i
Tii XY)(S
The functional to be minimized
2n
1i
Tii
jj
XY)(S
Ti
j
n
1i
Tii XXY2
ij
p
1kkik
j
Ti
j
XXX
p,,2,1j,0XXY)(S2
1ij
n
1i
Tii
j
p,,2,1j,0XXY ij
n
1i
Tii
Normal equations
n
1i
Tiiii
n
1i
Tii XYX0XXY
p1
12
11
X
X
X
)XY( T
11
p2
22
21
X
X
X
)XY( T
22
np
2n
1n
X
X
X
)XY( T
nn
0
0
0
Normal equations
npp2p1
2n2212
1n2111
XXX
XXX
XXX
Tnn
T22
T11
XY
XY
XY
0
0
0
p1
12
11
X
X
X
)XY( T
11
p2
22
21
X
X
X
)XY( T
22
np
2n
1n
X
X
X
)XY( T
nn
0
0
0
npp2p1
2n2212
1n2111
XXX
XXX
XXX
Tnn
T22
T11
XY
XY
XY
0
0
0
npp2p1
2n2212
1n2111
XXX
XXX
XXX
n
2
1
Y
Y
Y
0
0
0
Tn
T2
T1
X
X
X
Normal equations
npp2p1
2n2212
1n2111
XXX
XXX
XXX
n
2
1
Y
Y
Y
0
0
0
Tn
T2
T1
X
X
X
Tn
T2
T1
X
X
X
np2n1n
p22221
p11211
XXX
XXX
XXX
np2n1n
p22221
p11211
XXX
XXX
XXX
p
2
1
X
0XXYXXYX TTT
Normal equations
Normal equations
is of full rank, i.e. is regular X XX T
YXXXˆ T1T)n,OLS(
Ordinary Least Squares (odhad metodou nejmenších čtverců)
(Please, keep this formula in mind, we shall use it many, many times.)
0XXYXXYX TTT
XXYX TT YXXX TT
YXXXˆ T1T)n,OLS(
Ordinary Least Squares (odhad metodou nejmenších čtverců)
0XYHaving recalled the model and substituting it here ,
we arrive at )X(XXXˆ 0T1T)n,OLS(
T1T0T1T XXXXXXX
T1T0 XXX
Ordinary Least Squares (odhad metodou nejmenších čtverců)
T1T0)n,OLS( XXXˆ
T1T0)n,OLS( XXXˆ
Definition
An estimator where LY)X,Y(~ )X(LL
is matrix, is called the linear estimator .)np(
)n,L( 1̂
n
1i
TiiR
XYminarg p
- estimate Odhad metodou nejmenší absolutních odchylek
)(ri 1L
Roger Joseph Boscovich (1757)
Pierre Simon Laplace (1793)
Galileo Galilei (1632)
)n,L( 1̂
n
1i
TiiR
XYminarg p
- estimator Odhad metodou nejmenší absolutních odchylek
Does it exist at all?
)(ri 1L
Let be a sequence of r.v’s,
. Then is the best linear unbiased estimator .
If moreover , and ‘s are independent, is consistent. If further
where is a regular matrix, then
where
.
1ii }{ ,,0 ij
2jii
)n,SLO(̂
)n(OXX T )n(O)XX( 11T
)n,SLO(̂
QXXlim Tn
1
n
Q
)0))ˆ( 0),( ,(n
nOLS N(L n
120),( ))ˆ((cov QnOLS n
Theorem
),0(2
ij is Kronecker delta, i.e. if and for .1ij ji 0ij ji
What is to be learnt from this lecture for exam ?
The Ordinary Least Squares (OLS) – principle and existence.
Properties of OLS and conditions necessary for them.
Alternative estimating method.
All what you need is on http://samba.fsv.cuni.cz/~visek/Econometrics_Up_To_2010
