8/9/2019 Winer Filter
1/42
Week 3 ELE 774 - Adaptive Signal Processing 1
WIENER FILTERS
8/9/2019 Winer Filter
2/42
ELE 774 - Adaptive Signal 2Week
!o"ple#-val$ed stationar% &at least '(s(s() stoc*astic processes(
Linear discrete-ti"e +ilter, w , w ., w 2, ((( &IIR or FIR &in*erentl% sta/le))
%&n) is t*e esti"ate o+ t*e desired response d&n) e&n) is t*e esti"ation error , i(e(, di++erence /'( t*e +ilter o$tp$t and t*e
desired response
Linear 0pti"$" Filtering1 State"ent
8/9/2019 Winer Filter
3/42
8/9/2019 Winer Filter
4/42
ELE 774 - Adaptive Signal 4Week
Linear 0pti"$" Filtering1 State"ent
0pti"i8ation &"ini"i8ation) criterion1 .( E#pectation o+ t*e a/sol$te val$e,
2( E#pectation &"ean) s9$are val$e,
( E#pectation o+ *ig*er po'ers o+ t*e a/sol$te val$e
o+ t*e esti"ation error(
:ini"i8ation o+ t*e :ean S9$are val$e o+ t*e Error &:SE) is "at*e"aticall%
tracta/le(
Pro/le" /eco"es1
3esign a linear discrete-ti"e +ilter '*ose o$tp$t %&n) provides an
esti"ate o+ a desired response d&n), given a set o+ inp$t sa"ples $&),$&.), $&2) (((, s$c* t*at t*e "ean-s9$are val$e o+ t*e esti"ation error
e&n), de+ined as t*e di++erence /et'een t*e desired response d&n) and
t*e act$al response, is "ini"i8ed(
8/9/2019 Winer Filter
5/42
ELE 774 - Adaptive Signal ;Week
Principle o+ 0rt*ogonalit%
Filter o$tp$t is t*e convol$tion o+ t*e +ilter IR and t*e inp$t
8/9/2019 Winer Filter
6/42
ELE 774 - Adaptive Signal $adratic F$nc( = !onve# F$nc(
:ini"$" is attained '*en
&radient '(r(t( opti"i8ation varia/le
' is 8ero()
8/9/2019 Winer Filter
7/42
ELE 774 - Adaptive Signal 7Week
3erivative in co"ple# varia/les
Let
t*en derivation '(r(t( 'k is
6ence
or
??? @1 real, '*% ???
8/9/2019 Winer Filter
8/42
ELE 774 - Adaptive Signal Week
Principle o+ 0rt*ogonalit%
Partial derivative o+ @ is
Bsing and
6ence
8/9/2019 Winer Filter
9/42
ELE 774 - Adaptive Signal CWeek
Principle o+ 0rt*ogonalit%
Since , or
T*e necessar% and s$++icient condition +or t*e cost +$nction @ to
attain its "ini"$" val$e is, +or t*e corresponding val$e o+ t*eesti"ation error eo&n) to /e ort*ogonal to eac* inp$t sa"ple t*at
enters into t*e esti"ation o+ t*e desired response at ti"e n(
Error at t*e "ini"$" is $ncorrelated 'it* t*e +ilter inp$t?
A good /asis +or testing '*et*er t*e linear +ilter is operating in its
opti"$" condition(
8/9/2019 Winer Filter
10/42
ELE 774 - Adaptive Signal .Week
Principle o+ 0rt*ogonalit%
!orollar%1
I+ t*e +ilter is operating in opti"$" conditions &in t*e :SE sense)
W*en t*e +ilter operates in its opti"$" condition, t*e esti"ate o+ t*e
desired response de+ined /% t*e +ilter o$tp$t %o&n) and t*e
corresponding esti"ation error eo&n) are ort*ogonal to eac* ot*er(
8/9/2019 Winer Filter
11/42
ELE 774 - Adaptive Signal ..Week
:ini"$" :ean-S9$are Error
Let t*e esti"ate o+ t*e desired response t*at is opti"i8ed in t*e:SE sense, depending on t*e inp$ts '*ic* span t*e space
i(e( & ) /e
T*en t*e error in opti"al conditions is
or
Also let t*e "ini"$" :SE /e &D)
6W1 tr% to derive t*is
relation +ro" t*e corollar%(
8/9/2019 Winer Filter
12/42
ELE 774 - Adaptive Signal .2Week
:ini"$" :ean-S9$are Error
Nor"ali8ed :SE1 Let
:eaning
I+ is 8ero, t*e opti"$" +ilter operates per+ectl%, in t*e sense t*at
t*ere is co"plete agree"ent /'( d (n) and ( &0pti"$" case)
I+ is $nit%, t*ere is no agree"ent '*atsoever /'( d (n) and
&Worst case)
8/9/2019 Winer Filter
13/42
ELE 774 - Adaptive Signal .Week
Wiener-6op+ E9$ations
We *ave &principle o+ ort*ogonalit%)
Rearranging
'*ere
Wiener-6op+
E9$ations&set o+
in+inite e9n(s)
8/9/2019 Winer Filter
14/42
ELE 774 - Adaptive Signal .4Week
Wiener-6op+ E9$ations
Sol$tion Linear Transversal &FIR) Filter case
: si"$ltaneo$s e9$ations
8/9/2019 Winer Filter
15/42
ELE 774 - Adaptive Signal .;Week
Wiener-6op+ E9$ations &:atri# For")
Let
T*en
and
8/9/2019 Winer Filter
16/42
ELE 774 - Adaptive Signal .
8/9/2019 Winer Filter
17/42
ELE 774 - Adaptive Signal .7Week
S$/stit$te =
Re'riting
Error-Per+or"ance S$r+ace
8/9/2019 Winer Filter
18/42
ELE 774 - Adaptive Signal .Week
Error-Per+or"ance S$r+ace
>$adratic +$nction o+ t*e +ilter coe++icients = conve# +$nction, t*en
or
Wiener-6op+
E9$ations
8/9/2019 Winer Filter
19/42
ELE 774 - Adaptive Signal .CWeek
:ini"$" val$e o+ :ean-S9$are Error
We calc$lated t*at
T*e esti"ate o+ t*e desired response is
6ence its variance is
T*en
At wo(
& J min
is independent o+ w)
8/9/2019 Winer Filter
20/42
ELE 774 - Adaptive Signal 2Week
!anonical For" o+ t*e Error-Per+or"ance S$r+ace
Re'rite t*e cost +$nction in "atri# +or"
Ne#t, e#press @&w) as a per+ect s9$are in w
T*en, /% s$/stit$ting
In ot*er 'ords,
8/9/2019 Winer Filter
21/42
ELE 774 - Adaptive Signal 2.Week
!anonical For" o+ t*e Error-Per+or"ance S$r+ace
0/servations1 @&w) is 9$adratic in w,
:ini"$" is attained at wGwo,
@"in is /o$nded /elo', and is al'a%s a positive 9$antit%,
@"inH =
8/9/2019 Winer Filter
22/42
ELE 774 - Adaptive Signal 22Week
!anonical For" o+ t*e Error-Per+or"ance S$r+ace
Trans+or"ations "a% signi+icantl% si"pli+% t*e anal%sis,
Bse Eigendeco"position +or R
T*en
Let
S$/stit$ting /ack into @
T*e trans+or"ed vector v is called as t*e principal a#es o+ t*e s$r+ace(
a vector
!anonical +or"
8/9/2019 Winer Filter
23/42
ELE 774 - Adaptive Signal 2Week
!anonical For" o+ t*e Error-Per+or"ance S$r+ace
'.
'2wo
@&wo)G@"in
@&w)Gc c$rve
v.&.)
v2&2)
@"in
@&v)Gc c$rve
>
Trans+or"ation
8/9/2019 Winer Filter
24/42
ELE 774 - Adaptive Signal 24Week
:$ltiple Linear Regressor :odel
Wiener Filter tries to "atc* t*e +ilter coe++icients to t*e "odel o+ t*e
desired response, d&n)(
3esired response can /e generated /%
.( a linear "odel, a
2( 'it* nois% o/serva/le data, d&n)
( noise is additive and '*ite(
:odel order is m, i(e(
W*at s*o$ld t*e lengt* o+ t*e Wiener +ilter /e to ac*ive "in( :SE
8/9/2019 Winer Filter
25/42
ELE 774 - Adaptive Signal 2;Week
:$ltiple Linear Regressor :odel
T*e variance o+ t*e desired response is
J$t 'e kno' t*at
'*ere wo is t*e +ilter opti"i8ed '(r(t( :SE &Wiener +ilter) o+ lengt* :(
.( Bnder+itted "odel1 :K"
Per+or"ance i"proves 9$adraticall% 'it* increasing :( Worst case1 :G,
2( !riticall% +itted "odel1 :G"
woGa, RGR",
8/9/2019 Winer Filter
26/42
ELE 774 - Adaptive Signal 2
8/9/2019 Winer Filter
27/42
ELE 774 - Adaptive Signal 27Week
E#a"ple
Let t*e "odel lengt* o+ t*e desired response d&n) /e , t*e a$tocorrelation "atri# o+ t*e inp$t $&n) /e &+or conse9( sa"ples)
T*e cross-correlation o+ t*e inp$t and t*e &o/serva/le) desiredresponse /e
T*e variance o+ t*e o/serva/le data &desired response) /e
T*e variance o+ t*e additive '*ite noise /e
We do not kno' t*e val$es
8/9/2019 Winer Filter
28/42
ELE 774 - Adaptive Signal 2Week
E#a"ple >$estion1
a) Find @"in +or a &Wiener) +ilter lengt* o+ :G.,2,,4
/) 3ra' t*e error-per+or"ance &cost) s$r+ace +or :G2
c) !o"p$te t*e canonical +or" o+ t*e error-per+or"ance s$r+ace(
Sol$tion1
a) 'e kno' t*at and t*en
8/9/2019 Winer Filter
29/42
ELE 774 - Adaptive Signal 2CWeek
E#a"ple Sol$tion, /)
8/9/2019 Winer Filter
30/42
ELE 774 - Adaptive Signal Week
E#a"ple
Sol$tion, c) 'e kno' t*at '*ere +or :G2
T*en
v.&.)
v2&2)
@"in
8/9/2019 Winer Filter
31/42
ELE 774 - Adaptive Signal .Week
Application !*annel E9$ali8ation
Trans"itted signal passes t*ro$g* t*e dispersive c*annel and a
corr$pted version &/ot* c*annel noise) o+ #&n) arrives at t*e receiver(
Pro/le"1 3esign a receiver +ilter so t*at 'e can o/tain a dela%ed
version o+ t*e trans"itted signal at its o$tp$t(
!riterion1 .( Mero Forcing &MF)
2( :ini"$" :ean S9$are Error &::SE)
Filter, w!*annel, h
3ela%, O
#&n) %&n)
#&n-O)
&n)8&n)
-
8/9/2019 Winer Filter
32/42
ELE 774 - Adaptive Signal 2Week
Application !*annel E9$ali8ation
::SE cost +$nction is1
Filter o$tp$t
Filter inp$t
!onvol$tion
!onvol$tion
8/9/2019 Winer Filter
33/42
ELE 774 - Adaptive Signal Week
Application !*annel E9$ali8ation
!o"/ine last t'o e9$ations
!o"pact +or" o+ t*e +ilter o$tp$t
3esired signal is #&n-O), or
!onvol$tion
Toeplit8 "atri# per+or"s convol$tion
8/9/2019 Winer Filter
34/42
ELE 774 - Adaptive Signal 4Week
Application !*annel E9$ali8ation
Re'rite t*e ::SE cost +$nction
E#panding &data and noise are $ncorrelated E#&n)v&k)QG +or all n,k)
Re-e#pressing t*e e#pectations
8/9/2019 Winer Filter
35/42
ELE 774 - Adaptive Signal ;Week
Application !*annel E9$ali8ation
>$adratic +$nction = gradient is 8ero at "ini"$"
T*e sol$tion is +o$nd as
And @"in is
@"in depends on t*e design para"eter O
8/9/2019 Winer Filter
36/42
ELE 774 - Adaptive Signal
8/9/2019 Winer Filter
37/42
ELE 774 - Adaptive Signal 7Week
Application Linearl% !onstrained
:ini"$" - ariance Filter
Pro/le"1 2( We 'ant to design a /ea"+or"er '*ic* can resolve an
incident 'ave co"ing +ro" angle o &'it* a scaling +actor g),
'*ile at t*e sa"e ti"e s$ppress all ot*er 'aves co"ing +ro"
ot*er directions(
8/9/2019 Winer Filter
38/42
ELE 774 - Adaptive Signal Week
Application Linearl% !onstrained
:ini"$" - ariance Filter
Alt*o$g* t*ese pro/le"s are p*%sicall% di++erent, t*e% are"at*e"aticall% e9$ivalent(
T*e% can /e e#pressed as +ollo's1
S$ppress all co"ponents &+re9( or dir( ) o+ a signal '*ile
setting t*e gain o+ a certain co"ponent constant &o or o)
T*e% can /e +or"$lated as a constrained opti"i8ation pro/le"1
!ost +$nction1 variance o+ all co"ponents &to /e "ini"i8ed)
!onstraint &e9$alit%)1 t*e gain o+ a single co"ponent *as to /e g(
0/serve t*at t*ere is no desired response?(
8/9/2019 Winer Filter
39/42
ELE 774 - Adaptive Signal CWeek
Application Linearl% !onstrained
:ini"$" - ariance Filter
:at*e"atical "odel1 Filter o$tp$t U Jea"+or"er o$tp$t
!onstraints1
8/9/2019 Winer Filter
40/42
ELE 774 - Adaptive Signal 4Week
Application Linearl% !onstrained
:ini"$" - ariance Filter
!ost +$nction1 o$tp$t po'er = 9$adratic = conve# !onstraint 1 linear
:et*od o+ Lagrange "$ltipliers can /e $tili8ed to solve t*e pro/le"(
Sol$tion1 Set t*e gradient o+ @ to 8ero
0pti"$" /ea"+or"er 'eig*ts are +o$nd +ro" t*e set o+ e9$ations
si"ilar to Wiener-6op+ e9$ations(
o$tp$t po'er constraint
8/9/2019 Winer Filter
41/42
ELE 774 - Adaptive Signal 4.Week
Application Linearl% !onstrained
:ini"$" - ariance Filter
Re'rite t*e e9$ations in "atri# +or"1
6ence
6o' to +ind Bse t*e linear constraint1
to +ind
T*ere+ore t*e sol$tion /eco"es
For o,w
o is t*e linearl% !onstrained :ini"$"-ariance &L!:) /ea"+or"er
For o, wo is t*e linearl% !onstrained :ini"$"-ariance &L!:) +ilter
8/9/2019 Winer Filter
42/42
:ini"$"-ariance 3istortionless Response
Jea"+or"erVFilter
3istortionless = set gG., t*en
We can s*o' t*at &6W)
@"in represents an esti"ate o+ t*e variance o+ t*e signal i"pinging on t*eantenna arra% along t*e direction (
enerali8e t*e res$lt to an% direction &ang$lar +re9$enc% )1
"ini"$"-variance distortionless response &:3R) spectr$" An esti"ate o+ t*e po'er o+ t*e signal co"ing +ro" direction An esti"ate o+ t*e po'er o+ t*e signal co"ing +ro" +re9$enc%