A Study on System Identification Using Wavelet … · A Study on System Identification Using Wavelet Transformation Wook Jin Baek, Jeong Woo Han, Sung Ju Kang

HWAHAK KONGHAK Vol. 39, No. 2, April, 2001, pp. 163-175(Journal of the Korean Institute of Chemical Engineers)

��

��†��*

�� , *��(1999� 10 20� ��, 2001� 2 20� ��)

A Study on System Identification Using Wavelet Transformation

Wook Jin Baek, Jeong Woo Han, Sung Ju Kang† and Chang Bock Chung*

Department of Chemical Engineering, *Faculty of Applied Chemical Engineering, Chonnam National University, Kwangju 500-757, Korea(Received 20 October 1999; accepted 20 February 2001)

� �

��(Fourier) �� (wavelet) �� (de-noising, ��, !�

"#$� %& ')�( )*+ ,-. /,0 1). 2 3 de-noising� �� 4567 8,�� 3� �9.(

:; /�+ ��< =>0 1). De-noising ?@A B,�A shrinkage CDE �� 6,0A threshold F GH�A "

I� JK1). L ��(A (. )M NO PA QRE ST� UV� WX �� threshold Y �Z 6,[\ ]@

^ (. _`� , 2 8,a. de-noising� �� Y �Z �,�� ST bc+ QRE 2de f� QR^ [gh $

i(system identification)� 6,[j 2 Ok (. _`�l).

Abstract − The wavelet transformation, which was developed in order to overcome the defects of traditional Fourier trans-

formation, is applied to many fields of study in various ways-for example, de-noising, data compression and mathematic appli-

cations such as solving partial differential equations, etc. De-noising is one of the main application areas of the wavelettransformation and has been studied by many researchers. The effect of de-noising depends upon the shrinkage function and

the method of choosing the threshold value for the function. The objective of this work is to analyze the results of applying var-

ious threshold algorithms according to characteristics for signals and noise level. By applying the de-noising to the system

identification, we compared the performances of signals which went through the de-noising process with those of signals with-

out de-noising.

Key words: Wavelet, De-noising, Shrinkage, Threshold, System Identification

†E-mail: [email protected]

1. � �

�� , ��

�� !"�� #$ sharp spike% ��

% &'( )�* �+) �,� �-.(Gibb's phenomena)� �

� �,. % /0)�1 2 34 �� 56789:, de-noising,

1;<=, �>?@, ABC�, D�EF, GHI JKFL MN O P

Q IR�S ,TU VWX YV7� �,[1]. Z [ de-noising� 2

34 ��L \]�E V IRXS Donoho[2-4], Johnstone[3-6], Gao

[7-11] O� LM Y6U !_� à7� �,. bD� �cd% ��

è f� ABX UKg� � 2 34 �� V)P bD -#

% )� Jh !_789: PQ i��jk 5678,.

lKmn� _)� Jh9X opq 1;% V)� Black-Box m

nr Jh� 'V)� st, \uIL v- lKL opq 1;� ,T

U wxL bD� y�)� �9zX K{U lKmn� _)� |}

,. 2 34� \U ~� ��7�S 2 34 �� V)P �

S ��U ��E wxL bD� �(�) C�U AB% �� ,

$� lKmn� _)� �U �� F� ~� !_� �& � � '

� 6]78,. Palavajjhala O(1996)� � 2 34 ��(discrete

wavelet transform)� V)P prefilter �K� #� AB% �� F

�� V)�9:[12], Nikolaou O(1998)� 2 34 �� V)

� AB <=� �M FIR Jh� �>�� !_% )�,[13]. �U

Carrier O(1998)� 2 34 �� -|� ��% �U lK * �L

F�� V)��[14], Flehmig O(1998)� noisy �� @K A

B�L s�� F�)� !_� 2 34� V)�,[15].

� !_�S� Table 1� K�U PQ � 2 34 de-noising i�

�jk� lK mn� _)� �� F� !_� �V)P bD -#

�K� #� AB% VU st bD -# �K� #¡ �� AB

% VU stL �¢� £¤)��, PQ de-noising i��jL �¢

W £¤)P �¥,. Z�� ABL �� ¦§ de-noising �

163

164 ��

% �� F� �K� �Vg� �L �¢� i©�� )P SX ,

§ �� ª 5L �«mn� V)��, bDL ¬�� \U

�� i©�� )P, ,TU bD-AB£(Noise-to-Signal Ratio, NSR)

% �� bD� ®®L �« mn� ¯�)�9: de-noising�K�S

VU 2 34 ��L U JhE Multi-Resolution Analysis(MRA)L °

±(level)� ¦§ �¢� £¤)�,.

2. � �

2-1. ��

��kL ²³� VU ABL cd� MN� H 1800�\ ´

Joseph Fourier� KLU sine� cosine� V)P ��% ]�)� �

�� S �W78,. ZQ$ �� L ��% _�)�

sine, cosine�� © !" AB sharp spike MN�

�-% �� , �+U ��L µ¶� \ª7

8,[1]. �U �·� cd� K�% ¸�� ]�¹ � º� ». -

�7| �·� cd� K�% �� $¼½ � �� Short Time Fourier

Transform(STFT) 56789$ �U M>W(resolution)� �K7

� ».� �`,. % /0)�1 �� ,T):

cd�-�· ]� �¢U 2 34 �� 5678�, 1980�\ [

� � S. Mallat� I. Daubechies O� LM �"WX 6�7| PQ

IR�S ^V7� �,.

2 34 �� m(¾) 2 34(mother wavelet)L scaling� trans-

lation�K� LM ,TU ��% ¿�)� �� cd� �· K

�% ¸�� +)À ]�¹ � �,� Á.� �� ,. � scale

� �· K�% y�)� translation� cd� K�% y�U,[1, 16].

2-1-1. !" 2 34 ��(Continuous Wavelet Transform, CWT)

CWT� ©Â Ã� F (1)XS KL7� Ä� MN)�1 )� A

B x(t) m 2 34 Å �� ψ(t)L convolution wx%

�9:, *� lÆ0Ç�(complex conjugate) ,.

(1)

P�S s� scale È5�� : τ� translation È5�� ,(s, τÉR).

2-1-2. � 2 34 ��(Discrete Wavelet Transform, DWT)

CWT% ��)� �MS� �( �K µ¶),. F (2)� LM ®

®L È5 ��k� �(�Ê F (3)� � 2 34 ��(Discrete

Wavelet Transform, DWT)Å U,.

(2)

(3)

��9X s0 τ0L �� 2 1� 'V)� m, k� ]�((sampling)

�K� 'V7� È5�� ,.

DWT� ËÌ�% V)P CWTL �� ¢)À )�9$ �

» CWTL ]�(Í �� )zX µ¶U �� Î© £¨

Ï� ,. QU ».� �Ð)�1 1989� Mallet� LM DWT% �

, ÑÒ� ¨Ï�9X ��¹ � �� Ó� i��jE MRA Jh

-Ô78,[1, 16-18].

2-1-3. Multi-Resolution Analysis(MRA)

De-noising�� 9X �" � 2 34(fast discrete wavelet)

i��jE MRA� 'VÍ,. MRA� 2 34 ��L �Õ9X Ö�

(filter)% V):, Ö�kL ×¤�� V)P ��Ø� ��Ù ÚE

Jh ,. Ö�L wx� 'V)� m 2 34� ¦Å ²K7: SX

×¤)� �� Ö�(low-pass filter)�¹� )� scaling ��[F (4)] �

� Ö�(high-pass filter)�¹� )� 2 34 ��[F (5)]% 'VU,.

(4)

(5)

(6)

P�S h(m)� �� Ö� �� g(m)� �� Ö� �� ,. Ö�

��k� dyadic down sampling �K� #� ABL convolution� L

M ��7: KÛ(% �M � 'VÍ,. m 2 349X 'VU

Daubechies 3 2 34� Fig. 1� $¼Ü8,.

Fig. 2(a) (b)� F (4) (5)% ÝÞßWX W�U Ä : °±� ¦

ÅS IM(decomposition)�K� �_�(reconstruction)�K� #¡À

Í,. IM �K�S �� µ�� LM IMÍ AB% approximation �

�(cA)Å )� �� µ�� LM IMÍ AB� detail ��(cD)Å U,.

� IMÍ ® �� 5�% +�9X ÚPc� dyadic down-sampling

�K� #¡: IMÍ approximation�� ,D °±� �a)�*

VÍ,. �_� �K� IMÍ ��k� ²³)� �K9X IM�K�

S +�9X Ú|à 5�% áÂ >xX �á)� dyadic up-sampling

�K� #â AB% 0áU,.

°±� ¦§ ]�( �K� LM �· M>W� �+7� filtering�

K� LM cd� ·ã �+Í,. ¦ÅS °±� ¦Å �·� cd�

� \M ,TU M>W% �À 7�*, � MRAÅ �� [¶U

CWTx τ s,( ) 1

s-------- x t( )ψ∗ t τ–

s---------

dt∫=

s s0m= τ kτ0s0

m=,

DWTx m k,( ) 1

s0m

----------- x t( )ψ∗ s0m–t kτ0–( ) td∫= m k, Z∈

φ x( ) h m( )m Z∈∑ 2φ 2x m–( )=

ψ x( ) g m( ) 2φ 2x m–( )m Z∈∑=

g m( ) 1–( )mh 1 m–( )=

2

Table 1. Applied threshold method for each shrinkage

ShrinkageThreshold method

Minimax Universal SURE Hybrid GCV

Hard √ √ √ √Soft √ √ √ √ √Non-Negative Garrote √ √ √ √ √Firm √

Fig. 1. ‘Daubechies 3’ wavelets.

�� 39� �2� 2001� 4�

�� 165

ä� Í,[17, 18].

2-2. De-noising

De-noising� ��9X bD� å AB� £M cd�� ¬,� .

� æÔ)P bD� -#)� Jh ,.

2 34 �� VU de-noising �K� ¬À 3»�X ç| `

,[2, 5, 7].

1 »� −IM�K°±� ßè)�, ßèÍ °± IM�K� �a

2 »� − Detail ��k� threshold �K� �V

®®L °±� \M threshold Jh� ²K)�, shrinkage �� V

3 »� −�_� �K

�KÍ detail �� éL approximation ��% V)P � ��

ª êë »�� de-noising �K�S vì�9X [¶U »�XS threshold

Jh�S ��Í threshold�E λ� V)P shrinkage �� LM

�cd K�% y�)� detail ��k� �KU,.

2-2-1. Threshold �L ßè

Shrinkage �� V�� U λ�� _)� Jh ,. Threshold

� íî f9� P�Ù bD� Î y�¹ Ä �, �\X threshold

� íî ¬� ABL [¶U K�� ïv7| ð ñ |� I��

R�¹ Ä ,. ¦ÅS ® °±� \M �+U threshold �� ²K)�

Jh� [¶U �-� Í,. � \U PQ Jhk ��7| ��

� !_�S� minimax, universal, SURE, hybrid, GCV Jh� 'V)

�,.

�Minimax

Minimax risk bound% �ÇX )� λ�� U,.

®®L shrinkage �� \U λ�k� Gao� LM ��Í �� '

V)�,[7-9].

�Universal

F (7)� Ã detail ��L 5�� LM ��7zX �� K £¤

� ·»)� bDL ¬�� f� st� �¢ ò,� ióô �,[6].

(7)

P�S nj� °± j�S detail ��L 5� ,.

�Stein’s Unbiased Risk Estimate(SURE)

SURE criterion� �ÇX )� �� ßèU,[3, 6, 10, 19].

SURE Jh� �� 0b)� I�� ñ |�% �ó)� �,

³��E Jh ,. MRAL °±� ¦Å �� 7| : bDL ¬�

� õ�Þ �¢ ò,� ióô �,. jêë °±�SL SURE criterion

� F (8)X ]�Í,.

SURE criterion :

(8)

P�S #� �ö� �÷)� ��kL 5� � xøy� ª � [�S

f� �� ßè)� �B ,.

�Hybrid

©Â� KLÍ hybrid KL (I), (II)� LM bDL ¬�� f9�

SURE Jh ò �9zX universal Jh� ßè)�, Z ù��

SURE universal Jh [ f� λ�� ßè)� �KÍ SURE Jh

,[3]. jêë °±L st ,D� Ã� Jh� LMS λ�� ßèU,.

(I) E st: bDL ¬�� f� stX universal Jh�

ßè.

(II) E st: bDL ¬�� ð stX ��S �úU J

h� ¦Å λ�� ßè

(9)

�Generalized Cross-Validation(GCV)

!"�� -X )zX soft shrinkage non-negative garrote shrinkage

� �V7: !"�� hard shrinkage�� V¹ � º� thresh-

old Jh ,. GCV�� F (10)� Ã detail��L ��X ç|

ô �� bD� \U '�F µ¶ º,. ® °±� \M

GCV ��% �ÇX )� δ�� )� % threshold �(λ)9X

VU,. ×û �Ç�� ü� �� L Jh Å ¹ � ��

��K £¤� 0b),. � ® °±�S δ� 0�Su� cDjL �\

�� ý�X )� �� 'V)�� þÿ I¹ h(golden section method)

� V)P GCV ��L �Ç�� ²K)�,[20-24].

(10)

P�S nj,0� °±j�S 0�� detail ��L 5� � cDj,δ� °

± j�S c|` threshold �(δ)� V)P soft, non-negative garrote

shrinkage� LM �KÍ detail �� ,. ||x||2� -� norm� $¼Ü

� �B ,.

2-2-2. Shrinkage ��

� !_�S� � � shrinkage ��% 'V)�,. Hard shrinkage

� keep-or-kill Jh9X, detail ��L +\� λ�, f� st detail

��% 09X �KU,. Soft shrinkage� shrink-or-killXS detail ��

L +\� λ�, f� st detail ��% 09X )� Z ùL detail

��k� soft shrinkage �� LM ÚPc� Û� ,. Soft shrinkage

λ 2 nj( )log=

SURE λ cD,( ) nj 2 # i: cDi λ≤{ } cDi λ∧( )2

i 1=

nj

∑+⋅–=

Sj2 Qj nj⁄≤

Sj2 Qj nj⁄≤

Sj2 nj

1– cDi2 1–( )

i 1=

nj

∑= , Qj nj( )3 2⁄2log=

GCVj δ( )

1nj

---- cDj cDj δ,– 2

nj 0,

nj

------- 2

-----------------------------------=

Fig. 2. (a) Decomposition process of MRA, (b) Reconstruction processof MRA.

HWAHAK KONGHAK Vol. 39, No. 2, April, 2001

166 ��

� £¤� ð �� % Ú� � hard shrinkage� £M ñ |

�� ¬À $¼�,. Hard shrinkage� !"� � �� soft shrink-

age�, I� ¬� ÔKU G ,[3, 4, 7]. kL ».� �Ð)�1

�, f� ñ |� ÔK�� Non-Negative Garrote shrinkage

� -Ô78,[8]. Firm shrinkage� semi-soft shrinkageÅ�W ): hard

shrinkage soft shrinkage% �³U [· wx ,. Minimax Jh��

'V)�9: Gao� LM ��Í �� 'V)�,. Detail ��L +\

� λ1�, f� st detail ��% 09X )� λ2� �, ð st�

detail ��% �é): $� st� soft shrinkage�� LM detail

�� KU,[9].

® Jh� LM detail ��% �K)� �U Fk� ©Â Ã,.

�Hard

(11)

�Soft

(12)

�Non-Negative Garrote

(13)

�Firm(Semi-soft)

(14)

P�S sign� uB% ²KM c� signum function ,.

2-2-3. Scale

Shrinkage ��k� threshold �� LM �t7� �� +U λ�Lßè� ©c [¶),. °± Lé�� MRA�S bD �t� I

y% ¦Ò� white noiseE st� °±� \M bDL �� ó)

�©W 7$ colored noise% �� st�� °±� ¦§ detail ��L >

~~�% �óMR U,[11]. ¦ÅS °±� ¦Å bDL ¬�% �óU

threshold �L �� µ¶)À 7�* % scale� V)P �KU,.

F (15)% V)P _U �KÍ ]G% ®®L threshold�� M

�VU,. ��9X white noise� st�W °±� �ó)P scale�

VU threshold �� )� Ä �� ³),� ióô �,[6].

(15)

P�S MAD� 09Xu�L median absolute deviation � 0.6745�

�t� Iy% �WÞ )� �KE1 ,.

2-3. �� (System Identification)

�� F�� opq * �Xu� �� \U ¸� mn� �K

)� � � mnrL U �9X �Ù black box modeling Å U,.

opq * �Xu� lK mn� ²K)� �MS� ,DL 4»�%

#¡À Í,[25, 26].

»� 1. opq 1; ��

lK� \U ò� mn� �� MS� M� lK� \U �IU K

�% y�)� opq * �� µ¶),.

»� 2. mnL ßè

mn _�% ßè)� ßèÍ mn� \U � �·!� ²KU

,. »� opq�L ��(Í ßw mn _�� oq u(t) pq y(t)

% !~�� F (16)9X ]�� ,[26].

(16)

F (16)�SL A(q), B(q), C(q), D(q) � F(q)� q−1� \U ,�F9

X ©Â Ã ]�Í,.

P�S nk� �·!, e(t)� white noise% $¼Ü� q−1� Backward

shift operatorX ?% k�, q−1y(t)=y(t−1)L ~�% �E,.

� !_�S� È5 �� B% V)� Finite Impulse Response(FIR)

mn� B F% V)� Output Error(OE) mn� 'V)�,.

»� 3. lK mnL È5 �� K

ßèÍ mn� LM ��Í pq * �� v«� LM �|` pq

* �� ³)WÞ ��L È5 ��k� �� MS� ��U �

»� µ¶),. �� F��S� QU �»�� loss function

Å ): ��9X F (17)L wx% �� -��(Mean Square

Error, MSE)L wx% ��,.

(17)

P�S N� mn �K� 'VÍ * �L 5�, y(t)� @Kpq¡,

� �· t�SL �K pq� LHU,.

»� 4. mn validation

�KÍ mn 'V�� IÙ ò�% ��)� »�X mn�

�K)� �K(»� 3)� 'V7 �� * �% VU,. Validation

�K� mn � ²K�W ~�Í,. �% ��S �� F

��K� aU �, �÷¹�U validation²�% �9� [»)� �L

mn _�% ßèU,. �� ¦§ loss function

L �Ç KW% VU,[25].

2-3-1. FIR mn

lK� \U '�F(�· !, mn � O) µ¶ º,� Á.

� �� 9$ Î� �L È5 ��k� µ¶X U,� ».� ��

�,. FIR mn� �M �� · !� ,§ mn _�(OE, BJ O)�

V¹ � �9: F (18)X ]�Í,[25, 26].

(18)

2-3-2. OE mn

OE mn� ßw � �· mn� \M �Á ×û� � £¤� ·

»U Jh ,[25, 26].

(19)

OE mn� lK� \U '�F µ¶): �� FIR mn� V)

P �·!� ��,. OE mnL loss function� F (20)9X ]�¹ �

�9: mn È5�� \M £ßw ,. mn� \U È5 ��k

L �� loss function� �Ç( �9XS ²K7�* loss function È

5 ��k� \MS £ßw zX £ßw ��INh� VMR U,.

Thard cD λ;( ) 0 for cD λ≤cD for cD λ≤

=

Tsoft cD λ;( ) 0 for cDλ≤sign cD( ) cD λ–( ) for cD λ≥

=

TGarrot cD λ;( ) 0 for cDλ≤

cD 1 λ cD⁄( )2–{ } for cD λ≤

=

TFirm cD λ1; λ2,( )

0 for cDλ1≤

sign cD( )λ2 cD λ1–( )

λ2 λ1–----------------------------- for λ1 cD λ2≤<

cD for cDλ2>

=

σjMAD cD j( )

0.6745---------------------------=

A q( )y t( ) B q( )F q( )-----------u t nk–( ) c q( )

D q( )------------e t( )+=

A q( ) 1 a1q1–+Λ a+ naq

na–B q( ) b1q 1–

+Λ bnbqnb–

,+=,+=

F q( ) 1+f1q1– +Λ+fnfq

nf– C q( ) 1 c1q 1– +Λ+cncqnc– ,+=,=

D q( ) 1 d1q1– +Λ d+ ndq

nd–+=

V loss1N---- y t( ) y t( )–( )2

t 1=

N

∑ 1N---- ε2 t( )

t 1=

N

∑= =

y t( )

y t( ) B q( )u t nk–( )=

y t( ) B q( )F q( )-----------u t nk–( ) e t( )+=

VOE1N---- εOE

2 t( ) 1N---- y t( ) y t( )–[ ]2

t 1=

N

∑=t 1=

N

∑=

�� 39� �2� 2001� 4�

�� 167
(20)
3. Simulation

2,0005L Pseudo-Random Binary Sequence(PRBS)% oq9X )�

Table 2L È5 �� F (19)�S bD �� -ùU OE mn� V

)P oq9Xu� pq AB% ¿�)P % å ABX 'V)�,.

� �«mn 1� �«mn 2L SX ,§ �� ª 5L AB%

¿�)�9: ® mnL �� ,D� Ã,[27].

�«mn 1: �· ! 2E 3�

�«mn 2: �· ! º� 3 � ^ �

¿�Í å AB� NSR 5, 10, 30, 50, 75, 100%% �WÞ �t� I

y% �� white noise� �)P @K pq AB(measured output signal)

% ¿�)�9: NSRL KL� F (21)� Ã,.

(21)

1N----= y t( ) B q( )

F q( )-----------u t nk–( )–

2

t 1=

N

∑

NSR %( ) Variance of NoiseVariance of True Signal---------------------------------------------------------------- 100 %( )×=

Fig. 3. True(solid lines) and various measured signals(dashed lines) of test model #1.

Fig. 4. True(solid lines) and various measured signals(dashed lines) of test model #2.

Table 2. OE parameters applied on each test model

B F

Model #1 [0 0 0 0.075-0.0425 0.005] [1-2.4 1.91-0.504]Model #2 [0-0.175-0.365-0.18] [1-2.46 2.0-0.5376]


168 ��

bD ¯�Í @KAB� ®®L de-noising i��j� �V)��

�¢L !WX� ��-��% 'V)�,. MRA �K�SL °±�

°±� ��" ��U �, ®®L NSR� ¦Å MSE� f� ��L

°±� ßè)�,.

bD -#Í AB% �� F�� V¹ � 2,0005L 1; [ 1,024

(210)5� mn �K� 'V)�� $� 9765� mn validation� '

V)�,.

OE mn� VU validation�K�S ®®L È5 �� \U global

minimum �� ü� �M �0�� µ¶),. �\ 50êL �0 ��

� �a)�9: 50ê Ü� �# �� ³) �� AB� �ó

\>�S -ù)�,. �# �9X� gradient norm[26, 28] 10−2�

, f� st% 'V)�,.

bD� �t� Iy% �� % 6¿�" ¿�)�� Ã�

NSR� �� bD ÅW bDL 6¿ $S� ¦Å MSE� %ÅÀ Í,.

¦ÅS mà �KL MSE� 50-100êL bD 6¿�K� #â �IÙ �

#�Ê � MSEL �� 'V)P PQ � Jhk� £¤)�,.

®®L å mn� ,TU NSR� �WÞ bD� ¯�)P ¿�U @

KAB% Fig. 3� 4� $¼Ü8,.

�«mn 1� �«mn 2Xu�L å pq AB� \U cd� è

L ~�% �� V)P Fig. 5� $¼Ü8,. Z&�S �

� �«mn 2� �«mn 1� £)P å AB 1'� �cd uI� >

�Ù Î y�)� noisy signal(� i � �,.

� !_�S� MATLAB� V)P simulation� �a)�9: Wavelet

Toolbox System Identification Toolbox% V)�,.

4. ��

4-1. �� de-noising

MRA% VU de-noising �KL °±� \U �� i© ��

M °±� ��S de-noising� �a)�9: de-noising �¢L

Fig. 5. Frequency vs. amplitude plot using fast fourier transform.

Table 3. Simulation results of MRA level with minimum average MSEfor various NSR values

NSR(%) 5 10 30 50 75 100

LevelTest model #1 2 2-3 3 3 3-4 4Test model #2 1 2 2 3 3 4

Fig. 6. Effect on level of MRA for test model #1(Soft shrinkage case with various threshold methods). “circle” =Minimax, “ square” =Universal,“ triangle up” =SURE, “ diamond”=Hybrid, “ triangle down” =GCV.

�� 39� �2� 2001� 4�

�� 169


Table 4. MSE of de-noised signal for test model #1 (Universal threshold case with various shrinkage methods)

NSR(%)MSE

(Without de-noising)Level Hard Soft

Non-Negative Garrote

5 Mean : 0.0986 2Mean 0.0283 0.0281† 0.0283

Std 0.0014 0.0014 0.0014

10 Mean : 0.1972 2Mean 0.0528 0.0529 0.0525†

Std 0.0026 0.0026 0.0026

30 Mean : 0.5912 3Mean 0.1052 0.1041† 0.1041†

Std 0.0069 0.0065 0.0065

50 Mean : 0.9856 3Mean 0.1549 0.1532† 0.1532†

Std 0.0102 0.0097 0.0097

75 Mean : 1.4788 3Mean 0.2185 0.2156† 0.2157

Std 0.0169 0.0158 0.0158

100 Mean : 1.9708 4Mean 0.2638 0.2603† 0.2603†

Std 0.0174 0.0160 0.0160

Table 5. MSE of de-noised signal for test model #1 (Soft shrinkage case with various threshold methods)

NSR(%)MSE

(Without de-noising)Level Minimax Universal SURE Hybrid GCV

5 Mean : 0.0986 2Mean 0.0283 0.0281† 0.0286 0.0281† 0.0286

Std 0.0014 0.0014 0.0015 0.0014 0.0017

10 Mean : 0.1972 2-3Mean 0.0531(2) 0.0529(2) 0.0524(3) 0.0525(2) 0.0521(3) †

Std 0.0027 0.0026 0.0042 0.0026 0.0044

30 Mean : 0.5912 3Mean 0.1062 0.1041† 0.1067 0.1041† 0.1067

Std 0.0072 0.0065 0.0087 0.0065 0.0101

50 Mean : 0.9856 3Mean 0.1569 0.1532† 0.1598 0.1532† 0.1603

Std 0.0107 0.0097 0.0134 0.0097 0.0172

75 Mean : 1.4788 3-4Mean 0.2216(3) 0.2156(3) † 0.2270(3) 0.2156(3) † 0.2273(4)

Std 0.0177 0.0158 0.0194 0.0158 0.0257

100 Mean : 1.9708 4Mean 0.2676 0.2603 0.2661 0.2599† 0.2671

Std 0.0189 0.0160 0.0264 0.0264 0.0346

The index in parentheses indicates MRA level

Fig. 7. Results of validation based on FIR method for test model #1(Soft shrinkage case with various threshold methods).

170 ��

!WX� å AB de-noising �K� #� AB ' L MSE% 'V

)�,. �\ MRAL °±� 5X g� � �� MSE(average MSE)�

�Á f� °±� Table 3� K�)�,. � !_�S 'VU PQ i�

�j� °±� ¦Å �VU ²�X� MSE �L � �9$ ��

MSE� �ÇE °±� #L Ã� s�� ,. \]�9X soft

shrinkageL st �«mn 1L PQ °±� \U de-noising²�% Fig. 6

� $¼Ü8,.

Table 3�S �) bDL ¬�� õ�Þ MRA�K�S� �� *

� °±� µ¶X �� i � �� ¸�U NSR� \M �«mn 2�

�«mn 1� £)P +� °±� 'V)PR �� i � �8,. Ä

� �«mn 2� �«mn 1� £M å AB 1'� noisy��

zX MRA�K�S °± *©,� ¦Å bD-� ©.Å å ABL

�cd K�� / -#7� �� 0 ²�% �E,� ¿®Í,.

Table 4� universal threshold% 'VU stX �«mn 1� PQ �

shrinkage Jh� �Vg� �L ²�% $¼19: ,§ threshold

Jh� 'VU stW £2U s�� ,. Table 4�S i � �)

�«mn 1L st !"�� hard shrinkage� £)P !"�E

soft shrinkage non-negative garrote shrinkage� �� MSE �� ð

� º8� � $� �¢� �E,. Z�� «mn 2� \MSW

Ã� ²�% �8,. ®®L Tablek� �� MSE(Mean9X ]�) ]

G(StdX ]�)% �¡X $¼Ü89: �B ‘†’% V)P �Á

f� �� MSE% _�)�,.

Table 5�� \]�9X soft shrinkage��% 'V)P PQ �

threshold Jh� �Vg� stL MSE% �«mn 1L st� \MS $

Fig. 8. Results of validation based on FIR method for test model #2(Soft shrinkage case with various threshold methods).

Fig. 9. Results of validation based on OE method for test model #1(Soft shrinkage case with various threshold methods).

�� 39� �2� 2001� 4�

�� 171


Fig. 10. Results of validation based on OE method for test model #2(Soft shrinkage case with various threshold methods).

Fig. 11. Model validation for test model #1(Soft shrinkage case withSURE threshold method). “heavy solid line”=True signal, “heavydashed line”=Measured signal, “circle”=Predicted signal(with-out de-noising), “light dashed line”=De-noised signal, “square”=Predicted signal(with de-noising).

Fig. 12. Model validation for test model #2(Soft shrinkage case withSURE threshold method). “heavy solid line”=True signal, “heavydashed line”=Measured signal, “circle”=Predicted signal(withoutde-noising), “light dashed line”=De-noised signal, “square”= Pre-dicted signal(with de-noising).

172 ��

¼Ü89:, ,§ shrinkage�SW £2U s�� ,. ThresholdL s

t �«mn 1� universal� hybrid Jh � ò� ²�% � � �9:,

�«mn 2L st� +� NSR�S� universal Jh ò� NSR 30%

>�S� SUREJh � ò� ²�% $¼3 Ä9X {E� )�,.

4-2. ��

FIR mn� OE mn� 'V)P �� F� �K� �a)�9:

PQ � de-noising �K� #� AB de-noising �K� #¡ �

� AB� \U �� F� �¢� £¤)�,.

OE mn� 'V)�* �|S 2 34 de-noising �% �')�

�M de-noising� U st Z4 �� st mª å ABL mn

� �· !� 'V)�,.

®®L �« mnk� \U validation²�E �� MSE% \]�9

X soft shrinkageL st� \M Z&(Fig. 7-10)� ](Table 7-10)X $

¼Ü8� validation plot� Fig. 11� 12� $¼Ü8,.

FIR Jh� VU st� Fig. 7-8� Table 7-8� $¼Ü8,. ª �

«mn mª de-noising �K� #� AB� de-noising� #¡ ��

AB� £M 56 � ò� �¢� �� mà shrinkage threshold�

S NSR ��¹�Þ de-noising� U st� �� F� �¢ �

ò©,� {E¹ � �8,.

�U de-noising �KL MRA°±� \U �� i© �� )P

�Á ò� validation� �� °±� Table 6� K�)�,. �«mn 1

L st Table 3L de-noising °±� £¤)� �Á ò� lKmn� �

� �MS� ��L de-noising °±�,� U »� KW +� °±�

ßèMR �� i � �,. � Table 3L °±� 'VU st�� de-

noising �K�S å AB L MSE� f� v-X� å ABL K�

% >�uI 7� �� QU ²�% �E Ä9X ¿®Í,. �«m

n 2W �«mn 1� ä'U ²�% �P c�* �«mn 1� £M >

\�9X +� °±� ßèMR �� i � �8,. Ä� �«mn 2

L AB ��S ü� � �9: å AB 1'� �cd% Î y�

)� noisy AB� de-noising �K�S å ABL K�� ï>7� �

�� +� °±� ßè)P ABL K�� ï>7� Ä� JMR ¹

Ä9X ¿®Í,.

OE Jh� VU st� Fig. 9-10� Table 9-10� $¼Ü8,. �«

mn 1� °± 2(NSR 5%E st) ) �� °± 3(NSR 30% >E

st) )�S de-noising� #� AB% 'VU st� Z4 �� A

Table 6. The optimal MRA levels in model validation of FIR method

NSR(%) 5 10 30 50 75 100

LevelModel #1 2 2 2 2 3 3Model #2 1 1 2 2 2 3

Table 7. Average MSE in model validation of FIR method for test model #1(Soft shrinkage case with various threshold methods)

NSR(%) Without de-noising Level Minimax Universal SURE Hybrid GCV

5Mean : 0.0302

Std :0.0022

1 Mean 0.0233 0.0233 0.0233 0.0233 0.0234Std 0.0011 0.0011 0.0011 0.0011 0.0011

2Mean 0.0215† 0.0215† 0.0215† 0.0215† 0.0216Std 0.0009 0.0009 0.0009 0.0009 0.0009

3Mean 0.0329 0.0335 0.0311 0.0334 0.0304Std 0.0014 0.0015 0.0020 0.0015 0.0029

4Mean 0.0931 0.0943 0.0858 0.0943 0.0832Std 0.0024 0.0023 0.0099 0.0022 0.0122

30Mean : 0.0963Std : 0.0138

1 Mean 0.0521 0.0520 0.0522 0.0520 0.0523Std 0.0065 0.0065 0.0065 0.0065 0.0068

2Mean 0.0340 0.0339† 0.0342 0.0339† 0.0343Std 0.0042 0.0042 0.0043 0.0042 0.0043

3Mean 0.0393 0.0396 0.0383 0.0396 0.0377Std 0.0038 0.0039 0.0042 0.0039 0.0043

4Mean 0.0976 0.0989 0.0865 0.0988 0.0840Std 0.0070 0.0070 0.0124 0.0070 0.0157

50Mean : 0.1444Std : 0.0213

1 Mean 0.0731 0.0728 0.0731 0.0728 0.0731Std 0.0113 0.0111 0.0112 0.0111 0.0114

2Mean 0.0449 0.0448 0.0452 0.0448 0.0451Std 0.0061 0.0062 0.0062 0.0062 0.0064

3Mean 0.0442 0.0444 0.0433† 0.0443 0.0436Std 0.0074 0.0074 0.0072 0.0074 0.0079

4Mean 0.0987 0.1006 0.0870 0.1005 0.0854Std 0.0077 0.0075 0.0154 0.0074 0.0181

100Mean : 0.2704Std : 0.0502

1 Mean 0.1316 0.1313 0.1318 0.1313 0.1317Std 0.0276 0.0277 0.0278 0.0277 0.0278

2Mean 0.0740 0.0735 0.0744 0.0735 0.0746Std 0.0167 0.0168 0.0170 0.0168 0.0175

3Mean 0.0570 0.0568 0.0572 0.0567† 0.0572Std 0.0155 0.0156 0.0155 0.0156 0.0156

4Mean 0.1061 0.1076 0.0990 0.1074 0.0966Std 0.0155 0.0152 0.0180 0.0153 0.0202

�� 39� �2� 2001� 4�

�� 173

Table 9. Average MSE in model validation of OE method for test model #1(Soft shrinkage case with various threshold methods)


5Mean : 0.0193Std : 0.0015

1Mean 0.0183 0.0183 0.0183 0.0183 0.0183

Std 0.0003 0.0003 0.0003 0.0003 0.0003

2Mean 0.0180 0.0179† 0.0179† 0.0179† 0.0180

Std 0.0002 0.0002 0.0003 0.0002 0.0003

3Mean 0.0235 0.0241 0.0224 0.0241 0.0221

Std 0.0010 0.0009 0.0013 0.0008 0.0016

4Mean 0.0507 0.0528 0.0467 0.0528 0.0447

Std 0.0035 0.0019 0.0053 0.0019 0.0065

30Mean : 0.0267Std : 0.0106

1Mean 0.0211 0.0211 0.0211 0.0211 0.0210

Std 0.0028 0.0028 0.0028 0.0028 0.0028

2Mean 0.0195† 0.0195† 0.0195† 0.0195† 0.0196

Std 0.0012 0.0011 0.0012 0.0011 0.0013

3Mean 0.0248 0.0250 0.0238 0.0249 0.0236

Std 0.0023 0.0025 0.0026 0.0024 0.0026

4Mean 0.0524 0.0543 0.0463 0.0540 0.0445

Std 0.0062 0.0061 0.0082 0.0058 0.0094

50Mean : 0.0345Std : 0.0160

1Mean 0.0229 0.0230 0.0230 0.0230 0.0229

Std 0.0053 0.0054 0.0054 0.0054 0.0053

2Mean 0.0209 0.0207† 0.0209 0.0207† 0.0208

Std 0.0022 0.0019 0.0022 0.0019 0.0019

3Mean 0.0248 0.0252 0.0241 0.0252 0.0241

Std 0.0030 0.0030 0.0030 0.0030 0.0030

4Mean 0.0495 0.0533 0.0435 0.0532 0.0437

Std 0.0076 0.0058 0.0092 0.0058 0.0099

Table 8. Average MSE in model validation of FIR method for test model #2(Soft shrinkage case with various threshold methods)


5Mean : 0.0186Std : 0.0015

1Mean 0.0181† 0.0182 0.0182 0.0182 0.0181†

Std 0.0012 0.0011 0.0011 0.0011 0.0012

2Mean 0.0238 0.0240 0.0237 0.0240 0.0237

Std 0.0013 0.0013 0.0014 0.0013 0.0013

3Mean 0.0431 0.0441 0.0417 0.0441 0.0412

Std 0.0014 0.0014 0.0024 0.0014 0.0033

30Mean : 0.0749Std : 0.0087

1Mean 0.0429 0.0429 0.0429 0.0429 0.0429

Std 0.0059 0.0059 0.0059 0.0059 0.0059

2Mean 0.0362 0.0364 0.0360† 0.0364 0.0361

Std 0.0048 0.0048 0.0048 0.0048 0.0048

3Mean 0.0475 0.0485 0.0454 0.0485 0.0446

Std 0.0051 0.0050 0.0057 0.0050 0.0064

50Mean : 0.1214Std : 0.0208

1Mean 0.0629 0.0629 0.0629 0.0629 0.0629

Std 0.0104 0.0104 0.0105 0.0104 0.0104

2Mean 0.0467 0.0468 0.0466† 0.0468 0.0466†

Std 0.0075 0.0075 0.0077 0.0075 0.0076

3Mean 0.0529 0.0537 0.0522 0.0537 0.0520

Std 0.0052 0.0054 0.0052 0.0053 0.0054

100Mean : 0.2373Std : 0.0328

1Mean 0.1085 0.1083 0.1087 0.1083 0.1087

Std 0.0162 0.0163 0.0162 0.0163 0.0162

2Mean 0.0708 0.0705 0.0711 0.0705 0.0713

Std 0.0124 0.0127 0.0121 0.0127 0.0121

3Mean 0.0662 0.0666 0.0651† 0.0666 0.0655

Std 0.0098 0.0098 0.0094 0.0098 0.0101


174 ��

B% 'VU st� £)P ò� validation� �9:, �«mn 2� NSR

100% >� � °± 1�S� � ò� validation ²�% � � �

D� i � �,. QU ²�� S ��U ñ Ã ABL ��

S áE� ü� � �,. ¦ÅS �«mn 2(� ^ �) Ã å AB

� noisy �� ABOEJh� 'V)P lKmn� _)� st

+� NSR�S� de-noising å ABL K�% 7À )P �Ùó �

¨�% ½ � �9zX 'V� cL% MR � Ä9X ¿®Í,. ZQ

$ bDL ¬�� ð st� å ABL �cd uI� £M bD �

Î -#7zX de-noising �K� #� AB� � ò� validation ²

�% �À � Ä9X ¿®Í,. � NSR ¬�ÅW MRA °±�

+À )P å ABL K�% �\U9X �éMR ¹ Ä9X ¿®Í,.

�� F�� |S ,§ shrinkageL stW soft shrinkageL st

ä'U ²�% �8,.

� !_� 'VÍ ®®L mnk� FIR� OE Jh mª NSR �

�¹�Þ de-noising �� 8: de-noising �K� #� AB%

VU st� Z4 �� st� £M �, 9öU mn� ��

8,. », �«mn 2� ABL ��> OE Jh� V)�� +�

NSR�S� de-noising �% �� |ó:� {E)�,.

5. � �

PQ � 2 34 de-noising i��j� V)P bD-# �¢�

~)P £¤)�� bD� -#U AB Z4 �� AB% V)P

�� F��K� �V�" Z ²�% £¤)�,. ABL ��, bD

L ¬� Z�� MRA °±L �� i© �� )P SX ,§ ��

� �� AB% 'V)�� ,TU ¬�L NSR� °±� 'V)�,.

2 34 de-noisingL st shrinkage Jh $ threshold Jh� ¦

ÅS bD-# �¢� ;·L � �9$ ð � ºD� {E)

�� ABL ��, bDL ¬� Z�� MRA °±� ¦§ �W #L

ä'�� {E)�,.

�� F�L st FIR�S� å ABL noisy �� ~� º de-

noising �K� #� AB% VU st� Z4 �� st� £MS

Table 9. Continued.


100Mean : 0.0526Std : 0.0382

1Mean 0.0289 0.0286 0.0287 0.0286 0.0289

Std 0.0097 0.0092 0.0095 0.0092 0.0097

2Mean 0.0239† 0.0239† 0.0240 0.0239† 0.0239†

Std 0.0039 0.0040 0.0039 0.0040 0.0039

3Mean 0.0273 0.0277 0.0269 0.0277 0.0269

Std 0.0055 0.0051 0.0052 0.0051 0.0048

4Mean 0.0523 0.0562 0.0489 0.0561 0.0485

Std 0.0122 0.0109 0.0125 0.0109 0.0137

Table 10. Average MSE in model validation of OE method for test model #2 (Soft shrinkage case with various threshold methods).

NSR (%) Without de-noising Level Minimax Universal SURE Hybrid GCV

5Mean : 0.0077†

Std : 0.0021

1Mean 0.0115 0.0116 0.0116 0.0116 0.0115

Std 0.0005 0.0005 0.0005 0.0005 0.0005

2Mean 0.0200 0.0202 0.0199 0.0202 0.0200

Std 0.0007 0.0007 0.0009 0.0007 0.0007

3Mean 0.0435 0.0461 0.0418 0.0460 0.0414

Std 0.0021 0.0013 0.0032 0.0013 0.0038

30Mean : 0.0089†

Std : 0.0013

1Mean 0.0116 0.0117 0.0116 0.0117 0.0116

Std 0.0014 0.0014 0.0014 0.0014 0.0014

2Mean 0.0201 0.0205 0.0196 0.0205 0.0197

Std 0.0014 0.0015 0.0018 0.0015 0.0018

3Mean 0.0410 0.0445 0.0382 0.0442 0.0373

Std 0.0048 0.0042 0.0062 0.0043 0.0068

50Mean : 0.0115†

Std : 0.0044

1Mean 0.0118 0.0120 0.0120 0.0120 0.0119

Std 0.0022 0.0022 0.0022 0.0022 0.0022

2Mean 0.0205 0.0208 0.0197 0.0208 0.0200

Std 0.0027 0.0025 0.0029 0.0025 0.0028

3Mean 0.0428 0.0461 0.0412 0.0460 0.0414

Std 0.0061 0.0052 0.0068 0.0052 0.0068

100Mean : 0.0158Std : 0.0088

1Mean 0.0133 0.0130† 0.0133 0.0130† 0.0133

Std 0.0031 0.0028 0.0032 0.0028 0.0031

2Mean 0.0208 0.0219 0.0208 0.0219 0.0207

Std 0.0039 0.0044 0.0045 0.0044 0.0045

3Mean 0.0412 0.0453 0.0390 0.0455 0.0391

Std 0.0072 0.0075 0.0076 0.0074 0.0079

�� 39� �2� 2001� 4�

�� 175

nd

00

ve

09

tic

).

-

cal

by

ical

by

P.:

n,

-

)

ll,

e

-

NSR ��¹�Þ �� 9öU mn� �� 8,.

OE�S� �«mn 1�S� de-noising� #� AB% VU st�

�� 9öU mn� �� 89$ �«mn 2 Ã å ABL �

� noisyU st� de-noising �Ùó ò �� ¨�% $¼<�

i � �8,.

� �

� !_� 1996�W �=\ ¤ >!_£ á� L)P ç|?

9:, � �' @A.,.

� ��

cDj,δ : the vector of shrunk coefficients for given threshold δ at level j

e(t) : white noise

k, m : sampling parameter

N : the number of data used model estimation

nj,0 : the number of zero elements at level j

nk : time delay

s : scale parameter

t : time

u(t) : input signal at time t

V loss : loss function

||x||2 : square norm of x

xøy : minimum of x and y

y(t) : output signal at time t

: estimated output at time t

Z : integer

# : the number of data

� ��

λ : threshold value

: estimated standard deviation at level j

τ : translation parameter

��

* : complex conjugate

��

j : the level of MRA

δ : threshold value minimizing GCV function

��

1. Graps, A.: IEEE Computational Science and Engineering, 2(2), 50

(1995).

2. Donoho, D. L.: “Wavelet Shrinkage and W. V. D.: A Ten-minute Tour,”

Technical report 416, Dep. of Statistics, Stanford University(1993).

3. Donoho, D. L. and Johnstone, I. M.: J. Amer. Statist. Assoc., 90(432),

1200(1995).

4. Donoho, D. L. and Johnstone, I. M.: Biometrika, 81(3), 425(1994).

5. Johnston, I. M., Kerkyacharian, G. and Picard, D.: J. Roy. Statist.

Soc., B(57), 301(1995).

6. Johnstone, I. M. and Silverman, B. W.: J. Roy. Statist. Soc., B(59),

319(1997).

7. Bruce, A. and Gao, H. Y.: “WaveShrink: Shrinkage Functions a

Thresholds,” Technical report, StatSci Division, MathSoft, Inc., 17

Westlake Ave. N, Seattle, WA98109.9891(1995).

8. Gao, H. Y.: “Wavelet Shrinkage Denoising Using the Non-Negati

Garrote,” The Wavelet Digest, 6(3), (1997).

9. Gao, H. Y. and Bruce, A. G.: Statistica Sinica, 7(4), 855(1997).

10. Gao, H. Y.: “Threshold Selection in WaveShrink,” Technical report,

StatSci Division, MathSoft, Inc., 1700 Westlake Ave. N, Seattle, WA981

(1997).

11. Gao, H. Y.: “Wavelet Shrinkage Smoothing for Heteroscedas

Data,” The Wavelet Digest, 6(3), (1997).

12. Palavajjhala, S., Motard, R. L. and Joseph, B.: AIChE J., 42(3), 777

(1996).

13. Nikolaou, M. and Vuthandam, P.: AIChE J., 44(1), 141(1998).

14. Carrier, J. F. and Stephanopoulos. G.: AIChE J., 44(2), 341(1998).

15. Flehmig, F., Von Watzdorf, R. and Marquardt, W.: Comp. Chem. Eng.,

22, Suppl., S491(1998).

16. Chui, C. K.: “An Introduction to Wavelets,” Academic Press(1992

17. Akansu, A. N. and Haddad, R. A.: “Multiresolution Signal Decom

position: Transform, Subbands, and Wavelets,” Academic Press(1992).

18. Strang, G. and Nguyen, T.: “Wavelets and Filter Banks,” Wellesley-

Cambridge Press(1996).

19. Nason, G. P.: “Wavelet Regression by Cross-validation,”Techni

Report 447, Dep. of Statistics, Stanford University(1994)�

20. Weyrich, N. and Warhola, G. T.: In S.P. Singh, editor, Approxima-

tion Theory, Wavelets and Applications, NATO ASI Series C(454),

523(1995)�

21. Jansen, M., Malfait, M. and Bultheel, A.: Signal Processing, 56(1),

33(1997).

22. Jansen, M., Uytterhoeven, G. and Bultheel, A.: “Image De-noising

Integer Wavelet Transforms and Generalized Cross Validation,”Techn

Report TW264, Dep. of Computer Science, K. U. Leuven(1997)�

23. Jansen, M. and Bultheel, A.: “Multiple Wavelet Threshold Estimation

Generalized Cross Validation for Data with Correlated Noise,” Technical

Report TW250, Dep. of Computer Science, K. U. Leuven(1997)�

24. Press, W. H., Teukolsky, S. A., Vetterling, W. T. and Flannery, B.

“Numerical Recipes in C, the Art of Scientific Computing,”2nd ed

Cambridge: Cambridge University Press(1992)�

25. Zhu, Y. C. and Backx, T.: “Identification of Multivariable Industrial Pro

cesses for Simulation, Diagnosis and Control,”Springer-Verlag(1993�

26. Ljung, L.: “System Identification:Theory for the User,”Prentice-Ha

Englewood Cliffs, N. J(1987)�

27. Kang, S. J.: “System Identification with Modified Finite Impuls

Response Models: Comparison with Other Identification Methods,” Ph.

D. Dissertation, University of Missouri-Columbia, MO, USA(1994)�

28. Ljung, L.: “System Identification Toolbox User’s Guide,” The Math

Works, Inc.(1997)�

y t( )

σj


Documents

A Study on System Identification Using Wavelet … · A Study on System Identification Using Wavelet Transformation Wook Jin Baek, Jeong Woo Han, Sung Ju Kang