SIPE - Lecture 1. Introduction

System Identification &

Parameter Estimation(SIPE)

Wb 2301Lecture 1: introduction

Alfred Schouten

Lecture 1 February 6. 2007

People

Lectures•Erwin de Vlugt ([email protected])•Alfred Schouten ([email protected])•Frans van der Helm ([email protected])

Assignments and ‘SIPE helpdesk’(on appointment, by e-mail):•Jasper Schuurmans ([email protected])•Winfred Mugge ([email protected])

mailto:[email protected]





http://cms2.tudelft.nl/live/pagina.jsp?id=c07a7db1-3a00-4195-968f-86e86e3919e9&lang=en

http://cms2.tudelft.nl/live/pagina.jsp?id=41b5cf7d-583f-4602-a9a6-f8c107429c2a&lang=en

http://cms2.tudelft.nl/live/pagina.jsp?id=9ad1b082-64b3-4be6-abe3-421fdd79eb01&lang=en

http://cms2.tudelft.nl/live/pagina.jsp?id=16c21c57-5b50-40cd-836d-f234f8eed30a&lang=en


Course info• Tuesday 1th+2nd hour (8.45 – 10.30)• February 6 – June 5, 2007 (15 lectures)• Room E, Mechanical Engineering• Blackboard:

• Announcements• Lecture Notes• Assignments• Chapters Reader• Demonstration programs (Matlab)• Matlab History

• 7 ECTS => 7*28 = 196 hours, for 15 lectures • Work-load => approx. 13 hours/week !!!!!!


Assignments

• Most lectures are closed with an assignments• Assignments are due for the next lecture (via Blackboard)

• Question hour• Questions on assignments, Matlab programming, and the course in general• Friday, 10.45 hours• Room B, Mechanical Engineering• Jasper Schuurmans and Winfred Mugge

⇒ Question hour is the only option for questions!If you have ‘long’ questions: please send email with questions in advance


Goal of the course

• How to approach a priori unknown, dynamical systems?• Non-parametric representation in frequency domain

• Linearization• Understanding most important dynamic characteristics

• Model structure + parameters• Estimation of model parameters• Validation of the model

• Students should acquire:• Intuition and understanding: Lectures• Theoretical background: Reader• Practical skills: Assignments


System

EMG

position

recordedsignals

Sinkjaer, Andersen & Larsen (1996)Joint moments

Sensorsignals

Muscleforces

Internalsignals


Signals

•Thick lines: normal gait

•Thin lines: perturbed gait

position

Moments

EMG Tibialis Anterior(front side ankle)

EMG Soleus(back side ankle)


Model

Anklejoint

position force

EMG

Parameters:mass

viscoelasticitymuscle propertiesfeedback via CNS

(muscle force)


anklejoint

position

moment

EMG

manipulator(servo)

perturbedposition

muscle ++

+ -

Model

interpretationof task

CNS


Intuition:What is a model?

• Models: Physical, mental, statistical, psychological, etc• Mathematical models:

• Goals: fundamental knowledge, control, simulation, etc.• Quantitative hypothesis:

• Theory ⇒ Hypothesis ⇒ Model ⇒ Validation

• Input and output signals• Quantitative relation between input and output• Model structure, model parameters

• Models in this course:• Input and output signals are time-signals• Dynamic relation


System identification and parameter estimation

Unknownsystem

Inputsignal

Outputsignal

ModelPredicted

output

+

-

Parameter estimation

Unknownsystem

Inputsignal

Outputsignal

System identification


Model validation

Unknownsystem

Inputsignal

Outputsignal

ModelPredicted

output

+

-Validation

N.B. Do not use the same input - output combinationfor parameter estimation and for validation :

!! Fitting ≠ Validation !!


System identification &Parameter estimation

• Zadeh (1962):Identification is the determination on the basis of input and output, of a system within a class of systems, to which the system under test is equivalent.

• Parameter estimation is the experimental determination of values of parameters that govern the dynamic and/or non-linear behaviour, assuming that the structure of the model is known.


Wb 2301: System identification:

• Practical approach!!• Intuitive knowledge

• Lectures

• Practical examples in Matlab• demo programs and exercises• simulations from class room, ‘history’

• Home work: assignments• Assistance of PhD teaching assistants (question hour)

• Jasper Schuurmans, Winfred Mugge


Wb 2301: System identification:

• Mathematical background (available on Blackboard)• Papers and book chapters (Pintelon & Schoukens: System identification)• Additional material (in Dutch):

• Reader Wb 2307: Signal theory (Dankelman & Van Lunteren)• Chapters 1 and 2 of Wb 2301: System Identification (Stochastic theory)

• English version of reader is in progress

• Class assignments, each lecture, deadline next lecture!• PhD assistants: Jasper Schuurmans, Winfred Mugge

• Final assignment: Analysis of research data• 1) intrinsic and reflexive feedback mechanisms• 2) control mechanism of coronary circulation• 3) manual control task: identification of human controller• written report

• Written exam


Grading

• Final grade• 25% average of class assignments• 25% final assignment• 50% written examination


Related courses

Previous• Wb 2207: Systeem- en Regeltechniek 2 (SR 2)• Wb 2310: Systeem- en Regeltechniek 3 (SR 3)

Related• SC4110: System identification (Bombois & van den Hof)

• (Linear) control theory

• Wb 2301: System identification & parameter estimation:• Research & design• including parameter estimation


Goals• Analysis unknown (dynamic) systems:

• Time domain• Frequency domain

• Modeling of systems• Parameter estimation

• Optimization methods• Validation

• Non-linear modeling• non-linear dynamic models• expert systems• fuzzy models• neural networks


• Science: Battle against noise• Repeat the experiment

• Noise cancels out • Improve Signal-to-Noise Ratio

• Reduce Noise• Better Signals: Concentrate power at specified frequencies

• Estimate the noise• Use noise-filters and ‘subtract’ the noise

? y(t)

u(t)

n(t)

v(t)

Analysis problem


Analysis problem

• Given: u(t) and y(t) are measurable input and output signals• Requested: description of system

Problem: Noise n(t) is unknown

• Solutions:• Filtering: If there is no overlap in frequencies of v(t) and n(t)• Averaging: Repetitive measurements

case 1 case 2u(t) deterministic stochasticn(t) stochastic stochasticy(t) stochastic stochastic

? y(t)u(t)

n(t)

v(t)


Filtering

• Assumption: No overlap in frequencies of v(t) and n(t)• v(t): Low frequencies, signal content• n(t): High frequencies, noise

• Low-pass filter with cut-off frequency to discriminate v(t) and n(t)• Which cut-off frequency?• What if the assumption is not correct?

? y(t)u(t)

n(t)

v(t)


Averaging

• Assumption: n(t) is stochastic and has zero mean• u(t) deterministic (step, pulse, sinusoid):

repeating u(t)Not periodic: Average response on step or pulsePeriodic: (sum of) sinusoids

• u(t) stochastic:u(t) is not repeatable:

more advanced mathematical tools needed

? y(t)u(t)

n(t)

v(t)


Stochastic theory

• Averaging over non-repeatable signals is impossible• Stochastic theory:

• not the individual realizations• but statistical properties• like probability functions

• Relation between stochastic signals x(t ;ζ) and y(t;ζ)• Probability function:

• Probability density function:

• Function of τ: Time-shift between signals is result of dynamicrelation !!

• Dynamic relation must be described by differential equation

{ }F x y x t x y t yxy( , ; ) Pr ( ; ) ( ; )τ ζ τ ζ= ≤ + ≤ ∩

{ }dyytyydxxtxxdxdyyxf yx +≤+<+≤<= );( );(Pr);,( ζτζτ ∩


Stochastic theory

Example: noise, 0-50 Hz

Input signal

Output signal


• Many realizations of input and output signals are needed for a proper estimate of the probability density functions

• In reality: One, sufficiently long, realization is thought to be representative of many realizations:

Ergodicity

Ergodicity


Stochastic signals

Signal properties:•probability•mean, μ•standard deviation, σ

Right: normal or gaussiondistribution.

fxy

2

21

21)(

⎟⎠⎞

⎜⎝⎛ −

−= σ

μ

πσ

x

x exf


2D Probability density function

{ }dyytyydxxtxxdxdyyxf yx +≤+<+≤<= );( );(Pr);,( ζτζτ ∩


• Probability for certain values of y(t) given certain values of x(t)

• Co-variance of y(t) with x(t): y(t) changes if x(t) changes• No co-variance between y(t) and x(t): probability density

function is circular• Covariance between y(t) and x(t): probability density function

is ellipsoidal

• No co-variance between y(t) and x(t):• no relation exist• transfer function is zero !

2D Probability density function


Cross-product function Rxy(τ)

• Cross-product function Rxy(τ)

• Auto-product function Rxx(τ)

∫∫∞

∞−

∞

∞−

=+= dxdyyxxyftytxER xyxy );,()}()({)( τττ

∫∫∞

∞−

∞

∞−

=+= 212121 );,()}()({)( dxdxxxfxxtxtxER xxxx τττ


Cross-covariance function Cxy(τ)

• Cross-covariance function Cxy(τ)

• Auto-covariance function Cxx(τ)

∫∫∞

∞−

∞

∞−

−−=−+−= dxdyyxfyxtytxEC xyyxyxxy );,())((})()()({()( τμμμτμτ

∫∫∞

∞−

∞

∞−

−−=−+−= 212121 );,())(()})()()({()( dxdxxxfxxtxtxEC xxxxxxxx τμμμτμτ


Identification: time-domain vs. frequency-domain

H(t)u(t)

n(t)

y(t)

•y(t) = h(t)*u(t) + n(t) = ∫ (h(t’)*(u(t-t’)*dt’ + n(t)•Unknown system: Impulse response of h(t’)•Mostly: Direct model parametrization

Y(ω) = H(ω)*U(ω) + N(ω)Unknown system: Transfer function H(ω) for number of frequencies


u(t), y(t)

‘non-parametric’model

parametricmodel

U(ω),Y(ω)

non-parametricmodel

parametricmodel

Identification: time-domain vs. frequency-domain

ARXARMAEtc.

FrequencyResponseFunction(FRF)


Time-domain vs. Frequency-domain

x(t), y(t)

Rxy(τ)

Cxy(τ)

Kxy(τ)

X(ω), Y(ω)

Sxy(ω)

Γxy(ω)

input, output

Cross-productfunction

Cross-covariancefunction

Cross-correlationfunction

input, output

cross-spectraldensity

coherence

Fourier TransformationTime Domain Frequency Domain


FRF vs. time-domain models

• Non-parametric in frequency-domain, parametric in time-domain (black-box model with not interpretable parameters).


Input and output signal

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5-1.5

-1

-0.5

0

0.5

1

1.5

• input: u(t) = sin(ωt)• output: y(t) = A*sin (ωt + ϕ)• amplitude A and phase ϕ

A

ϕ


Fourier transformation:time-domain vs. frequency-domain

• y(t) is an arbitrary signal••• symmetric part: cos(ωt)• anti-symmetric part: sin(ωt)

∫ −= dtetyY tjωω *)()()sin(*)cos()()( tjteimeree tjtjtj ωωωωω −=+= −−−

-5 -4 -3 -2 -1 0 1 2 3 4 5-2

-1.5

-1

-0.5

0

0.5

1

1.5

2


Fourier-transformation

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5-1.5

-1

-0.5

0

0.5

1

1.5

∫ −= dtetyY tjωω *)()(( ) ( )HzHzty 5sin5.0sin)( +=

Y(0.5Hz): 5 Hz signal will be averaged out


Fourier coefficients

∫ −= dtetyY tjωω *)()(

)sin(*)cos()()( tjteimeree tjtjtj ωωωωω −=+= −−−

)(.)()( ωωω bjaY +=

Fourier coefficients: a(ω) + j.b(ω)


Inverse Fourier Transformation

)cos(.)sin(*)(.)cos().()(

111

1111

ϕωωωωω

+=+=

tAtbjtaty

))()(arctan(

)()()(

1

11

21

2111

ωωϕ

ωωω

ab

baA−

=

+=


time-domain vs. frequency-domain

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5-1.5

-1

-0.5

0

0.5

1

1.5

-0.5 0 0.5 1 1.50

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

y(t) = sin(0.5t) Y(ω) = δ(ω-0.5)


Time domain:impulse response

y(t) = h(t)*u(t)


Frequency domain:Bode-diagram

second order system: mass-spring-damper

M = 1 Kg

B = 1 Ns/m

K= 1 N/m

Y(ω) = H(ω)*U(ω)


Assignment 1:Write your own Fourier Transform

•Continuous domain

•Discrete domain

∫ −= dtetyY tjωω *)()(

X t x en kj kn N

k

N

= −

=

−

∑Δ . /2

0

1π

Education

SIPE - Lecture 1. Introduction