27/10/2009 IVR Herrmann 1 IVR: Control Theory OVERVIEW Given desired behaviour, determine control signals Inverse models: – Inverting the forward model

27/10/2009 IVR Herrmann 127/10/2009 IVR Herrmann

IVR: Control Theory

OVERVIEW

• Given desired behaviour, determine control signals

• Inverse models:– Inverting the forward model for simple linear dynamic system– Problems for more complex systems

• Open loop control: advantages and disadvantages• Feed forward control to deal with disturbances


Control 2Keypoints:• Given desired behaviour,

determine control signals• Inverse models:

– Inverting the forward model for simple linear dynamic system

– Problems for more complex systems• Open loop control: advantages and

disadvantages• Feed forward control to deal with disturbances


The control problem

• For given motor commands, what is the outcome? → Forward model

• For a desired outcome, what are the motor commands? → Inverse model

• From observing the outcome, how should we adjust the motor commands to achieve a goal?

→ Feedback control

Motor command

Robot in environment

OutcomeGoal Action


x(t) is an unknown functionf is a given analyticalexpression

Differential Equations

)()( txatx

txfdt

tdx

00 xtx

Using known relations between quantities and their rate of change in order to predict the behaviour of these quantities

Equation:

Initialcondition:

Examples:

)()( txctx 0)(sin)( tg

t

)(txc

v

v

x

Pendulum equation simplified pendulum equation

and an initial condition (see last lect.)

Matrix form of thesimplified equationEach of these: two initial conditions, e.g. x(0)=x0 and v(0)=v0


Differential equation of a two-link planar arm

r=2

Equations are classical physics.Here copied from the paper:


Solving differential equations numerically*

1. Given and

2. Choose a small step size Δt

3. Start at t = 0 and x(0) = x0

4. Iterate x(t+Δt) = x(t) + Δt dx/dt until t = T

txfdt

tdx 00 xx

*This is the most simple way of integrating a differential equation. Even for small step sizes considerable errors may accumulate.

x(t)x

tΔt

old value

approximatenew value

tangent to x(t)

t t+Δt


Example from last lecture: Electric Motor

Simple dynamic example – We have a process model:

Solve to get forward model s(0)=s0:

• Derivation of this andmore general cases using e.g. Laplace transformation

Battery voltage

VB

Vehicle speed

v?

VB

IR

e

dt

ds

k

MRskVB

21

t

MR

kk

k

Vs B 21

1

exp1


Solving differential equations numerically

1. Given and

Rewrite:

2. Choose a small step size Δt

3. Start at t = 0 and s(0)= s0

4. Iterate s(t+Δt) = s(t) + Δt ds/dt until t = T

00 ss dt

dsR

k

MskVB

21

sRM

kkV

RM

k

dt

dsB

212

Example from last lecture:


The control problem• Forward: Given the control signals, can we predict

the motion of the robot?

Motor command


Forward model

Predicted output

• Inverse: Given the desired motion of the robot Can we determine the right control signals?

• Control theory aims at unique or easy exact solutions• Difficult in the presence of non-linearities or noise

Actual output?

Desired output


Open loop control

Examples:• To execute a memorised trajectory, produce

appropriate sequence of motor torques • To obtain a goal, make a plan and execute it

– means-ends reasoning could be seen as inverting a forward model of cause-effect

• ‘Ballistic’ movements such as saccades

Motor command


Inverse model


Open loop control

• Potentially cheap and simple to implement e.g. if solution is already known.

• Fast, e.g. useful if feedback would come too late.

• Benefits from calibration e.g. tune parameters of approximate model.

• If model unknown, may be able to use statistical learning methods to find a good fit e.g. neural network.


Open loop control

• Neglects possibility of disturbances, which might affect the outcome.

• For example:– Change in temperature may change the friction in

all the robot joints.– Or unexpected obstacle may interrupt trajectory

Motor command


Inverse model

Disturbances


Feed-forward control

• One solution is to measure the (potential) disturbance and use this to adjust the control signals.

• For example– Thermometer signal alters friction parameter.– Obstacle detection produces alternative trajectory.

Motor command


Inverse model

DisturbancesMeasurement


Feed-forward control

• Can sometimes be effective and efficient.

• Requires anticipation, not just of the robot process characteristics, but of possible changes in the world.

• Does not provide or use knowledge of actual output (for this need to use feedback control)


Open loop control:

Motor command


Inverse model

Feed-forward control:

Motor command


Inverse model


Motor command


Control Law


Feed-back control:

Disturbances ?


Combining control methods

• In practice most robot systems require a combination of control methods– Robot arm using servos on each joint to obtain

angles required by geometric inverse model – Forward model predicts feedback, so can use in

fast control loop to avoid problems of delay– Feed-forward measurements of disturbances can

be used to adjust feedback control parameters– Training of an open-loop system is essentially a

feedback process


Feedback control

• Example: Greek water clock

• Requires constant flow to signal time

• Obtain by controlling water height

• Float controls inlet valve


Feedback control

• Example: Watt governor

Steam engine

Drive shaft

Steam valve


Compare: room heating• Open loop: for desired

temperature, switch heater on, and after pre-set time, switch off.

• Feed forward: use thermometer outside room to compensate timer for external temperature.

• Feedback: use thermometer inside room to switch off when desired temperature reached.NO PREDICTION REQUIRED!


Motor command


Control Law


Control laws in feed-back control

• On-off: switch system if desired = actual• Servo: control signal proportional to difference

between desired and actual, e.g. spring:

Desired

output

Actual

output


Servo control

Simple dynamic example:

Control law:

So now have new process:

Battery voltage

VB

Vehicle speed

s?

dt

ds

k

MRskVB

21

ssgoal)( goal ssKVB

dt

ds

k

MRskKKs

dt

ds

k

MRskssK

21goal

21goal

)(

)(

212

1 )(7.0

kkK

MR

gain=K

1

goal

kKKs

s

With steady state:

And half-life:


dt

ds

k

MRskKKs

dt

ds

k

MRskssK

21goal

21goal

)(

)(

? ?

Say we have sgoal =10.

What does K need to be for s∞ = 10?

If K=5, and s∞ = 8, and the system takes 14 seconds to reach s=6 starting from s=0, what is the process equation?



Motor command


Control Law


Complex control law

• May involve multiple inputs and outputs

• E.g. ‘homeostatic’ control of human body temperature

– Multiple temperature sensors linked to hypothalamus in brain

– Depending on difference from desired temperature, regulates sweating, vasodilation, piloerection, shivering, metabolic rate, behaviour…

– Result is reliable core temperature control

Desired

output

Actual

output


Negative vs. Positive Feedback• Examples so far have been negative feedback:

control law involves subtracting the measured output, acting to decrease difference.

• Positive feedback results in amplification:– e.g. microphone picking up its own output.- e.g. ‘runaway’ selection in evolution.

• Generally taken to be undesirable in robot control, but sometimes can be effective – e.g. in exploratory control, self-excitation– model of stick insect walking (Cruse et al 1995)– (requires negative feedback from the environment)


• 6-legged robot has 18 degrees of mobility.

• Difficult to derive co-ordinated control laws

• But have linked system: movement of one joint causes appropriate change to all other joints.

• Can use signal in a feed-forward loop to control 12 joints (remaining 6 use feedback to resist gravity).


Advantages

• Major advantage is that (at least in theory) feedback control does not require a model of the process.

– E.g. thermostats work without any knowledge of the dynamics of room heating (except for time scales and gains)

• Thus controller can be very simple and direct

– E.g. hardware governor does not even need to do explicit measurement, representation and comparison

• Can (potentially) produce robust output in the face of unknown and unpredictable disturbances

– E.g. homeostatic body temperature control keeps working in unnatural environments


Issues• Requires sensors capable of measuring

output– Not required by open-loop or feed-forward

• Low gain is slow, high gain is unstable• Delays in feedback loop will produce

oscillations (or worse)• In practice, need to understand process to

obtain good control:– E.g. James Clerk Maxwell’s analysis of dynamics

of the Watt governor• …more next lecture


Cybernetics

• Feedback control advanced in post WW2 years (particularly by Wiener) as general explanatory tool for all ‘systems’: mechanical, biological, social…

• E.g. William Powers –

“Behaviour: the control of perception”

• Jay Wright Forrester –

“World dynamics” – modelling the global economic-ecological network


Further reading:Most standard robotics textbooks (e.g. McKerrow) discuss forward and inverse models in great detail.

For the research on saccades see:

Harris, C.M. & Wolpert, D.M. (1998) Signal dependent noise determines motor planning. Nature, 394, pp. 780-184.

For an interesting discussion of forward and inverse models in relation to motor control in humans, see:

Wolpert, D.M. & Ghahramani, Z. (2000) “Computational principles of movement neuroscience” Nature Neuroscience 3, pp. 1212-1217.

Wolpert D.M., Ghahramani Z, & Flanagan J. R. (2001) Perspectives and problems in motor learning. Trends in Cognitive Sciences 5:11, pp 487-494.


Further reading on feedback control:Most standard robotics textbooks (e.g. McKerrow) discuss feedback control in great depth.

For the research on stick insect walking see:

Cruse, H., Bartling, Ch., Kindermann, T. (1995) High-pass filtered positive feedback for decentralized control of cooperation. In: F. Moran, A. Moreno, J. J. Merelo, P. Chacon (eds. ), Advances in Artificial Life, pp. 668-678. Springer 1995

Classic works on cybernetics:

Wiener, Norbert: (1948) Cybernetics. or Control and Communication in the Animal and Machine, MIT Press, Cambridge

Powers, WT: (1973) Behavior, the Control of Perception, Aldine, Chicago

Forrester, JW (1971) World Dynamics, Wright and Allen, Cambridge

Documents

27/10/2009 IVR Herrmann 1 IVR: Control Theory OVERVIEW Given desired behaviour, determine control signals Inverse models: – Inverting the forward model