Better motion control using accelerometer/gyroscope sensor fusion

Gabor Pallergaborpaller@gmail.com

Sfonge Ltd.http://www.sfonge.com

Where were we?

● Droidcon 2011, London: Motion recognition on Android devices● http://mylifewithandroid.blogspot.com/2011/10/my-

presentation-about-motion.html

● Processing only the accelerometer for motion recognition

Acceleration

Acceleration caused bythe change of direction

v1

v2dV

Acceleration caused by the change of velocity

v1

v2

dV

a=ΔVΔ t

Extract motion information from accelerometer data

● Accelerometer data is a vector, having 3 axes (x,y,z)● This vector has the following components:

● Gravity acceleration– Pointing toward the center of the Earth

– Value of about 10 m/s2

– That's what we measure when the accelerometer is used to calculate tilt

● Any other acceleration the device is subject to– Added to the gravity acceleration

– “Disturbs” tilt measurement in gaming (swift movements cause acceleration) – hence the reason for gyroscopes

– Can be used for movement detection

Measured acceleration

Absolute value

● x, y, z: acceleration vector components● g – value of the gravity acceleration (can be

approximated as 10)

a=√x2+ y2+ z2−g

Snap – one wayMovement starts: accelerating

Movement ends: decelerating

Droidcon 2011 flashback

● Conclusions:● Power consumption is a problem● Some neat functionality can be implemented by doing

pattern recognition on the acceleration vector's absolute value

● In general case the gravity and motion acceleration components cannot be separated

● You can try to use an additional sensor like the gyro to help the separation

Gyroscope

● Very new phenomenon as gyroscopes suitable for consumer electronic devices appeared very recently

● First appearance: Wii Motion Plus accessory, 2009 June

● First Android smart phone: Nexus S (end of 2010)

● Pros:

● Not sensitive to gravity● Cons:

● Currently supported only by high-end Android phones● Drift problems (more about that later)

Compass

● Measures the device orientation wrt. the magnetic vector of the Earth

● This vector points toward the magnetic center of the Earth

– It has a component that points to the magnetic North pole – that's what we use for orientation

– Beware of the z component! (also called magnetic inclination). If the device is not held horizontally, the downward vector element influences the measurement

● Pros:

● Can be used to deduce gravity, not sensitive to motion acceleration

● Widely available in Android devices

● Cons:

● Requires calibration

● Sensitive to metal objects, magnetic fields (e.g. electric motors)

This time it is gyroscope only

Gyroscope

Gyroscope measurement data

● Measures rotation around 3 axes● More exactly: measures rotation speed (angular

velocity) around the axes

v x=Δφ

Δ t

Getting the rotation angle

● Get the angle difference

● Get the absolute angle

Δφ=v x Δ t

φ '=φ+Δφ

Drift

Noise

Gyro as support sensor

● Because of accumulating error, gyro alone can be rarely used

● But● The accelerometer has no accumulated error but

has the gravity component problem● The gyro has accumulated error but is not sensitive

to gravity

● Sensor fusion: the use of multiple sensors so that they compensate each other's weaknesses

Accelerometer-gyro fusion

● The easy way● Use the virtual sensors that calculate gravity and

linear acceleration from multiple sensors

● The hard way● Process raw accelerometer and gyroscope data to

yield the motion information you need

Virtual sensors

Gravity and motion accelerationdeduced from the accelerometerand the gyroscope

Roll/pitch/yaw from the compass

Drift-compensated gyroscope

Drift-compensated gyroscope

The hard way

● Why would you go the hard way?● Sensor fusion co-processing provided by the phone

is not precise enough or can have undesirable properties (like auto-calibration in Nexus S)

● Virtual sensors are not available (is there any such case with gyro-equipped phone?)

● You would like to understand how it works and what to expect from built-in sensor fusion

● Just for the fun of it :-)

What we want

● Remember: accelerometer measures the sum of gravity and motion acceleration

● Kills two use cases:● If you need device tilt, the motion acceleration

component corrupts the measurement● If you want motion acceleration, it is impossible to

subtract the gravity acceleration in a general case

● Separate gravity and motion acceleration with the help of the gyroscope

Idea

Idea in words

● Pick a reliable gravity vector measurement (make sure that there's no motion then)

● If you detect motion (more about later), rotate the previous gravity vector using the gyroscope data and use it as gravity vector estimation

● Subtract this gravity vector estimation from the measured acceleration – this yields the motion acceleration

Updating the gravity vector estimation

● The gravity vector estimation has to be updated time to time as rotation angle errors accumulate

● If we detect an acceleration measurement where there is no motion acceleration, we can take it as new reliable gravity vector estimation

● Remember slide #7: if the absolute value of the accelerometer output is close to the Earth's gravity, we can assume that there's no motion → the gravity vector estimation can be updated with the current accelerometer output

Implementation

● Example program: http://www.sfonge.com/forum/topic/example-application-accelerometergyroscope-processing-android

Now what?

3D linear acceleration signal of a well-known motion

Recognizing motion

● 3D linear acceleration signals are not so intuitive

● Motion recognition:● Record acceleration pattern of reference motion

and compare with these references● Convert from acceleration domain to something

more intuitive like velocity– Accelerometer/gyroscope bias will become linearly

growing drift after you integrate the acceleration signal!

Walking with swinging hand

Walking with steady hand

Cutting corners

Conclusions

● Each sensor has strengths and weaknesses● Combine them and they compensate each

other● Some sensor fusion is already built-in● If not → don't worry, come up with your own, it's

fun!● Motion recognition based on 3D linear

acceleration signal is much more exact than doing the same from 1D signal

Questions?