Upload
others
View
6
Download
0
Embed Size (px)
Citation preview
WiGest: A Ubiquitous WiFi-based Gesture Recognition System
Heba Abdelnasser†, Moustafa Youssef‡, and Khaled A. Harras*Alexandria Univ.†, Egypt-Japan Univ. of Sc. and Tech.‡, Carnegie Mellon Univ.*
Motivation
● Rendering touch input – user is wearing gloves.
● Wet/dirty hands.
Motivation
● Rendering touch input – user is wearing gloves.
● Wet/dirty hands.
● Special sensors● E.g. Samsung Galaxy S5
WiGest Approach
● Free the user from specialized sensors● Using the ubiquitous WiFi● Works with any phone
● Leveraging natural human movements to control devices
Basic Idea
• Leveraging the impact of hand motion on the received WiFion the phone to control WiFi-enabled devices.
WiGest Example
Challenges
• WiFi is a noisy signal• Avoiding false positives – human
interference• Handling the variations of
gestures and their attributes
• Energy consumption• No training
– Handle different humans– Same human a different times
-36
-33
-30
-27
-24
1 2 3 4 5 6 7 8
RS
SI
dB
m
Time in Seconds
-41
-40
-39
-38
-37
-36
-35
-34
0 1 2 3 4 5 6 7 8 9 10
RS
SI
dB
m
Time in Seconds
Far
Near
Agenda
Introduction• WiGest System• Results• Conlusions
Agenda
Introduction• WiGest System• Results• Conlusions
WiGest System
WiGest System
Primitive Extraction
Primitive Extraction
1. Denoising– Using Wavelet denoising
2. Edges detection,– Extract different primitives.
3. Parameters extraction,– Magnitude: near/far.– Speed: fast/slow.
-36
-33
-30
-27
-24
1 2 3 4 5 6 7 8
RS
SI
dB
m
Time in Seconds
Input Output
Noise Reduction
Time in sec
Scale
2.5 3 3.5 4 4.5 5 5.5 6 6.5 7
5
10
15
20
25
30
35
-36
-33
-30
-27
-24
1 2 3 4 5 6 7 8
RS
SI
dB
m
Time in Seconds
Noise Reduction
• Removing noise using wavelet denoising– It can preserve signal details while filtering out the noise
and variations in the signal– Consists of three stages:
• Decomposition the signal to approximation and detail coefficients.
• Thresholding detail coefficients• Reconstruct the denoised signal by adding approximation and
detail after thresholding
Noise Reduction
Raw signal After denoising
-33
-30
-27
-24
1 2 3 4 5 6 7 8
RS
SI
dB
m
Time in Seconds
-36
-33
-30
-27
-24
1 2 3 4 5 6 7 8
RS
SI
dB
m
Time in Seconds
Edges Extraction
• Detecting gesture edges using wavelet analysis– In detail (high-pass filter) coefficients
• Falling edge causes a local maxima• Rising edge causes a local minima
Edges Extraction
Denoised signal
DWT detail coefficient
-33
-30
-27
-24
1 2 3 4 5 6 7 8
RS
SI
dB
m
Time in Seconds
-6
-4
-2
0
2
4
6
1 2 3 4 5 6 7 8
Time in Seconds
Edges Extraction
Denoised signal
DWT detail coefficient
-33
-30
-27
-24
1 2 3 4 5 6 7 8
RS
SI
dB
m
Time in Seconds
Edges Extraction
Denoised signal
DWT detail coefficient
Edges Extraction
Denoised signal
DWT detail coefficientExtracted primitives
Parameters Extraction
Magnitude
-41
-40
-39
-38
-37
-36
-35
-34
0 1 2 3 4 5 6 7 8 9 10
RS
SI
dB
m
Time in Seconds
Far
Near
-39
-38
-37
-36
-35
0 2 4 6 8 10 12 14 16 18
RS
SI
dB
m
Time in Seconds
Slow Fast
Speed
Parameters Extraction
Magnitude
-41
-40
-39
-38
-37
-36
-35
-34
0 1 2 3 4 5 6 7 8 9 10
RS
SI
dB
m
Time in Seconds
Far
Near
-39
-38
-37
-36
-35
0 2 4 6 8 10 12 14 16 18
RS
SI
dB
m
Time in Seconds
Slow Fast
Speed
System Logic Flow
Segmentation
• Silence period determines the start and end of a gesture
• Preamble (unique signature), to start the communication channel
Frequency =
3 /2 Hz,
Count = 2
Frequency =
5 /2 Hz,
Count = 3
-39
-36
-33
-30
0 1 2 3 4 5 6 7 8
RSSI
dBm
Time in SecondsSilence period Gesture delimiter
Pattern Encoding
• The extracted gesture primitives are converted to string sequence:
– Rising edge : +– Falling edge : -– Pause : 0
• Gesture matching– String matching to identify the gesture
Extracted gesture pattern : - + - +Gesture : Infinity
Gesture Identification
Extracted gesture pattern : - + - +Gesture : Infinity
Preamble • Energy efficiency• Human interference
and noise
Pattern encoding• Efficient matching• Error tolerance
Gestures ParametersFrequency =
3 /2 Hz,
Count = 2
Frequency =
5 /2 Hz,
Count = 3
-39
-36
-33
-30
0 1 2 3 4 5 6 7 8
RSSI
dBm
Time in Seconds
Count• Number of gesture
repetitions
• E.g. double click and single click
Frequency• Number of gesture
repetitions per unit time
• E.g. speed of basketball
dribble
System Logic Flow
Action Mapping
• At the end, the extracted gesture is mapped to an
application action .
Gesture Action
Application
Agenda
IntroductionWiGest System• Results• Conlusions
Implementation and Evaluation
• Test environments• Typical apartment• Engineering building at our campus
• More than 1000 experiment• Off-the-shelf WiFi-equipped devices• Hardware:
• Cisco Linksys
• Android-based cell phone.
• HP laptop
39 ft
34 ft
115 ft
49 ft
Experiments Performed
• Distance effect for different scenarios• Orientation• Number of APs• Human interference
• Whole home case study
• More results in the paper
Distance Impact
0.5
0.6
0.7
0.8
0.9
1
3 7 10 13 16 20 23 26 39 52 62-70
-60
-50
-40
-30
Acc
ura
cy
Av
erag
e R
SS
I
Distance in feet
Accuracy Average RSSI
0.5
0.6
0.7
0.8
0.9
1
3 7 10 15 20 26 30 39 56-70
-60
-50
-40
-30
Acc
ura
cy
Av
erag
e R
SS
I
Distance in feet
Accuracy Average RSSI
0.5
0.6
0.7
0.8
0.9
1
7 10 13 16 20 30 36-70
-60
-50
-40
-30
Acc
ura
cy
Av
erag
e R
SS
I
Distance in feet
Accuracy Average RSSI
• 87% accuracy for distances up to 26ft
• Using a single AP
Orientation Impact
• Overall accuracy of 90.5%• The highest accuracy is in West – body not blocking the
signal
0
0.2
0.4
0.6
0.8
1
North South East West Overall
Acc
ura
cy
Number of APs Impact
0
0.2
0.4
0.6
0.8
1
1 APs 3 APs 5 APs 7 APs
Acc
ura
cy
Number of APs used in voting
• Accuracy 96% using 3 APs
• Reaches 100% with seven APs
Human interference
• Accuracy 89% in the presence of four
interfering humans
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
Det
ecti
on
Acc
ura
cy
Number of interfering humans
Whole Home Media-player Case Study
• Accuracy of 96% using 2 APs
0 .9
0 .96
0 .93
0 .97
0 .99
1
0 .98 0.02
0.01
0.025
0.03
0.04
0.090.005 0.005
0.005
0.005
0.03 0.005
0
0 0 00
0 0000
0 0000
00 0000
0000
000
00
0
0
0
00
00
nu
ll
Ac
tu
al
Ac
tio
n P
erfo
rm
ed
Act ion Classif ied
39 ft
34 ft
Agenda
IntroductionWiGest SystemResults• Conlusions
Conclusion
• WiGest is a gesture recognition system• Robust to noise in the environment and interfering humans• Does not require any training• Energy-efficient• Works with any WiFi-enabled device
• Primitives detection accuracy is 87.5%– Using a single AP
• Accuracy increases to 96% using three overheard APs.
Future Work
• Leveraging CSI for finer grained gestures• Other wireless technologies, e.g. cellular and
Bluetooth• Other applications
Thank You
Come and see our demo tomorrow10am to 1pm
wrc-ejust.org wrc_ejust wrc.ejust