Indoor LocalizationIndoor localization platform providing high accuracy could enable a host of applications Targeted Location Based Advertising Indoor Navigation (e.g. Airport Terminals)

Indoor Localization

Qian Zhang

Indoor localization platform providing high accuracy could enable a host of applications

Targeted Location Based Advertising

Indoor Navigation (e.g. Airport Terminals)

Real Life Analytics (Gym, Office, etc..)

Applications of Indoor Localization

Lots of Technologies!

Ultrasonic time of flight

Stereo camera

Ad hoc signal strength

Physical contact

WiFi Beacons

Infrared proximity

Laser range-finding

Array microphone

Floor pressure Ultrasound

• Technologies to be covered in this Chapter:

• Wireless-based solution

• VLC-based solution

• Multi-source based solution

Agenda

01

02

03

Wireless-based Solutions

Multi-source based solutions

VLC-based solutions

Wireless Technologies for Localization Name Effective Range Pros Cons

GSM 35km Long range Very low accuracy

LTE 30km-100km

Wi-Fi 50m-100m Readily available; Medium range

Low accuracy

Ultra Wideband 70m High accuracy High cost

Bluetooth 10m Readily Available; Medium accuracy

Short range

Ultrasound 6-9m High accuracy High cost, not scalable

RFID & IR 1m Moderate to high accuracy

Short range, Line-Of-Sight (LOS)

NFC

• Fingerprinting: Radar

• Fingerprinting: PinLoc

• SpotFi: Decimeter Level Localization using WiFi

• Push the Limit of WiFi based Localization for Smartphones

• Accurate RFID Positioning in Multipath Environments

Fingerprinting

• Mapping solution

• Address problems with multipath

• Better than modeling complex RF propagation pattern

Fingerprinting

SSID (Name) BSSID (MAC address) Signal Strength (RSSI)

linksys 00:0F:66:2A:61:00 18

starbucks 00:0F:C8:00:15:13 15

newark wifi 00:06:25:98:7A:0C 23

• Easier than modeling

• Requires a dense site survey

• Usually better for symbolic localization

• Spatial differentiability

• Temporal stability

Fingerprinting

Received Signal Strength (RSS) Profiling Measurements

• Construct a form of map of the signal strength behavior in the coverage area

• The map is obtained: – Offline by a priori measurements

– Online using sniffing devices deployed at known locations

• They have been mainly used for location estimation in WLANs

• Different nodes: – Anchor nodes

– Non-anchor nodes

– A large number of sample points (e.g., sniffing devices)

• At each sample point, a vector of signal strengths is obtained – jth entry corresponding to the jth anchor’s transmitted signal

• The collection of all these vectors provides a map of the whole region

• The collection constitutes the RSS model

• It is unique with respect to the anchor locations and the environment

• The model is stored in a central location

• A non-anchor node can estimate its location using the RSS measurements from anchors

Received Signal Strength (RSS) Profiling Measurements

RADAR: An In-Building RF-Based User Location and Tracking system

Paramvir Bahl and Venkata N. Padmanabhan

•Functional Components • Base Stations (Access Points)

• Mobile Users

•Fundamental Idea in RADAR • Signal Strength is a function of the receiver’s location

• Road Maps

•Techniques to build the Road Maps • Empirical Method

• Radio Propagation Model

•Search Techniques • Nearest Neighbor in Signal Space (NNSS)

• NNSS Avg.

• Viterbi-like Algorithm

Data Collection

• Key Step in the proposed approach

• Records the Radio Signal as a function of the user location

• Off-Line Phase • Construct/validate models for signal propagation

• Real-Time Phase (Infer location of user)

• Every packet received by the base station, the WiLIB extracts • Signal Strength

• Noise floor at the transmitter

• Noise floor at the receiver

• MAC address of the transmitter

Data Processing

• Traces collected from the off-line phase are unified into a table consisting of tuples of the format

[ x, y, d, ss(i), snr(i) ] I € {1,2,3}

• Search Algorithm • NNSS

• NNSS – Avg.

• Viterbi-like Algorithm

• Layout Information

Algorithm and Experimental Analysis

Empirical Method

• 280 combinations of user location and orientation (70 distinct points, 4 orientations on each point)

• Uses the above empirical data recorded in the off-line phase to construct the search space for the NNSS Algorithm

• Algorithm (Emulates the user location problem) • Picks one location and orientation randomly

• Searches for a corresponding match in the rest of the 69 points and orientations

•Comparison with • Strongest Base Station

• Random Selection

Error Distance Values

• Multiple Nearest Neighbor • Increases the accuracy of the Location Estimation

Figure : Multiple Nearest Neighbors T – True Location

G – Guess N1,N2,N3 - Neighbors

N1

N3 N2

G

T

Empirical Method (Cntd. )


• Impact of Number of Number of Samples • Accuracy obtained by all the samples can be obtained if only a few samples

are taken

• Impact of User Orientation •Off-line readings for all orientations is not feasible •Work around is to calculate the error distance for all combinations

No. Of Real-Time Samples Error Distance degradation

1 30%

2 11%

3 4%

• Tracking a Mobile User

• Analogous to the user location problem

• New Signal Strength data set

• Window size of 10 samples

• 4 Signal Strength Samples every second

• Limitation of Empirical Method

• To start off with needs an initial signal strength data set

• Relocation requires re-initialization of the initial data set


Radio Propagation Model

• Introduction • Alternative method for extracting signal strength information

• Based on a mathematical model of indoor signal propagation

• Issues • Reflection, scattering and diffraction of radio waves

• Needs some model to compensate for attenuation due to obstructions

• Models • Rayleigh Fading Model : Infeasible

• Rician Distribution Model : Complex

• Wall Attenuation Factor

Wall Attenuation Factor

Radio Propagation Model (Cntd. )

• Advantages: • Cost Effective

• Easily Relocated

Conclusion

• RF-based user location and tracking algorithm is based on • Empirically measured signal strength model

• Accurate

• Radio Propagation Model

• Easily relocated

• RADAR could locate users with high degree of accuracy

• Median resolution is 2-3 meters, which is fairly good

• Used to build “Location Services” • Printing to the nearest printer

• Navigating through a building

While most WiFi based localization schemes operate with signal strength based information at the MAC layer, PinLoc recognizes the possibility of leveraging detailed physical (PHY) layer information

Fingerprinting Wireless Channel

• 802.11 a/g/n implements OFDM – Wideband channel divided into subcarriers

– Intel 5300 card exports frequency response per subcarrier

Frequency subcarriers

1 2 3 4 5 6 7 8 9 10 39 48

phase and magnitude over 30 subcarriers richly capture the scattering in the environment

• Two key hypotheses need to hold:

Temporal

• Channel responses at a given location may vary over time

• However, variations must exhibit a pattern – a signature

1.

Spatial

• Channel responses at different locations need to be different 2.

Is WiFi Channel Amenable to Localization?

channel responses from multiple OFDM subcarriers can be a promising location signature

• Measured channel response at different times – Using Intel cards

cluster2

cluster2

cluster1

cluster1

Observe: Frequency responses often clustered at a location

Variation over Time

But not necessarily one cluster per location

cluster2

cluster2

cluster1

cluster1

2 clusters with different

mean and variance

But not necessarily one cluster per location

• Measured channel response at different times – Using Intel cards

Variation over Time

Unique clusters per location

How Many Clusters per Location?

Do all 19 clusters occur

with same frequency?

Most

frequent

cluster

2nd

most

3rd

4th Others

3 to 4 clusters heavily dominate, need to learn these signatures

Unique clusters per location

Cluster Occurrence Frequency

Spatial

• Channel responses at different locations need to be different 2.

Clusters with different

mean and variance

Is WiFi Channel Amenable to Localization?

Temporal

• Channel responses at a given location may vary over time

• However, variations must exhibit a pattern – a signature

1.

Location Signature

What is the Size of a Location?

● Localization granularity depends on size ● RSSI changes in orders of several meters (hence, unsuitable)

Cross correlation with signature at reference location

Channel response changes every 2-3cm

3 cm apart

2 cm apart

Define “location” as 2cm x 2cm area, call them pixels

What is the Size of a Location?

● Localization granularity depends on size ● RSSI changes in orders of several meters (hence, unsuitable)

Will all pixels have unique signatures? But …

Real (H(f))

Im (

H(f

))

Self

Similarity

Cross

Similarity > Max ( )

Pixel 1

Pixel 2

Pixel 3

For correct pixel localization

Self

Similarity

Cross

Similarity > Max ( ) 0 -

Self – Max (Cross)

AP1


AP2


AP1 and AP2

67% pixel accuracy even with multiple APs

Opportunity:

- Humans exhibit natural (micro) movements

- Likely to hit several nearby pixels

- Combine pixel fingerprints into super-fingerprint

67% accuracy inadequate …

can we improve accuracy?

Intuition: low probability that a set of pixels

will all match well with an incorrect spot

From Pixels to Spots

Combine pixel fingerprints from a 1m x 1m box.

Spot

Pixel

2cm

PinLoc: Architecture and Modeling

Test Data

Parameters: (wK, UK, VK)

Variational Inference (Infer.NET)

PinLoc measures the CFRs at spots of interest during the training phase and tries to identify as many of the unique clusters as possible during a war-driving period

Per pixel signature

Real (H(f))

Im (

H(f

))

Per spot signature

Real (H(f))

Im (

H(f

))

• Evaluated PinLoc (with existing building WiFi) at:

– Duke museum

– ECE building

– Café (during lunch)

• Roomba calibrates

– 4m each spot

– Testing next day

– Compare with Horus (best RSSI based scheme)

PinLoc Evaluation

• 90% mean accuracy, 6% false positives • WiFi RSSI is not rich enough, performs poorly - 20% accuracy

Accuracy per spot False positive per spot

Performance

SpotFi: Decimeter Level Localization using WiFi

Manikanta Kotaru, Kiran Joshi, Dinesh Bharadia, Sachin Katti

Stanford University

Requirement for Ideal Localization System

1. Easily Deployable

• Commercial WiFi chips



• No hardware or firmware change

4



• No hardware or firmware change

• No User Intervention

5

2. Universal

• Localize any WiFi device

• No specialized sensors

3. Accurate

1 m

• Error of few tens of centimeters

State-of-the-art

System Deployable Universal Accurate

RADAR, Bahl et al, ’00

HORUS, Youssef et al, ’05

ArrayTrack, Xiong et al, ’13

PinPoint, Joshi et al, ’13

CUPID, Sen et al, ’13

LTEye, Kumar et al, ’14

Phaser, Gjengset et al, ’14

Ubicarse, Kumar et al, ’14

SpotFi, Kotaru et al, ’15

System Overview

Localization - Overview

Challenge - Multipath

Solving The Multipath Problem

State-of-the-art

Model signal on antennas alone Model signal on both antennas and

subcarriers

SpotFi

Sub

carr

iers

Antennas

𝒇𝟏

𝒇𝟐

𝒇𝟑

𝒇𝟒

Overall Architecture

• SpotFi collects CSI and RSSI measurements from all the APs that can hear the packet transmitted by the target • SpotFi calculates the ToF and AoA of all the propagation paths from the target to each of the APs • SpotFi then identifies the direct path between the target and the AP that did not undergo any

reflections • SpotFi estimates the location of the target by using the direct path AoA estimates and RSSI

measurements from all the APs

Step 1: Resolve Multipath

𝜽𝟏, 𝝉𝟏

𝜽𝟐, 𝝉𝟐

Equal Distance

Line

Signal Modeling

Phase

Distance travelled by the WiFi signal

Ph

ase

1 / frequency

0

𝐏𝐚𝐭𝐡 𝐃𝐢𝐟𝐟𝐞𝐫𝐞𝐧𝐜𝐞 =𝟐𝝅

𝒘𝒂𝒗𝒆 𝒍𝒆𝒏𝒈𝒕𝒉∗ (𝑷𝒉𝒂𝒔𝒆 𝑫𝒊𝒇𝒇𝒆𝒓𝒆𝒏𝒄𝒆)

Equal Phase Line

Signal Modeling – AoA (Angle of Arrival)

Signal Modeling – AoA (Angle of Arrival)

Uniform linear array consisting of M antennas:

• For AoA of θ, the target’s signal travels an additional distance of d*sin(θ) to the second antenna in the array compared to the first antenna

• This results in an additional phase of -2π*d*sin(θ)*f/c at the second antenna

Signal Modeling - AoA

Define Φ1 = e−

𝑗2𝜋𝑑sin𝜃1𝑐

𝑓

𝜽𝟏

1

Γ1 is complex attenuation of the path. Φ1 depends on AoA

Phase at the antenna 1: 𝑥1 = Γ1 Phase at the antenna 2: 𝑥2 = Γ1Φ1 Phase at the antenna 3: 𝑥3 = Γ1Φ1

2 2 3

Say There Are Two Paths…


𝑥1 = Γ1 𝑥2 = Γ1Φ1 𝑥3 = Γ1Φ1

2


𝑥1 = Γ1 + Γ2 𝑥2 = Γ1Φ1 + Γ2Φ2 𝑥3 = Γ1Φ1

2 + Γ2Φ22

Problem Statement

CSI - Known

𝑥1 = Γ1 + Γ2 𝑥2 = Γ1Φ1 + Γ2Φ2 𝑥3 = Γ1Φ1

2 + Γ2Φ22

Problem Statement

𝑥1 = Γ1 + Γ2 𝑥2 = Γ1Φ1 + Γ2Φ2 𝑥3 = Γ1Φ1

2 + Γ2Φ22

Parameters - Unknown

Problem Statement

Number of paths (or AoAs) < Number of antennas (or equations)

𝑥1 = Γ1 + Γ2 𝑥2 = Γ1Φ1 + Γ2Φ2 𝑥3 = Γ1Φ1

2 + Γ2Φ22

Typical Indoor Multipath

That’s A Problem

State-of-the-art Commodity WiFi chips

Number of antennas/equations should be at least 5

How To Obtain More Equations?

Model signal on both antennas and subcarriers

Sub

carr

iers

Antennas

𝒇𝟏

𝒇𝟐

𝒇𝟑

𝒇𝟒

𝒇𝟏

𝒇𝟐

Each Subcarrier Gives New Equations

Define 𝜴𝟏 = 𝒆−𝒋𝟐𝝅 𝒇𝟐−𝒇𝟏 𝝉𝟏

Γ1 is complex attenuation of the path. Ω1 depends on incoming signal ToF

Phase at first subcarrier: 𝑥1 = Γ1 Phase at second subcarrier: 𝑥2 = Γ1Ω1

Signal Modeling – ToF (Time of Flight)

Estimate both AoA and ToF

More number of equations in terms of parameter of our interest


At first subcarrier, for 3 antennas

𝑥1 = Γ1

𝑥2 = Γ1Φ1

𝑥3 = Γ1Φ12

At second subcarrier, for 3 antennas

𝑦1 = Γ1Ω1

𝑦2 = Γ1Φ1Ω1

𝑦3 = Γ1Φ12Ω1


At first subcarrier, for 3 antennas

𝑥1 = Γ1 + Γ2

𝑥2 = Γ1Φ1 + Γ2Φ2

𝑥3 = Γ1Φ12 + Γ2Φ2

2

At second subcarrier, for 3 antennas

𝑦1 = Γ1Ω1 + Γ2

𝑦2 = Γ1Φ1Ω1 + Γ2Φ2Ω2

𝑦3 = Γ1Φ12Ω1 + Γ2Φ2

2Ω2

𝑥1 = Γ1 + Γ2

𝑥2 = Γ1Φ1 + Γ2Φ2

𝑥3 = Γ1Φ12 + Γ2Φ2

2

𝑦1 = Γ1Ω1 + Γ2

𝑦2 = Γ1Φ1Ω1 + Γ2Φ2Ω2

𝑦3 = Γ1Φ12Ω1 + Γ2Φ2

2Ω2

Problem Statement

Sub

carr

ier

1

Sub

carr

ier

2

CSI - Known

𝑦1 = Γ1 + Γ2

𝑦2 = Γ1Φ1 + Γ2Φ2

𝑦3 = Γ1Φ12 + Γ2Φ2

2

𝑦1 = Γ1Ω1 + Γ2

𝑦2 = Γ1Φ1Ω1 + Γ2Φ2Ω2

𝑦3 = Γ1Φ12Ω1 + Γ2Φ2

2Ω2

Problem Statement

Sub

carr

ier

1

Sub

carr

ier

2

Parameters - Unknown

𝑥1 = Γ1 + Γ2

𝑥2 = Γ1Φ1 + Γ2Φ2

𝑥3 = Γ1Φ12 + Γ2Φ2

2

𝑦1 = Γ1Ω1 + Γ2

𝑦2 = Γ1Φ1Ω1 + Γ2Φ2Ω2

𝑦3 = Γ1Φ12Ω1 + Γ2Φ2

2Ω2

Problem Statement

Sub

carr

ier

1

Sub

carr

ier

2

Number of equations =

Number of Subcarriers x

Number of Antennas

AoA, ToF Estimates

𝜽𝟏, 𝝉𝟏

𝜽𝟐, 𝝉𝟐

Step 2: Identify Direct Path

𝜽𝟏, 𝝉𝟏

𝜽𝟐, 𝝉𝟐

𝜽𝟏, 𝝉𝟏

AoA, ToF Estimates

𝜽𝟏, 𝝉𝟏

𝜽𝟐, 𝝉𝟐

Use Multiple Packets

𝜽𝟏, 𝝉𝟏

𝜽𝟐, 𝝉𝟐

Use Multiple Packets

Direct Path Likelihood

Higher weight

Higher weight

Higher weight

Lower weight

Lower weight

• Smaller ToF


Higher weight

Lower weight

Lower weight

Lower weight

• Smaller ToF

• Tighter Cluster

Lower weight


Higher weight

Higher weight

Lower weight

Lower weight

• Smaller ToF

• Tighter Cluster

• More Packets

Lower weight

Highest Direct Path Likelihood

Step 3: Localize The Target

𝜽𝟏, 𝝉𝟏

𝜽𝟐, 𝝉𝟐

𝜽𝟏, 𝝉𝟏

Use Multiple APs

Direct Path AoA = 45 degrees Signal Strength = 10 dB

Direct Path AoA = 10 degrees Signal Strength = 30 dB

Find location that best explains the AoA and Signal Strength

at all the APs

Direct Path AoA = -45 degrees Signal Strength = 20 dB

Use Different Weights

Use different weights for different APs

Direct Path AoA = 45 degrees Signal Strength = 10 dB Direct Path Likelihood

Direct Path AoA = 10 degrees Signal Strength = 30 dB Direct Path Likelihood

Direct Path AoA = -45 degrees Signal Strength = 20 dB Direct Path Likelihood

Evaluation

52 m

Testbed

Access point Target

AP Locations Target Locations

0

0.2

0.4

0.6

0.8

1

0.05 0.5 5

Emp

iric

al C

DF

Localization Error (m)

Indoor Office Deployment

52 m

16 m

0.4 m

ArrayTrack Ubicarse SpotFi

0.3 m 0.4 m 0.4 m

AP Locations Target Locations

Stress Test – Obstacles Blocking The Direct Path

AP Locations Target Locations 52 m

0

0.2

0.4

0.6

0.8

1

0.05 0.5 5

Emp

iric

al C

DF


Stress Test – Obstacles Blocking The Direct Path

1.3 m

AP Locations Target Locations 52 m

Effect of WiFi AP Deployment Density

0

0.2

0.4

0.6

0.8

1

0.05 0.5 5

Emp

iric

al C

DF


3 APs

4 APs

5 APs

0.8 m

Conclusion

• Deployable: Indoor Localization with commercial WiFi chips

• Accurate: Accuracy comparable to state-of-the-art localization systems which are not suitable for wide deployments

• Universal: Simple localization targets with only a WiFi chip

Push the Limit of WiFi based Localization

for Smartphones

Hongbo Liu, Yu Gan, Jie Yang, Simon Sidhom, Yan Wang, Yingying Chen

Department of Electrical and Computer Engineering Stevens Institute of Technology

Fan Ye IBM T. J. Watson Research Center

The Need for High Accuracy Smartphone Localization

Shopping Mall Airport

Help users navigation inside large and complex indoor environment, e.g., airport,

train station, shopping mall.

Understand customers visit and stay patterns for business

Train Station

Smartphone Indoor Localization - What has been done?

Contributions in academic research

Commercial products

Localization error up to 10 meters

Google Map Shopkick

Locate at the granularity of stores

WiFi indoor localization

High accuracy indoor localization

WiFi enabled smartphone indoor localization

RADAR [INFOCOM’00], Horus [MobiSys’05], Chen et.al[Percom’08]

Cricket [Mobicom’00], WALRUS [Mobisys’05], DOLPHIN [Ubicomp’04], Gayathri et.al [SECON’09]

SurroundSense [MobiCom’09], Escort [MobiCom’10], WILL[INFOCOM’12], Virtual Compass [Pervasive’10]

Is it possible to achieve high accuracy localization using most prevalent WiFi infrastructure?

0

5

10

15

20

25

30

35

40

45

AP 1 AP 2 AP 3 AP 4

6 - 8 meters ~ 2 meters

Root Cause of Large Localization Errors

Permanent environmental settings, such as furniture placement and walls.

Transient factors, such as dynamic obstacles and interference.

Am I here?

I am around here.

32: [ -22dB, -36dB, -29dB, -43dB ]

48: [ -24dB, -35dB, -27dB, -40dB]

Orientation, holding position, time of day, number of samples

Physically distant locations share similar WiFi Received Signal Strength !

Rec

eive

d S

ign

al S

tren

th

(dB

m) WiFi as-is is not a suitable candidate for high accurate

localization due to large errors

Is it possible to address this fundamental limit without the need

of additional hardware or infrastructure?

Inspiration from Abundant Peer Phones in Public Place

Increasing density of smartphones in public spaces

Provide physical constraints from nearby peer phones

How to capture the physical constraints?

Target

Peer 1

Peer 2

Peer 3

Basic Idea

WiFi Position Estimation Acoustic Ranging

Interpolated Received Signal Strength Fingerprint Map

Exploit acoustic signal/ranging to construct peer constraints Target

Peer 1 Peer 2

Peer 3

• Peer assisted localization

• Fast and concurrent acoustic ranging of multiple phones

• Ease of use

System Design Goals and Challenges

Exactly what is the algorithm to search for the best fit position and quantify the signal similarity so that to reduce large errors?

How to design and detect acoustic signals?

Need to complete in short time.

Not annoy or distract users from their regular activities.

Rigid graph construction

Sound signal design

Acoustic signal detection

System Work Flow

Identify nearby peers

Beep emission strategy

Only phones close enough can detect recruiting signal

Peer phones willing to help send their IDs to the server

Employ virtual synchronization scheme based on time-multiplexting

Deploy extra timing buffers to accommodate variations in the reception of the schedule at different phones, e.g., 20 ms

Peer recruiting & ranging

Peer assisted localization


WiFi position estimation


Minimizing the impact on users’ regular activities

Fast ranging

Unobtrusive to human ears

Robust to noise

Change point detection

Correlation method

16 – 20 KHz

ADP2

Lab Train Station Shopping Mall Airport

HTC EVO

Construct the graph G and G’ based on initial WiFi position estimation and the acoustic ranging measurements.

Graph G based on WiFi position estimation

Rigid Graph G’ based on acoustic ranging




WiFi position estimation



System Work Flow

System Work Flow





WiFi position estimation Peer assisted localization

Graph Orientation Estimation Translational Movement

WiFi based graph Acoustic ranging graph

• Prototype Devices

• Trace-driven statistical test Feed the training data as WiFi samples

Perturb distances with errors following the same distribution in real environments

Prototype and Experimental Evaluation

ADP 2 HTC EVO

• Localization performance across different real-world environments (5 peers)

Localization Accuracy

Peer assisted method is robust to noises in different environments

Median error 90% error

Lab Train Station Shopping Mall Airport

• Overall Latency

• Energy Consumption

Overall Latency and Energy Consumption

Negligible impact on the battery life

• e.g., with additional power consumption at about 320mW on HTC EVO - lasts 12.7 hours with average power of 450mW

Pose little more latency than required in the original WiFi localization about 1.5 ~ 2 sec

• Peer Involvement

• Movements of users

• Triggering peer assistance

Discussion

Provides the technology for peer assistance

Up to users to decide when they desire such help

Do not pose more constraints on movements than existing WiFi methods

Affect the accuracy only during sound-emitting period

• Happens concurrently and shorter than WiFi scanning

Use incentive mechanism to encourage and compensate peers that help a target’s localization

• Leverage abundant peer phones in public spaces to reduce large localization errors

• Exploit minimum auxiliary COTS sound hardware readily available on smartphones

• Demonstrate our approach successfully pushes further the limit of WiFi localization accuracy

Conclusion

Aim at the most prevalent WiFi infrastructure

Do not require any special hardware

Utilize much more accurate distance estimate through acoustic ranging to capture unique physical constraints

Lightweight in computation on smartphones

In time not much longer than original WiFi scanning

With negligible impact on smartphone’s battery life time

Accurate RFID Positioning in Multipath Environments

Jue Wang & Dina Katabi ACM Sigcomm 2013

RFIDs

Battery-free RF stickers with unique IDs

RFIDs

5-cent stickers to tag any and every object

Reader’s range is ~15m

Imagine you can localize RFIDs to within 10 to 15 cm!

No more customer checkout lines

If we can locate RFID to within 10 to 15cm

The Challenge: Multipath Effect

Localization uses RSSI or Angle-of-Arrival (AoA)

But, signal bounces off objects in the environment

Angle of signal is not the direction of the RFID

Multipath propagation limits the Accuracy of RFID localizations

PinIt

Accurate RFID localization (e.g., 10 to 15cm) even in

multipath and non-line-of-sight settings

• Focuses on proximity to reference RFIDS

• Exploits multipath effects to increase accuracy

PinIt Exploits Multipath Signals from nearby RFIDs propagate along closer paths and experience similar reflections

PinIt Exploits Multipath Signals from nearby RFIDs propagate along closer paths and experience similar reflections

Nearby RFIDs have similar profiles with smaller shifts in the peaks

Implementation & Evaluation

• Implemented a PinIt Reader in USRP

• Commercial off-the-shelf RFIDs

• Mounted the antenna on an iRobot that slides back and forth

Positioning Accuracy

• 200 RFIDs deployed on the shelves in the library spaced by 15 cm

PinIt improve the accuracy by 6x in comparison to AoA and 10x in comparison to RSSI

Automatic Checkout

Five items in two adjacent baskets at checkout

Which Items Belong to Which Basket?

Is the Cookie Bag in the Orange or Blue Basket?

i

i+2

i

i i

time time

Why Dynamic Time Warping (DTW)?

Any distance (Euclidean, Manhattan, …) which aligns the i-th point on one time series with the i-th point on the other will produce a poor similarity score.

A non-linear (elastic) alignment produces a more intuitive similarity measure, allowing similar shapes to match even if they are out of phase in the time axis.

C

Q

C

Q

How is DTW Calculated?

KwCQDTW

K

k k1min),(

Every possible warping between two time series, is a path though the matrix. We want the best one…

(i,j) = d(qi,cj) + min{ (i-1,j-1), (i-1,j ), (i,j-1) }

This recursive function gives us the minimum cost path

Warping path w

C

Q

One more note

Warping path w

The time series can be of different lengths..

C Q

Is the Noodle in the Orange or Blue Basket?

Brief Summary

• PinIt provides accurate RFID positioning even in multipath

and NLOS settings

• It uses DTW to compare RFID multipath profiles

• It enables new applications including eliminating checkout

lines, object tracking in libraries and pharmacies, smart

homes, …

Agenda

01

02

03



VLC-based solutions

Wearables Can Afford: Light-weight Indoor Positioning with Visible Light

Zeyu Wang, Zhice Yang, Jiansong Zhang, Chenyu Huang,Qian Zhang

Hong Kong University of Science and Technology

Indoor Localization

• Fingerprinting: Use wireless signal (WiFi, FM, Sound, etc.) to construct the fingerprint map

Indoor Localization


• Dead reckoning: Use inertial sensors to calculate moving path

Indoor Localization



Accuracy is not enough (~several meters)

Indoor Localization



• Dedicated localization infrastructure

Indoor Localization



• Dedicated localization infrastructure

Complex and high-cost to handle RF multipath

Visible Light Positioning


…1

…1

...1…


…1

…1

...1…

…2

…2

...2…

• Visible Light Positioning (VLP) is an emerging positioning technique that based Visible Light Communication (VLC) – Light bulbs are densely deployed

Location anchors are ubiquitous

– Light beam is very directional

No multipath, localization is simple and accurate

– More…

• Light is free of radio wave

• Positioning through light bulbs is green in energy


How VLC generally works?

• Modulate Light Intensity

Normal Light

Modulated Light

Time

Problem in VLC: Flickering

10Hz 100Hz >1000Hz

Consequence: Overhead in Client

• Additional Receiving Device – Using customized light sensor that

requires cumbersome calibration[1]

• High Computational Overhead – Using very high resolution camera to

extract the roller shuttering patterns[2]

>1000Hz

[1] L. Li etc. “Epsilon: A visible light based positioning system” in NSDI’14 [2] Y.-S. Kuo etc. “Luxapose: Indoor positioning with mobile phones and visible light” in Mobicom’14

Must be LED

These overhead can hardly be afforded in wearables. Can they be eliminated?

Idea: Flickering-free Modulation

• Instead of changing the intensity, we modulate information by changing the polarization of light Human eyes CANNOT perceive changes in polarization

Therefore low baud rate in transmitters

Therefore low decoding overhead in clients

PIXEL

Review the display mechanism of LCD !

Back Light

Polarizing Film

PIXEL: One Pixel from LCD

Polarizing Film Eyes

0V


Back Light

Polarizing Film


Voltage

Liquid Crystal

5V


Liquid Crystal

Back Light

Polarizing Film


Camera

Eyes

VLC Transmitter

PIXEL: VLC Transmitter

Voltage

Liquid Crystal

Back Light

Polarizing Film

Polarizing Film

Eyes

Locatio

n…

Sun

PIXEL: VLP Architecture

Polarizing Film

VLC Transmitter

… Lo

cation

…

… Lo

cation

…

… Lo

cation

…

…

Challenge: User Mobility

SNR

45° 135°

Challenge: User Mobility (Cont.)

Receiving Direction

Voltage “Low”

Voltage “High”

Received Light Intensity

Solution: Dispersion

Solution: Dispersion (Cont.)

Disperse the Polarization of Different Colors into Different Directions

Dispersor

Solution: Dispersion (Cont.)

Receiving Direction

Received Color

SNR

45° 135°

Voltage “Low”

Voltage “High”

Positioning Method

1 3

2

1

2

3

Challenges: Less Beacons

• Existing methods for camera-based VLC localization require multiple beacon lamps(3 or more) being captured in a single image

• Field Test: 2 or less beacon lamps can be captured by the front camera in normal holding position Portable cameras do not have wide Field of View

The ceiling of buildings is normal limited to several meters.

Example: 3m below, camera of iPhone 6 can only cover 3*3m2 of the ceiling.

Challenges: Less Beacons (Cont.)

Challenges: Less Beacons (Cont.)

1 2

Location Ambiguity

• The position of the receiver has 6 degrees of freedom: 3 in location and 3 in 3D orientation.

• Each received beacon adds 2 AoA constraints to the position and orientation.

• The gravity sensor adds 2 constraints to the 3D orientation.

Two beacons are enough

Solution: Sensor Assisted Localization

1 2

Location Ambiguity

Gravity

Solution: Sensor Assisted Localization

Implementation

• VLC Transmitter

– Polarizing film ($0.001/cm2)

– LCD with only one pixel ($0.03/cm2)

– Glass box with optical rotation liquid

– 14Hz Baud Rate

– Location Beacon

• 5bit Preamble + 8bit Location ID + 4bit CRC

• Client

– Polarizing film ($0.001/cm2)

– Android App with VLC decoding and VLP algorithm

• Smart phone: Galaxy S II (1.2GHz CPU, 8 Megapixel Camera)

• Wearable: Google Glass

Evaluation-VLC

VLC Transmitter

𝜃 30

25

20

15

10

5

0

SNR

(d

B)

0 20 40 60 80 100 120 140 160 180

Receiver's Orientation 𝜃 (degree)

w/o dispersor

with dispersor

30 × 40

60 × 80

Evaluation-VLC

30

25

20

15

10

5

0

SNR

(d

B)

1 2 3 4 5 6 7 8 9 10 11 12 13

Distance (m)

120 × 160

VLC Transmitter

𝑑

Evaluation-VLP

1 2

3

4 5

7 8

6

1.8m

2.4m

1

0.8

0.6

0.4

0.2

0 0 10 20 30 40 50

Positioning Error (cm)

CD

F

Google Glass 300MHz Google Glass 600MHz Google Glass 800MHz

Evaluation-VLP

1

0.8

0.6

0.4

0.2

0 0 50 100 150 200 250

VLP Processing Time Cost (ms)

CD

F Samsung Galaxy SII 1200MHz

Conclusion

• We introduce a light weight VLC method that based on modulating light’s polarization

• We propose to use optical rotation material/dispersor to hand users’ mobility

• We implement and evaluate the VLP system, and results show submeter accuracy can be achieved in both smart phone and wearables.

Agenda

01

02

03



VLC-based solutions

SurroundSense: Mobile Phone Localization via Ambience Fingerprinting

Ionut Constandache, Martin Azizyan and Romit Roy Choudhury

Context

Pervasive wireless connectivity

+

Localization technology

=

Location-based applications

Location-Based Applications (LBAs)

• For Example: – GeoLife shows grocery list when near Walmart

– MicroBlog queries users at a museum

– Location-based ad: Phone gets coupon at Starbucks

• iPhone AppStore: 3000 LBAs, Android: 500 LBAs

Location-Based Applications (LBAs)

• For Example: – GeoLife shows grocery list when near Walmart

– MicroBlog queries users at a museum

– Location-based ad: Phone gets coupon at Starbucks

• iPhone AppStore: 3000 LBAs, Android: 500 LBAs

• Location expresses context of user – Facilitates content delivery

Location is an IP address As if for content delivery

Thinking about Localization

from an application perspective…

Emerging location based apps need

place of user, not physical location

Starbucks, RadioShack, Museum, Library

Latitude, Longitude

We call this Logical Localization …

Can we convert from

Physical to Logical Localization?

Can we convert from


State of the Art in Physical Localization:

1. GPS Accuracy: 10m

2. GSM Accuracy: 100m

3. Skyhook (WiFi+GPS+GSM) Accuracy: 10m-100m

Widely-deployable localization technologies have errors in the range of several meters

Can we convert from


State of the Art in Physical Localization:

1. GPS Accuracy: 10m

2. GSM Accuracy: 100m

3. Skyhook (WiFi+GPS+GSM) Accuracy: 10m-100m

Several meters of error is inadequate

to logically localize a phone

Physical Location Error

Several meters of error is inadequate

to logically localize a phone

RadioShack Starbucks

Physical Location Error

The dividing-wall problem

Contents

• SurroundSense

• Evaluation

• Limitations and Future Work

• Conclusion

It is possible to localize phones by sensing the ambience

Hypothesis

such as sound, light, color, movement, WiFi …

Sensing over multiple dimensions extracts more information from the ambience

Each dimension may not be unique,

but put together, they may provide a

unique fingerprint

B A C D E

Should Ambiences be Unique Worldwide?

F G

H J

I

L M N

O

P Q

Q R

K

SurroundSense

• Multi-dimensional fingerprint – Based on ambient sound/light/color/movement/WiFi

Starbucks

Wall

RadioShack

Should Ambiences be Unique Worldwide?

B A C D E

F G

H J

I

K L

M N O

P Q

Q R

GSM provides macro location (strip mall) SurroundSense refines to Starbucks

+

Ambience Fingerprinting

Test Fingerprint

Sound

Acc.

Color/Light

WiFi

Logical Location

Matching

Fingerprint Database

=

Candidate Fingerprints

GSM Macro Location

SurroundSense Architecture

Fingerprints

• Sound:

(via phone

microphone)

• Color:

(via phone

camera)

Amplitude Values -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

No

rmal

ized

Co

un

t

0.14

0.12

0.1

0.08

0.06

0.04

0.02

0

Acoustic fingerprint

(amplitude distribution)

Color and light fingerprints on HSL space

Ligh

tnes

s

1

0.5

0

Hue

0

0.5

1 0 0.2 0.4 0.6

0.8 1

Saturation

Fingerprints

• Movement: (via phone accelerometer)

Cafeteria Clothes Store Grocery Store

Static

Moving

Fingerprints • Movement: (via phone accelerometer)


Static

Moving

Queuing

Fingerprints



Static

Queuing Seated

Moving

Fingerprints



Static

Pause for product browsing

Moving

Fingerprints



Static

Pause for product browsing

Short walks between product browsing

Moving

Fingerprints



Static

Walk more

Moving

Fingerprints



Static

Walk more Quicker stops

Moving

Fingerprints


• WiFi: (via phone wireless card)


Static

ƒ(overheard WiFi APs)

Moving

Discussion

• Time varying ambience – Collect ambience fingerprints over different time windows

• What if phones are in pockets? – Use sound/WiFi/movement

– Opportunistically take pictures

• Fingerprint Database – War-sensing

Contents

• SurroundSense

• Evaluation


• Conclusion

Evaluation Methodology

• 51 business locations – 46 in Durham, NC

– 5 in India

• Data collected by 4 people – 12 tests per location

• Mimicked customer behavior

Evaluation: Per-Cluster Accuracy

Cluster

No. of Shops

1 2 3 4 5 6 7 8 9 10

4 7 3 7 4 5 5 6 5 5

Acc

ura

cy (

%)

Cluster

Localization accuracy per cluster


Cluster

No. of Shops

1 2 3 4 5 6 7 8 9 10

4 7 3 7 4 5 5 6 5 5

Acc

ura

cy (

%)

Cluster


Multidimensional sensing


Cluster

No. of Shops

1 2 3 4 5 6 7 8 9 10

4 7 3 7 4 5 5 6 5 5

Fault tolerance

Acc

ura

cy (

%)

Cluster



Cluster

No. of Shops

1 2 3 4 5 6 7 8 9 10

4 7 3 7 4 5 5 6 5 5

Acc

ura

cy (

%)

Cluster

Localization accuracy per cluster Sparse WiFi APs


Cluster

No. of Shops

1 2 3 4 5 6 7 8 9 10

4 7 3 7 4 5 5 6 5 5

No WiFi APs

Acc

ura

cy (

%)

Cluster


Evaluation: Per-Scheme Accuracy

Mode WiFi Snd-Acc-WiFi Snd-Acc-Lt-Clr SS

Accuracy 70% 74% 76% 87%

Evaluation: User Experience

Random Person Accuracy

Average Accuracy (%) 0 10 20 30 40 50 60 70 80 90 100

1

0.9 0.8

0.7

0.6

0.5

C

DF

0.4

0.3 0.2

0.1

0

WiFI

Snd-Acc-WiFi

Snd-Acc-Clr-Lt

SurroundSense

Economics forces nearby businesses to be different

Not profitable to have 3 coffee shops

with same lighting, music, color, layout, etc.

SurroundSense exploits this ambience diversity

Why does it work?

The Intuition:

Contents

• SurroundSense

• Evaluation


• Conclusion

Limitations and Future Work

• Energy-Efficiency

• Localization in Real Time

• Non-business locations


• Energy-Efficiency – Continuous sensing likely to have a large energy draw

• Localization in Real Time




• Localization in Real Time – User’s movement requires time to converge




• Localization in Real Time – User’s movement requires time to converge

• Non-business locations – Ambiences may be less diverse

Contents

• SurroundSense

• Evaluation


• Conclusion

SurroundSense

• Today’s technologies cannot provide logical localization

• Ambience contains information for logical localization

• Mobile Phones can harness the ambience through sensors

• Evaluation results: – 51 business locations,

– 87% accuracy

• SurroundSense can scale to any part of the world

End of This Chapter

Documents

Indoor LocalizationIndoor localization platform providing high accuracy could enable a host of applications Targeted Location Based Advertising Indoor Navigation (e.g. Airport Terminals)