Ricky Jacobrjacob@cs.nuim.ie

HapticGPS: Integrating haptic feedback to pedestrian navigation applications

http://www.cs.nuim.ie/~rjacob/

“A location-based service (LBS) is an information or entertainment service, accessible with mobile devices through the mobile network and utilizing the ability to make use of the geographical position of the mobile device”

Source: Wikipedia, April 2011

What is a Location-Based Service?

What is Haptics?

… is a tactile feedback technology

… takes advantage of our ‘sense of touch’

… by applying forces, vibrations, and/or motions to the user through a device.

‘Visualization’ that isn’t visual

Touch20 times faster than vision

We can notice two stimuli just 5 ms apart

‘First sense’ to develop when we are born,

We are highly sensitive to vibration up to 1000 Hz(most sensitive at 250 Hz)

and the last sense used before death

We can sense displacements on our palm as low as 0.2 microns in length

Direction in which objects press the skin is not perceived by touch

Where is haptics used?

From Surgery…

… to Wii games

Pedestrian Navigation using mobile• Maps on mobile with route overlay• Textual description• Audio feedback• Landmark based navigation• 3D models• Augmented Reality navigation

HapticGPS: Integrating haptic feedback to pedestrian navigation applications

HapticenabledDevice

Spatial Data Non-Spatial

Data

Real-timeData

DATA FUSION(Enhanced data)

Algorithms

HapticInteraction

ModelFeedback

FeedbackInteraction

Haptic Interaction Model

Sabrina Paneels and Jonathan C. Roberts

Haptic visualization process

OpenStreetMap

Device used

Cloudmade API

Our application framework

Routing service

Spatial data

GPS, compass, accelerometer

Haptic feedback usingvibration alarm in the phone

Server Client

Use

r

This is where we got the data from

Schematic diagram of HapticGPS

Scans the area for a waypoint

Selects destination

Moves in that direction

User is at a location

Gets an alert when he reaches a waypoint

Scans the area for next waypoint

Continues till he reaches destination

How does it work?

Gets an alert when he points to the waypoint

Test Path

OSM: Mapnik Tiles Zoom to waypoint

Walking direction of user

Difficult way point

Orientation of the Phone

Start Point

End Point

Area 1

Area 2

Area 3

User behaviour while walking

Area 1 Area 2 Area 3

Walking across open areas and bicycle parking

Walk on open areas

• Users can turn off vibration alert if they feel battery charge is low

• Switch to color coded buttons for information

• The display operates as follows:

Color coded buttons

Reached a waypoint, now scan the area to find next waypoint

Pointing in the direction of the next way point

You are yet to reach a way point

User comments• User 4: “I liked this application as it did not need me to know

how to use a map on the mobile phone”.

• User 5: “I did not look into the screen of the phone even once during the entire trip, and with so many people around on the path, it was useful as I was always moving and very rarely was standing still”.

• User 3: responded in relation to the decision making required at the critical points in the routes. “If I was at a position in the route where there were many choices I got a little frustrated if I was receiving negative feedback from the phone”.

• User 5: “I would only use the application in places where I was a complete stranger as I am likely to take random shortcuts upon routes that I am very familiar with”.

Some key findings and beneficial outcomes

• Users take more time to decide where to go when at a complex road intersection/junction

• People tend to walk across open areas: we need routing algorithms that will take this into consideration

• Provide navigation for people in noisy, crowded and/or unfamiliar environments.

• The user can interact with the real environment and people around rather than the mobile device.

• Haptic feedback enables a ‘heads-up’ interaction between the user and the mobile device.

“Crowded streets of India”

Visually impaired

A snowy day

Where/When to use Haptics?

A sunny day Hand full with shopping bags

A rainy day

T. Amemiya

…developed a haptic direction indicator, which will help blind pedestrians intuitively and safely escape from dangerous area by means of haptic navigation

L.R. Elliott and J.B.F. van Erp – [ Vibrotactile waist belt ]

• a soldier in combat, a firefighter trying to navigate a smoke-filled building• a hiker traversing new terrain• a tourist trying to find a local landmark while enjoying scenery,

navigate without having to hold and look at a display—thus allowing him or her the freedom to attend to whatever is most important at that time.

M. Pielot

… tactile wayfinder freed the participants’ attention but could not keepup with the navigation system in terms of navigation performance

S. Robinson

…pedestrian navigation using bearing-based haptic feedback. The benefits listed by using a haptic feedback based system includes the ‘heads-up’ approach.

There is scope to improve pedestrian navigation applications by including another modality - Haptics

Indoor localization and positioning

Walking followed by standing still

Understand user movements

Walking downstairs from first floor

Point2Query: Straight line

Point2Query: Across a region

Point2Query: ‘L’ shaped

Comments and Feedback!

rjacob@cs.nuim.ie

• Do you see Haptics being used as an alternate modality when vision based Location-based services are so popular?

• What other information can be conveyed to the user?

• Where else can haptics be used? On smartphones - is it useful outside the domain of pedestrian navigation?