Presenters: Brendan O’Leary and Lee StearnsPresenters: Brendan O’Leary and Lee Stearns

Coauthors: Gemstone Team Vision, Bobby Bobo, Rama Chellappa, and Cha-Min Tang

Coauthors: Gemstone Team Vision, Bobby Bobo, Rama Chellappa, and Cha-Min Tang

Team Vision is: Christopher Cheng, Brendan O’Leary, Lee Stearns, Steve Caperna, Junghee Cho, Victoria Fan, Avishkar Luthra, Jansen Sheng, Andrew Sun, Roni Tessler, Paul Wong, and Jimmy Yeh

Team Vision is: Christopher Cheng, Brendan O’Leary, Lee Stearns, Steve Caperna, Junghee Cho, Victoria Fan, Avishkar Luthra, Jansen Sheng, Andrew Sun, Roni Tessler, Paul Wong, and Jimmy Yeh

The White CaneThe White Cane Other Commercial Navigation SolutionsOther Commercial Navigation Solutions

Objects and Landmarks to IdentifyObjects and Landmarks to Identify

Our Goal: A Complete Navigation System for the BlindOur Goal: A Complete Navigation System for the Blind

GPSAlgorithm

Computer Vision

Algorithm

GPS UnitInertial

Navigation

Speech OutputPush-to-TalkVoice Input

Camera

Audio Feedback

GraphImage

DatabaseAudio Interface

Hot Swappable Templates

Hot Swappable Templates

Too few featuresToo few features

Too few matchesToo few matches

No consensusNo consensus

Object LocationObject Location

The SIFT Computer Vision Algorithm

The SIFT Computer Vision Algorithm

A Demonstration of Our Speech MenuA Demonstration of Our Speech Menu

Throat Microphone with Push-to-Talk ButtonThroat Microphone with Push-to-Talk Button

A Sample Grammar FileA Sample Grammar File

The Discrete Speech InterfaceThe Discrete Speech Interface

The Continuous Beacon InterfaceThe Continuous Beacon Interface

An Early Test of Our Vision AlgorithmAn Early Test of Our Vision Algorithm

Sign Experiment SetupSign Experiment Setup 3D Object Experiment Setup3D Object Experiment Setup

Locating a Restroom SignLocating a Restroom Sign

Locating a CaneLocating a Cane

Initial Results:

Discrete Mode:

Average Location Time: 16 sec

Continuous Mode:

Average Location Time: 12 sec

With Consistent Tracking: 5 sec

Initial Results:

Discrete Mode:

Average Location Time: 16 sec

Continuous Mode:

Average Location Time: 12 sec

With Consistent Tracking: 5 sec

Some Future PeripheralsSome Future Peripherals

Headphones that Don’t Block

External Sounds

Headphones that Don’t Block

External Sounds

Higher Resolution Camera with

Better Auto-Focus

Higher Resolution Camera with

Better Auto-Focus

Current Test SystemCurrent Test System

CameraFeed

CameraFeed SIFTSIFT BBFBBF RANSACRANSAC 3D

Transform3D

TransformObject

LocationObject

Location

11 22 33 44 55 66

Questions?Questions?

Our test system:

Toshiba Satellite A75-S2311 Windows XP Professional (32-bit)

Service Pack 23.3 GHz Pentium 4 processor1.5 GB of RAM

First-generation Logitech QuickCam for Notebooks Pro (mounted on sunglasses)

USB microphoneOver-the-ear headphones

Our test system:

Toshiba Satellite A75-S2311 Windows XP Professional (32-bit)

Service Pack 23.3 GHz Pentium 4 processor1.5 GB of RAM

First-generation Logitech QuickCam for Notebooks Pro (mounted on sunglasses)

USB microphoneOver-the-ear headphones