Wearable Computing Past, Present and Future

WearableWearable ComputingComputingPastPast, Present and , Present and FutureFuture

PresentedPresented by by Jeremiah Scholl, with help from Mikael Jeremiah Scholl, with help from Mikael DruggeDrugge

OverviewOverview

PurposePurposeThe The earlyearly daysdaysDefinition and Definition and usersusersTechnologyTechnology and and gadgetsgadgetsReal Real lifelife applicationsapplications

PurposePurpose

Why are you here today?Why are you here today?–– Get Get introducedintroduced to to wearablewearable

computingcomputing..–– WhatWhat existsexists, , whatwhat cancan be be

donedone todaytoday??–– WhatWhat is the is the currentcurrent statestate

of the art?of the art?–– WhereWhere areare wewe headingheading??

The The EarlyEarly DaysDays

””HowHow diddid it start?it start?””

In the In the BeginningBeginning......

m1

Slide 5

m1 This is a photography.How do you create a scene like this?

That's what a guy named Steve Mann was also thinking about... 1970smdr, 2003-8-23

Steve MannSteve Mann

1970s, 1970s, prepre--laptoplaptop, , earlyearly computercomputer era.era.BuildingBuilding computerscomputershehe couldcould wearwear..InventorInventor of of wearablewearablecomputingcomputing..

Steve Mann Steve Mann -- BiographyBiography

1991: 1991: StartedStarted the the ””WearableWearable ComputingComputingProjectProject”” at MIT.at MIT.1995: 1995: WorldWorld’’ss firstfirstcovertcovert wearablewearablecomputercomputer –– cameracameraand display and display concealedconcealedin in ordinaryordinary eyeglasseseyeglasses..1997: PhD from MIT in 1997: PhD from MIT in the the fieldfield hehe himselfhimself hadhadinventedinvented..TodayToday, 2003: Works at , 2003: Works at University of Toronto.University of Toronto.

Steve Mann Steve Mann -- EvolutionEvolution

Evolution of Evolution of MannMann’’ss wearablewearable computerscomputers..

mdr8

Slide 8

mdr8 Mention some anecdotes about the pictures:

a) 4cm CRT monitor. Antennaes transmit video, processed at a remote location, sent back.

b) Apparatus for painting with light.

c) 1.5cm CRT monitor.

d) ...

e) Glasses with camera and display concealed.Twiddler used as input device.Mikael Drugge, 2003-8-25

Definition and Definition and UsersUsers

””WhatWhat is it and is it and whywhy useuse it?it?””

WhatWhat is a is a WearableWearableComputerComputer??

LaptopLaptop??–– ...in a backpack?...in a backpack?–– ......withwith a a headhead--mountedmounted display?display?

PDA?PDA?CommercialCommercial system?system?OffOff--thethe--shelfshelf componentscomponents??WristWrist watchwatch??

WhatWhat is a is a WearableWearableComputerComputer??

WristWrist watchwatch runningrunningLinux and XFree86.Linux and XFree86.ClockClock and video and video conferencingconferencingapplicationapplication..

WhatWhat a a wearablewearablecomputercomputer is is dependsdependson on youryour definition...definition...–– ButBut it must be it must be

wearablewearable!!

WhyWhy useuse WearablesWearables??

TheyThey areare wearablewearable!!–– AlwaysAlways withwith youyou, not as , not as easilyeasily forgottenforgotten..

InstantInstant access, information access, information anywhereanywhere..–– LaptopLaptop requiresrequires timetime to to prepareprepare for for useuse..–– PDA PDA requiresrequires useuse of of bothboth youryour hands.hands.

WearablesWearables cancan becomebecome a part of a part of youyou..–– Transparent Transparent useuse, not just , not just ””a a thingthing””..

mdr10

Slide 12

mdr10 The question that the previous slide brings forward is rather simple - Why use wearables? What do they have to offer that other computers have?Mikael Drugge, 2003-8-25

WhoWho usesuses WearablesWearablestodaytoday??

TechniciansTechnicians–– BlueprintsBlueprints, etc., etc.

FieldField workersworkers–– Access to information Access to information

given by given by remoteremote experts.experts.MilitaryMilitary personnelpersonnel–– SoldiersSoldiers, , monitoringmonitoring

healthhealth, , equipmentequipment, etc., etc.–– MapsMaps and and terrainterrain..–– InfrastructureInfrastructure ((sewerssewers, ,

roads) in urban areas.roads) in urban areas.ResearchersResearchers

mdr11

Slide 13

mdr11 For example, in an aircraft there is limited space.Mikael Drugge, 2003-8-25

WhoWho willwill useuse WearablesWearablesin the in the futurefuture??

MedicalMedical workersworkers??FirefightersFirefighters??PolicePolice departmentdepartment??WarehouseWarehouse clerksclerks??RegularRegular peoplepeople??YouYou??

mdr12

Slide 14

mdr12 Medical workers. We mention this later on...

Fire fighters. Mention how it could work like.

Police department. Speculate. WLAN coverage?

Warehouse clerks. "Do we have this in store?"Mikael Drugge, 2003-8-25

TechnologyTechnologyand and GadgetsGadgets

””WhatWhat dodo youyou wantwantto to wearwear todaytoday??””

WhatWhat dodo youyou needneed for a for a wearablewearable computercomputer??

DependsDepends, , butbut thesethese parts parts areare commoncommon1.1. HeadHead--mountedmounted display.display.2.2. Camera Camera recordingrecording viewview..3.3. Audio, Audio, e.ge.g. speaker and . speaker and micmic..4.4. Input Input devicedevice, , e.ge.g. . keyboardkeyboard..5.5. The The computercomputer itselfitself..6.6. Network Network connectionconnection.. 44

11 22

33 5&65&6

HeadHead--mountedmounted Display Display (HMD)(HMD)

Small Small screenscreen, , typicallytypicallycoveringcovering oneone of of youryoureyeseyes..Works like an Works like an ordinaryordinarymonitor, monitor, providingproviding an an image image floatingfloating in the in the air in front of air in front of youyou..LCD or TFT LCD or TFT technologytechnology..–– UsedUsed to be a real CRT in to be a real CRT in

the the oldold daysdays..

HeadHead--mountedmounted Display Display (HMD)(HMD)

Transparent displaysTransparent displays–– AllowsAllows augmentedaugmented realityreality, ,

wherewhere virtualvirtual information information overlapsoverlaps the real the real worldworld..

OpaqueOpaque displaysdisplays–– Less sensitive to the Less sensitive to the

backgroundbackground noisenoise..

StateState of the art.of the art.–– SmallestSmallest..–– AdvancedAdvanced..

CameraCamera

AnyAny small small cameracamera..–– OrdinaryOrdinary webweb cameracamera..–– CustomCustom mademade cameracamera..

SuitableSuitable placementplacement–– HeadHead, , followsfollows useruser’’ss gazegaze..–– ShoulderShoulder, , moremore stablestable..

Input Input devicedevice

KeyboardKeyboard–– CanestaCanesta’’ss IR IR keyboardkeyboard..–– ArmArm--strappedstrapped keyboardkeyboard..–– FrogPadFrogPad..–– TwiddlerTwiddler chordingchording

keyboardkeyboard..

MouseMouse–– TwiddlerTwiddler, , againagain..

MaybeMaybe wewe needneed to to useusesomethingsomething else?else?BrainGateBrainGate

Input Input devicedevice

GesturesGestures–– The GestureThe Gesture pendant pendant -->>

For controlling smart homesFor controlling smart homes–– Fingers for dialing numbersFingers for dialing numbers

VoiceVoice recognitionrecognition–– SuitableSuitable at at timestimes, , butbut not as a not as a

genericgeneric solution for solution for everythingeverything..

MultiMulti--modalmodal interfacesinterfaces–– CombiningCombining severalseveral typestypes of input, of input,

e.ge.g. . voicevoice and and gesturesgestures..

WhatWhat elseelse??

Output Output devicedevice

SightSight –– Visual outputVisual output–– HMD, HMD, wristwrist watchwatch......

Hearing Hearing –– Audio/sound/Audio/sound/speechspeech//musicmusic..–– Speakers, Speakers, earplugearplug//headsetheadset......

TouchTouch –– TactileTactile feedbackfeedback–– Example: The radar Example: The radar vestvest, , allowingallowing a person to a person to

””feelfeel”” objectsobjects in the in the surroundingsurrounding..

TasteTaste and and smellsmell –– Not Not thatthat commoncommon, , yetyet..

The The ComputerComputer itselfitself

AnythingAnything small small butbutpowerfulpowerful enoughenough!!–– LaptopLaptop or or TabletPCTabletPC–– PC104 (a small PC)PC104 (a small PC)–– XybernautXybernaut–– PDA, PDA, iPAQiPAQ, , ToshibaToshiba–– Or Or somethingsomething elseelse......

Network Network connectionconnection

BenefitsBenefits of of havinghaving a a networknetwork–– Access to the Internet.Access to the Internet.–– CommunicationCommunication..

Wireless Wireless networknetworkconnectionconnection–– WaveLANWaveLAN, IEEE802.11b, IEEE802.11b–– GPRS or UMTS (3G)GPRS or UMTS (3G)–– BluetoothBluetooth–– InfraInfra--redred

Putting it all Putting it all togethertogether......

NowNow youyou havehave all all gadgetsgadgets youyou needneed..–– ButBut howhow dodo youyou connectconnect themthem??

WiresWires–– HavingHaving tootoo manymany wires wires cancan becomebecome ratherrather cumbersomecumbersome..–– Solution: Solution: IntegrateIntegrate intointo the normal the normal clothingclothing..

WirelessWireless–– ComfortableComfortable, , butbut easiereasier to to eavesdropeavesdrop on.on.

Body Body networknetwork–– SendSend signals by signals by usingusing youryour human body as a human body as a conduit.conduit.

Stick your finger in your ear for audioStick your finger in your ear for audio–– E.gE.g. a . a TwiddlerTwiddler whichwhich is is onlyonly usableusable whenwhen in in youryour hand.hand.

PoweringPowering the the wearablewearable

PowerPower is a is a significantsignificant problem!problem!–– YouYou dondon’’tt wantwant to drag a to drag a powerpower cablecable

behindbehind youyou......–– All All devicesdevices consumeconsume powerpower..

TradeoffTradeoff betweenbetween functionalityfunctionality and and powerpower..

–– BatteriesBatteries nevernever last last longlong enoughenough..E.gE.g. a . a laptoplaptop cancan runrun ~2~2--4 4 hourshours..The Nobel The Nobel PrizePrize awaitsawaits..

ImportantImportant problem, problem, howhow to to solvesolve it?it?

Human Human poweredpowered devicesdevices

TypicalTypical powerpower consumptionsconsumptions–– DesktopDesktop computercomputer, 100W, 100W–– LaptopLaptop computercomputer, 10W, 10W–– EmbeddedEmbedded CPU, 1WCPU, 1W

Human body Human body usesuses ~120W.~120W.–– WeWe needneed, , letlet’’ss saysay,,

5W for a 5W for a wearablewearable..–– WhatWhat ifif wewe couldcould taptap

intointo thisthis powerpower......

Human Human poweredpowered devicesdevices

Examples of human Examples of human powerpower availabilityavailability–– Body heat, 0.6 Body heat, 0.6 –– 4.8W (4.8W (wetsuitwetsuit clothesclothes))–– BreathBreath, 0.4 , 0.4 –– 2.5W (pressure mask)2.5W (pressure mask)–– BloodBlood pressure, 0.2W (pressure, 0.2W (turbineturbine))–– LimbLimb motion, 0.3 motion, 0.3 –– 1.5W (1.5W (pulleyspulleys))–– Finger motion, 0.019W (Finger motion, 0.019W (keyboardkeyboard typingtyping))–– WalkingWalking, 5 , 5 –– 8W (8W (shoeshoe generator)generator)WalkingWalking is the best is the best methodmethod so far.so far.

CommercialCommercial wearableswearables

WhyWhy buildbuild it it yourselfyourself??Motorola and Motorola and FrogFrog DesignDesign’’ss””OffspringOffspring”” concept.concept.–– No longer makes No longer makes ””mobile phonesmobile phones””

Steve Mann and Siemens Steve Mann and Siemens arearealsoalso workingworking on a on a productproduct lineline..The market is still not The market is still not veryvery bigbig..–– ResemblesResembles the the earlyearly daysdays of of

personal personal computingcomputing..–– MuchMuch dodo--itit--yourselfyourself and and hackinghacking..

Our Our lecturelecture so far...so far...

NowNow youyou shouldshould havehave somesome ideaidea ofof–– WhoWho inventedinvented wearablewearable computingcomputing??–– WhatWhat is a is a wearablewearable computercomputer??–– WhatWhat dodo youyou needneed to to buildbuild oneone??–– WhatWhat technologytechnology existsexists todaytoday??

NextNext bigbig questionquestion......–– WhatWhat cancan it be it be usedused for?for?

Break...Break...

AfterAfter the break: Real the break: Real lifelife applicationsapplications, , research research issuesissues, , enhancingenhancing youryour vision, vision, buyingbuying sofassofas, , augmentedaugmented memorymemory, a , a giantgiant’’ss eyeseyes, , buildingbuilding villages, etc...villages, etc...

ApplicationsApplications andandUseUse in Real Lifein Real Life

””WhatWhat cancan be be donedone, really?, really?””

WhatWhat cancan youyou dodo withwith a a wearablewearable computercomputer??

EverythingEverything a a regularregular computercomputer cancan do.do.HoweverHowever, the , the contextcontext differsdiffers..–– YouYou wearwear it and it and cancan movemove aroundaround..–– The The wearablewearable computercomputer is exposed to is exposed to

different different environmentsenvironments and and contextscontexts..–– Limitations in terms of hardware.Limitations in terms of hardware.–– ThisThis makes a makes a bigbig differencedifference......

ApplicationsApplications

MediatedMediated RealityReality–– ExperiencingExperiencing the the worldworld throughthrough thethe computercomputer..–– AllowsAllows computercomputer to process the to process the sensorysensory cuescues

beforebefore reachingreaching thethe useruser..E.gE.g. block . block commercialcommercial billboardsbillboards..

AugmentedAugmented RealityReality–– OverlayingOverlaying virtualvirtual information on the real information on the real worldworld..

E.gE.g. . allowallow architectsarchitects to to buildbuild virtualvirtual houseshouses..E.gE.g. the AR . the AR QuakeQuake or AR or AR PacmanPacman game.game.

BothBoth realitiesrealities cancan enhanceenhance youryour sensessenses..

Augmented Reality Pac Augmented Reality Pac ManMan

Applications Applications –– Augmented Augmented Memory (1/2)Memory (1/2)

Trivial example, finding Trivial example, finding youryour wayway..–– ””WhereWhere diddid I park I park mymy carcar??””

Camera on Camera on youryour body body recordsrecords the the wayway..ReplayReplay helpshelps youyou findfind youryour wayway back.back.–– OnlyOnly keykey eventsevents needneed to be to be recordedrecorded..–– Example: Example: IntersectionsIntersections at a at a carcar park.park.

Applications Applications –– Augmented Augmented Memory (2/2)Memory (2/2)

ElderlyElderly or or peoplepeople withwith poorpoor memorymemory..–– RememberRemember namename and and faceface of of peoplepeople..

Image Image processingprocessing cancan recognizerecognize a a faceface and and mapmap it to it to the the personperson’’ss namename and and affiliation.affiliation.

–– How should it be presented?How should it be presented?

ApplicationsApplications –– AidingAiding the the VisuallyVisually DisabledDisabled (1/3)(1/3)

SomeSome forms of forms of lowlow vision vision cancan not be not be alleviatedalleviated by by useuse of of ordinaryordinary glassesglasses..Solution?Solution?


UserUser wearswears nonnon--transparent transparent glassesglasseswithwith integratedintegrated displays, displays, experiencesexperiencesthe the worldworld throughthrough a a cameracamera..ComputerComputer processed video processed video streamstream..–– EnhanceEnhance contrastcontrast..–– AdjustAdjust colorscolors..–– NightNight vision.vision.–– EnlargedEnlarged viewview..


FisheyeFisheye lenselense for for reading text.reading text.

RemappingRemapping aroundaroundblind blind spotsspots..

ApplicationsApplications –– AdditionalAdditionalVision Tricks Vision Tricks (1/2)(1/2)

””EdgertonianEdgertonian”” eyeseyes–– FreezeFreeze--frameframe effecteffect, fast , fast shuttershutter..–– ThisThis enablesenables

Reading text on a Reading text on a tiretire of a of a speedingspeeding carcar..ClearlyClearly seeingseeing the rotor the rotor bladesblades of a of a helicopterhelicopter..CountingCounting the the numbernumber of of boltsbolts holdingholding an an airplaneairplane rotor rotor togethertogether in in midmid--airair..Plus lots of Plus lots of otherother interestinginteresting effectseffects..

mdr4

Slide 41

mdr4 In addition to the more down-to-earth uses for enhanced vision by use of video processing...

"Edgertonian eyes", like going to Kåren and experience stroboscopic lights, sort of... But with another use for it.Mikael Drugge, 2003-8-25

ApplicationsApplications –– AdditionalAdditionalVision Tricks Vision Tricks (2/2)(2/2)

GiantGiant’’ss eyeseyes..–– EnhancesEnhances depthdepth perception of distant perception of distant objectsobjects..

mdr6

Slide 42

mdr6 blablabla

But not very practical to use in everyday life,I guess...Mikael Drugge, 2003-8-25

ApplicationsApplications –– AA--LifeLife, , AvalancheAvalanche RescueRescue (1/2)(1/2)

HelpHelp rescuerescue avalancheavalanche victimsvictims..SurvivalSurvival chancechance–– 92% 92% afterafter 15 15 minutesminutes..–– 30% 30% afterafter 35 35 minutesminutes..Time is not the Time is not the onlyonly factorfactor–– OrientationOrientation, , headhead up or up or downdown..–– Air pockets, air Air pockets, air channelschannels..HowHow cancan wearableswearables helphelp rescuersrescuers??

mdr19

Slide 43

mdr19 Say something about small town villages, how they differ compared to the Big Cities. Advantages...

The scary situations, assault, robbery,triggered by non-natural readings.

This kind of "safety vest" exists today.Mikael Drugge, 2003-8-25

ApplicationsApplications –– AA--LifeLife, , AvalancheAvalanche RescueRescue (2/2)(2/2)

EachEach victimvictim wearswears a a small sensor.small sensor.–– TiltTilt sensorsensor–– HeartHeart raterate–– BloodBlood/oxygen /oxygen saturationsaturation

iPAQiPAQ gets gets readingsreadings..–– Sorts and Sorts and prioritizesprioritizes..

RescuersRescuers get get adviceadvice on on wherewhere to start to start diggingdigging..–– IncreasedIncreased survivalsurvival..

mdr20

Slide 44




Applications Applications –– Social Social softwaresoftwareUsually designed for urban settings. Usually designed for urban settings.

Interface to groups or individuals.Interface to groups or individuals.Safety netSafety netFriend finderFriend finderFamiliar StrangersFamiliar Strangers

ApplicationsApplications –– SafetySafety NetNet, , Creating Villages in the CityCreating Villages in the City

Social Social networknetwork of of wearablewearable usersusers..Biosensors monitor Biosensors monitor useruser’’ss body.body.–– HeartHeart raterate, , perspirationperspiration, , breathbreath raterate..–– Alert Alert friendsfriends in in casecase of of abnormalabnormal valuesvalues..

ExampleExample–– Start sending video in Start sending video in scaryscary situations.situations.

SuchSuch ””safetysafety vestsvests”” existexist alreadyalready..

mdr7

Slide 46




Applications Applications –– Friend Friend finder (1/3)finder (1/3)

Connects people with common interests.Connects people with common interests.Works for you automatically while you walk Works for you automatically while you walk around and meet people.around and meet people.Imagine you want to buy a sofaImagine you want to buy a sofa……–– Look at sellers on a bulletin board?Look at sellers on a bulletin board?

Can you really trust those people?Can you really trust those people?–– From a friend of yours?From a friend of yours?

Trustworthy, but may not have a sofa.Trustworthy, but may not have a sofa.–– From a friend of a friend of a friend?From a friend of a friend of a friend?

Well, thatWell, that’’s an awful lot of persons to asks an awful lot of persons to ask…… So how do So how do you solve that?you solve that?

Applications Applications –– Network of Network of Buyers and Sellers (2/3)Buyers and Sellers (2/3)

Use wearable computers!Use wearable computers!Buyer announces, electronicallyBuyer announces, electronically–– ““I want to buy a sofa.I want to buy a sofa.””

Someone else announcesSomeone else announces–– ““I have a sofa to sell.I have a sofa to sell.””

When meeting with a friend, you When meeting with a friend, you exchange your shopping lists.exchange your shopping lists.

Applications Applications –– Network of Network of Buyers and Sellers (3/3)Buyers and Sellers (3/3)

X wants a sofa!

X wants a sofa!

Z sells a sofa

X can now buythe sofa from Z

Familiar strangers Familiar strangers

Who are familiar strangers?Who are familiar strangers?–– People that People that ““knowknow”” each other from a each other from a

common location but that choose to common location but that choose to ignore each other.ignore each other.

Information about familiar strangers Information about familiar strangers may help us gain information about may help us gain information about our surroundings.our surroundings.

ConclusionsConclusions andandSummarySummary

””WhatWhat havehave wewe learntlearnt todaytoday??””

ConclusionsConclusions and and SummarySummary

WearableWearable computingcomputing–– InventedInvented by Steve Mann in the 1970s.by Steve Mann in the 1970s.

TechnologyTechnology–– HMD, I/O HMD, I/O devicesdevices, , networksnetworks, , powerpower, etc., etc.

ApplicationsApplications–– AugmentedAugmented memorymemory..–– MediatedMediated realityreality..–– EnhancedEnhanced sensessenses..–– AvalancheAvalanche rescuerescue..–– Market Market networknetwork..–– HealthHealth--carecare..

Research Research issuesissues–– Interaction, Interaction, mergingmerging withwith pervasivepervasive computingcomputing..–– MuchMuch moremore remainsremains......

Documents

Wearable Computing Past, Present and Future