Embed Size (px)
Citation preview
1
Arpan Pal
Innovation Lab, TCS
May 17, 2015
Context-aware Mobile Phones - towards Ubiquity
Agenda
• Ubiquity and Internet-of-Things• Mobile Phone based Sensing• Smart Solutions using Mobile Phone Sensors• Ongoing Work
Ubiquity and Internet-of-Things
4
Ubiquitous Computing
“Ubiquitous computing enhances computer use by making many computers available throughout the physical environment, but making them effectively invisible to the user”
At present Ubiquity is viewed as a Consumer phenomenon – However widespread adoption of ubiquitous devices among Enterprise stakeholders will drive Enterprises to ubiquity.
5
Smart Solutions using Ubiquity
• Smartness comes from Context-awareness• Context-awareness comes from
• Sensing one’s environment (anything)• Analyzing the sensed data to make meaningful inferences• Actuate / Render value-added information from the inferred analytics
• anytime, anywhere (on Mobile, TV, PC, Tablet)
• Ubiquity CycleSense -> Analyze -> Actuate
• Need to do all the above while preserving the privacy
• Need for easy-to-use HMI to make technology hidden to background
6
Internet-of-Things (IoT) – towards Ubiquity
Internet
Sensors (Location, Audio , Camera, …)
Gateway (Stand-alone Device / Mobile Phones)
Backend Server & database
Portal
USB, CAN, Zigbee, BT, NFC, WiFi
GSM/GPRS, 3G
BAN
KIN
GB
AN
KIN
G
INSU
RA
NCE
INSU
RAN
CE
AG
RIC
ULT
UR
EA
GRI
CULT
URE
HEA
LTH
CARE
HEA
LTH
CAR
E
GO
VER
NM
ENT
GO
VER
NM
ENT
UTI
LITY
UTI
LITY
MA
NU
FACT
URI
NG
MA
NU
FACT
URI
NG
TRA
NSP
ORT
TRA
NSP
OR
T
APPLICATION SERVICESAPPLICATION SERVICES
INFRASTRUCTURE PLATFORMINFRASTRUCTURE PLATFORM
INTERNET INTERNET
GATEWAYGATEWAY
ACT
UA
TEA
CTU
ATE
SEN
SESE
NSE
AN
ALY
ZEA
NA
LYZE
Mobile Phone based Sensing
8
Mobile Phones as Sensor and Gateway
Smart Phones
InternetWiFi, 2G/GPRS, 3G
On-board sensorsAccelerometer, GPS,
CompassCamera, Microphone
Internet
Sensors (Location, Audio , Camera, …)
Gateway (Stand-alone Device / Mobile Phones)
Backend Server & database
Portal
USB, CAN, Zigbee, BT, NFC, WiFi
GSM/GPRS, 3G
Other sensor interfaces through
Bluetooth and USB
9
Mobile Phones as Sensor and Gateway
Basic and Feature PhonesInternet2G/GPRS
On-board sensorsMicrophone, Camera
Internet
Sensors (Location, Audio , Camera, …)
Gateway (Stand-alone Device / Mobile Phones)
Backend Server & database
Portal
USB, CAN, Zigbee, BT, NFC, WiFi
GSM/GPRS, 3G
10
Sensor Penetration in Indian Mobile Phones
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
Bluetooth
USB
Edge
GPRS
Wifi
3G
Camera
GPS
Accelarometer
Digital Compass
Basic Phones
% Penetration for sensors and network interfaces
x100
11
Sensor Penetration in Indian Mobile Phones
% Penetration for sensors and network interfaces
0 0.2 0.4 0.6 0.8 1
Bluetooth
USB
Edge
GPRS
Wifi
3G
Camera
GPS
Accelarometer
Digital Compass
Feature Phones
x100
12
Sensor Penetration in Indian Mobile Phones
% Penetration for sensors and network interfaces
0 0.2 0.4 0.6 0.8 1
Bluetooth
USB
Edge
GPRS
Wifi
3G
Camera
GPS
Accelarometer
Digital Compass
Smart Phones
x100
13
Sensor Penetration in Indian Mobile Phones
% Penetration for sensors and network interfaces
0 20 40 60 80 100
Bluetooth
USB
Edge
GPRS
Wifi
3G
Camera
GPS
Accelarometer
Digital Compass
Consolidated Market Penetration
14
Power Consumption using Sensors
Source: Nericell: Rich Monitoring of of Road and Traffic Conditions using Mobile SmartphonesPrashant Mohan et. al., Microsoft Research, SenSys 2008, North Carolina, USA
Smart Solutions using Mobile Phone Sensors
16
Pothole Detection using Accelerometer
“The Pothole Patrol: Using a Mobile Sensor Network for Road Surface Monitoring”Jakob Ericsson et.al., MIT Labs, MobiSys 2008, Colorado, USA
17
Audio Sensing using Microphone
“SoundSense: Scalable Sound Sensing for People-centric Applications on Mobile Phones”Hong Lu et.al., Dartmouth College, MobiSys 2009, Krakow, Poland
Applications• Daily Diary• Music Classification• Honk Detection
18
Bike Sensing using external Sensors
“The BikeNet Mobile Sensing System for Cyclist Experience Mapping”Eisenman et.al., Columbia Univ, SenSys 2007, Sydney, Australlia
Applications• Fitness Sensing• Cyclist Experience• Pollution Monitoring
Ultrasound based Tracking in Smart Stores
• Ultrasound beacon transmitters used for location marks / landmarks in a retail store
• Registered Customers carrying Mobile phones can receive these beacons as a record of visiting the designated location
• The visit records are uploaded for loyalty discounts, behavior analysis etc.• The audio frequency used is in the in-audible regions around 20 KHz. • A wide range of cell phones can support this mode of communication and can
enable location sensing in smart spaces
20
Context-aware TV using Microphone and Camera
QR Code
OCR
Ultrasound Internet
Applications• Delivering TV context-aware information and ads to mobile phones / tablets
21
OCR using Mobile Phone Camera Data
“Shah, P.; Karamchandani, S.; Nadkar, T.; Gulechha, N.; Koli, K.; Lad, K.; , "OCR-based chassis-number recognition using artificial neural networks," Vehicular Electronics and Safety (ICVES), 2009 IEEE International Conference on , vol., no., pp.31-34, 11-12 Nov. 2009
Applications• Automatic scan of Car Identifier Number for Insurance Application
Edge Detection
Horizontal Segmentation
Vertical Segmentation
22
Eye Tracking using Camera
“EyePhone: Activating Mobile Phones with Your Eyes”Emiliano Mullizo et.al., Dartmouth College, MobiHeld 2010, New Delhi, India
Applications• Novel Mobile PhoneHMI
23
Brain-Computer Interaction
“NeuroPhone: Brain-Mobile Phone Interface using a Wireless EEG Headset”Campbell et.al., Dartmouth College, MobiHeld 2010, New Delhi, India
Applications• Novel Mobile Phone HMI
Ongoing Work
25
Use Case Focus
Internet of ThingsSmart Home --> Smart Car --> Smart CityBA
NKI
NG
BA
NKI
NG
INSU
RA
NCE
INSU
RAN
CE
AG
RIC
ULT
UR
EA
GRI
CULT
URE
HEA
LTH
CARE
HEA
LTH
CAR
E
GO
VER
NM
ENT
GO
VER
NM
ENT
UTI
LITY
UTI
LITY
MA
NU
FACT
URI
NG
MA
NU
FACT
URI
NG
TRA
NSP
ORT
TRA
NSP
OR
T
APPLICATION SERVICESAPPLICATION SERVICES
INFRASTRUCTURE PLATFORMINFRASTRUCTURE PLATFORM
INTERNET INTERNET
GATEWAYGATEWAY
ACT
UA
TEA
CTU
ATE
SEN
SESE
NSE
AN
ALY
ZEA
NA
LYZE
Smart City
Smart Home
Smart Buildings
Smart VehicleSmart Infra
ParticipatorySensing
Smart Stores
26
Application Focus
App Usage
Pothole / Traffic Jam Detection Smart City Municipal Service / Intelligent Transportation System
Driver Behavior Rogue Driver Detection / Dynamic Insurance
Ultrasound-based localization Smart Tourism / Smart Store
Audio Surveillance using Mobile Phones / Video Surveillance
Law Enforcement
Context-aware Advertisement Target Advertisement delivered to User
Multimodal HMI (Audio and Gesture)
Rich and easy-to-use User Interfaces
27
Platform Focus
Component Purpose
Phone Middleware Java Based Service Container and Device Management
Phone Analytics Statistical and Analytical Processing of Sensor Data – low footprint, low computation power
Privacy Privacy preservation through aggregation, anonymization and access control
Back-end Platform Device Management, Control and Sensor Data Storage
Back-end Analytics Machine Learning, Reasoning, Stream Processing and Large data Handling
