Upload
arpan-pal
View
24
Download
0
Embed Size (px)
Citation preview
Experience certainty.
Copyright © 2011 Tata Consultancy Services Limited
Signal Processing, Communication and Computing Aspects of Internet-of-Things - Research Challenges
Arpan PalHead of ResearchInnovation Lab, Kolkata
3 Experience certainty.
Human-in-Loop Cyber-physical Systems
Humans
Physical Objects
and Infrastruct
ure
Computing Infrastruct
ure
Perso
nal
Conte
xt
Disco
very
PhysicalContext Discovery
4 Experience certainty.
Signal
Processing
Internet-of-Things - towards Intelligent Infrastructure
Sense
Extract
Analyze
Respond
Learn
Monitor
IntelligentInfra
@Home
@Building
@Vehicle@Utility
@Mobile
@Store
@Road
“Intelligent” (Cyber) “Infrastructure” (Physical)
APPLICATION SERVICES
BACK-END PLATFORM
INTERNET
GATEWAY
Internet-of-Things (IoT) Framework
Sense
Extract
Analyze
Respond
Communication
Computing
5
Integrated Platform for Intelligent Infrastructure
People Feedback & Emotions
Social Media
Integrated Services
Sensors & IoTPlatform
Traditional Monitoring & Control Systems Citizen Data
Smart Integration Platform
Transportation Healthcare Electricity
WaterPublic Safety Tourism
Smart Integrated Services
Sense
Analyze
Extract
Respond
Intelligence
Smart Domain Services
Community
etc.
Sense: People Activity, Appliances, Vehicles , Road, Home/Bldg, Utility Infrastructure
Detect gas leakage/water contamination : mobilize rescue team, suggest optimum route
Divert Road Traffic in case of Water Pipeline Burst
Correlate Electricity/Water /Gas consumption patterns
Intelligent Integration Platform
Integrated Intelligent Services
RIPSAC – Real-time Integrated Platform Services & Analytics for Cyber-physical Systems
6
Context Discovery
Physical Context Discovery(What is happening where and when)
• Localization and Spatiotemporal data fusion
• Sensor Informatics (Noise Cleaning, Disaggregation, Feature Extraction, Machine Learning)
• Semantic Interop. and Analytics of Sensor Data – Semantic Sensor Web
• Model driven Analytics – CPS modeling
• New modes of Sensing (5 senses computing)
• New Platforms for Sensing (UAV, Mobile Robots, Mobile Phones – Participatory Sensing)
7
Context Discovery
Human Context Discovery(Who is doing what, where and when, who is thinking what)
- Identity, Location, Activity, Physiology, Psychology
• Unobtrusive Sensing (Mobile phones and Surveillance cameras – Kinect)
• Non-invasive Physiological Sensing
• Psychology sensing via EEG
• Model driven Analytics - Human behavior modeling
• Sensing from Social Networks
8
Challenges
Signal Processing• Preserving the Battery power of edge devices• Extracting information from ambient-noise-corrupted sensor data
Low-power computing algorithms Adaptive Signal Processing for Ambient Noise Removal
Communication• Reducing the cost of Communication while preserving reliability
and security Secure, Lightweight yet Reliable communication protocols over
Internet• Preserving the Privacy
Quantifying Privacy vs. Utility measures
Computing• How to do big data analytics in Cloud• Distributed Computing - Bringing Edge into the Grid• Semantic Model of Physical World and Human World – Human-in-
loop CPS
10
Ubiquitous Healthcare – Elderly / Chronic Patient Monitoring
ECG
Blood PressureMonitor
Pulse OxyMeter
Healthcar
e Portal
Mobile phone as medical
gateway
Web Request
PatientRecords
Health Center / Home
Expert Doctor
Social Network
“Sensor Observation Service based Medical Instrument Integration”, SMART 2012
Requirement from NUH (Peritonial Dialysis) and SingHealth (Elderly Care)
11
Ubiquitous Healthcare – Research on Frugality and Usability
Healthcar
e Portal
Mobile phone as medical Sensor and
Gateway
Web Request
PatientRecords
Health Center / Home
Expert Doctor• Replace Medical Sensors with Mobile Phone
Sensing• Activity and Localization using Mobile Phones• Activity and Identification using Kinect Camera• Social Media as a Soft sensor• Multimodal Fusion
Social NetworkIdentification Localization
Activity DetectionPhysiological Sensing
“Mobile Healthcare Infrastructure for Home & Small Clinic”, Mobilehealth @ Mobihoc 2102
12
PPG based Pulse Measurement using Phone Camera
Subject1 Subject2 Subject3
Actual Detected Actual Detected Actual Detected
68 66 66 63 85 84
2.9% 4.5% 1.1%
Sources of noise:i. improper finger
placementii. imparting
excessive pressureiii. finger movement
Challenge: too much noise
13
PPG based Pulse – Proposed Robust Algorithm
Rejection FSM– Detect onset of good signal– Continuous consistency check– Reject unusable input data
o Feedback: notify user– Once enough signal received,
perform FFT
“A Robust Heart Rate Detection using Smart-phone Video”. MobileHealth @ Mobihoc 2013
14
Blood Pressure and ECG Monitoring from PPG
Extract PPG Features
BP Ground Truth Create BP Model
Extract PPG Features
Predict BP levels
(SP, DP, PP)
Extract ECG Features
Create ECG Parameters’ Model
Predict ECG Parameters
ECG Parameters
BP levels(SP, DP, PP)
Training Phase
Testing Phase
Data set Pd Ps PP-diff < 15
Standard dataset (14 features) 92.9% 74.7% 77.9%
TCS dataset - add height, weight, age
99.3% 82.7% 85.5%
BP Level
Pd Ps
Very Low < 50 < 70
Low 50-65 70-100
Normal 65-90 100-135
High 90-100
135-160
Very High
> 100 >160
“Estimation of Blood Pressure Levels from Reflective Photoplethysmograph using Smart Phones” – IEEE BIBE 2013“Estimation of ECG Parameters using Photoplethysmography” – IEEE BIBE 2013“HeartSense – Estimating Blood Pressure and ECG from Photoplethysmograph using Smart Phones” – Demo @ Sensys2013
15
Mobile Phone based Activity Detection for Wellness
Activity Detection– Uses Accelerometer Data– Gyroscope and Magnetometer for orientation
correction– Step Count, Stride Length Estimation– Walking, Brisk Walking, Running
Classification
Peak Detection and Step Validation using IPA;Calculating Step cycle lengths for all valid steps in the window
Classification of window activity using step frequencies derived from step cycle lengths
16
Mobile Phone based Activity Detection for Wellness
Noise Cancellation and pre-processingCalorie Count from Step Count and Type of Activity
UbiHeld - Ubiquitous Healthcare Monitoring System for Elderly and Chronic Patient”, Recognize2Interact @ UbiComp 2013
17
Mobile Phone based Indoor Localization – Geo Fencing and Proximity
Indoor Localization– Initial Referencing through GPS and Magnetometer– Inertial Navigation through accelerometer, gyro,
magnetometer – Improved accuracy through Stride Length, Kalman Filter /
Particle Filter– Tracking of non-smartphones via Bluetooth– Augmentation with Wi-Fi Triangulation
Location ID LOC I LOC II LOC III
Actual (ft) 2 4 6
Estimated (ft) 1.95 4.16 6.27
% Error 2.5 4.12 4.4
“BlueEye A System for Proximity Detection Using Bluetooth on Mobile Phones”, PUCAA @ �UbiComp 2013
Geo-fencing through Magnetometer
Proximity Sensing via Bluetooth
Based on RSSI Done by capturing RSSI in android in respect to
another phone Able to model the constants in equation as a
function of distance in given environment
18
Inertial Navigation– Step Count + Stride Length (personalized model)– Gyroscope and Magnetometer-corrected Inertial Navigation
Wi-Fi based Triangulation– Based on RSSI of known location of 3 or more access points– Attenuation modeling of the building
Fusion, Tracking and Correction– Kalman Filter based Tracking– Particle Filter based Correction
Mobile Phone based Indoor Localization – Inertial and Wi-Fi
Other Requirements• Colleague Finder in Large Offices• Shopper Localization in Retail Stores• Emergency Evacuation in Large Buildings
19
Kinect Based Human Identification
Human Identification– Skeleton Model Based / Depth
based– 20 joints of skeleton data for a
person captured at 30 frames per sec in side way walking pattern
• 2D Camera with IR depth sensor
• Excitation by IR light pattern
• Directional Mic.
• Human Identification • Gait cycle detection• Feature extraction from
skeleton joints• Training• Recognition
“Pose Based Person Identification Using Kinect”, IEEE SMC 2013“Stabilization of Cluster Centers over Fuzziness Control Parameter in Component-wise Fuzzy C-means Clustering”, Fuzz IEEE 2013“Feature Selection by Differential Evolution Algorithm - A Case Study in Personnel Identification”, CEC 2013
20
Kinect Based Activity Detection
H H NH NH
Human and non-Human Classification
“Human Localization from Kinect Captured Data for Activity Recognition at Home”, HomeSys @ Ubicomp 2013
21
Social Media as a Soft-sensor for Healthcare
Support Community Discovery– Use hidden community detection by applying
NLP on posts to create the social graph to identify the undeclared community for a given disease
Disease onset discovery like dementia or Alzheimer's disease or psychological disorders from social network posts– Search for patterns in posts to detect possible
symptoms to diseases– E.g. - sentiment analysis on posts will give
whether the given post’s emotion is positive or negative. If the emotions are cycling between positive and negative extremes with some periodicity, probably the person has bipolar disorder
“Using Social Network Graphs for Search Space Reduction in Internet of Things”, Ubicomp 2013
Prototyping in TCS own internal social network platform - Knome
23
Cognitive Load on Human Brain
Add in your mind:
23+45=?
1846890129 + 2374609823=?
How to Measure Cognitive Load
User Study Can be biased and it is Indirect
measurement
Biological Response Unbiased & more reliable method
ECG Pupil diameter Skin conductivity, other EEG (this is a more direct way as it
measures brain activity directly) Cheaper that fMRI, PET and other brain
activity measurement means
Application Personalize education based on
real-time measurement of one’s cognition state through EEG signals
Possibly a better measure for testing understanding of a subject during a course during taking a test
Getting unbiased feedback from subject on user interface design
Stress during critical operation like ATC.
24
Two class Cognitive Load – User Experience Testing
• People are given two different types of Onscreen Keyboards to use (one with easy and other is more complex to use for text entry)
• Corresponding EEG is recorded ((14 channel Emotiv).
Results shows clear classification of these two cases based on measuring cognition while subject is using a particular keyboard
Constraints: • Subject training required• Only two level classification is addressed
“Evaluation of Different Onscreen Keyboard Layouts using EEG signals”, SMC 2013“Unsupervised Approach for Measurement of Cognitive Load using EEG Signals”, BIBE 2013
25
Multi class Cognitive Load – Task Difficulty Testing
• People are given to analyze pseudo codes of three different difficulty levels (Low, Medium and High).
• EEG response is recorded (14 channel emotiv device).
Results shows clear formation of clusters each for corresponding task difficulty level with some overlaps.
Constrains: Multi class approach ( no continuous score ) Subjects are heavily constrained while they work out
Easy Task Difficult Task
“EEG-Based Fuzzy Cognitive Load Classification”, Fuzz IEEE 2013
27 Experience certainty.
5 senses Computing - 3D Reconstruction with 2D images from mobiles
• Low cost solution for 3D reconstruction from multiple 2D images captured from mobile device.
• Derive the motion information from the inbuilt sensors of the mobile phone and then aid in increasing the accuracy of the 3D reconstruction.
• Support heterogeneous and homogeneous objects • Future research focus on the multi-modal fusion of the 3D information
and the other sensors’ data for a given object or environment
Applications• Agro-advisory Service• Remote Diagnostics of Machines• Remote Healthcare
29
Communication - Constrained Application Protocol (CoAP)
– Provides RESTful web interface suitable for constrained devices
– Ideally to run on unreliable transport (e.g., UDP)– Confirmable (CON) mode for optional reliability with an
ACK feed backo Retransmissions ensure best-effort delivery
– Non-confirmable (NON) mode for unreliable deliveryo No ACK – no retransmission
– Supports HTTP like request/response– Supports resource-observe
o A variant of publish/subscribeo Useful for real-time updates
– Supports both unicast and multicast
30 Experience certainty.
CoAP vs. HTTP
Constrained Object Access Protocol (CoAP) Improved version of CoAP using dynamic network condition sensing Lightweight security protocols sensor authentication and data delivery
Use suitable lightweight application protocol between edge devices and core network
• http://people.inf.ethz.ch/mkovatsc/californium.php• Ralf Koetter, Muriel Medard, 2003 IEEE/ACM transaction http://web.mit.edu/medard/www/NWCFINAL.pdf• Bandyopadhyay, S. and Bhattacharyya, A. Lightweight Internet protocols for web enablement of sensors using constrained gateway devices. In Proc.
International Conference on Computing, Networking and Communications (ICNC), 2013, San Diego, CA, IEEE(2013), 334 – 340
31
Fog Computing – the Grid with an Edge
• Flavio Bonomi et.al. MCC2012, Helsinki, Finland
Intelligent Systems - Intelligence comes from Analytics Need for crunching huge amount of sensor data and respond in real-
time Needs large computing infrastructure in cloud Another option is to distribute computing load to the edge devices
Edge Devices computing power remain unused most of the timeFree Computing resource for the gridPotentially millions of ~1GHz Processors on the grid depending upon use case
Energy cost at edge is typically at consumer rates << Energy cost at cloud which is at Enterprise ratesEnergy cost account for 50% of Data Center Opex
32
Fog Computing - Solution Approach
• Agent-based grid Computing using CONDOR• Need for agents in diverse types of edge devices via a common
framework
• Min-Jen Tsai, ,Yuan-Fu Luo , Expert Systems with Applications, Volume 36, Issue 7, Sept. 2009, Elsevier
33
Privacy – Smart Meter Data
Activity monitoring Advantage: Personalized services and recommendation like theft
detection, elderly monitoring (University of Virginia’s ALARMNET, Harvard’s CodeBlue)
Privacy issue: Leads to private data (smart meter data) leakage
[1] www.winlab.rutgers.edu/~gruteser/papers/fp023-roufPS.pdf
[1]
34 Experience certainty.
Privacy Preservation
• Information-theoretic approach for sensitivity analysis of sensor data• Requirement based application utility measurement• Balancing of Privacy vs. Utility
Privacy
Utility
Privacy Preservati
on Tool
35 Experience certainty.
Results: Sensitivity Analysis and Detection
[5] J. Zico Kolter and Matthew J. Johnson, "REDD: A public data set for energy disaggregation research," SustKDD, 2011.
[5]
36 Experience certainty.
Semantic Query and Analysis on Spatiotemporal Sensor Data
Analytics Engine
Time Series
Database
RIPSAC
Sensor Manufacturer
Sensor KnowledgeDatabase
(Domain and Resource Ontology)
Provision Sensors & Actuators
Algorithm Catalog
Algorithm(s) Selection &Execution
Raw Sensor Data
App Developer
Query on Sensor Property & Capability
User
Algorithm DiscoveryRegister Sensor
Specification Metadata
Sensor Installer
Sensors & Actuators
Algorithm provider
Concept Developer
Algorithm Registration
Algorithm Instantiation
38
Innovation@TCS - Innovation Labs
Bangalore, India1
TCS Innovation Labs - Bangalore
Chennai, India2
TCS Innovation Labs - ChennaiTCS Innovation Labs - RetailTCS Innovation Labs - Travel & HospitalityTCS Innovation Labs - InsuranceTCS Innovation Labs - Web 2.0TCS Innovation Labs - Telecom
Cincinnati, USA3
TCS Innovation Labs - Cincinnati
Delhi, India4
TCS Innovation Labs - Delhi
Hyderabad, India5
TCS Innovation Labs - HyderabadTCS Innovation Labs - CMC
Kolkata, India6
TCS Innovation Labs - Kolkata
Mumbai, India7
TCS Innovation Labs - MumbaiTCS Innovation Labs - Performance Engineering
Peterborough, UK8
TCS Innovation Labs - Peterborough
Pune, India9
TCS Innovation Labs - TRDDC - Process EngineeringTCS Innovation Labs - TRDDC - Software EngineeringTCS Innovation Labs - TRDDC - Systems ResearchTCS Innovation Labs - Engineering & Industrial Services
1 2
3
4
597
6
8
2000+
Associates in Research, Development and Asset Creation
19 Innovation Labs
39
Academic Co-Innovation Network (COIN )
Fostering joint research and innovation through a mutually beneficial alliance between TCS and academia
Academic context
Thoughts and research towards disruptive InnovationKnowledge exchange and people development
Industry-oriented Business context
innovation scalability of academia context of real-world problems
Collaborativeresearch
environment
Collaboration Mechanisms•MoU based Alliances•Sabbaticals – Academia to TCS Innovation Lab and TCS Innovation Lab to Academia•TCS Research Scholar Program•Masters and PhD Internships
Joint publications and IPRs
40
Innovation Lab Kolkata, at-a-Glance
Research Areas• Sensor Signal Processing• 2D/ 3D Image / Video
Processing • Protocols, Security and
Privacy• Parallel and Distributed
Computing• Stream Processing and
Reasoning• System Modeling and
Identification• Sematic Sensor Web• Social Media Analytics
Academic Collaborations• Singapore Management University (iCity
Platform)• Indian Statistical Institute
(Protocol/Privacy/Security, Image / Video Processing)
• IIT Kharagpur (Analytics, Semantic Processing)
• IIT Bombay (Energy and Utilities)• Jadavpur University (Signal Processing)
Higher Studies
• PhD - 4• Masters - 4
Total Researchers - 37