http://mobilityresearchcenter.org © 2009 Carnegie Mellon CyLab. All rights reserved.

ARCHITECTURE

Intelligent Meeting OrganizerArrange Meetings QuicklyAlejandro Rivera, Kathleen Yang, Neha Pattan, Martin Griss

PROBLEM• Meeting scheduling

– with multiple participants• Decision factors

– Changing participants schedules– Optimal meeting venue– Inflexible room resources (video, teleconference) – Various personal preferences

SOLUTION• Virtual Secretary

– Learns your personal preferences– Manages your daily agenda– Negotiates meetings for you– Re-schedules meetings– Knows your location, recommends a suitable venue– Reminds you of scheduled meetings

• Users:– Doctors, Patients, Anyone!

Mobile ClientPhilosophy:

• “Request it and forget it”

Advantages:

• Always with you wherever you go

• Offline capabilities

• Easy to useFunctionality:

• Location-aware: GPS & GSM Cell ID

• Offline capabilities

Environment:

• Cell phoneXMPP

HTTP

SMS

Communication protocols

SecretaryPhilosophy:

• “Arrange it and notify user”Environment:

• Personal Computer (laptop)

• Company’s central serverAdvantages:

• Computing power

• Access to personal data

• Privacy: “Don’t share personal information”Functionality:

• Communicates with other secretaries

• Negotiate meetings

• Tracks scheduled meetings

STATUS• Arrange meetings with multiple participants• Get meeting parameters from user and mobile

sensors• Negotiate meetings between secretaries based on

users agenda (Google Services)

TECHNOLOGIES• Mobile client:

– Python for Symbian S60– XMPP / HTTP / SMS– Google Maps service

• Server side:– Java Agents Development

Framework (JADE)– Google Calendar/Contacts

services– XMPP

Secretary Negotiation

User Virtual Secretary Calendar

NEXT STEPS• Add indoor location capabilities.• Indicate attendees location on a map prior to the

meeting.• Predict user behaviors based on historic data• Introduce advanced user preferences that will feed

the rule system used by the secretary to negotiate meetings.

• Create different types of secretaries and behaviors to fit different types of users.

Sponsors: Cylab, Nokia

© 2009 Carnegie Mellon CyLab. All rights reserved.http://mobilityresearchcenter.org

Usable Security and Privacy for Context-Aware Mobile Applications

How the medical world can benefit?

Diwakar Goel, Eisha Kher, Shriya Joag , Martin Griss, Anind Dey

GOAL

Improve end-users’ ability to privately and securely interact with their environment and data while mobile.

PROBLEM

METHODOLOGY

Camera in conjunction with GPS and cell-id :

1.User travels to a region.2.Receives a QR Code valid for a limited time based on his location.3.Uses this token along with QR codes (static or dynamic) at service site.

• To access a region• To access sensitive information

4.Authenticates (event registered with server)• Logged on the server

. Expires code as per lifetime

CHALLENGES

• Manual focus required• Move processing to server?

• Have to pause to capture• Varying phone screen resolutions

BENEFITS

• No other hardware needed• Camera available on most phones• Screens are everywhere!

• Inexpensive to generate• Can be changed on the fly• Revoking access easier• Easier for administration

• Minimizes human errors in security• One time setup!

Interacting with a smart environment challenges users: • how do you provide input? • how do you perceive output?• what can the system do for you?• how will it do it for you?…

Mobile users face particular challenges as they move:• realizing differences between the environments• discerning what they can and cannot do and what data they may have access to

SOLUTION

Making the various sensors on the phone and in the environment:• mutually negotiate and authenticate each other• to access services, gain privileges and take implicit actions.

•Accelerometer•Camera•Microphone•GPS

•Bluetooth•Wi-Fi•GPRS/3G/EDGE•Cell-ID

•Battery•IMEI•Memory•SW Version•IMEI

Calendar, Email, Call / SMS history

© 2009 Carnegie Mellon CyLab. All rights reserved.http://mobilityresearchcenter.org

IMSafeLeveraging IMS Innovation™ and Android through a Mobile Health Application

Alain Kajangwe, Chandani Desai, Diwakar Goel, Patrick Maniraho Kristoffer Gronowski, Martin Svensson

PROBLEMHow do you:

Constantly stay aware of medical facilities nearby? Communicate crucial medical information to paramedics when unconscious? Effectively locate the closest person in your safety net? Get information from other IP devices when you cannot

access your phone? Keep track of your medication and lab appointments?

SOLUTION

Intelligent location-based services using Google Maps Securely store crucial medical information directly on your phone for ease of access Store – forward – broadcast presence information through IMS Innovation servers Multimedia synchronization through IMS

IP Multimedia Subsystem Not Just Text Powerful ways of forking Being adopted by mobile operators Presence notification Supports subscribe – notify (push) Better security architecture Messages can have different priority levels XDMS (XML Document Management Server) XCAP (XML Configuration Access Protocol)

IMS:fills the gap

ANDROID

Open Source Interesting libraries and constant updates to APIs Designed to work with next-generation mobile technologies Easy interfacing with 3rd party sensors and devices

FEATURES

FUTURE WORK:

Real time inventory status from pharmacies nearby Dedicated device to trigger emergency actions Interface additional condition sensors with to phone Use sensors and robust communication technologies for diagnosis

Enhance native contact list with IMS presence Fallback to other communication mechanisms e.g. SMS Synchronization with other multimedia devices Emergency actions Medication and lab appointment scheduler Medicine inventory management

http://mlt.sv.cmu.edu © 2009 Carnegie Mellon CyLab. All rights reserved.

Mobile Natural Language Technologies

Joy Ying Zhang, CMU SVjoy.zhang@sv.cmu.edu

Mobile Speech Translator• Pandora: Two-way statistical speech

translation system for mobile devices (PDA, cellphone)

• Data-driven: easy to train for new language pairs and domains

• Domain: travel/medical/force protection• Languages:

English/Spanish/Chinese/Japanese/Thai ...

Mobile Sign Translator Automatically detect text area in captured image

Optical character recognition (OCR): picture to text

OCR error correction as a translation task Apply machine translation on foreign text

On-going Research•Context-aware speech translation (translate according to where you are, who you are)

•Self-adapting translation system

ReferencesZhang and Vogel, Pandora: a large-scale two-way statistical machine translation system for hand-held devices, MT Summit XI, 2007Zhang and Huang, Mining translations of OOV terms from the web through cross-lingual query expansion, SIGIR 2005.Chang, Zhang, Vogel and Yang, Enhancing image-based Arabic document translation using a noisy channel correction model, MT Summit XI, 2007Yang, Gao and Zhang, Towards automatic sign translation, HLT 2001.

Speech Translator for 3D Virtual World

• Multilingual speech conversation in the virtual 3D world (Second Life)

• Simulates what users will do in real life• Automatic language detection• Automatic translation

Capturer Module

Interactive Module

Camera Image

Image Commands

Commands

Results

Visual Output

Detection & Recognition &

Translation Module User

Audio Output

User Input

http://mobilityresearchcenter.org © 2009 Carnegie Mellon CyLab. All rights reserved.

PROBLEM• GSM doesn’t work well indoors• Zero-setup is better• Fewer sample fingerprints should work• Fingerprint pollution by signal fluctuation

Enhancement of Wi-Fi Indoor LocationingTony Lin, Ilya Landa, Martin Griss, Joy Zhang

SOLUTION• Indoor locationing technique• Using existing Wi-Fi access points (AP)• Enhanced Redpin algorithm is introduced• Dominant AP filter is adopted

FUTURE WORK• Tracking with user activities• More RF sources in addition to Wi-Fi,

such as BlueTooth• More pilots applications

CONCLUSION• Enhanced Redpin can provide

efficient accuracy even with fewer training data

• Applying the dominant AP filter can gain better results by reducing noise

USER SCENARIOS• Tracking medical personnel in a hospital

– To study personnel movement in a hospital– To find the nearest relevant person

• Tracking elders at home or senior center– To determine current location of a person– To view movement history– To detect abnormalities in a person’s position or movement

EXPERIMENT• Five algorithms were used to do comparison

– Naïve Bayes Classifier– Support Vector Machine (SVM) – K Nearest Neighbor (KNN)– Redpin– Enhanced Redpin

• Different training data size was measured• Adjacent areas were measured• Applying AP filter to reduce the signal noise• Different number of APs was measured• 1,000 fingerprints were used• 10% for testing data, 90% for training data• For each experiment, 100 runs were executed

  Bayes SVM KNN Redpin Enhanced Redpin

Accuracy 61% 81% 80% 86% 87%

Accuracy after apply filters

10% 90% 89% 90% 91%

Figure 1: Average accuracy of non-filtering and filtering data for each algorithm

Figure 2: Average accuracy for every training data size

Figure 3: Average accuracy of adjacent areas Figure 4: Average accuracy of different number of APs

0.4

0.45

0.5

0.55

0.6

0.65

0.7

0.75

0.8

0.85

0.9

50 100 150 200 250 300 350 400 450 500 550 600 650 700 750 800 850 900

Size of training data

Acc

urac

y

Naïve BayesSVMKNNRedpinPMI Redpin

0.50.55

0.60.65

0.70.75

0.80.85

0.90.95

1

0 1 2 3The number of increased adjacent rooms

Acc

urac

y

Naïve BayesSVMKNNRedpinPMI Redpin

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1 2 3 4 5 6 7 8

The number of access points

Acc

ura

cy

SVMKNNRedpinPMI Redpin

Sponsor: CyLab

© 2009 Carnegie Mellon CyLab. All rights reserved. http://mobilityresearchcenter.org

Social ShopAds

HealthDine Measurement

Problemv Provide monitoring, feedback, and encouragement for the user to comply with clinician or care giver-set goals.

v Sensors on mobile devices collect and analyze data to determine user activities.

v Monitor for compliance and give feedback to the user.

v Provide summaries and notifications to the clinician or caregiver.

v Ensure all communications are secure.

v Ensure user has control over what information is given to whom.

v Ensure security and privacy should not be over burdening on either the mobile device or the user.

Status

v Built a prototype on N95 using AES and on a serverv Initial rule-based privacy model

• Privacy Rules Definition: Through web portal• User Mediation: Through device, application created

for device, accessed through Wi-Fi .• Artificial Intelligence: Learning done by server, based

on rules and user mediation.v Evaluated several security schemes

• Measure efficiency of different encryption schemes.• Authenticating users of the system• Generic curve of data size versus CPU usage• Ensuring security of the keys, creation and distribution

of new keys

Next Stepsv Explore different learning schemes

v Test prototype in multi-user campus setting

v Location Tracking to provide better medical data

v More efficient power consumption

v Study of comparison of individual privacy rules with the social group

v Scalability of system for large number of sensors and large quantities of data being sent

http://mobilityresearchcenter.org

Solution & Benefits

v To determine the most optimal solution for encryption of datav Create user-defined privacy specificationsv A machine learning paradigm that learns from the previous decisions of the usersv Measuring overheads for the transmission of the data and make the process more efficientv Ensuring a reliable system in all media with respect to wireless communication of the data over large distances.v Key infrastructure element in remotely supervised, home- based patient rehabilitationv Better and less expensive alternative for rehabilitation.v No supervised rehabilitation or monitoringv Early detection of abnormal conditions

Sponsor: CyLab

Real Time Health Monitoring SystemSecurity and Privacy

Sweta Deivanayagam, Anusha Nagarajan, Yash Shroff, Ashish TulsainiMartin Griss, Asim Smailagic, Dan Siewiorek

© 2009 Carnegie Mellon CyLab. All rights reserved.http://mobilityresearchcenter.org

A Mobile Application To Monitor the ElderlyOmar Abdul Baki, Tony Lin, Martin Griss, Joy Zhang

ProblemCurrent solutions which monitor the homebound elderly are either too costly or too intrusive.

Solution1. An “always-on” mobile application2. Learns behavior patterns3. Responds to erratic readings 4. Classifies activities according to :

a) Locationb) Movement and posture

Scalable Accurate Non-Intrusive Low-Cost Usable

Preliminary Results

Prototype in Mobile Python for the Nokia N95 :• Uses online clustering for learning• Detects falls• Recognizes repeated behavior as normal• Detects movement from room to room• Detects changes in postures (standing,

sitting, lying down)

Future Work

• Factor time of day and day of week into learned behavior patterns

• Use other available sensors (such as light and microphone)

• Replace thresholding with more advanced abnormality recognition techniques

Prompt: Are you all right?

Continue Monitorning

YES NO

Sponsors: CyLab, Panasonic, Nokia

© 2009 Carnegie Mellon CyLab. All rights reserved.http://mobilityresearchcenter.org

Personal Messaging AssistantProvide focused and timely healthcare information messagesDeepthi Madamanchi, Sumalatha Komarraju, Senaka Buthpitiya & Martin Griss

ProblemHealthcare providers interact with many peoplethroughout the day. Currently many support peopleprocess messages, reminders and follow-ups. • Increasing use of mobile devices and mail by

patients and caregivers around the clock• Clinics and hospitals offering email and web

access services to patients• Support personnel have limited working hours • Increasing delays in providing medical help in

emergency and critical conditions.• Possible loss of business and eventual distrust

of a particular HMO or healthcare provider.• Providers work in multiple hospitals with

different policies and modes of patient interaction How PMA works

Next Steps• Handle more message types and sources

• Process forward, file, folder, calendar rules

• Enhance intelligence and statistical message content analysis

• Deploy PMA on a single secure personal service center

• Larger scale test and pilot

• Add features for other mobile professionals

SolutionPartially automate processing of message traffic to mobile devices. • An intelligent, context and content-aware

messaging system which can act more effectively/rapidly than a human assistant.

• “Triage” health-related email content to match context, importance, and urgency.

• Deliver the right message at the right place and right time based on context and preference of the healthcare provider or patient.

Scenario• Dr. Bhatt uses e-mail to follow up on his cardiac

patients’ post-release from hospital, sends drug orders to pharmacies, and replies to (some of) his patients’ queries.

• These tasks have different levels of urgency and importance.

• PMA can direct the most appropriate message(s) to the doctor based on his current context (location and activity) or to his nurse or colleagues.

Features of PMA• Context Aware

Processes e-mail based on your location and activity into consideration

• IntelligentWorks out importance and urgency of emails (based on your context and preferences)

• AdaptiveLearns your priorities and preferences

• ExtensibleSupports new contexts (e.g., provider signs up for a weekly medical conference)

Results• Functional prototype using Jess rules, and

MySQL on server, accessing email from Gmail inbox

• Sends selected prioritized messages via SMS (or HTTP) to N95 phone

• Accepts user context & tags, and GPS location

Sponsor: CyLab

Motivation Emergencies like earthquakes, may have

individuals trapped in unsafe buildings or spaces.

Limited manpower and risk to the lives of emergency response workers impede rescue efforts.

Traditional static sensor networks, need impractical number of nodes to cover large areas simultaneously.

Robots have high per unit cost. They can be deployed only in limited numbers thus offering lower coverage.

SensorFlyCollaboratively-mobile Miniature Flying Sensor Network

Aveek Purohit and Pei Zhang

Maximum access to unknown areas Cannot rely on infrastructure Self recovery from failures

Challenges

Platform

Mapped Safe Area

Base/Charging Station

Replacement

Low Power

SensorFly: Overview

SensorFly Middleware Architecture

First Generation SensorFly: Low-cost compass sensor to determine the

orientation Accelerometer to detect collisions with

obstacles Sonic sensor for altitude detection V1 SensorFly node weight = 29 grams Energy usage for V1 SensorFly node

Future Work: Flight control and navigation with minimal sensing. Energy efficient network protocols for the controlled mobile SensorFly environment. Collaborative 3D localization

© 2009 Carnegie Mellon Cylab. All Rights Reserved.http://mobilityresearchcenter.org

Motors

Acc

Compass

Sonic

Radio

Altitude Control

Navigation

Scheduler Network ManagementTask

Management

Applications

Hardware / Peripheral Drivers

Location Engine

Operation Mode Power UsageHovering 6.1 WForward flight 7.2 WData Tx/Rx/Sensing 310/330/225 mWProcessing only 150 mWIdle ~1 mW

Sponsor: Cylab

http://www.cmu.edu/silicon-valley/research/smartspaces.html © 2009 Carnegie Mellon CyLab. All rights reserved.

SmartSpaces Mobile Health and Telehealth CapabilitiesMartin Griss (PI), Ray Bareiss, Patricia Collins, Ed Katz, Mike Smith

Indicator 1 – Number of Older Americans

• 81% prefer to live independently• 30% would “rather die” than live in a nursing home• Limited mobility • Declining health• Home health expenditures in the U.S. are less than in other developed countries

Elders need assistance at home

Mobile blood pressure monitor

• Tracks vital signs

• Notifies family/friend, health care provider of abnormalities

• Notification ranges settable by family/friend, health care provider

Exercise monitor

• Tracks motion throughout the home (RFID, Wi-Fi)

• Reminds patient of need to exercise

• Optionally, provides exercise instructions

• Notifies family/friend of patient’s failure to move

• Notification information is settable by recipient

Diet monitor

• Tracks selection of grocery products

• Reminds patient of dietary restrictions

• Optionally, provides nutrition information

• Notifies family/friend of patient’s failure to comply

• Notification settable by recipient

Home HomePortal

Home HomePortal

Home HomePortal

ServiceCenter

ConciergeService

ServiceService

ServiceService

ServiceService

ServiceService

Analytics

Family & Friends

Concierge

Caregivers

Ecosystem Architecture(future work)

Context-Aware, Agent-Based System

• Supports aggregation of information across homes

• Supports initiative-taking agents

• Supports autonomous collaboration by agents

• Concierge mediates some services to optimize health care provider’s time

• Family & friends offer first-line support to patient when appropriate

Solution• Integrated, context-aware, agent- based, mobile system• Multiple modalities (voice, touch, vision, RFID)

Results

Sponsors: SAP, Panasonic, Nokia, Motorola