551 Smart Phone Long Handout

8/3/2019 551 Smart Phone Long Handout

1/82

Dong Xuan (CSE/OSU) / 2009

Design and Implementation of

Smartphone-based Systems andNetworking

Dong Xuan

Department of Computer Science and Engineering

The Ohio State University, USA


2/82

Dong Xuan (CSE/OSU) / 2010 2

Outline

Smartphones Basics

Mobile Social Networks

E-Commerce

E-Health

Safety Monitoring

Future Research Directions


3/82


A smartphone is a mobile phone offering advancedcapabilities, often with PC-like functionality

Hardware (Apple iPhone 3GS as an example)

CPU at 600MHz, 256MB of RAM 16GB or 32GB of flash ROM

Wireless: 3G/2G, WiFi, Bluetooth

Sensors: camera, acceleration, proximity, light

Functionalities

Communication News & Information

Socializing

Gaming

Schedule Management etc.

Smartphone Basics

3


4/82


Smartphones are popular and will become more popular

Smartphone Popularity

4


5/82


Smartphone Accessories

5


6/82

Dong Xuan (CSE/OSU) / 2010 6

Smartphone Features

Communication/Sensing/Computation

Inseparable from our human life


7/82


Our Smartphone Systems

E-SmallTalker [IEEE ICDCS10]:senses information published byBluetooth to help potential friends findeach other (written in Java)

E-Shadow [IEEE ICDCS11]: enablesrich local social interactions with

local profiles and mobile phone

based local social networking tools

P3

-Coupon [IEEE Percom11]:automatically distributes electronic

coupons based on an probabilistic

forwarding algorithm

7


8/82


Our Smartphone Systems

Drunk Driving Detection [Per-Health10]: uses smartphone (GoogleG1) accelerometer and orientationsensor to detect

Stealthy Video Capturer [ACMWiSec09]: secretly senses itsenvironment and records video via

smartphone camera and sends it to athird party (Windows Mobileapplication)

Download & Run Video sent by Email Captured Video

8


9/82


Exemplary System I:E-SmallTaker Small Talk

A Nave Approach

Challenges

System Design

Implementation and Experiments

Remarks

9


10/82


Small Talk People come into contact opportunistically

Face-to-face interaction

Crucial to people's social networking Immediate non-verbal communication

Helps people get to know each other

Provides the best opportunity to expand social network

Small talk is an important social lubricant

Difficult to identify significant topics

Superficial

10


11/82


A Naive Approach of Smartphone-

based Small Talk Store all users information, including each users full contact

list

User report either his own geo-location or a collection ofphone IDs in his physical proximity to the server using internet

connection or SMS

Server performs profile matching, finds out small talk topics

(mutual contact, common interests, etc.) Results are pushed to or retrieved by users

11


12/82


However Require costly data services (phones internet

connection, SMS)

Require report and store sensitive personalinformation in 3rdparty

Trusted server may not exist

Server is a bottleneck, single point of failure, target ofattack

12


13/82


E-SmallTalker A Fully

Distributed Approach No Internet connection required

No trusted 3rdparty

No centralized server

Information stored locally on mobile phones

Original personal data never leaves a users phone

Communication only happens in physical proximity

13


14/82


Two Challenges How to exchange information without establishing a Bluetooth

connection Available data communication channels on mobile phones

Cellular network (internet, SMS, MMS), Bluetooth, WiFi, IrDA Bluetooth is a natural choice

Bluetooth connection needs users interaction due to security reasons

How to find out common topics while preserving users privacy No pre-shared secret for strangers

Bluetooth Service Discovery Protocol can only transfer limited serviceinformation

14


15/82


System Architecture Context exchange Context encoding and matching

Context data store

User Interface

15


16/82


17/82


Context Encoding Example of Alices Bloom

filter

Alice has multiple contacts,

such as Bob, Tom, etc. Encode contact strings,

Firstname.lastname@phone

_number, such as

Bob.Johnson@5555555555

and

Tom.Mattix@6141234567

17


18/82


Implementation J2ME

about 40 java classes, 127Kb jar file

On real phones Sony Ericsson (W810i), Nokia (5610xm, 6650, N70, N75,N82)

18


19/82


Experiments Settings

6 phones, n=150, k=7, m=1024 bits, default distance=4m, average of10 runs

Performance Metrics Discovery time: the period from the time of starting a search to the time of

finding someone with common interest, if there is any

Discovery rate:percentage of successful discoveries among all attempts

Power consumption

Factors Bluetooth search interval Number of users

Distance

19


20/82


Experiment Results Minimum, average and maximum discovery time are

13.39, 20.04 and 59.11 seconds respectively

Always success if repeat searching, 90% overall ifonly search once

Nokia N82 last 29 hours when discovery interval is

60 seconds

20


21/82


Related Work Social network applications on mobile phones

Social Serendipity

Centralized, Bluetooth MAC and profile matching, SMS, strangers

PeopleTones, Hummingbird, Just-for-Us, MobiLuck, P3 Systems, Micro-Blog,

and Loopt Centralized, GPS location matching, Internet, existing friends

Nokia Sensor and PeopleNet

Distributed, profile, Bluetooth / Wifi connection, existing friends

Private matching and set intersection protocols

Homomorphic encryption based

Too much computation and message overhead for mobile phone Limitations

Require costly data services (phones internet connection, SMS)

Require report and store sensitive personal information

Bottleneck, single point of failure, target of attack

21


22/82


Remarks Propose, design, implement and evaluate the E-SmallTalker

system which helps strangers initialize a conversation

Leveraged Bluetooth SDP to exchange these topics without

establishing a connection

Customized service attributes to publish non-service related

information.

Proposed a new iterative commonality discovery protocol based on

Bloom filters that encodes topics to fit in SDP attributes to achieve a

low false positive rate

22


23/82


Exemplary System II:E-Shadow

Concept

Application Scenario

Goals and Challenges

System Design


Remarks

23


24/82


Concept Motivation

Importance of Face-to-Face Interaction

Prevalence of mobile phones

Distributed mobile phone-based local social

networking system

Local profiles Mobile phone based local social interaction tools


25/82


Application Scenario: Conference


26/82


27/82


Layered Publishing

Spatial Layering

WiFi SSID at least 40-50 meters, 32 Bytes

Bluetooth Device (BTD) Name 20 meters, 2k Bytes

Bluetooth Service (BTS) Name 10 meters, 1k Bytes

Temporal Layering For people being together long or repeatedly

Erasure Code


28/82


E-Shadow Publishing Procedure

Valve Generator

Information

Filter

Database

Sensor

Feedback

User

Maual

Input

Online

Data

Mining

BT

Device

BT

Service

WiFi


29/82


Matching E-Shadow with its Owner

Intuitive Approach: Localization

However, imprecision beyond 20-25 meters


30/82


Human Direction-driven Localization

Direction more important than distance Human observation

A new range-free localization technique RSSI comparison: Less prone to errors

Space partitioning: Tailored for direction decision


31/82


Walking Route and Localization We allow users to walk a distance

Triangular route: A->B->C in (a), for illustration purposes

Semi-octogonal route: A->B->C->D->E in (c), more natural

Take measurements on turning points

Calculate the direction through RSSI comparison and space

partitioning


32/82


Implementation

Information

Publishing Module

Database

Generator

Buffers

Control Valve Broadcasting

Interfaces

Retrieval &

Matching Module

Receivers

Localization

Decoding & Storage

Sensing Module

User Interface


33/82


Evaluations (1)-Time & Energy

E-Shadow Collection Time

WiFi SSID: 2 seconds

BTD: 12-18 seconds

BTS: 25-35 seconds

E-Shadow Power

Consumption

3 hours in full performanceoperation

>12 hours in typical situation


34/82


Evaluations (2)-Localization

3 Outdoor Experiments:

Open field campus

2 Indoor Experiments:

Large classroom


35/82


Evaluation (3)-Simulations

Large-Scale Simulations:

Angle deviation CDFs

12 times of exemplary

direction decisions


36/82


Related Work Centralized mobile phones applications

Social Serendipity

Centralized, Bluetooth MAC and profile matching, SMS, strangers

Decentralized mobile phone applications

Nokia Sensor

Distributed, profile, Bluetooth / Wifi connection, existing friends

E-Smalltalker

Distributed, no Bluetooth / Wifi connection, strangers

Localization techniques for mobile phones applications

GPS

Virtual Compass

peer-based relative positioning system using Wi-Fi and Bluetooth radios

Limitations Privacy compromise

Unable to capture the dynamics of surroundings

No mapping between electronic ID and human face

Localization techniques either not pervasive or not accurate for long range

36


37/82


Remarks

Propose, design, implement and evaluate the E-Shadow

system which lubricates local social interactions

E-Shadow concept

Layered publishing to capture the dynamics of surroundings

Human-assisted matching that works for mapping E-Shadow with its

owner in a fairly large distance

Implementing and evaluating E-Shadow on real world mobile phones

37


38/82


Exemplary System III:P3-Coupon

Coupon Distribution

A Nave Approach

Challenges

System Design


Remarks

38


39/82


Electronic Coupon Distribution

Electronic coupons

Similar to paper coupons

Can be stored on mobile phones

Two distribution methods

Downloading from Internet websites Need to define target group

Limited coverage

Hard to maintain dynamic preferences lists on central databases

Peer to Peer Distribution No special destination/target group

More coverage

More flexible user-maintained preferences list

39


40/82


A Naive Approach of Peer-to-Peer

Coupon Distribution A store periodically broadcast the coupon

Users within broadcast range receive the coupon

User can decide whether to use, forward or discard the coupon

Users forward the coupon to others in physical proximity

Forwarders IDs are recorded in a dynamically expanding list

The coupon is used by some user

The store reward all users who have forwarded the coupon

40


41/82


However

Require manually establishing wireless connections Cumbersome

Not prompt Not possible for coupon forwarding among strangers

Require recording the entire forwarding path Potential privacy leakage

Discourage users forwarding incentives

41


42/82


Challenge

How to design a prompt coupon distribution

mechanism that

Incentivize coupon forwarder appropriately for keeping thecoupons circulating

Preserve the privacy of coupon forwarders

42


43/82


P3-Coupon A Probabilistic

Coupon Forwarding Approach Probabilistic sampling on forwarding path

Keep only one forwarder for each coupon: NO privacy leakage

Probabilistically flip ownership at each hop

Accurate approximation of coupon rewards plenty of chances of interpersonal encounters

Accurate bonus distribution with 50 coupons and 5000 people

Adaptive to different promotion strategies

Flip-once model Always-flip model

No manual connection establishment Connectionless information exchange via Bluetooth SDP

43


44/82


System Architecture Store Side

A central server for broadcasting and redeeming coupons

Client side Coupon forwarding manager, coupon exchange, coupon data store, user

interface

44


45/82


Probabilistic Forwarding Algorithm

Always-Flip Model

The coupon ownership keeps flipping with certain probability at each hop.

Good at assigning relative bonuses affected by the whole path lengths

E.g. the parent forwarder receives k times the bonus given to children forwarders

The flip probability can be calculated in advance by the store, once k is fixed, usingthe following formula

45


46/82


Probabilistic Forwarding Algorithm

Extension: Flip-Once Model

Once flipped, a coupons ownership remain the same in a forwarding path.

Good at assigning absolute bonuses irrelevant of the number of followingforwarders

E.g. hop 1 user gets 10%, hop 2 user gets 5%, etc. The flip probability can be calculated in advance by the store using the following

formula

46


47/82


Coupon Format

Coupon description Product description

Discounts

Coupon issuer

Coupon code

Start/end date

Coupon forwarder information The current owner

Digital signature Prevent forging fraud coupons

47


48/82


Implementation

J2ME about 17 java classes, 1390Kb jar file

On real phones Samsung (SGH-i550), Nokia (N82, 6650, N71x)

48


49/82


Experiments

Experimental evaluations Coupon forwarding time

Power consumption

Simulation evaluation Number of Coupon holders vs. Time

Distribution saturation time vs. Number of Seeds

Coupon ownership distribution for probabilistic sampling

Deviation between theoretical and actual bonus (Always-Flip, Flip-Once)

Factors Number of coupons

Number of users

Number of initial coupon holders

49


50/82


51/82


Remarks

Propose, design, implement and evaluate the P3-Coupon

system which helps prompt and privacy preserving coupon

distribution

Probabilistic one-ownership coupon forwarding algorithm

Implement the system on various types of mobile phones

Extensive experiments and evaluations show that our approach

accurately approximate the theoretical coupon distribution in which the

whole forwarding path needs to be recorded

Practical for real-world deployment

51

52


52/82


Exemplary System IV Drunk Driving

DetectionMotivation

Our Contributions

Detection Criteria

Our System

Related Work

Implementation and EvaluationRemarks

52

52

53


53/82


MotivationCrashes caused by alcohol-impaired driving pose a

serious danger to the general public safety and health

13,041 and 11,773 driving fatalities happened in 2007 and

2008* 32% of the total fatalities in these two years*

Drunk driving also imposes a heavy financial burden on

the whole society

Annual cost of alcohol-related crashes totals more than $51billion*** Data from U.S. NHTSA (National Highway Traffic Safety Administration)

** Data from U.S. CDC (Central of Disease Control)

53

53

54


54/82


Motivation

Detection of drunk driving so far still relies on visual

observation by patrol officers

Drunk drivers usually make certain types of dangerous maneuvers

NHTSA researchers identify cues of typical drunk driving behavior

Visual observation is insufficient to prevent drunk driving

The number of patrol officers is far from enough

The guidelines are only descriptive and qualitative

Usually, it is too late when drunk drivers are stopped by officers

It is essential to develop systems actively monitoring drunk

driving and to prevent accidents

54

54

55


55/82


Our Contributions

Propose utilizing mobile phones as a platform for

active drunk driving detection system

Design a real-time algorithm for drunk drivingdetection system using mobile phones

Simple sensors required only

i.e., accelerometers and orientation sensors

Design and implement a mobile phone-based activedrunk driving detection system

Reliable, Non-intrusive, Lightweight and power efficient, and

No extra hardware and service cost

55

55

56


56/82


Cues for Drunk Driving DetectionCues related to lane position maintenance problems

E.g., weaving, drifting, swerving and turning with a wide radius

Cues related to speed control problems

E.g., accelerating or decelerating suddenly, and braking erratically

Cues related to judgment and vigilance problems

E.g., driving with tires on lane marker, slow response to traffic signals

56

56


57/82

58


58/82


Drunk Driving Detection Criteria

Focus on the first two categories of cues

They correspond to higher probabilities of drunk driving

Map them into patterns of acceleration

Probability of drunk driving detection goes higher while

the number of observed cues increases

58

Driversproblems in

maintaining lane

position

Abnormal lateral

movements

Patterns oflateral

acceleration of

vehicles

Drivers

problems in

controllingspeed

Abrupt speed

variations

Patterns of

longitudinal

acceleration ofvehicles

58

59


59/82


Our System

9

59

60


60/82


Implementation

Develop the prototype system on Android G1 phone with

accelerometer and orientation sensor

Implement the prototype in Java, with Eclipse and Android 1.6

SDK

The whole prototype system can be divided into five major

components

User interface System configuration Monitoring daemon

Data processing Alert notification

60

61


61/82


Evaluation - Testing Data Collection

Test data

72 sets of data with simulated drunk driving related behaviors

- Weaving, swerving, turning with a wide radius

- Changing speed erratically (accelerating or decelerating) 22 sets of data for regular driving

- Each one for 5 to 10 minutes

Mobile phone positions in the vehicle

61

62


62/82


Evaluation - Detection Performance

Study the accuracy of detecting drunk driving related behaviors

In terms of false negative and false positive

Study performance in the special case, such as the phone slides in the vehicle

during driving Slides has obvious impacts on detection accuracy

May add additional calibration procedure to solve it (future work)

62

63


63/82


Evaluation Energy Efficiency

Curves of battery level states during mobile phone running

Phone runs without drunk driving detection system

Monitoring daemon of system keeps running, sensing and doing the pattern

matching on the monitoring results

63

64


64/82


Related WorkDriver vigilance monitoring and driver fatigue prevention

Monitoring the visual cues of drivers to detect fatigue in driving

Installed cameras just in front of drivers are potential safety hazard

Monitoring through vehicle-human interface

Capture fatigued or drunk driving through monitoring interactions

Low compatibility, vehicles need to couple with auxiliary add-ons

Detect abnormal driving through GPS and acceleration dataPattern matching with GPS and acceleration data

However, GPS data are not always available

64

65


65/82


Remarks

First to propose utilizing mobile phones as a platform for

developing active drunk driving detection system

Design and implement an efficient detection system based on

mobile phone platforms

Experimental results show our system achieves good detection

performance and power efficiency

In the future work, to improve the system with additional

calibration procedure and by integrating all available sensingdata on a mobile phone such as camera image

65


66/82


Exemplary System V: Stealthy Video

Capturer Background

SVC Overview

Challenges

Our Approaches

Experimental Evaluations

Remarks

66


67/82


Background

More and more private information is entrusted to

our friend, the 3G Smartphone, which is getting

more and more powerful in performance anddiversified in functionality.

67


68/82


SVC Overview

Almost every 3G Smartphone is equipped with a

camera and the wireless options, such as 3G

networks, BlueTooth, WiFi or IrDA. These wireless connections are good enough to

handle certain types of video transmission.

We turn 3G Smartphones into an online stealthy

video-recorder.

68


69/82


System Architecture

69


70/82


Challenges

Stealthily install SVC into 3G Smartphones Windows Hiding

Infection Method

Collect the video information from 3GSmartphones DirectShow Controls

Data Compressing Send the video file to the SVC intender

File Sending

70


71/82


Infection Method

To embed SVC in a 3G Smartphone is called a

infection process.

We employ Trojan horse for downloads as theinfection approach.

Our experimental SVC is hidden in the game

of tic-tac-toe that we develop in WindowsMobile environment.

71


72/82


The Scenario of Tic-Tac-Toe

72


73/82


Triggering Schemes

Triggering Algorithm is designed to determine when

to turn on the video capture process and send the

captured video to make SVC stealthier and get moreuseful information.

Three scenarios are under consideration.

The first scenario is tracking.

The second scenario is related with political or businessespionage.

The third scenario is a hybrid one, where SVC is used for

much diversified everyday purposes.

73


74/82


Applications

Suspects tracking

Kids care

74


75/82


Kids tracking

75


76/82


Implementation

76


77/82


Experimental Evaluations:

Power Consumption Power curve

77


78/82


Experimental Evaluations:

CPU and Memory Usage CPU and Memory

78


79/82


Remarks

The initial study exploited from SVC will draw wide

attentions on 3G Smartphones privacy protection and

open a new horizon on 3G Smartphones securityresearch and applications.

We are currently investigating the modeling of smart

spyware from the study of spear and shield.

79


80/82


A Summary

80


81/82


Future Research Directions

Smartphone-based Systems and Networking Mobile social networking, e-commerce, e-health, safety

monitoring etc.

Easy to start and exciting but too many competitors, lack ofscientific depth

Smartphone Core Improvement Multitasking, power management, efficient local

communication protocol, accurate localization,security/privacy protection

Deep but hard to start

81


82/82

Final Remarks

Smartphones have brought significant impacts

to our daily life.

We present five exemplary systems on mobilesocial networking, e-commerce, e-health and

safety.

Research and development on smartphoneswill be hot.

Documents

551 Smart Phone Long Handout