E0-245: ASP

Lecture 19: Multimedia and Android Components

Dipanjan Gope

2

Dipanjan Gope

• Location Sensors - Theory of location sensing

- Package android.location

• Physical Sensors - Sensor Manager

- Accelerometer

- Gyroscope

- Magnetometer

- Sensor fusion

• NFC

• Multimedia - Camera

- Microphone

Module 2: Android Sensor Applications

3

Dipanjan Gope

Activity

Views

Intent

ContentProvider

BroadcastReceiver

Service

Coverage

4

Dipanjan Gope

• Greg Milette, Adam Stroud: Professional Android Sensor Programing, 2012, Wiley India

• Reto Meier: Professional Android 4 application development. John Wiley & Sons, 2012.

References

5

Dipanjan Gope

Content Provider

https://bestandroidtraining.wordpress.com/2013/03/30/android-project-structure/

6

Dipanjan Gope

android.database.sqlite

class SQLiteDatabase - insert - query - delete

- ….

abstract class SQLiteOpenHelper - onCreate

- onUpgrade

class ContentValues class Cursor

7

Dipanjan Gope

Quick Guide Database Usage

-http://www.androidhive.info/2011/11/android-sqlite-database-tutorial/

8

Dipanjan Gope

Step 0: Database content

….

9

Dipanjan Gope

Step 1: Schema and SQLiteOpenHelper

10

Dipanjan Gope

Step 2: Add to DB

11

Dipanjan Gope

Step 3: Read from DB

12

Dipanjan Gope

Step 4: Update DB

13

Dipanjan Gope

Step 5: Delete DB

14

Dipanjan Gope

DB Flow

15

Dipanjan Gope

• Wrapper over DB for decoupling data and UI apps

ContentProvider

Interface ContentProvider:

16

Dipanjan Gope

• Design DB, schemas, URIs

• Extend ContentProvider

• Implement interface functions

• Register in manifest

Steps: ContentProvider

17

Dipanjan Gope

ContentProvider: Demo

18

Dipanjan Gope

• Permissions .. Permissions .. Permissions

• Same query

Usage From Another App

19

Dipanjan Gope

Permissions: CP Creator App

1

2

3

20

Dipanjan Gope

Permissions: CP Access App

1

2

21

Dipanjan Gope

Native Android Content Providers

Media Store

Browser

Contacts

Calendar

Call Log

22

Dipanjan Gope

• Application continues to run when UI is invisible

• Has higher priority than inactive Activities - lesser chance of getting killed when system requires resources

• Can be configured to restart when resources are re-available

• Service priority can be raised to foreground Activity

Service

23

Dipanjan Gope

Service

http://developer.android.com/guide/components/services.html

24

Dipanjan Gope

Service Class

Ref: [2]

25

Dipanjan Gope

Start/Stop a Service

Ref: [2]

26

Dipanjan Gope

Multimedia

27

Dipanjan Gope

• Camera - photos and videos

- other applications: barcode, face recognition, pulse rate detection

• Microphone - record audio

- other applications: Text-to-speech, voice activation

• Speaker - play audio

- other applications: audiogram

Multimedia: Components

28

Dipanjan Gope

Audio: Speaker

29

Dipanjan Gope

Speaker Application: Audiogram

30

Dipanjan Gope

Audio: Microphone

31

Dipanjan Gope

Heart Sounds: Phonocardiography

After basic Noise Filtering

Recorded Heart Sound:

S1 S2

Mobile Phone Sensor: Microphone

32

Dipanjan Gope

Heart Rate Detection

0 0.5 1 1.5 2 2.5

x 105

-0.5

0

0.5

0 0.5 1 1.5 2 2.5

x 105

0

0.005

0.01

0 20 40 60 80 100 120 140 160 180 2000

200

400

80.7495 Hz

33

Dipanjan Gope

Heart Sounds: Phonocardiography

S3

S4

Healthy Heart Sound

Third Heart Sound

Fourth Heart Sound

Mp3 from Littmann Heart Sounds

34

Dipanjan Gope

Video/Image: Camera

Ref: [2]

35

Pulse Detection from Mobile Phone

Video Recording Face Recognition RGB Components

ICA Analysis FFT Pulse From Spectrum

Pre-processing - Detrend - Zscore

Post-processing - Smooth - Bandpass Filter - Spline

M.Z. Poh, D.J. McDuff, R.W. Picard, Opt Express. 18(10) pp. 10762–10774, 2010.

36

Pulse Detection from Mobile Phone

(a) RGB components extracted from 1280x720 resolution video capture from a Samsung Galaxy S3 mobile phone at 30 frames per second over a period of 30 seconds. (b) The FFT of (a) lacks prominent peaks in spectrum. (c) The 3 components extracted from the RGB traces using JADE ICA analysis algorithm (d) The FFT of (c) contains distinct peaks in the spectrum from which pulse-rate is obtained

37

Accuracy Worsens With Distance

0.4 0.6 0.8 1 1.2 1.4 1.6 1.8

x 104

-2.5

-2

-1.5

-1

-0.5

0

0.5

Total Pixel Size of Detected Face

log

10

(Err

or)

WebCam Results

MTLB_dtrnd_schm1

MTLB_dtrnd_schm2

MTLB_dtrnd_schm3

TRVN_dtrnd_schm1

TRVN_dtrnd_schm2

TRVN_dtrnd_schm3

38

Dipanjan Gope

Activity

Views

Intent

ContentProvider

BroadcastReceiver

Service

Coverage

39

Dates

• Final Exam: 30th April 10-1pm

• Final Project submission: same day

- Codes or apk file