SIF: A Selective Instrumentation Framework for Mobile Apps

Shuai Hao, Ding Li, William G.J. HalfondRamesh Govindan

Background

2

App Behaviors are Complex

Source: www.zdnet.com, 3/7/2013

Source: www.zdnet.com, 6/6/2012

, 2/9/2013

3Motivation Design Implementation Evaluation

Motivation

Many Studies on App Behavior

AppInsight[OSDI’12]

AppInsight[OSDI’12]

4Motivation Design Implementation Evaluation

Motivation

Many Studies on App Behavior

FreeMarket[NDSS’12]

AppInsight[OSDI’12]

FreeMarket[NDSS’12]

5Motivation Design Implementation Evaluation

Motivation

Many Studies on App Behavior

AppInsight[OSDI’12]

FreeMarket[NDSS’12]

All use app instrumentation

Dr. Android[SPSM’12]

Dr. Android[SPSM’12]

6

Server

Client AppApp

Cloud

Phone

In-App Billing

API

In-App Billing

Motivation Design Implementation Evaluation

Motivation

App Instrumentation Process is Complex

1 2 3 4

56Original

Instrumented

FreeMarket

It raises the level of abstraction for performing app instrumentation.

Goal

7

Instrumentation Framework for Mobile Apps

Approach

8

Instrumentation Framework for Mobile Apps

Instrumentation Framework

High-level Instrumentation

Specification

App

App

Instrumented

Find critical paths to debug slow apps

Study vulnerability of In-App Billing service

Support fine-grained permissions

Remove ads from app

Trace apps’ access to both user location and Internet

Binary? Instrumentwhat?

Modify appfunctionality?

R1: Instrument binaryR2: Instrument at different levels of granularity:bytecode, method, path

Requirements

9

Survey of Recent Research

R3: Permit user-specified instrumentation

Task

AppInsight

FreeMarket

Dr. Android

AdCleaner

Location Auditor

bytecode, method, path

bytecode

bytecode

bytecode

path

Requirements

10

Survey of Recent Research

R1: Instrument binary

R2: Instrument at different levels of granularity: bytecode, method, path

R3: Permit user-specified instrumentation

Challenges

11

Finding the Right Abstractions

R2: Instrument at different levels of granularity: bytecode, method, path

How to find the high-level abstractions for what to instrument?Gmail

Involves ~150,000path!How to succinctly specify code paths of

interest?

R3: Provide user-specified instrumentation

Arbitrary user-level instrumentation can change execution time of instrumented app

1

2

3 4

5

6

Challenges

12

Providing Overhead Feedback

CFG

1

2

3 4

5

6

BADInstr. Spec.

1

2

3 4

5

6

GOODInstr. Spec.

How to provide quick offline overhead feedback?

Contributions

Novel and flexible abstractions for specifying instrumentation points

Support for user-specified instrumentation with quick overhead feedback

Extensive evaluation that demonstrates expressivity and efficiency

13

SIF: Selective Instrumentation Framework

14Motivation Design Implementation Evaluation

SIF Workflow

Instrumenter

SIFScript

AppApp

Instrumented

Overhead Estimator

UserOverhead Feedback

SIF

RefineHigh

execution time

15

Task

AppInsight

FreeMarket

Dr. Android

AdCleaner

Location Auditor

Insert before/after code point

Replacecode point

Track pathsA à B

Motivation Design Implementation Evaluation

SIF Abstractions

Survey of Recent Research

Operations on set of instructions that share common attributesCodepoint Set AbstractionSet of dynamic traces between two

user-specified codepointsPath Set Abstraction

16

User-specifiedinstrumentation

Motivation Design Implementation Evaluation

SIFScript

AdCleaner

Instrumentation code

Specify codepoint filter

Hook with user defined code

Select target codepointsPlace instrumentation

Before After

17Motivation Design Implementation Evaluation

SIFScript

Location Auditor

1

2

3 4

5

C1

2

3 4

5

B

1

34 5

6

A2

GPS

Internet

1

2

3 4

5

D

Internet

1

2

4

1

2

3

34 5

2GPS

Internet

1

2Internet

Find API invokes to GPSFind API invokes to INTERNETPlace instrumen-tation

Motivation Design Implementation Evaluation

Realizing Path Set Abstraction

18

1

2

3 4

5

C1

2

3 4

5

B

1

3

4 5

6

A2

GPS

Internet

1

2

3 4

5

D

Internet

Reachability Analysis

Find potential set of methods M• Invoke instructions• Class hierarchy4 5

4

M = {A, B, C, D}

Motivation Design Implementation Evaluation

Realizing Path Set Abstraction

19

Path Profiling

• Ball-Larus approach• Extension to inter-

procedural profiler

1

3

4 5

6

2

1

3

4

6

2

1

3

5

6

2

Control FlowGraph

Path 1 Path 2

Runtime Statistics- Path 1: 3 times- Path 2: 5 times

Unfolding path statistics

Record call-site info

P1,P2,P2,P1,P2,P2,P1,P2

Call Site 2

P1,CS1,CS2,P2,CS1,…

Call Site 1

Motivation Design Implementation Evaluation

Realizing Path Set Abstraction

20

Path Stitching

Reproduce inter-procedural paths by simulated call stack analysis1

2

3 4

5

C1

2

3 4

5

B

1

3

4 5

6

A2

GPS

Internet

1

2

3 4

5

D

Internet

5

4

1

2

3 4

5

C1

2

3 4

5

B

1

3

4 5

6

A2

GPS

Internet

1

2

3 4

5

D

InternetA: P2, CS2C: P2, CS2

D: P1

Motivation Design Implementation Evaluation

Estimating Overhead Feedback

21

CFG

1

2

3 4

5

6User Code

Overhead Instructions inserted by SIF User-specified instrumentation

code

Introduced Overhead ∑ 𝑓𝑟𝑒𝑞𝑖∗𝑐𝑜𝑠𝑡𝑖instruction i in

6 User Code

and

Dynamic execution statisticse.g. by workload generator

Profiled estimates for instruction execution time

22Motivation Design Implementation Evaluation

Evaluation

Overview

CompletenessClassical tasks: e.g. timing profiler

Recent research: e.g. AppInsight, FreeMarket

Conciseness #Source lines of code

EfficiencyInstrumentationtime

Runtime overhead

Accuracy of overhead feedback

23Motivation Design Implementation Evaluation

Evaluation

Conciseness of SIF

Location AuditorAppInsight

AlgoProfFreeMarket

Privacy LeakageAdCleaner

Fine-grained PermissionFlurry-like Analytics

Call Graph ProfilerTiming Profiler

0 10 20 30 40 50 60 70 80 90 100

Source lines of code

SIF abstractions permit concise specifications

24Motivation Design Implementation Evaluation

Evaluation

Efficiency of SIF: Runtime Overhead

Privacy Leakage

AdCleaner

Fine-grained Permission

Flurry-like Analytics

Call Graph Profiler

Timing Profiler

92% 93% 94% 95% 96% 97% 98% 99% 100%

Original AppOverhead by SIF

Normalized Cost

SIF introduces <5% runtime overhead

0.8

4.4

0.6

1.3

1.0

0.4

25Motivation Design Implementation Evaluation

Evaluation

Efficiency of SIF: Accuracy of Overhead Feedback

SIF’s estimation error is within 7%

Privacy Leakage

AdCleaner

Fine-grained Permission

Flurry-like Analytics

Call Graph Profiler

Timing Profiler

0% 2% 4% 6% 8%

Estimation Error

26

Conclusion

Abstractions for specifying instrumentation points Support for user-specified instrumentation with quick overhead

feedback Extensive evaluations that demonstrates expressivity and

efficiency

SIF: A Selective Instrumentation Framework

Future Work

Port SIF to other platformsUse SIF to study various kinds of app behaviors Extend overhead estimator to support more sophisticated

instrumentation