Improving and Controlling User-Perceived Delays in Mobile Apps Lenin Ravindranath By the time it loads, the church service is over. Too slow!!! Slow responsiveness,

Improving and Controlling User-Perceived Delays in Mobile Apps

Lenin Ravindranath

By the time it loads, the church service is over. Too slow!!!

Slow responsiveness, long load time.

So slow. Did an intern write this app??

Slower than a snail.

horrifically slow.Uninstalled.

MIT

So slow that my screen already dims before it's loaded

~ Two Million Apps

> 500,000 Developers

Too slow - killing the usefulness when you really need to go

There’s an App for that

But it’s slow

“So slow. Did an intern write this app??”

“Slower than a snail even on 3G.”

“Slow and unresponsive like mud”

“Sluggish and freezes my HTC phone.”

“Very very slow compared to even browsing web.”

“Consistently 3 seconds behind where I touch.”

“Loading GPS data is *** slow”

• Diverse environmental conditions– Network connectivity, GPS signal quality,– Location, Sensor conditions etc.

• Variety of hardware and OS versions• Wide range of user interactions and inputs

Hard for developers to test

Performance problems are inevitable in the wild

Significant diversity in the wild

• App store is brutally competitive

Need for improving and controlling performance

• Response time matters– Users are impatient– 100ms delay can cost substantial drop in revenue at Amazon– Similar observations from Google and Bing

Help app developers

Find a bar

User Interaction Rendering

User-perceived delay

Monitor in the hands of user

Improve and control the user-perceived delay


AppInsightOSDI ‘12

Monitor performance in the wild

What is the user-perceived delay?

Where is the bottleneck?

Developer

Feedback

• Users with slow network – Poor Performance

• Users with fast network – Not producing the best quality result






Developer

Feedback

Deal with uncontrolled variability






Developer

Feedback

TimecardSOSP ‘13

Adapt processing to conditions in the wild

How much time has already been spent?

How much time will be spent?

RuntimeEstimatio

n

Adapt

High delays High delaysReduce

Tightly control user-perceived delay






Developer

Feedback

TimecardSOSP ‘13




RuntimeEstimatio

n

Adapt

Tightly control user-perceived delay

Low delays Low delaysImprove result






Developer

Feedback

TimecardSOSP ‘13



RuntimeEstimatio

n


AppInsight

Timecard

Minimal developer effort Readily deployable

- No changes to the OS or runtime

Negligible overhead

Product Impact at Microsoft, Facebook

Used by dozens of developers

AppInsight

Timecard

App

AppInstrumented

Automatic Binary

Instrumentation

User Transaction

TrackingMonitor app execution across async boundariesin a light-weight manner

AppInsight Monitor performance in the wild



Developer

Feedback

Significant barrier for most app developers

• No platform support• Only option is to instrument your app– Manage your own logging infrastructure

Users

App

Instrumenter

AppInstrumented

Analysis

Developer

Feedback

Server

Traces

Performance

Improved

Developer

AppInsight

AppStore

• Highly interactive, UI centric– Single UI thread that should not be blocked

• Most tasks are performed asynchronously– Asynchronous APIs for Network, Sensor, IO etc.– Computation performed on background threads

Highly Asynchronous Programming Pattern

Monitoring App Performance is Challenging

Tracing async code is challenging

Internet

Location

Mood Around Me

Nearby Tweets

GPS

sad

“Just got engaged :)”

“My life sucks”

“At last its sunny. But I have to sit and prepare my talk :(”

“I am sad that there is no more how I met your mother”

ClickHandler() { location = GPS.Sample(); url = GetUrl(location) tweets = Http.Get(url); result = Process(tweets); UI.Display(result);

}

GetUrl(location){ ... }

Process(tweets){ ... }

ClickHandlerStart

ClickHandlerEnd

LogStart();

LogEnd();

Thread

User-perceived Delay

Hypothetical Synchronous Code

User Interaction

Mood Around Me

sad

ClickHandler() { GPS.AsyncSample(GPSCallback);}

GPSCallback(location) { url = GetUrl(location); Http.AsyncGet(url, DownloadCallback);}

DownloadCallback(tweets) { result = Process(tweets); UI.Dispatch(Render, result);}

Render(result){ UI.Display(result);}



Render

UI Thread

Background Thread

UI Dispatch

ClickHandlerStart

System

ProcessDownloadCallback

User Interaction

Asynchronous Code

Background Thread

GPS

Network

ClickHandlerEnd

Async Http CallAsync GPS Call

GPSCallback

Render

UI Thread

UI Dispatch

ClickHandlerStart

System

Async GPS Call

DownloadCallback

User Interaction

Background Thread

Network

Async Http Call

GPSCallback

GPS

Background Thread

User Transaction

Transaction timeUser-perceived Delay

UI Thread

Background Thread

Background Thread

User Transaction

Apps are highly asynchronous

Where is the bottleneck?Focus development efforts

30 popular apps200,000 transactions

– On average, 10 asynchronous calls per user transaction

Up to 7000 edges

– There are apps with 8 parallel threads per transaction

UI Thread

Background Thread

Background Thread

Background Thread

Twitter

Thread Wakeup

Thread Blocked

Fire

Fire

Render

Callback

Callback

Nearby Tweets

Nearby PostsTwitter

Facebook Process Posts

Process Tweets

Facebook

User Interaction

Aggregate

GPS

GPS Callback

Mood Around Me

UI Thread

Background Thread

Background Thread

Background Thread

Thread Wakeup

Thread Blocked

Fire

Fire

RenderUser TransactionUser Interaction

Aggregate

GPS

GPS Callback

Critical PathOptimizing the critical path reduces the user perceived delay

CallbackFacebook

CallbackTwitter

Twitter

FacebookProcess Posts

Process Tweets

Users

App

Instrumenter

AppInstrumented

Analysis

Developer

Feedback

Server

Traces

Developer

AppInsight

AppStore

Low Overhead

Capturing User Transactions

UI Thread

Background Thread

Background Thread

User Transaction

Instrumentation impacts app performance– They are already slow enough!!!

Scarce Resources– Compute, Memory, Network, Battery

• User InteractionsCapture

UI Thread

Background Thread

Background Thread

User Transaction

User Interaction

Event Handler

• User Interactions• Thread Execution

Capture

UI Thread

Background Thread

Background Thread

User Transaction

ClickHandlerStart

ClickHandlerEnd

RenderStart

DownloadCallback Start

End

GPSCallback Start

GPSCallback End

RenderEnd

• User Interactions• Thread Execution• Async Calls and Callbacks

Capture

UI Thread

Background Thread

Background Thread

User Transaction

UI Dispatch Call

Download Callback

Async GPS Call

GPS Callback

Async Http Call

UI DispatcherCallback

• User Interactions• Thread Execution• Async Calls and Callbacks• UI Updates

Capture

UI Thread

Background Thread

Background Thread

User Transaction

UI Update

UI Thread

Background Thread

Background Thread

Background Thread

• User Interactions• Thread Execution• Async Calls and Callbacks• UI Updates• Thread Synchronization

Fire

Fire

Thread Blocked Thread Wakeup

Capture

User Transaction


Capture

UI Thread

Background Thread

Background Thread

User Transaction

• Trace every method– Prohibitive overhead

• Enough to log thread boundaries• Log entry and exit of Upcalls

Capturing Thread Execution

Upcalls

System

App

UI Thread

Background Thread

Background Thread

ClickHandlerStart

ClickHandlerEnd

RenderStart

End

GPSCallback Start

GPSCallback End

RenderEnd


ClickHandler() { GPS.AsyncSample(GPSCallback);}GPSCallback(location) { url = GetUrl(location); Http.AsyncGet(url, DownloadCallback);}DownloadCallback(tweets) { result = Process(tweets); UI.Dispatch(Render, result);}Render(result){ UI.Display(result);}

• Event Handlers are Upcalls• Function pointers point to potential Upcalls

– Callbacks are passed as function pointers

<Button Click=“ClickHandler” />

Identify Upcalls

Instrument Upcalls• Rewrite app

– Trace Upcalls

ClickHandler() { Logger.UpcallStart(1); GPS.AsyncSample(GPSCallback); Logger.UpcallEnd(1);}

GPSCallback(location) { Logger.UpcallStart(2); url = GetUrl(location); Http.AsyncGet(url, DownloadCallback); Logger.UpcallEnd(2);}

DownloadCallback(tweets){ Logger.UpcallStart(3); result = Process(tweets); UI.Dispatch(Render, result); Logger.UpcallEnd(3);}

Render(result){ Logger.UpcallStart(4); UI.Display(result); Logger.UpcallEnd(4);}



UI Thread

Background Thread

Background Thread

ClickHandlerStart

ClickHandlerEnd

RenderStart


End

GPSCallback Start

GPSCallback End

RenderEnd

Low Overhead

7000 times less overhead

UI Thread

Background Thread

Background Thread

UI Dispatch Call

Download Callback

Async GPS Call

GPS Callback

Async Http Call

UI DispatcherCallback

Async Calls and Callbacks

Async Calls and Callbacks• Log Callback Start

– We capture start of the thread

• Log Async Call

• Match Async Call to its Callback

GPSCallback(location) { Http.AsyncGet(url, DownloadCallback); Logger.AsyncCall(5);}

DownloadCallback(tweets) { Logger.UpcallStart(3); ....}

Download Callback

Async Http Call

Detour Callbacks

System

App

Http.AsyncGet

Async Call

DownloadCallback

DownloadCallback(tweets){}

Callbacks

Detour Callbacks

DetourCallback(tweets){

DownloadCallback(tweets);}

class DetourObject {

}

MatchId = 3

obj.DetourCallback

MatchId = 3

MatchId = 3

Http.AsyncGet(url, DownloadCallback); Http.AsyncGet(url, obj.DetourCallback);

obj = new DetourObject(DownloadCallback, MatchId++);

System

App

Http.AsyncGet

Async Call

DownloadCallback

DownloadCallback(tweets){}

while(...) {

}

Detour Callbacks




}

MatchId = 4

obj.DetourCallback

MatchId = 4

MatchId = 4


System

App

Http.AsyncGet

Async CallDownloadCallback(tweets){}

while(...) {

}

Http.AsyncGet(url, obj.DetourCallback);

Detour Callbacks




}

MatchId = 5

obj.DetourCallback

MatchId = 5

MatchId = 5


System

App

Http.AsyncGet

Async CallDownloadCallback(tweets){}

while(...) {

}

Http.AsyncGet(url, obj.DetourCallback);

• Detour Callbacks• Delayed Callbacks– Track Object Ids

• Event Subscriptions

Async Calls and Callbacks

Low Overhead


Capture

UI Thread

Background Thread

Background Thread

User Transaction

Users

App

Instrumenter

AppInstrumented

Analysis

Developer

Feedback

Server

Traces

Developer

AppInsight

AppStore

Users

App

Instrumenter

AppInstrumented

Analysis

Developer

Feedback

Server

Traces

Developer

AppInsight

AppStore

Aggregate Analysis

Group similar transactions– Same transaction graph

• Outliers– Points to corner cases

• Highlight common critical paths– Focus development effort

• Root causes of performance variability– Highlight what really matters in the wild

Async GPS Call Async Http Call

Network

GPS

Device

Compute

Users

App

Instrumenter

AppInstrumented

Analysis

Developer

Feedback

Server

Traces

Developer

AppInsight

AppStore

Developer FeedbackWeb based Interface

• Long User Transactions• Critical Path• Aggregate Analysis– Outliers– Common Case– Factors affecting

Users

App

Instrumenter

AppInstrumented

Analysis

Developer

Feedback

Server

Traces

Developer

Deployment

AppInsight

AppStore

Deployment

• 40 apps in the Windows Phone store• 30 to few thousand users• 6 months to 1 year of data• > Million user transactions

AppInsight Overhead

Battery

<1%

Binary Size

1.2%

Memory

2%

Network

4%

Compute

0.02%

Low Overhead

• Impact on app performance is minimal• Low resource consumption

Based on 6 months of data from 30 apps in the wild

User Transactions and Critical Path

• 15% of the user transactions took more than 5 seconds!– 30% took more than 2 seconds

• Top 2 edges responsible for 82% of transaction time

• Critical path analysis– Focus on less than half the edges for 40% of transactions

• Aggregate analysis – Pointed to factors creating performance variability– GPS, Device, Network, Sensors

Focus development efforts

App

Instrumenter

AppInstrumented

Analysis

Developer

Feedback

Server

Traces

AppInsight

Developer

Improved

Users

AppStore

App: My App

Problem: UI hog

AppInsight:• High performance variability in UI thread• Abnormal latencies only at the start of the session• System loading DLLs in the critical path– Up to 1 second

Fix:• Preload the DLL asynchronously when the app starts– Dummy call to DLL

Developer Case Study

App: Popular Search App

Problem: Radio wakeup delay

AppInsight:• High performance variability before network call• Radio wakeup delay in critical path• Dominating when users are on cellular networks– More than 2 seconds

Fix:Wakeup radio as the user starts typing before invoking the transaction



App: Popular Professional App

Problem: Slow transactions

AppInsight:• Custom instrumentation in the critical path– Affecting user perceived delay

Users

App

Instrumenter

AppInstrumented

Analysis

Developer

Feedback

Server

Traces

AppInsight

Developer

AppStore

Server-based User Transactions

Core processing of the user transaction is at the serverCore processing

Server

Input and sensors

User Interaction

App

Send request

NetworkParse and

renderReceiveresponse

App

Network

GPS Start

Http Request

GPS Callback

ClickHandler

UI Dispatcher

Download Callback

Server

Network + Server

• Joint app-server instrumentation– Azure services


Request Handler Start Request Handler End

Network Network

Server Threads

Spawn Workers

Send ResponseRequest Handler

GPS Start

Http RequestClick

Handler

Download CallbackGPS Callback

UI Dispatcher


Identify and fix bottlenecks at the app and the server

Network

Server

NetworkHTTP[“x-AppInsight-Id”]

Server


App App

NetworkNetwork

Uncontrollable and Variable Delays

Reading sensors

Radio state

Device type

(3G, 4G, WiFi, LTE, ..)RTT, tput, TCP, DNS

Device type

Server


App App

NetworkNetwork


Reading sensors

Radio state

Device type


Device type

Significant variability in user-perceived delays

Server


App App

NetworkNetwork


Reading sensors

Radio state

Device type


Device type

Significant variability in user-perceived delays

• Users with high uncontrollable delays– Poor user-perceived delays

• Users with low uncontrollable delays– Not producing the best quality result

Server


App App

NetworkNetwork


Reading sensors

Radio state

Device type


Device type

Tightly control user-perceived delays

Adapt to conditions in the wild

Server


App App

NetworkNetwork


AdaptDo less or more processing

Send less or more data

Meet end-to-end deadline

Trade-off on quality or quantity of the result


Request Response

Deadline

Server processing

• Trade-off quality of result for processing time– More time to process, better quality results

Server

Controlling the Server Delay

Request Response

Deadline

WorkerWorker

Worker

Worker

Worker

Server


Request Response

Deadline

WorkerWorker

Worker

Worker

Worker

Better result

Server


Deadline

Server

Server processing


• Servers keep fixed deadlines– No visibility into external delays

Server


App App

NetworkNetwork



Adapt

Timecard

App App

Server

GetElapsedTime();

PredictRemainingTime(responseSize);

Time elapsed since user interaction

Predicted downlink & app processing delay

Timecard

App App

Server

GetElapsedTime();


Adapt Processing Time

Desired user-perceived delay

Timecard

App App

Server

GetElapsedTime();



Trade-off on quality of the result


Timecard

App App

Server

GetElapsedTime();


Adapt Response

PredictRemainingTime(10KB);

PredictRemainingTime(5KB)

PredictRemainingTime(15KB);


Timecard

App App

Server

GetElapsedTime();


Adapt Response


Timecard

App App

Server

GetElapsedTime();


Better quality result


Timecard

App App

Server

GetElapsedTime();


• Analysis of 4000 apps– 80% of them are single request-response transactions

Challenges

Server

App App

Challenges

UI Thread

Background Thread

GPS Start

Http RequestGPS Callback

ClickHandler

Server

App

UI Dispatcher

Download Callback

AppHighly asynchronous

Server Threads

Challenges

Spawn Workers

Send Response

ServerRequest Handler

UI Thread

Background Thread

GPS Start


ClickHandler

App

UI Dispatcher

Download Callback

AppHighly asynchronous

Challenges

Server

App App

GetElapsedTime();

No single reference clock

Variable network delaysand processing delays


Transaction

Highly asynchronous

Timecard

Server

App App

GetElapsedTime();


Transaction

Transaction Tracking

TimeSync Delay Predictors

Timecard

App Instrument

er

Service

Developer

Timecard.dll

GetElapsedTime();



App

AppInstrumented

AppStore

Timecard

Server

App App

GetElapsedTime();


Transaction



Server Threads

Spawn Workers

Send Response


Transaction

UI Thread

Background Thread

GPS Start


ClickHandler

App

UI Dispatcher

Download Callback

App


Server Threads

Spawn Workers

Send Response


UI Thread

Background Thread

GPS Start


ClickHandler

App

UI Dispatcher

Download Callback

App


TC Transaction context (TC) attached to every thread

– Carry along timestamps, transaction information

TC

TC

TC

TC

TC

TC

TC

TC

GetElapsedTime();

Timecard

Server

App App

GetElapsedTime();


Transaction



TimeSync

GPS

• Sync with Cell Tower– Off my several seconds to minutes

• GPS– Does not work indoors

• Probing

Probe

• Typical probing techniques don’t work well– Radio wakeup delays and queuing delays– Errors more than 100ms – Energy inefficient (network radio tail energy)

TimeSync

• Radio-aware, network-aware probing– Probes when radio is active and network is idle

TimeSync in Timecard

Server

App App

TimeSync in Timecard

Probe

UI Thread

Background Thread

GPS Start


ClickHandler

App

~5ms errorEnergy efficient

Timecard

App App

Server

GetElapsedTime();


Predicting Remaining Time

App App

Server


Downlink delay App processing delay

Predicting Downlink Delay

LatencyThroughput

Loss rate

TCP window state• Most transfers are short– Median – 3KB– 90% percentile – 37KB

Analysis of 4000 apps

• Cellular networks– High-bandwidth, rare-loss,

high-latency,

• 99% transfers over HTTP– TCP window state matters -> multiple RTTs

Response size


TCP Window

1 extra RTT


Response sizeLatency

TCP parameters

Determined by RTT Number of round trips


• Use recent estimate of RTT– Use time sync probes

Read TCP window state

• Read TCP window state at serverMiddlebo

x

• Do not work with middleboxes– Split-TCP connection

• TCP window state at the middlebox matters– No easy way to estimate

TCP window state

Build an empirical data-driven model

RTT

Number of round trips

Predicting Downlink DelayPredictRemainingTime(responseSize);• Data-driven model

– Response size– Recent RTT– Network provider– Bytes already transferred in

the TCP connection o Proxy for TCP window state at the

middlebox

Middlebox

Downlink Delay Predictor

Cellular Median error - 17 ms, 90th percentile - 86 ms

Wi-Fi Median error - 12 ms, 90th percentile - 31ms

Learn

Predicting App Processing Delay

App App

Server


• Parsing and rendering delay– Correlated with response size– Correlated with device type

Timecard

App App

Server

GetElapsedTime();



Predicted downlink & app processing delay

Mobile Ads ServiceContextual ads to mobile apps

Mobile Ads Service

Fetch and process ads for keywords

Extract keywords Render adSend keywords

Ad

Mobile Ads Service


Ad

Ad providerAd provider

Ad provider

Ad provider

Ad provider

A

Keywords

Mobile Ads Service


Ad

Mobile Ads Service


Ad

GetElapsedTime();



Mobile Ads Service

Within 50ms of the target delay 90% of the time

With Timecard

Without Timecard

Target delay

User-perceived delay (ms)

• ~200,000 transactions

Timecard

App App

Server

GetElapsedTime();


Adapt Resources Used


Timecard

App App

Server

GetElapsedTime();


Request Prioritization

AppInsight

Timecard

Identify performance bottleneckso Critical path analysis

Control user-perceived delayso Elapsed time estimationo Remaining time prediction

User transaction trackingo Asynchrony, Time Sync

By the time it loads, the church service is over. Too slow!!!

Slow responsiveness, long load time, and crashity crash crash.

horrifically slow.Uninstalled.

Crashes every time. DO NOT DOWNLOAD. RIP-OFF!!

Slow and Crashes often. Fix. I will give 5 stars instead of 1.

So slow that my screen already dims before it's loaded

Too slow - killing the usefulness when you really need to go

Crashes all the time. Should be called Crashing News.

Crashed 3 times before I gave up

Slow or no network crashes the app immediately.

UI Thread

Background Thread

Background Thread

Exceptionat GetSentiment()DownloadCallback()

Stack trace

GetSentiment()……DownloadCallback()

Http.AsyncGet(url)

GPSCallback()

ClickHandler()

GPS.AsyncSample()

User Interaction

Exception Path

App Crashes

App Store

Slow and Crashes often. DO NOT DOWNLOAD. RIP-OFF!!

VanarSena MobiSys ‘13

• Fast UI automation– Based on transaction tracking– 4x - 40x reduction in testing time

Scalable Testing Infrastructure

• Automatic dynamic testing – Using an army of monkeys

Uncovered ~3000 crash bugs in 1100 apps in the app store

Apps

OS

Sensors Network

VanarSenaMobiSys ‘14


TimecardSOSP ‘13

ProcrastinatorMobiSys ‘14

Wireless ESP

NSDI ‘11

VTrackSenSys ‘09

CTrackNSDI ‘11

SmartAdsMobiSys ‘13

CodeInTheAir

HotMobile ‘12

CombineMobiSys ‘07

SixthSense

MobiSys ‘08

DAIRMobiSys ‘06

App—aware Operating Systems

NectarOSDI ‘10

Wearables

Mobile Support

Best Paper

Best Paper Nominee

Backup

• 0.1% runtime overhead

• Less than 1% memory overhead

• Less than 1% network overhead

• Negligible battery overhead

Timecard Overhead

Documents

Improving and Controlling User-Perceived Delays in Mobile Apps Lenin Ravindranath By the time it loads, the church service is over. Too slow!!! Slow responsiveness,