Embed Size (px)
Citation preview
Analyzing the Network Friendliness of Mobile Applications
version 1.0
Analyzing the Network Friendliness of Mobile Applications
Issue: 1.0
Date: 2012-07-05
Author: Song Jiantao
Email: [email protected]
Summary ..............................................................................................................................1
1 Preface ..............................................................................................................................2
2 Application Network Friendliness Optimization System .....................................................4
3 Method for Analyzing the Network Friendliness of Applications ........................................6
4 Apps Insider ....................................................................................................................10
4.1 Description ...................................................................................................................................................10
4.2 Apps Insider Case Study ................................................................................................................................12
5 Healthy Development of the Application Network Friendliness Optimization System .......14
6 References .......................................................................................................................16
1
Summary
In recent years, the development of smartphones has made great progress. Mobile application development for smartphones has become the next big thing in a post-PC era. But since most developers are lacking in their understanding of mobile networks and the behaviors of applications on mobile networks, these applications may be unfriendly towards mobile networks and cause the following problems: high device power consumption, frequent signaling storms on mobile networks, low utilization efficiency of network resources, deterioration in user experience, and threats to user privacy and network security.
To guide the development of network-friendly applications, the GSM Association has released Guidelines for Development of Network Friendly Applications, and hosted a “Smarter App Challenge” event. To further cultivate an application network friendliness optimization system, Huawei has provided the Apps Insider, an automatic tool for analyzing the network friendliness of applications. This white paper introduces the methodology of analyzing network friendliness, describes the Apps Insider toolkit, and gives a case study.
The Apps Insider assesses the network friendliness from the perspective of user experience and network impact respectively, including assessment indexes like user experience, device power consumption, signaling consumption, traffic consumption, connection consumption, multi-radio capability (UMTS/LTE/Wi-Fi), privacy and security.
The Apps Insider adopts the client/server model: clients collect the measurement results and report them to the server; the server calculates the average network friendliness score of each application based on the score of each assessment index and its weight; then it ranks applications by network friendliness. By analyzing the key network friendliness indexes, the Apps Insider provides suggestions on application development and network optimization to improve network friendliness.
Industry partners including developers and operators are welcome to use the Apps Insider in Huawei’s mLAB to improve the network-friendliness of applications. Huawei’s mLAB will also regularly release the network friendliness ranking of popular applications and provide suggestions for optimizing their network friendliness. Through continuous efforts to analyze, assess, optimize, and manage the network friendliness of mobile applications, win-win outcomes for operators, application developers, device vendors, network equipment suppliers, and end users will be made possible.
2
1 Preface
In recent years, the development of smartphones has made great progress. According to Gartner, Inc., worldwide smartphone sales to end users soared to 472 million units in 2011, up 58% from the previous year. Mobile application development for smartphones has become the next big thing in a post-PC era. Most mobile applications are developed based on the experience of fixed network applications. As a result, these applications may be unfriendly towards mobile networks and cause the following problems: high device power consumption, frequent signaling storms on mobile networks, low utilization efficiency of network resources, deterioration in user experience, and threats to user privacy and network security. To develop friendly applications for mobile networks, developers must have a comprehensive understanding of mobile networks and the behaviors of applications on mobile networks.
On UMTS networks, UE (user equipment) has two basic operation modes: idle and connected. In idle mode, the UE is in standby mode, no service is running, and the UE is not connected to the Universal Terrestrial Radio Access Network (UTRAN). When the Radio Resource Control (RRC) connection is set up, the UE is switched to the connected mode. In connected mode, the UE has the following states: Cell-DCH, Cell-FACH, and Cell/URA-PCH. In the Cell-DCH state, the UE has a dedicated channel (DCH or HSPA) and consumes more power to transmit data at a faster rate. In the Cell-FACH state, the UE consumes less power as data is transmitted at a lower rate. In the Cell/URA-PCH state, the UE has no data transmitted on either the uplink or the downlink and needs only the same amount of power as idle mode to retain the connection.
To efficiently utilize wireless resources and reduce power consumption, RRC state transitions can be performed by using the dynamic channel allocation algorithm based on the occupied radio link control (RLC) buffer. Signaling consumption varies according to the state transition path. To avoid a ping-pong state transition, the DCH2FACH transition is triggered when the deactivated timer T1 (2 to 10 seconds) expires after data transmission is completed on the DCH/HSPA. Similarly, the FACH2PCH transition is triggered when the deactivated timer T2 (2 to 10 seconds) expires.
The following is an example of the network-unfriendliness of a mobile application: A social network application obtains the updates of friends by frequently polling the server (known as a “heartbeat”). The heartbeat has little impact on fixed networks. On a wireless network such as the UMTS, however, a complete signaling process must be performed for each heartbeat, for every upward transition (PCH2FACH, FACH2DCH) and downward transition (DCH2FACH, FACH2PCH) of the RRC state on the signaling plane. The user may obtain no valuable information from the heartbeat (his/her friends may make no updates during heartbeat intervals), and the power and traffic are simply wasted. Frequent heartbeats cause frequent RRC state transitions, consuming large
3
quantities of signaling resources and increasing the CPU load of the interface board on the control plane.
Currently, network friendliness design and the optimization of mobile applications have gained extensive attention in both academic and industry fields. The GSM Association has created the Guidelines for Development of Network Friendly Applications. AT&T’s research institute has developed the Mobile Application Resource Optimizer (ARO) in cooperation with University of Michigan. The AT&T ARO consists of a data collector and data analyzer. The data collector tests applications by using a UE and records TCP sessions, HTTP packets and contents, screen video (3 fps), user input, battery level, and GPS/camera/Bluetooth usage, and sends the test results to the data analyzer. The data analyzer analyzes the impacts of mobile applications on wireless networks and device power consumption based on the wireless network model and device power consumption model.
Huawei’s mLAB provides a wireless network environment that integrates GSM, UMTS, LTE and Wi-Fi networks with the Apps Insider, an automatic tool for analyzing the network friendliness of applications. With the wide use of smartphones, malware is used to intrude on users’ privacy and attack wireless networks, posing great threats to mobile users and network security. The Apps Insider assesses user experience, power consumption, signaling consumption, traffic consumption and connection consumption of applications in terms of multi-radio (UMTS/LTE/Wi-Fi) capability, privacy and security.
4
AnalysisUse the Apps Insider to
analyze the UMTS/LTE/
Wi-Fi throughput rate,
RRC state transition
process, DL/UL signaling
consumption, terminal
power consumption,
connection consumption,
and privacy and security,
and save the analysis
results into the database.
Define baselines for assessing
the user experience, terminal
power consumption,
signaling consumption, flow
consumption, connection
consumption, multi-radio
capability, and privacy and
security, compare the analysis
results with the baselines,
score the network friendliness
of each application according
to the weight of each index,
and arrange the order of
applications according to the
aggregated score.
Identify key network
friendliness indexes
based on the comparison
result of each network
friendliness index with
its baseline, analyze the
key network friendliness
indexes, and provide
suggestions on application
development and network
optimization.
Establish an open
application network
friendliness assessment
center based on the
mLAB and formulate
regulations for assessing
and testing the network
friendliness of applications
to help developers publish
applications on application
stores, such as App Store
and Google Play.
Assessment
Optimization
Management
2 Application Network Friendliness Optimization System
Currently, most mobile applications are developed based on the experience of fixed network applications. Unfortunately, developers tend to typically lack an understanding of mobile networks and the behavior of applications on mobile networks. As a result, these applications may be unfriendly towards mobile networks and cause the following problems: high terminal power consumption, frequent signaling storms on mobile networks, low utilization efficiency of network resources, deterioration in user experience, and threats to user privacy and network security. Huawei’s mLAB aims to build an application network friendliness optimization system that can analyze, assess, optimize and manage the network friendliness of mobile applications to create win-win outcomes for operators, application developers, device vendors, network vendors and end users.
Figure 2-1 Application network friendliness optimization system
Operator
Operator
Terminal
Terminal
Network
Network
User
Fighting alone
Win-Win
User
Application
Application
5
● Analysis
Analyzes the following data and saves the analysis results into a database: Traffic Consumption (DL/UL, UMTS/LTE/Wi-Fi), RRC state transition, session number consumption, device power consumption, privacy and security measures.
● AssessmentDefine baselines for assessing user experience(key QoE indexes for different application categories, e.g. MOS for VoIP, Page Loading Latency for web browsing), device power consumption, signaling consumption, traffic consumption, session number consumption, multi-radio capability, as well as privacy and security, and compares the analysis results with the baselines, scores the network friendliness of each application according to the weight of each index, and arranges the order of applications according to their aggregated score.
● OptimizationIdentifies key network friendliness indexes based on comparison results and provide suggestions on application development and network optimization.
● ManagementBased on statistical analysis, app platforms like the Apple App Store and Google Play would be able to formulate required network friendliness rules and criteria for published apps. End users would also be able to select apps according to their friendliness ranking.
6
3 Method for Analyzing the Network Friendliness of Applications
The method for analyzing the network friendliness of applications is as follows: Collect data from the client and assistant tools (RNC signaling tracing tool, Agilent power consumption analyzer and Wireshark packet catcher), assess the applications in terms of user experience, device power consumption, signaling consumption, traffic consumption, connection consumption, multi-radio capability (UMTS/LTE/Wi-Fi), as well as privacy and security, then score the network friendliness of each application according to the weight of each index, and arrange the order of applications by aggregated score.
Figure 3-1 Method for analyzing the network friendliness of applications
Application /Classification
Weight of assessment indexes
Data collection and processing
Assessment and ranking
Assessment indexes
Optimization suggestions
Whatsapp/IM Wi-Fi DL/UL throughput rate traffic consumption Weight of traffic consumption
Assess the network friendliness of applications
User experience assessment report
Network impact assessment report
Network optimization suggestions
Application development suggestions
iMessage/IM3G/LTE DL/UL throughput Connection consumption Weight of connection
consumption
Rank applications in each type
viber/VoIP RRC state transition User experience Weight of user experience
Identify key network friendliness indexes
Youtube/Video
Duration of each RRC state Multi-radio capability Weight of multi-radio
Facebook/SNS
Signaling quantity Signaling consumption Weight of signaling consumptionSafari/Web
Power consumption analysis model
Terminal power consumptionWeight of terminal power
consumptionGmail/MAIL
APP security analysis Privacy and security Weight of privacy and securityiCloud/Cloud
7
The procedure for analyzing the network friendliness of applications is as follows:
Step 1 Classify applications and set the typical usage profile. In this step, select the applications to be tested, record the phone model, operating system version, name, and version of each application, and classify the applications. Applications can be classified into the following types and their corresponding usage profile could be predefined.
Step 2 Collect and process raw data. In this step, select various test scenarios including foreground with no user input, typical usage scenarios(multiple predefined usage profiles), and background, collect the raw data with assistant tools (air interface signaling analyzer, Gi interface IP packet analyzer, and Agilent power consumption analyzer) and the data reported by devices, then save the data into the database.
Calculate assessment indexes from raw data and save in database, re-generate baseline for each index.
Application
CategoryExamples Usage Profiles in Active State: Key Characteristics
Instant Messaging
(IM)
WhatsApp, MSN, QQ,
iMessage
Send/Receive Messages Per Hour
Average Message Size(MB)
Average Messaging Interval(Seconds)
Social Networking
Services (SNS)
Facebook, Twitter, Sina
Average User Profile Browsing Times Per Hour
Average Number of Posts/Comments Per Hour
Sent/Received with/without Pictures(Y/N)
Video Netflix, Youtube, Youku
Video File Formats(MP4/FLV/H.264/…)
Average Video File Size(MB)
Average Video Duration(Minutes)
VoIP Skype, Viber Average Call Duration(Seconds)
Web BrowsingChrome, IE, Safari,
Firefox, Opera, UCWeb
Average Page Size(MB)
Average Dwell Time on a Page(Seconds)
Cloud ServiceDropbox, Google Drive,
iCloud, SkyDrive
File Type of Syncing(DOC/PICTURE/CONTACTS/…)
Average File Size(MB)
Average File Syncing Interval(Seconds)
Email Hotmail, Gmail
Average Mail Size(MB)
Average Number of Mails Sent/Received Per Hour
Average Mail Transfer Interval(Seconds)
8
Figure 3-3 Device power consumption and signaling consumption estimation
based on RRC state transition
260mA
116mA
IDLE2D
F2D
P2F
D2F T1 = 2~10s
T2 = 2~10s
T3
F2P P2IDLE
3mA
RRC State Transition Process
Signaling Quantity
P2F;F2P 7
P2F;F2D;D2F;F2P 28
IDLE2D; D2F; F2P; P2IDLE 35
DCH/HSPA
FACH
PCHIDLE
Power consumption baseline in each RRC state
Signaling consumption
NOTE:
1. User Experience as assessment indexes excludes device power consumption, privacy and security, includes voice quality for VoIP service, page loading latency for web browsing, and video quality for video service.
2. Power consumption of a device can be calculated based on the power consumption baseline of each RRC state and the RRC state transition process instead of using the power consumption tool which is more efficient but less accurate than using power consumption reported by the device/test tool.
NOTE:
Power consumption baseline in each RRC state (Y axis) is the default value (based on mLAB test results) for estimation and could be changed with configuration GUI.
Figure 3-2 Mapping from Raw Data to Assessment Indexes and from Indexes to
Criteria
Collected Raw Data
Assessment Indexes
Assessment Criteria
Wi-Fi DL/UL throughput (bps)
3G/LTE DL/UL throughput (bps)
RRC state transition
Duration of each RRC state (ms)
Signaling quantity
User experience*
Traffic consumption
Power consumption analysis model
Multi-radio capability
Session consumption
MOS measurement
Signaling consumption
Network impact
TCP/UDP connections
Device power consumption*
User experience
TLS/SSL/ Encryption
Privacy and security
Privacy and security
9
● Assess the network friendliness of each application and arrange the order of applications by network friendliness. In this step, calculate the overall network friendliness score of each application based on the score by each assessment index and weight of each assessment index, and arrange the order of applications by network friendliness in each type.
● Set the weight of each assessment index according to the application type.
Step 4 Provide optimization suggestions. In this step, analyze the key network friendliness indexes, and provide suggestions on application development and network optimization to improve the network friendliness of applications.
NOTE: More black area means higher weight
IM App1 Scores
indexes baseline score
Video App2 Scores
indexes baseline score
SNS App3 Scores
indexes baseline score
Step 3 Score and Ranking ● Compare with baseline to score the network friendliness of each application by
each index. Each index is evaluated by its ranking sorted by the network impact and user experience, and the score of each index is calculated by its ranking result. The higher the network impact, the smaller the score. We set the score range from 0 to 5, with a maximum score being 5.
App Category
Network Impact User ExperiencePrivacy & Security
Traffic Session Signaling Multi-RadioUser
ExperiencePower
ConsumptionPrivacy & Security
IM
VoIP
Web
Video
SNS
*
10
4 Apps Insider
To accurately analyze the network friendliness of smartphones and applications, Huawei’s mLAB provides a wireless network environment that integrates GSM, UMTS, LTE and Wi-Fi networks with the Apps Insider, an automatic tool for analyzing the network friendliness of applications.
4.1 Description
The Apps Insider adopts the client/server model. The Apps Insider clients collect the measurement results and report them to the server. The Apps Insider server calculates the average network friendliness score of each application based on the score of each assessment index and its weight, and ranks the applications by network friendliness. By analyzing the key network friendliness indexes, the Apps Insider provides suggestions on application development and network optimization to improve the network friendliness of applications.
Figure 4-1 Framework of the Apps Insider
User experience assessment report
Network impact assessment report
Network optimization suggestions
Application development suggestions
Apps Insider server
Analyze the network
friendliness of applications
Select assessment
objects RNCNodeB
CN
mLAB Apps Insider toolkit
Apps Insider client
IM Video
SNS
VoIP
Gi interface IP packet analysis
Air interface signaling analysis
Power consumption
analysis
11
● Apps Insider clientThe Apps Insider clients include applications on Android, iPhone and Windows smartphones. Select the application to be assessed, for example: Sina Weibo. Classify the application, for example: Sina Weibo is a Social Networking Service (SNS). Set test scenarios such as foreground with no user input, normal use, background, start time and end time. Select assessment objects, such as the E2E delay, RTT, DNS response time, DL/UL traffic and power consumption. Report measurement results to the server. Display final assessment results.
● Apps Insider assistant toolsThe Apps Insider’s assistant tools include the air interface signaling analyzer, Gi interface IP packet analyzer, Agilent power consumption analyzer, and MOS analyzer.
● Apps Insider serverThe Apps Insider server collects information from the device, power consumption analyzer, and Gi interface IP packet analyzer, and saves the information into a database. Define baselines for assessing the network friendliness of applications in terms of user experience, device power consumption, signaling consumption, traffic consumption, connection consumption, multi-radio capability, and privacy and security, and provides suggestions on application development and network optimization.
12
● Rank applications by network friendliness.
Figure 4-2 Ranking of SNS applications by friendliness
4.2 Apps Insider Case Study
This case analyzes the network friendliness of three popular SNSs (AppA, AppB andAppC) on the iPhone and Android.
NOTE: User experience indexes vary by type of application. VoIP user experience is assessed by the MOS analyzer.
● Log in to AppA, AppB andAppC, and perform no operations within 20 minutes in the foreground.
Phone ModelOperating
System
Mobile
Application
Application
Version
Application
TypeTest Scenario
Test Duration
(Minutes)
iPhone4 iOS 5.0 AppA a SNS Foreground 20
iPhone4 iOS 5.0 AppB b SNS Foreground 20
iPhone4 iOS 5.0 AppC c SNS Foreground 20
Huawei Honor Android 2.3.6 AppA d SNS Foreground 20
Huawei Honor Android 2.3.6 AppB e SNS Foreground 20
Huawei Honor Android 2.3.6 AppC f SNS Foreground 20
Assessment Dimension Value RangeWeight (a default value configurable according to
application type)
Connection quantity (0, 5] 0.1
Traffic consumption (0, 5] 0.2
Signaling consumption (0, 5] 0.2
User experience* (0, 5] 0.1
Terminal power consumption (0, 5] 0.2
Multi-radio capability (0, 5] 0.1
Privacy and security (0, 5] 0.1
● Define the weight of each assessment index.
