Cross Platform Development tools for mobile applications ...754436/FULLTEXT01.pdfCross Platform Development tools for mobile applications, a performance and cost evaluation MÅRTEN

Cross Platform Development tools for mobileapplications, a performance and cost evaluation

MÅRTEN PÅLSSON

Master’s Thesis at CSCAuthor: Mårten Pålsson || [email protected]

Supervisor CSC: Sten Andersson || [email protected] Supervisor: Olle Bergling || [email protected]

Examiner: Anders Lansner || [email protected]

TRITA xxx yyyy-nn

AbstractSince the introduction of smart-phones, cellular phones havechanged from relatively simple communication devices tobecoming more akin to small computers in their own right.The smart-phone market is riddled with different platforms,with the most popular being Google’s Android, and Apple’siOS. Each of these platforms use their own operating sys-tems and technologies, making the smart-phone eco-systemhighly fragmented. This leads to problems for companieswishing to target more than one platform, since applica-tions developed for one device will not work on another.To solve this problem several different cross-platform de-velopment tools have appeared. These tools aim to providedevelopers with a solution along the lines of develop onceand deploy anywhere. Cross-platform development toolscome in many different forms and utilize different technolo-gies to allow applications to work on different platforms.

This project has evaluated two of these approaches inorder to to provide a suggestion on which is the most suit-able for a company wishing to develop applications for mul-tiple platforms. The evaluation has been based on a perfor-mance and cost analysis between a web-to-native wrapperand a source code translator.

The result of this work is two-fold. Firstly the poten-tial financial gains are based on the skill set of the com-pany in question, and no clear differentiation can be seenbetween the two approaches. Secondly, while source codetranslators provide better performance compared to web-to-native wrappers, web-to-native wrappers are more flexi-ble and thus generally a better alternative when developingapplications for multiple platforms.

ReferatEn prestanda- och kostnadsanalys mellan

korsplatformsverktyg för mobila applikationer

Mobiltelefoner har utvecklats snabbt de senaste åren. Frånatt vara relativt simpla verktyg mest inriktade på telefo-ni, så har de utvecklats till små datorer med avanceradfunktionalitet och hårdvara. Smart-phones, som de numerakallas, har tagit världen med storm och det finns ett stortantal olika plattformar på marknaden, varav de största ärAndroid och iOS. Varje plattform har ett eget operativsy-stem som är baserad på egna teknologier, vilket har letttill att smart-phone marknaden blivit väldigt fragmente-rad. Detta ställer till problem för företag som vill utvecklaapplikationer till flera platformar då en applikation somutvecklats till en platform inte fungerar på andra. För attlösa det här problemet så har korsplatformsverktyg trättfram. Dessa verktyg fungerar på många olika sätt och ut-nyttjar olika teknologier för att tillåta utvecklare att skapaapplikationer som går att köra på flera olika plattformar.

Det här projektet har jämfört två av dessa olika tillvä-gagångssätt, baserat på en prestanda- och kostnadsanalys,för att kunna föreslå vilket är det lämpligaste alternativetför ett företag som vill utveckla applikationer till flera plat-formar. Jämförelsen har gjorts mellan web-to-native wrap-pers och source code translators.

Resultatet av detta arbete är tvådealat. Den potentiellafinansiella vinsten är densamma för båda tillvägagångssät-ten och är baserad på den kompetens som redan finns in-om företaget. Vad gäller prestanda så presterar source codetranslators bättre än web-to-native wrappers. Däremot såär web-to-native wrappers mycket mer flexibla vilket gördem till ett generellt sett bättre alternativ att att användaför att utveckla appliaktioner till flera platformar.

Contents

1 Introduction 11.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.1.1 Smart-phone ecosystem . . . . . . . . . . . . . . . . . . . . . 21.2 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.3 Delimitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Frame of Reference 52.1 Native development . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.1.1 Apple iOS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.1.2 Google Android . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.2 Cross-platform development . . . . . . . . . . . . . . . . . . . . . . . 62.2.1 Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.2.2 Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

3 Method 133.1 Performance evaluation . . . . . . . . . . . . . . . . . . . . . . . . . 13

3.1.1 Application Design . . . . . . . . . . . . . . . . . . . . . . . . 143.1.2 PhoneGap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153.1.3 MoSync . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153.1.4 Android and iOS . . . . . . . . . . . . . . . . . . . . . . . . . 153.1.5 Experiment setup . . . . . . . . . . . . . . . . . . . . . . . . . 163.1.6 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

3.2 Cost evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163.2.1 Questionnaire . . . . . . . . . . . . . . . . . . . . . . . . . . . 173.2.2 Interview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

4 Results 194.1 Performance evaluation . . . . . . . . . . . . . . . . . . . . . . . . . 194.2 Cost evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

4.2.1 Interview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214.2.2 Questionnaire . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

5 Discussion 255.1 Performance evaluation . . . . . . . . . . . . . . . . . . . . . . . . . 25

5.1.1 PhoneGap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255.1.2 MoSync . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

5.2 Cost evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

6 Conclusion 296.1 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

7 References 31

Chapter 1

Introduction

1.1 Background

Since the introduction of smart-phones, cellular phones have changed from relativelysimple communication devices to becoming more similar to small computers. Theyhave acquired increased processing power and functionality such as GPS, Internetconnectivity and cameras. The smart-phone ecosystem is the home of several dif-ferent platforms with the most popular being Apple’s iOS, Google’s Android andWindows’ Windows Phone [20]. Each of these platforms have their own operatingsystems, development tools and programming languages.

For companies it is desirable to reach as many markets as possible to increasetheir profits. However, the variety in platforms and the differences between themmakes this an expensive prospect for companies developing mobile applications.Since each platform has its own programming language and run-time environment,applications developed for one platform cannot run on another. This means thatdeveloping an application for a specific platform requires intimate knowledge of thatparticular platform. It does not make things any easier that there are several differ-ent manufacturers producing devices for the same platforms - such as Samsung andHTC producing devices for Android. Companies wishing to target several platformswill have to acquire the competence to do so (such as programming languages andplatform architecture), as well as either dedicate the resources needed for developingthe same application in parallel for the different platforms, or spend twice as muchtime to develop the same application twice. This makes finding a way to developapplications from a single code base, that work on several different platforms, anattractive prospect.

Fortunately, there exists several different methods to develop cross-platform ap-plications. One way to do this is to develop a pure web based application that runsin the device browser. However, this approach has some inherent restrictions com-pared to native applications. Among these are the reliance on constant connectionto the Internet, as well as possible restrictions on which device level functionality(such as GPS and camera) the platform’s browser supports [17].

1

CHAPTER 1. INTRODUCTION

To address these problems several tools to develop native cross-platform appli-cations have emerged to offer a solution along the principle of “develop once, deployeverywhere”. There are numerous approaches, and combinations thereof, for thesecross-platform development tools (CPTs), but generally the different methodologiesused can be split into three broad categories; Web-to-native wrappers (an exampleis PhoneGap), source code translators (examples are XMLVM and MoSync) andrun-times (an example is RhoMobile)[17].

1.1.1 Smart-phone ecosystemThe smart-phone ecosystem is in a constant state of flux. An example is the riseand fall of Nokia’s operating system Symbian; from a dominating 52.4% marketshare in 2009 [27], to Q3 2013 when it only held a 7% market share [29]. In 2009Android was a newcomer to the smart phone market but was already predicted tobecome a big player in the future. Predictions were that Android would be secondbiggest actor on the smart phone market by 2013 after Nokia [27]. These predictionswere proven to be incorrect as by Q3 2013 Android held a dominant position onthe smart phone market with a 57% market share with iOS coming second, andSymbian – as stated above– only a shadow of its former glory.

0 10 20 30 40 50 60

Android

iOS

Symbian

BlackBerry OS

Windows Phone

Other

57

19

4

5

5

9

54

14

7

6

5

13 Q3 2013Q1 2013

Figure 1.1: Mobile OS growth in 2013. Source: (Global Web Index Q3 2013)

As can be seen in Figure 1.1 on mobile OS growth in 2013: Android and iOSare dominant with 57% and 19% market shares respectively, while everyone elseeither lost shares or stayed the same. Thus it makes sense for a software companydeveloping mobile applications to target Android first, and then expand to iOS.If an application were to be developed for both platforms it would reach a totalof 76% of the global smart phone market. In statistics provided by Priori Data(2014)[19], on the total amount of downloads in the USA of the 150 most popularapplications, it can be seen that only 8% for iOS and 0,3% for Android are paid for.These numbers are nowhere near conclusive, as you would need a complete overlook

2

1.2. PURPOSE

of the global application market, but they do give an indication of a trend. Thatis, that iOS users are more prone to pay for their applications than Android users.This could give developers added incentive to target multiple platforms, since eventhough Android has the largest user base, the iOS user base proves more lucrative.

1.2 PurposeThe purpose of this report is to evaluate different CPTs to find which approachis most suitable for a software development company wishing to develop nativeapplications targeting multiple platforms. The evaluation focus will lie on a costand performance comparison between using CPTs and native development.

1.3 DelimitationsThis work will evaluate one web-to-native approach and one source code translator(PhoneGap and MoSync respectively) and the native platforms targeted will be Ap-ple’s iOS and Google’s Android. The software development company that will beused as a basis for the evaluation will be Tekis AB. To further restrict the scope ofthe project the performance evaluation will be performed on a select group of hard-ware features; namely geolocation, magnetic field sensor (henceforth denominatedcompass) and file system access.

3

Chapter 2

Frame of Reference

2.1 Native development

Native application development refers to writing software for a specific platform, i.e.,a specific set of hardware, operating system (OS), processor and its architecture [21].Some advantages of native application development are: reliable performance, de-vice APIs, platform SDK, established distribution channels, and reliable support[16,21]. However, because of this specific development strategy, native applications willnot run on any but their native platform.

Native applications are executable files that run on a device and interact directlywith its OS [21]. This means that they can fully access device functionality (e.g.,calender and contacts) and hardware (e.g., camera and file system). The nativeAPIs also provide well developed abstractions for user interfaces (UI) and otheruser interactions and as such provide the best user experience possible [16].

2.1.1 Apple iOS

Apple’s iOS is a proprietary mobile operating system developed and distributedby Apple Inc [31]. It was originally released for iPhone and iPod touch devices in2007 [31] but has since been extended to the iPad family [12]. iOS is derived fromMac OS X [24] and shares many of its features. As such it is a Unix based systemand is the mobile version of the operating system (OS X) that Apple uses on itscomputers.

iOS applications are written in Objective-C [11], and to write and compile iOSapplications a Mac and Xcode is needed [13]. Xcode is Apple’s Integrated Develop-ment Environment (IDE). To test applications in Xcode’s internal emulator is free,but to test the application on a device, or to distribute it through the app store, apaid license is needed [14].

The iOS architecture is layered in the following descending order; the CocoaTouch layer (user interaction), the Media layer, the Core Services layer and the CoreOS layer [11]. This abstraction exists to relieve developers of having to communicate

5

CHAPTER 2. FRAME OF REFERENCE

with the hardware directly and encapsulates complex features such as threading andsockets [11].

2.1.2 Google Android

Android is a Linux based open source operating system [25] primarily designed fortouch screen devices such as tablets, smart-phones and smart watches. Developedby Android Inc and later purchased by Google, Android was unveiled in 2007 [31]together with the founding of the Open Handset Alliance. The Open HandsetAlliance is a group of companies led by Google to further open standards for mobiledevices [26]. The first smart-phone using the platform, the HTC Dream, was soldin 2008 [31].

Android code is released by Google under the Apache open source license [26].The Android Open Source Project (led by Google) is responsible for maintainingand further developing the platform [26]. Due to the open source nature of Android,and its popularity, there is a large community of developers writing applications forthe platform and extending its functionality [9].

Android applications are written in the Java programming language with theSDK providing support for packaging and compiling C and C++ code [10]. Ap-plications are compiled using the Android SDK and can be developed in Eclipseusing the Android Developer Tools plugin, or Google’s own Android Studio [10].All Android applications execute within their own virtual machine (VM), creatinga secure environment [26], ensuring that no applications can access other applica-tions’ data unless specific permission is given. In Android all applications are equal,with no differentiation between core and third party applications [7]. This meansthat users can fully tailor their phone to their needs and access a broad spectrumof applications and services.

2.2 Cross-platform development

There are several different cross-platform development tools (CPT) for developingcross-platform applications, using a variety of approaches.

2.2.1 Approaches

In a large online survey on cross-platform development done by VisionMobile [17]five distinct approaches were defined; Javascript frameworks, App factories, web-to-native wrappers, runtimes and source-code translators. These denominations willhenceforth be used since they adequately describe the different approaches. Appfactories will not be mentioned in this report as they target non-developers, andthis report focuses on low level software development.

6

2.2. CROSS-PLATFORM DEVELOPMENT

JavaScript frameworks

Javascript frameworks are tools to create web applications. Since all platforms havean integrated browser with HTML/CSS compatibility the simplest way to reachmultiple platforms is through web programming. All users can reach the applicationfrom any platform and you can reach a large market quickly. Accessing the webapplication requires an Internet connection, which is not always desirable dependingon the purpose of the application in question, or in areas with bad coverage. Anotherkey problem is that different platform browsers implement diffrent HTML5 APIs,for example the Android browser supports WebGL while iOS Safari does not [28].This means that developers will have to be aware of which functionality is or is notsupported in each browser. To address the issue of platform browser fragmentationthere exist tools such as jQuery Mobile [30]. jQuery Mobile and similar solutions areJavascript frameworks that aim to ease cross-platform web application developmentby providing a common API across all platforms.

Web-to-native wrappers

Web-to-native wrappers are tools that aim to deliver native applications using webtechnologies. An application created using a web-to-native wrapper is basically aweb application packaged inside of a native application shell. More specifically theapplications are written in HTML/CSS and Javascript, and execute within a We-bView on the device. A WebView is a native application containing a chromelessbrowser, i.e., a browser without navigation capabilities. Since all platforms sup-port exposing their native code to JavaScript code inside WebViews [16], packagedJavaScript API extensions can be used to access device level functionality beyondwhat the device’s browser is usually capable of, such as; notifications, accelerometer,compass and the file system [28].

To run truly native, and be free from constant internet connection, web-to-native wrappers bundle all necessary resource files into the native application [35].This allows the WebView to access all the resources it needs such as HTML andJavaScript files from the application itself, and function completely offline on thetarget device.

The most popular example of web-to-native wrappers is PhoneGap, with itsJavaScript API being used by other tools (e.g., WorkLight and AppMobi) [17].

Source code translators

Source code translators translate, or cross-compile if you will, the application’ssource code into native language code (e.g Objective C for iOS and C# for Win-dows Phone), intermediate bytecode or directly to low level machine code. Thiscode is then either built into an executable binary that can run on the device orcombined with a runtime [17]. Examples of source code translators are MoSync andXMLVM. XMLVM translates Java code into intermediate XML code that is thencross-compiled to the native code of the target platform [32].

7


Runtimes

A runtime is an execution environment that is located between the applicationand operating system and acts as a cross-platform compatability layer. A runtimeshields the application from the differences in the underlying platforms and containsa common API that the application can call to access device functionality. Run-times vary in their implementation and execute application code on the device usingseveral different methods including: virtualization, interpretation, just-in-time com-pilation and ahead of time compilation [17]. Examples of runtimes are Appcelerator,RhoMobile and Adobe Flex.

2.2.2 Tools

PhoneGap

PhoneGap is a web-to-native wrapper. The PhoneGap framework is essentiallya framework that allows web applications to function offline and to access devicefeatures. It usually consists of two parts; One part that includes the JavaScriptbusiness logic that drives the UI and its functionality, and one part that worksnatively to access and control the device [23]. Since PhoneGap only consists ofHTML/CSS, JavaScript and native code, PhoneGap applications can be developedusing the IDE of any platform it supports. A PhoneGap applications interfaceconsists of a single WebView which, upon launching the application, loads a HTMLstart-up page. Control is then passed to the WebView to allow the user to interactwith the web application. Javascript code executed from the WebView can pullresources from the resource files bundled with the application, or data from the webjust like a normal web application [34].

The UI consists entirely of HTML and CSS code and is specified in the HTMLsource files [34]. Switching between views and pulling resources as needed is im-plemented with JavaScript. PhoneGap can make requests for resources and pagesasynchronously, and manage the web page content dynamically [34]. This meansthat it can load resources into the HTML page only as they are needed, instead ofloading entire pages, which makes it possible to create applications that are moresimilar to native applications. Since PhoneGap uses HTML5 it is possible to storeall of the needed resource files on the device and thus remove the need for constantconnection to the web [34]. This makes the application native since it contains allthe resources it needs, e.g., SQL databases, stored on the device.

Since PhoneGap bundles all of the required HTML, CSS, and Javascript files,it is not susceptible to the single origin policy (which means that websites are onlyallowed to run scripts originating from the same site) since the application does nothave a domain.

Through PhoneGap’s plugin framework it is possible for users to add new fea-tures that are not currently available in the PhoneGap API [23]. To write sucha plugin two files are needed; a JavaScript file and a native language file. TheJavaScript file acts as the interface between the PhoneGap application and the plu-

8


gin as can be seen in Figure 2.1. Plugin functionalities are accessed by the Javascriptfile using JavaScript functions. The native language file includes the native platformcode to access the desired features or perform heavy tasks.

Figure 2.1: PhoneGap Architecture. Source: (IBM)

All of the platforms supported by PhoneGap have their own proprietary toolsfor packaging applications [34]. This means, for example, that they all use differentconfiguration files and folder structures. Also while the calls to the PhoneGap APIare the same for all platforms, the internal JavaScript code to access device featuresdiffers [34]. This makes it necessary for developers to have an instance of eachdevelopment environment in order to package and build their application for thedifferent platforms.

MoSync

MoSync SDK is an open source development environment which enables program-mers to develop applications for all of the major smart-phone platforms from a singlecode base. It allows developers to code and build applications through their ownIDE, based on the open source project Eclipse, in either C++ or HTML/CSS andJavaScript, or to use the two approaches together to create hybrid applications [2].

Applications created with MoSync are built to their target platform using GCC(GNU Compiler Collection), Pipe-Tool, and the MoSync Packager [4]. The MoSyncIDE uses GCC to compile the code to MoSync intermediate language which is thenfed into the Pipe-Tool. The Pipe-Tool combines the functions of a resource compiler,linker, and code optimizer. It optimizes the code, builds the code, links it with pre-built MoSync libraries and other resources, and then outputs byte-code or Java

9


which is then ready to be packaged with the appropriate runtime. The Packagerthen creates a runnable package for the target platform by combining the resultwith the appropriate runtime, or it produces a project that needs to be recompiledwith another tool. The latter case happens when trying to compile an applicationfor Windows Phone or iOS since both of these platforms require compilation in theirown native development environments (Visual Studio for WP and Xcode for iOS)to create an installable package for that platform [4].

MoSync has two primary runtime architectures, one for C++ and one for theweb-based languages. The design in both cases is very similar and consists mainlyof 6 components [4]:

Service layer Incorporates supporting functions such as file I/O, threading, net-working and memory management.

Application framework Holds the runtimes’s entry point. Its functionality varieson different platforms but it generally handles platform specific event man-agement, initialization and destruction.

Syscalls Implements all the basic features required for all platforms such as camera,images, graphics, audio, and networking. It is also responsible for interpretingMoSync resource files and supporting event management, initialization anddestruction.

Resource system Provides functionality for managing resources such as images,sounds and data. It dynamically creates and destroys resources as well asconditionally loading resources depending on the platform, e.g., the device’sscreen size.

Internal extensions Specifies the configuration of each OS.

Core Is responsible for the execution of MoSync applications and cooperates withthe syscall and resource system. There are three main cores, the VM core,recompiler core and generated core, but they all share a common interface.This allows all other modules to be shared across runtimes without modifica-tion. The VM core is a virtual machine that loads, interprets and executesbytecode. The recompiler core loads MoSync bytecode and recompiles it intoplatform specific code. The generated core does not interpret or recompilebytecode, instead it runs MoSync intermediate language code precompiledinto native code. The different cores are used for different platforms.

MoSync provides several different approaches for designing an application UI, basedon the developers skillset and preferences. These approaches include designing theUI in HTML/CSS, a NativeUI JavaScript API and a NativeUI C++ library [1].Developing the UI in HTML/CSS is identical to developing any other web applica-tion and provides an identical experience on all platforms [1]. Using the NativeUI

10


JavaScript API gives the developer access to native widgets such as tabs and but-tons directly from HTML or JavaScript [3]. The NativeUI C++ library providesfunctions and classes for control of native UI elements from C++ code [5]. Both theJavaScript API and the C++ library map against a widget C API in the SysCallmodule to produce the native widgets of the target platform [3, 5].

11

Chapter 3

Method

The purpose of this report is to evaluate different approaches used by CPTs, in or-der to be able to make a recommendation on which is most suitable for a companywishing to develop mobile applications. To do this a performance and cost evalua-tion has been performed between two different open source CPTs and native codingtechnologies for the iOS and Android mobile operating systems. The two CPTschosen for the task were PhoneGap and MoSync. PhoneGap was chosen becauseit is one of the most popular web-to-native CPTs on the market today. Its APIis used in several other web-to-native wrapper tools and it has a large user base.MoSync was chosen because it is one of the most popular open source source-codetranslators and it has a large user base.

3.1 Performance evaluation

According to Corral et al [18] the performance of a mobile application can be mea-sured with a number of parameters. These parameters are execution time, memoryusage, and battery consumption. Memory usage and battery consumption are be-coming less of a problem with modern devices since smart-phone hardware is becom-ing more and more potent. They can also be somewhat circumvented by competentprogrammers by implementing more efficient algorithms, something that is outsideof the scope of this project. The application footprint, i.e., the size of the appli-cation when installed on the device, can also be used as a parameter, but this isbecoming a diminishing problem as smart-phone storage space grows. The storagespace available in the original HTC Desire launched in 2010 was 512 MB [15] whilethe Samsung Note 3 released in 2013 has 32 GB available for the user [33]. Thisstudy will evaluate the execution time parameter, since it directly affects the userexperience. It also illustrates the differences in performance betweeen different ap-proaches, since it captures the overhead of when an application interacts with thesystem hardware.

Evaluating a performance metric is not as simple as measuring and noting thedata. A proper environment must be set up to fairly test the different CPTs on the

13

CHAPTER 3. METHOD

two platforms and appropriate methods for data evaluation must be used [22].To properly understand the impact on performance of the three different ap-

proaches – PhoneGap, MoSync, and native coding – a set of benchmarks wereimplemented. To do this a set of software routines were chosen, that utilise dif-ferent software and hardware resources on a mobile device. These features wereimplemented in an application identically coded in four different versions; one usingPhoneGap, one using MoSync and two coded using the native technologies for An-droid and iOS respectively. The aim was to create an environment where these threeapproaches, viz. PhoneGap, MoSync, and native coding, could be fairly compared.

3.1.1 Application DesignThe applications consist of a graphical user interface with a set of button used bythe controller to call methods that benchmark a given feature. The goal of eachmethod is to make a call to a software or hardware resource from the operatingsystem and wait for a response. The method measures the time from before makingthe call to after completion and records it in a file for later analysis.

In order to make a realistic comparison between the different approaches a se-lection of features from different system resources were chosen for benchmarking;the compass, geolocation and reading and writing to the file system. The compassbenchmark tests the performance of accessing hardware on the device. Geolocationtests both hardware access and network access as the devices location is calculatedthrough using both the GPS and network information. Finally manipulating thefile system tests the performance of the application’s data access.

Compass

The compass benchmark consists of requesting updates from the device’s magneticfield sensor, starting a 10 second timer, and counting the amount of heading updatesreceived from the sensor before the timer expires. This benchmark is designed tomeasure performance in a situation that simulates real world usage, where a useris more likely to request heading updates in short bursts, to get more accurateheading information, rather than requesting a single update. The benchmark isrun 20 times in order to gather enough information to be able to make a usefulstatistical analysis.

Geolocation

The geolocation benchmark is also, like the compass benchmark, designed to testperformance during conditions similar to real world use. Updates are requestedfrom the location sensors on the device, a timer is started, and the number of up-dates received until the timer expires are recorded. Unlike the compass benchmarkthe geolocation benchmark is run for a full minute. This is because it can takesome time for the sensor to connect to the GPS satellites. The applications wereprogrammed to receive updates from the sensor as quickly as possible, taking no

14

3.1. PERFORMANCE EVALUATION

consideration to battery consumption, and to discard cached locations to force thecontinuous acquisition of new ones. This is to fairly test the maximum sensor accessperformance of each approach, without taking software logic into the equation. Thisbenchmark was run 20 times.

File system

The file system benchmark is the simplest of the benchmarks, it reads or writes anarray of bytes from or to a file. Two array sizes were used to analyze the addedoverhead from using the CPT’s. The array sizes used were 32 bytes and 8 kilobytes.Each benchmark was run 100 times.

3.1.2 PhoneGap

The PhoneGap version of the application was coded using JavaScript, HTML andCSS. The different hardware and software routines used by the application wereimplemented using the PhoneGap API.

The first version of the PhoneGap application was coded for Android, but dueto the nature of the different operating systems a few changes had to be made forthe iOS version. The biggest change was the way the file system was handled.Since iOS does not allow applications to manipulate the file system outside its ownenvironment, which is allowed in Android, changes were made to the paths used bythe application. Otherwise only cosmetic changes were made to the code.

3.1.3 MoSync

The MoSync application was coded using C++ and like with the PhoneGap version,MoSync’s own API was used to access device resources.

The first version of the application was coded for Android and later ported toiOS. However MoSync AB has dropped development of the MoSync SDK due tobankruptcy. This meant that presently the MoSync SDK is not compatible with thenewest version (7.1) of iOS. Unfortunately this turn of events was not discovereduntil late in the implementation phase of the project so there was no time to finda solution or workaround. As such the MoSync version of the application was onlyimplemented for Android.

3.1.4 Android and iOS

The native versions of the application were implemented with Java for Android andObjective-C for iOS. The software routines were implemented according to bestpractice as proposed by the respective platforms.

15

CHAPTER 3. METHOD

3.1.5 Experiment setup

Each of the applications were run on a real mobile device, a Samsung Galaxy S3for the Android versions and an iPad 2 for the iOS versions. Each application canmake system calls to access specific resources on the device. Before each system calla time-stamp is taken and another directly after completion. This ensures that thewhole time span from request to acquisition is taken into account. Since millisecondsis the highest resolution available in JavaScript this is the resolution chosen for theexperiment.

3.1.6 Analysis

The analysis on the performance evaluation, between the different approaches, wasbased on the recommendations proposed by Fleming and Wallace [22]. First theresults must be normalized to a known machine in order to correctly representrelative system performance. Then the geometric mean is calculated to averagethe normalized numbers. These results can then be compared and used to drawconclusions.

Java was considered to be the known machine on the Android platform since itis the native environment, in the same way Objective-C was chosen as the knownmachine for iOS.

3.2 Cost evaluation

For a company not already developing mobile applications, adapting CPTs contrausing native coding techniques to develop applications would have different financialimpacts. To analyze this difference, the demands of the different approaches, andwhat it would take for a company to adopt them, must be understood. Since the dif-ferent approaches use different programming languages and (in the case of MoSync,Android and iOS) require different development environments, different skill setswithin a company directly affects the cost of adapting any of the approaches. Tofind out how these different approaches would affect Tekis AB a questionnaire wasdesigned and distributed among the company’s developers in order to find out whatform of programming skills and experience Tekis contains.

Another important factor is how the company tackles new projects and acquiresthe required knowledge for them. Accordingly an interview was held with the headdeveloper Pär Gillander.

The results of these two investigations were then combined and analyzed to givean estimation of the potential costs of using the different CPTs compared to nativedevelopment techniques to produce mobile applications.

16

3.2. COST EVALUATION

3.2.1 QuestionnaireThe following questionnaire was distributed to the software developers at Tekis AB.Each part of the multiple part questions were to be answered on a scale from 1 to5.

• How great is your skill in the following programming languages?

– C– C++– Objective-C– Java– JavaScript

• How much experience do you have with the following development environ-ments?

– Eclipse– Xcode– Visual Studio

• How much experience do you have developing software for the following mobileplatforms?

– Android– iOS

3.2.2 InterviewThe interview was an open interview on how Tekis approaches new projects, or-ganizes their development teams, and acquires new knowledge. A summary of theinterview can be found in section 4.2.

17

Chapter 4

Results

4.1 Performance evaluationThis section includes three tables, table 4.1, Table 4.2 and Table 4.3, showing the re-sults of the performance benchmarks. The arithmetic mean and standard deviationare included to show the distribution of the results. The results for the geoloca-tion and compass benchmarks are shown in number of updates per time interval(as described in the method) and the results for the other benchmarks are given inmilliseconds. The geometric mean for each benchmark has been normalized to theknown machine of the platform. As such Android and iOS has a geometric meanvalue of 1.

Benchmark: Geolocation Compass

Arithmetic mean (updates)

Android native 127.25 1885.95Android PhoneGap 12.25 4828.15Android Mosync 60.95 46.35

iOS native 62 364.2iOS PhoneGap 62.35 2073.65

Standard deviation (updates)


iOS native 0 1.196iOS PhoneGap 0.875 16.359

Geometric mean (normalized)

Android native 1 1Android PhoneGap 0.093 2.561Android Mosync 0.479 0.025

iOS native 1 1iOS PhoneGap 1.006 5.694

Table 4.1: Compass and geolocation benchmark

19

CHAPTER 4. RESULTS

Benchmark: Read 32B Read 8kB

Arithmetic mean (ms)


iOS native 0.420 0.666iOS PhoneGap 2.435 3.219

Standard deviation (ms)






Table 4.2: Reading from a file benchmarks

Benchmark: Write 32B Write 8kB

Arithmetic mean (ms)



Standard deviation (ms)






Table 4.3: Writing to a file benchmarks

20


4.2 Cost evaluationThis section includes a summary of the interview with Pär Gillander and of the datareceived from the questionnaire. The results from the questionnaire are displayedin bar charts organised by question.

4.2.1 InterviewHow much does educating a co-worker cost?

A rough estimation is that it takes about 200 work-hours before someone can startdelivering useful results when starting with a new system. Even more before he/shedelivers sustainable results, so about one months worth of wages. This scales linearlyso it takes about 400 work-hours for two people to start to deliver results.

How much time does it take?

This is very hard to say since it depends on the individual. Last year two newdevelopers were hired for a new project and it took them six weeks to producea finished product. It was not ready for distribution but it was functional andrelatively stable. Generally speaking it does not take very long for a developer tocome up to speed in a new area. Tekis uses a system where you learn while you workinstead of taking traditional courses. If someone does take a course in some areathey are expected to be able to work with the new task and produce results whenthey are finished. One thing they are trying however are on line courses. This allowsself-education during projects and for employees to adapt their own education totheir personal needs.

Projects

When a project is started that requires new knowledge the standard approach is tohire a consultant that is an expert within the required area. The consultant thenworks with the development team during the life time of the project to providesupport, know-how and educate the team. In this way the developers are edu-cated on the new system/technology and the company acquires a very productiveteam member. If it turns out that the new activity has a more long term place inTekis’ operation an expert is employed full time instead since the consultants expertknowledge disappears as soon as their contract expires.

21

CHAPTER 4. RESULTS

4.2.2 QuestionnaireResults from the questionnaire can be found in figure 4.1, figure 4.2, and figure 4.3

(a) C (Scale 1 - 5) (b) C++ (Scale 1 - 5)

(c) Objective-C (Scale 1 - 5) (d) Java (Scale 1 - 5)

(e) JavaScript (Scale 1 - 5) (f) HTML/CSS (Scale 1 - 5)

Figure 4.1: How great is your skill in the following programming languages?

22


(a) Eclipse (Scale 1 - 5) (b) Xcode (Scale 1 - 5)

(c) Visual Studio (Scale 1 - 5)

Figure 4.2: How much experience do you have with the following developmentenvironments?

23

CHAPTER 4. RESULTS

(a) Android (Scale 1 - 5) (b) iOS (Xcode 1 - 5)

Figure 4.3: How much experience do you have developing software for the followingmobile platforms?

24

Chapter 5

Discussion

5.1 Performance evaluation

5.1.1 PhoneGap

As can be seen in Table 4.1, Table 4.2 and Table 4.3 PhoneGap performs theworst in almost every benchmark. To understand the difference in performancecompared to the other approaches, the underlying structure of resource calls wereanalyzed. Whereas native coding technologies use native methods to call resourcesdirectly, JavaScript can only access system resources through an execution paththat implements at least one callback. To call native functionality, through thePhoneGap API, a JavaScript class in the WebView is called. This generates amessage that is caught by the PhoneGap native engine. The native engine executesthe native method and passes the result back to the specified JavaScript callbackmethod via a string message. This introduces overhead for calling methods sinceyou have to wait for the resource call to traverse the callback path, and for theresult to reach the original caller.

The only benchmarks where PhoneGap is the fastest is in writing to the filesystem on iOS, Table 4.3, and using the compass on both platforms, Table 4.1.It is interesting to note that while native Android is faster than iOS in the filesystem benchmarks, the Android version of PhoneGap is much slower than the iOSversion, as can be seen in Table 4.2 and Table 4.3. One reason for this could bethe way resources are accessed on the different systems. On iOS, calls to accessresources are highly restricted and usually only one alternative is provided, the waythe iOS developers decide is the best. On Android, however, there sometimes existseveral different ways to do the same thing, not all of which are equivalent regardingperformance. This means that on iOS the relative performance, represented by thegeometric mean, might generally be better than on Android because the PhoneGapAPI method calls may be sub-optimally implemented in the Android version.

While PhoneGap’s writing to a file benchmarks are only marginally faster thannative iOS, in the compass benchmark PhoneGap clearly outperforms both thenative platforms and MoSync, Table 4.1. PhoneGap receives two and a half times

25

CHAPTER 5. DISCUSSION

the number of updates compared to Android, and almost six times that of iOS.Since the native modules are implemented by PhoneGap themselves they couldhave included additional software logic to speed up the update frequency. Theycould, for example, be using cached values, that are returned to the caller if thecompass has not received new updates within a certain amount of time. It could alsobe the case that they implement deprecated methods that return updates faster.Since the performance differs for each system resource, a prospective developer mustcarefully consider the impact of using the different resources that their applicationneeds. Take for example a PhoneGap Android application, here the extra time ittakes to write to a file might not affect the general functionality of the application,as much as the extra time it takes to access the geolocation resource. Even though ittakes 191 times as long to write a 32 byte string to a file, Table 4.3, this only meansan extra overhead of 15 ms. 15 ms does not affect the overall user experience unlessI/O accesses are very frequent. On the other hand, the modest 10 times sloweraccess time on the geolocation resource, which results in 12 updates per minute, isa significantly bigger problem. Imagine using a GPS while driving a car and onlyreceiving updates every 5 instead of every 0.5 seconds. This translates to 139 metersbetween updates instead of only every 13 meters when traveling at 100 km/h.

Aside from the inherent overhead some of these performance issues can be solvedby implementing custom native plugins, where functionality and performance canbe optimized for the developers purposes. They can be used to perform CPU heavytasks where JavaScripts performance is just not good enough. They also allowdevelopers to extend the native functionality of the PhoneGap API which might benecessary in the future as more features are added to the platforms all the time.This however, requires that the developer writes native code, something that wewished to avoid by using PhoneGap in the first place.

5.1.2 MoSync

Since MoSync recompiles the code into native byte code, instead of wrapping itin an abstraction layer, it should suffer less from overhead and thus have betterperformance than PhoneGap. This holds true for every benchmark except thecompass where PhoneGap outperforms both Android and MoSync.

MoSync performs slightly worse than native Android in the geolocation andcompass benchmarks, Table 4.1. This is not too surprising since Android shouldrealistically be better at optimizing their compiled code than MoSync. What maybe surprising though is the fact that MoSync performs quite a bit better at all butone of the file system benchmarks, as can be seen in Table 4.2 and Table 4.3. Thiscould be explained by the fact that Android supports compiling C++ code natively.According to Android [8] this seldom benefits the applications performance, exceptin certain circumstances. One of these circumstances are when performing CPUheavy tasks, and since file I/O is a very CPU heavy task it comes as no surprisethat this is an area in which MoSync performs well. It should be noted that MoSynchas not revealed exactly how the recompilation works, but it can be speculated that

26


it is using the native compiler whenever possible.A downside with MoSync is its natural rigidity. Where PhoneGaps framework

is very flexible and can be modified and extended using custom native plugins,MoSync can only provide support for what they have already implemented. Since itrecompiles the code instead of just wrapping it and providing a way to communicatewith native code, the framework cannot be modified by the user. This becomes aproblem when developers wants to solve a problem using functionality that MoSyncdoes not currently support. MoSync and PhoneGap, as well as other companiesdeveloping CPTs, cannot compete with the resources of Apple and Google. Theywill seldom be able to stay up to date with all the new features. With PhoneGapthis problem can be circumvented in some cases by implementing it yourself throughplugins, while the very nature of MoSync as a source-code translator prohibits youfrom extending its functionality. An example of this is the inability to create newthreads or processes in MoSync[6], a fundamental feature for many developers.PhoneGap does not support threading either due to the fact that JavaScript doesnot, however if it is required a custom plugin could be created that implementsthreads.

5.2 Cost evaluation

To start developing applications targeting both iOS and Android using native codingtechnologies, Tekis would have to start two project groups due to the differencesbetween the two platforms. Following Tekis’ standard procedure when starting newprojects this would necessitate the hiring of an expert for each of the platforms. If,however, Tekis would use a CPT to produce the same application they would onlyneed one project group and one expert, which would effectively cut their costs inhalf. This is a very simplified argument but it serves to make the point that usingCPTs is a cost effective way of developing applications.

Since all the approaches use different programming languages and developmentenvironments they all require different skill sets. Thus the resources at the com-pany’s disposal must be analyzed when evaluating what it would take for a companyto start developing applications.

As can be seen by the results from the questionnaire in Figure 4.1, Tekis ABprimary development competence lies in web and Java programming. While fiveout of thirteen people answered that their skill with C and C++ is a 4 on a scalefrom 1 to 5, eight people answered 2 or less. The curve for both Java, JavaScriptand HTML/CSS looks much better with a larger amount of answers in the range 3to 5. The IDE which the company’s developers are most comfortable using is VisualStudio, with eleven out of thirteen answering 4 or more on how much experiencethey have with an IDE, Figure 4.2. Eclipse comes in second place with only fouranswering 4 or higher.

Since PhoneGap uses the coding languages that Tekis’ developers know bestand is compatible with the IDE that they are most comfortable with, this makes it

27

CHAPTER 5. DISCUSSION

a more suitable choice than MoSync for cross-platform development in regards todevelopment environment. This is because it would allow them to spend less moneyon education and less time on getting used to its development environment.

This of course does not necessarily hold true for other companies. As statedabove all companies have their own specialities and skill-sets, and thus need toperform their own analysis on which CPT best suits their own organisation.

28

Chapter 6

Conclusion

The choice on which approach to use, web-to-native wrappers or source code trans-lators, based on a performance and cost analysis is difficult. It depends mostly onwhat skill-set and competences the target organisation contains, and what kind ofapplications they are aiming to produce. While source code translators like MoSyncprovide better performance, making them good for performance demanding appli-cations such as games, the nature of how they function make them less flexible.Web-to-native wrappers, like PhoneGap, are much more flexible allowing devel-opers more freedom in designing their applications. The price for this flexibilityunfortunately, lie in their poor performance.

The loss of flexibility when using source code translators is an issue. Whenchoosing a CPT developers must be aware of which features that have not yetbeen implemented. Web-to-native wrappers with their modular approach, allowdevelopers to fill their customers’ needs by themselves while source-code translatorsonly allow the use of functionality that its developers have already implemented.

The potential economic gains are great when using CPTs to target multipleplatforms. How big these gains are depend on the techniques used by the CPT andskill set already existing within the company. The decision whether to use a source-code translator or web-to-native wrapper should only be made after considering therequirements of the application. If all the functionality requirements can be met bya source-code translator this is the better choice considering the performance gains.Generally however, web-to-native wrappers with their higher flexibility makes thema more stable and long term choice as a cross-platform development tool.

6.1 Future work

Cross platform development tools is an ever expanding subject. Understanding thecomplete performance implications of the different approaches is beyond the scopeof a thesis project. To improve upon this thesis however a more focused studyshould be performed. My recommendation is focusing on only one platform. Thiswould allow for the inclusion of more CPTs or a larger selection of system resources

29

CHAPTER 6. CONCLUSION

to be benchmarked. Including more CPTs in the study gives a broader spectrumof comparison while including a larger selection system resources allow for a deeperand more comprehensive comparison.

30

Chapter 7

References

[1] MoSync AB. Creating User Interfaces With MoSync. Visited in Jan 2014.url: http://www.mosync.com/docs/sdk/cpp/guides/ui/creating-user-interfaces-mosync/index.html.

[2] MoSync AB. MoSync SDK. Visited in Jan 2014. url: http://www.mosync.com/sdk.

[3] MoSync AB. The JavaScript NativeUI API. Visited in Jan 2014. url: http://www.mosync.com/docs/sdk/js/guides/nativeui/jsnativeui-library/index.html.

[4] MoSync AB. The MoSync Toolchain. Visited in Jan 2014. url: http://www.mosync.com/docs/sdk/tools/guides/architecture/toolchain/index.html.

[5] MoSync AB. The NativeUI C++ Library. Visited in Jan 2014. url: http://www.mosync.com/docs/sdk/cpp/guides/nativeui/using-nativeui-library/index.html.

[6] Mosync AB. Optimizing mobile applications. Visited in Feb 2014. url: http://www.mosync.com/docs/sdk/cpp/guides/testing/optimizing-mobile-applications/index.html.

[7] open handset alliance. Android. Visited in Feb 2014. url: %5C%5Chttp://www.openhandsetalliance.com/android_overview.html.

[8] Android. Android NDK. Visited in May 2014. url: https://developer.android.com/tools/sdk/ndk/index.html.

[9] Android. Android, the world’s most popular mobile platform. Visited in Feb2014. url: http://developer.android.com/about/index.html.

[10] Android. Developer Tools. Visited in Feb 2014. url: http://developer.android.com/tools/index.html.

[11] Apple. About the iOS Technologies. Visited in Feb 2014. url: https : / /developer . apple . com / library / ios / documentation / miscellaneous /conceptual/iphoneostechoverview/Introduction/Introduction.html.

31

CHAPTER 7. REFERENCES

[12] Apple. iOS 7 features. Visited in Feb 2014. url: https://developer.apple.com/ios7/#new.

[13] Apple. iOS Developer Library. Visited in Feb 2014. url: https://developer.apple.com/library/ios/referencelibrary/GettingStarted/RoadMapiOS/FirstTutorial.html#//apple_ref/doc/uid/TP40011343-CH3-SW1.

[14] Apple. iOS Developer Program. Visited in Feb 2014. url: https://developer.apple.com/programs/ios/.

[15] GSM Arena. HTC Desire. Visited in Feb 2014. url: http://www.gsmarena.com/htc_desire-3077.php.

[16] A. Charland and B. LeRoux. “Mobile application development: Web vs. na-tive”. In: Communications of the ACM 54.5 (2011), pp. 49–53.

[17] Andreas Constantinou et al. Cross-platform developer tools 2012. Tech. rep.VisionMobile, 2012.

[18] Luis Corral, Alberto Sillitti, and Giancarlo Succi. “Mobile Multiplatform De-velopment: An Experiment for Performance Analysis”. In: Procedia ComputerScience 10 (2012), pp. 736–743. issn: 1877-0509.

[19] Priori Data. Top apps and publishers report. Visited in Feb 2014. Jan. 2014.url: http://prioridata.com/products/free-reports/?top-apps-and-publishers-report.

[20] David H Deans. Global Shifts in the Smartphone Platforms Market. Aug. 19,2013. url: http://socialmediatoday.com/david- h- deans/1677421/global-shifts-smartphone-platforms-market.

[21] D. Dern. “Writing small [Tools and toys]”. In: Spectrum, IEEE 47.6 (June2010), pp. 14–15. issn: 0018-9235.

[22] Philip J. Fleming and John J. Wallace. “How Not to Lie with Statistics: TheCorrect Way to Summarize Benchmark Results”. In: Commun. ACM 29.3(Mar. 1986), pp. 218–221. issn: 0001-0782.

[23] Rohit Ghatol and Yogesh Patel. Beginning PhoneGap: Mobile Web Frameworkfor JavaScript. Apress, 2012. isbn: 978-1-4302-3903-1.

[24] Mark H. Goadrich and Michael P. Rogers. “Smart Smartphone Development:IOS Versus Android”. In: Proceedings of the 42Nd ACM Technical Symposiumon Computer Science Education. SIGCSE ’11. Dallas, TX, USA: ACM, 2011,pp. 607–612. isbn: 978-1-4503-0500-6.

[25] Google. Application Fundamentals. Visited in Feb 2014. url: http : / /developer.android.com/guide/components/fundamentals.html.

[26] Google. The Android Source Code. Visited in Feb 2014. url: http://source.android.com/source/index.html.

[27] Adrian Holzer and Jan Ondrus. “Mobile application market: A developer’sperspective”. In: Telematics and Informatics 28.1 (2011), pp. 22–31. issn:0736-5853.

32

[28] Mobile HTML5. HTML5 compatibility on mobile and tablet browsers withtesting on real devices. Visited in Jan 2014. url: http://mobilehtml5.org/.

[29] Global Web Index. Q3 GWI Launch: Mobile Operating Systems. Visited inFeb 2014. Oct. 2013. url: http://blog.globalwebindex.net/mobile-operating-systems.

[30] jQuery Mobile. A Touch Optimized Web Framework. Visited in Jan 2014. url:http://jquerymobile.com/.

[31] Masoud Nosrati, Ronak Karimi, and Hojat Allah Hasanvand. “Mobile Com-puting: Principles, Devices and Operating Systems”. In: World Applied Pro-gramming 2.7 (July 2012), pp. 399–408. issn: 2222-2510.

[32] Arno Puder. “Cross-Compiling Android Applications to iOS and WindowsPhone 7”. In: Mobile Networks and Applications 18.1 (2013), pp. 3–21.

[33] Samsung. Galaxy Note 3 specs. Visited in Feb 2014. url: http : / / www .samsung.com/se/consumer/mobil/mobil/smartphones/SM-N9005ZKENEE-spec.

[34] John M. Wargo. PhoneGap Essentials: Building Cross-Platform Mobile Apps.Addison-Wesley Professional, 2012. isbn: 0321814290.

[35] Alexander Zibula and Tim A. Majchrzak. “Cross-Platform Development Us-ing HTML5, jQuery Mobile, and PhoneGap: Realizing a Smart Meter Ap-plication”. In: Lecture Notes in Business Information Processing 140 LNBIP(2013), pp. 16–33.

33

Documents

Cross Platform Development tools for mobile applications ...754436/FULLTEXT01.pdfCross Platform Development tools for mobile applications, a performance and cost evaluation MÅRTEN