MOBILE APPLICATION TESTING

Overview IntroductionMobile testing Vs Mobile

App testingMobile Platforms Emulator & SimulatorChallenges in application

testingMobile application testingFramework Testing –

OverviewSummary

Introduction Mobile application testing is similar to software testing but

testing is perform on mobile instead of system. It can be automated or manual type of testing.

Mobile application testing is a process by which application developed for mobile devices is tested for its functionality, usability and consistency.

In application testing more about how application will perform on different platform , resolution network & hardware configuration.

ComparisonMobile Testing: Mobile testing is testing of feature of mobile or handset

given by company or manufacturer. Main focus is on mobile feature like calling feature , sms ,

battery testing , settings , phone book , Network compatibility etc.

Mobile Application Testing: Testing of application which is provided by third party to

user. It can be pre installed in mobile or can get from stores. It focus on application functionality, Performance and

adhering to guidelines provided by Platforms. Testing ensures application perform in same way as its was

designed.

Mobile Platforms• Before application testing, lets know about mobile platforms

for which apps are developed. Now days, there are lots of platform or operating system.

Continued…Platform which are on high priority in market are: Android (By Google) iOS (By Apple) Android OS – Android is based on Java programming

language. It is open source & Linux based operating system for mobile like Smartphone & tablet.

Latest Version: 7.0(Nougat) Developed by : Google iOS – iOS was previously known as iPhone Operating system.

It is the operating system which run on iPhone, iPad & iPod. Latest Version: 10.1 Beta 5 Developed by: Apple Inc.

Emulator & SimulatorAs a part of mobile application testing, we need to check if

the application is working as expected with all the majorly used devices like Android phones, iOS, other different types of tablets and iPads. But testing each & every physical device will very expensive to carry out testing.

So there is an smart alterative for testing : Emulator & Simulator.

Emulator -  Emulator is a program or device that permits programs written for one kind of computer to be used on another kind of computer.

Simulator - A mobile simulator is a software application for a personal computer which creates a virtual machine version of a mobile device.

Continued…

Disadvantage of Simulator/ Emulator:A simulator/emulator cannot provide the following features

− Mobile device battery Mobile device’s camera Difficult to mimic interruptions like incoming calls and

SMS. Not so much realistic simulation for mobile device memory

usage.

Lets discuss differences between simulator & Emulator:

Continued…

Emulator SimulatorThe emulator aims at

emulating or mimicking as close as possible the outer behaviour of an object.

Emulators are developed machine level assembly language.

They are suitable for debugging.

Emulator is not good in terms of performance. It is very slow.

It is re-implement of the original software

Simulator's objective is to simulate the internal state of an object as close as possible to the internal state of an object.

They are written in high level language

It is difficult to use for debugging purpose.

Simulator are faster in comparison to emulator.

It is just a partial re-implementation of the original software .

ChallengesTesting is fast-paced industry which is changing constantly.

Mobile apps has not only set target into consumer apps but also in commercial apps. Mobile apps should be rock solid rock before they are released to market. The movement towards mobile devices has brought a whole different set of challenges to the testing world.

6 key challenges that developer & tester are facing during the development cycle of application. They are:

1. Screen sizes. 2. Different OS versions. 3. Power Consumption. 4. Usability. 5. Connection Type. 6. Internationalisation

Continued…

1. Various screen size-  In Android world the verity of different aspect ratios & pixels can be overwhelming. Apple also bring a new screen size with launch of iphone6.  For testing it means that tester need to check on various devices that all the necessary screen elements are accessible with different screen sizes and aspect ratios. Example iOS developers are used to pixel perfect screen design, they now need to change their mindset to the adaptive screen design instead

Continued…Different range of mobile resolution :

2. Different OS Versions – As we discussed in previous slides, there are variety of platform for mobile apps. This wide range of operating system, is main huddle in successfully development of any mobile application. Also the version of OS, iOS users are known to be upgrading quickly to the newest versions. On the contrary, Android uptake has historically been very slow and the fragmentation is wide. This means that app developers need to support older OS versions and older API’s, and testers need to test for those.

Continued…User over different Operating system:-

Continued…3. Power Consumption- There are several types of batteries

used in different mobile devices (i.e. nickel cadmium/ lithium ion/ Nickel metal hybrid). When testing mobile apps we need to make sure that the power consumption is kept minimal and the app is developed by keeping the best practices in mind. The innovation in the battery storage capacity field hasn’t been as quick as in the app consumption. We are running lots of apps during the day and many apps are running on background without noticing user.  This all requires CPU cycles which on it’s turn require power and thus the batteries tend to dry.

Power Consumption test can be done manually; also there are some free tools available in the market such as Trepn Profiler, Power Tutor, and Nokia Energy Profiler. These are applications which can display the real-time power consumption on a smart phone or tablet.

Continued…4.Usability - When testing mobile apps it’s important to pay attention

to the size of click areas and making sure that all texts are readable without lenses. Mobile device screens are relatively small and there are always more data developer would like to present than possible to fit to the screen. It’s challenging to keep the interaction clean and simple for the user, and at the same time display all the necessary information. Font size and readability, clickable button are other challenging factors of usability.

5. Connection Type – Connection API’s on mobile platforms have been developed keeping those challenges in mind, the real world environment is still very much varying and interesting set of issues may occur.

There are several standards for mobile data connections (edge, umts, 3G, 4G) as well as for wi-fi. Sometimes there might be no connection available at all or the device is in flight mode. When users move around the connection type might change.

It’s important to test the bandwidth usage as not all carriers are supporting unlimited data volumes.

Continued…

6. Internationalisation – Many of the application are developed to be run on international market. Tester should take care of local settings , time zone according to target audience. Tester must also take account into translation of languages as per region. Right-to-left languages have always kept developers puzzled.

Though there are list of challenges, but above are common & general challenge that developer & tester face during development cycle. This challenges can be overcome & quality can be improve with help of testing expertise and using correct tools in suitable environment.

Mobile Application TestingLet us now start with different types of application testing.

1. Functional Testing - Functional testing is the most basic test for any application to ensure that it is working as per the defined requirements. Similar to other user-interface based applications, mobile applications require a number of human interactions in user scenarios.

Test team verifies that forms correctly submit and collect information from

users, that all components / function as expected, and that all links are

correct and functional.

Continued…

2. Compatibility Testing - Compatibility testing has got the highest stack when it comes to mobile application testing. The purpose of a mobile app compatibility test, to ensure application must run as desired in all operating system & different resolution.

Checklist of compatibility testing: * UI of the app must be as per screen size, no text or control

should be cut off. * All text should be readable in all resolutions device. * User should be able to suspend/ resume call or sms when

application is running. * Behavior of app when memory of device is almost full. * Responsive in both mode – Landscape & Portrait mode. * Application of response when no network is available.

Continued…

3. Localization Testing- Localization testing is the software testing process for checking the localized version of a product for that particular culture or locale settings.  The areas affected by localization testing are UI and content.

It is done to ensure the quality of a product for a particular target or local.

Advantages: * It helps reduce time for testing since its done for just on local. * It reduces overall testing and support costs. Characteristics: * Modules specific to Culture/locale-specific, language-specific,

and region-specific. * Installation and upgrading tests run in the localized

environment.  

Continued…

4. Performance Testing – It is a non functional testing. Mobile performance test covers client application performance, server performance, and network performance. It is important to make sure that the performance test scenarios cover all those areas. With the help of performance testing tools, it is not difficult to identify the existing networks, servers, and server-side application bottlenecks, given the predefined loads and transaction mix.

Continued…

5. Security Testing - Vulnerabilities to hacking, authentication, and authorization policies, data security, session management and other security standards should be verified as a part of mobile app security testing. Applications should encrypt user name and passwords when authenticating the user over a network.

One way to test security related scenarios is to route your mobile’s data through a proxy server like OWASP Zed Attack Proxy and look for vulnerabilities.

6. Stress Testing – Stress testing is to test the system behaviour under extreme conditions and is carried out till the system failure. Stress testing is a must to find exceptions, hangs, and deadlocks that may go unnoticed during functional and user interface testing.

Criteria of Stress testing: * Load your application with as much data as possible to try to

reach its breaking point.

Continued…

* Perform the same operations over and over again.* Perform the repeated operations at varying speeds – very quickly

or very slowly. * Leave your application running for a long period of time, both

interacting with the device and just letting it sit idle, or performing some automatic task that takes a long time, for example, a slideshow.

* Randomly send screen taps and keystrokes to your application.

7. Interrupt Testing – An application, while functioning, may face several interruptions like incoming calls or network coverage outage and recovery. It can also be defined as a branch of Mobile Application Testing that deals with --how an application reacts to an interruption and resumes to its previous state. The variety of devices, networks, configurations, etc. makes it more prominent for Mobile applications than the others.

Continued…

General interruption: 1. Incoming and Outgoing SMS and calls 2. Incoming Notifications from social

media, alarm, software update etc. 3. Network connection loss and restoration. 4. Plugging in & out of charging.

5. Battery if off or phone shut down while using application.

This list is not exhaustive but includes the most common scenarios.

8. Usability Testing - It is very important to get usability testing in place from the early stage of the application design, and it should not be done only when the application is completed. Usability testing requires heavy involvement of users, and the output might affect the application design, which is very difficult to change in the later stages of the project.

Continued…

Usability testing evaluates the application based on the following three criteria for the target audience:

* Efficiency: The accuracy and completeness with which specified users can achieve specified goals in a particular environment.

* Effectiveness: The resources expended in relation to the accuracy and completeness of goals achieved.

* Satisfaction: The comfort and acceptability of the work system to its users and other people affected by its use.

There are two types of mobile usability testing:1. Laboratory-based usability testing.2.Remote usability testing.

Continued…

9. Memory Leakage Testing - Mobile devices have very limited memory as compared to other computers, and mobile operating systems have a default behaviour to terminate applications that are using excessive memory and causing a poor user experience.

Memory testing is exceptionally important for mobile applications to ensure that each application maintains optimized memory usage throughout the user journey. It is recommended that we conduct memory testing on the actual target device, since the system architecture is different from an emulator to an actual device.

Continued…

10.Installation Testing - Installation testing verifies that the installation process goes smoothly without the user having to face any difficulty.

11. Uninstallation Testing - The basics of uninstallation testing can be summarized in one line as “Uninstallation should sweep out data related to the App in just one go”.

Framework Testing – OverviewA testing framework or more specifically a testing

automation framework is an execution environment to perform automated tests. It is the overall system in which the tests will be automated. It is defined as the set of assumptions, concepts, and practices that constitute a work platform or support for automated testing.

A testing framework is responsible for: * Defining the format in which to express expectations

* Creating a mechanism to hook into or drive the application under test

* Executing the tests and reporting the results Types of framework :- * Android Framework * iOS Framework

Continued…

Android framework: There are many framework in market but mainly are –

1. Robotium2. UIAutomater3. Appium4. Calabash5. Selendroid

Continued…

iOS framework:  Unlike Android, Apple does not license iOS for installation on non-Apple hardware. iOS and iOS applications are only installed on Apple devices . Framework available for iOS in market are –

1. Appium 2. Calabash 3. UI Automation 4. Earlgrey 5. XCTEST / KIF

SummaryConsidering the significant role the mobile applications is

playing nowadays play now days in our day to day life, testing of these applications are going to evolve and thus they require a lot of testing to make them work as required.

Designing the right test strategy, choosing the right mobile simulators, devices and mobile testing tools can make sure that we have 100% test coverage and help us include security, usability, performance, functionality and compatibility based tests into our test suites.

Thank You