Mobile Saturday. Тема 2. Особенности тестирования приложения...

Особенности тестирования приложения на Android

➔Specific functional ➔Performance➔Device park

selectionMykhailo Zhelieznov

QC Eng. at SoftServe

Agenda:

● Specific functional:

○ Application installation

○ Network access

○ Animation, Video, Audio

○ GUI testing

● Performance

● Device park

Application installation

Application installation

It’s important to check application installation, especially if it saves some specific data on device or registers to the server

Application installation

Device preparation

● Cleared device (after factory reset)

● Device with existing user’s data (apps installed, additional user’s data - documents, pictures, music etc.)

● Device OS upgrade

Application installation

● Settings > Backup and restore > Factory reset

● Reboot to recovery mode - Volume Down

+ Power (depends on Vendor) > wipe

data/factory reset

How to clear the device:

Application installation

Ways of application installing

● System apps, installed when system is initialized

● Install apps from Google Play Store

● Install apps using adb, "adb install xxx.apk"

● Copy xxxapp.apk to sdcard, browse the .apk file from your phone, double click, the app would be installed.

Application installation

What can be wrong while installing app?

● At least check installer is not corrupted

● Check requested permissions

● Check if application starts

● Check if all necessary data created. It can be pictures (e.g. WallPapers app), sounds, data base, other specific data saved in specific directory (need to be checked with developers)

Application installation

Other cases

● Check that Uninstalling works correct:

○ all necessary data is removed from device

○ application is unregistered from server

● Check that reinstalling of the same app doesn’t produce any issues

● New version of application can be available, check that upgrade successfully executes (this can be done in combination of OS upgrade)

Note: system apps & administrators can’t be removed, just stopped but not always

Application installation

Clear application data

As you use applications, they start storing files for reference later. These files are stored in an app "cache."

Settings > Applications > Select app (depends on vendor)

1

Clearing app data is a little more drastic.

You're wiping the cache, but also clearing

any and all settings that go along with that

app.

2

Application installation

‘Closing’ apps

Removing an entry in recent tasks will kill any background processes that exist for the process. It won't directly causes services to stop

For stop is a complete kill of the app -- all processes

are killed, all services stopped, all notifications

removed, all alarms removed, etc. The app is not

allowed to launch again until explicitly requested.

1

2

Network access

Need to check each connection type and switches between them

Network access

Disconnected WiFi Edge 3G AirPlane

Disconnected x x x x

WiFi x x x x

Edge x x x x

3G x x x x

AirPlane x x x x

20 elementary cases is just a beginning

State transitions / use cases, depended on requirements

Network access

●Disconnected > Edge > WiFi > Edge > WiFi > AirPlane mode > Disconnected > Wifi …

●Edge > Disconnected > AirPlane mode > Edge ...

● Downloading is in progress > connection type changed

● Wait for WiFi access before doing large data transfers, since WiFi transfers maximize battery usage compared to transfers over 3G or EDGE

● Cached files should be refreshed if they were changed on server. This can be done by manual downloading of resources or by Server push event

● Push event is received just after device back online. Downloading resume after device back online

Network access

Caching, downloads for offline work

Network access

● Downloaded data should be valid, encrypted if necessary

(files, push events etc.)

Animation, Video, Audio

Can be triggered:

● By user’s action○ Swipe, scroll○ Button press○ Tap on screen○ Device rotate○ Change settings

● On timeout● By event / push message receiving

Animation, Video, Audio

Can be interrupted (paused / stopped):

● By the same reasons they triggers (previous slide)● By incoming / outcoming call / sms● Status bar / menu opening, popup appearing● Low power resource● App move to BG > FG● Lock / Unlock device● If Audio / Video is playing in BG - it can be

interrupted by other applications sounds● Audio - headphones disconnection● Video - full screen enter / exit

Animation, Video, Audio

The best way to test the accuracy of Animation / Video / Audio - is simply watch / listen and compare with requirements. Additional tools can help to detect if playing started / paused, but won’t detect the incorrect displaying / sounding

Animation, Video, Audio

Graphical user interface testing

You can make a screenshot from your screen and then use any image editor which shows cursor location to calculate the sizes in pixel

GUI testing

Developer options can make your life much easy. Use next of them for GUI testing:

GUI testing

● Pointer location ● Show layout bounds

Dump view hierarchy for UI Automator + Show layout bounds GUI testing

140

580

A virtual pixel unit that you should use when defining UI layout, to express layout

dimensions or position in a density-independent way.

The density-independent pixel is equivalent to one physical pixel on a 160 dpi screen,

which is the baseline density assumed by the system for a "medium" density screen. At

runtime, the system transparently handles any scaling of the dp units, as necessary,

based on the actual density of the screen in use. The conversion of dp units to screen

pixels is simple:

px = dp * (dpi / 160).

For example, on a 240 dpi screen, 1 dp equals 1.5 physical pixels. You should always use

dp units when defining your application's UI, to ensure proper display of your UI on

screens with different densities.

GUI testing

Density-independent pixel (dp)

● find it in your device tech specification

● get display info using ADB:

○ $ adb shell dumpsys display

○ find ‘density’ in command output:

GUI testing

How to get your device density (Dots per inch)?

Performance

Performance

● Note the time of action, e.g.:○ application starting time○ screen switch○ list download / scroll○ any action triggered by user (for ex. on button press)

● Repeat same test few more times (e.g. 10) and calculate the average for better accuracy

● On different Android versions performance can be different as on the devices with a different characteristics

● ~100 switches between screens / ~100 screen rotates / application move BG then FG

● Load / Create a lot of files at once

Simple performance tests

Performance

Over and above you can use a simply ADB command-line tool to execute the actions by incredible scenario - like Monkey.

The basic syntax is:

$ adb shell monkey [options] <event-count>

For example, if your application package is com.myapp

and you want to generate 1000 gestures, with a delay of

500ms between each event, you have to use this

command:

$ adb shell monkey -p com.myapp --throttle 500 -v

1000

Simple performance tests

Performance

We can check the RAM usage without any additional tools● Go to Settings > Apps > Running

Consumption of resources

Performance

We can check Power Consumption the same way we did for RAM checking● Go to Settings > Battery

Consumption of resources

Consumption of resources

If you want a bit better tool with the same intuitive interface - go to Google Play store and find the App which will show all the data you need in the way you like (e.g. Simply System Monitor)

Performance

Performance

If power consumption is really important for you - buy additional hardware and software e.g. Monsoon Power Monitor

Consumption of resources

Device park

Device park

Android has a really big problem due to fragmentation of different vendors with different hardware and software installations as well as customized user interfaces.

Device park

1. Watches2. Smart glasses3. Home Appliances4. Cars5. Homes6. Cameras7. Smart TVs8. DECT phones9. Game consoles10. Mirrors (you read that right)

Android Everywhere: 10 Types of Devices That Android Is

Making Better

Device park

Gather as much information as possible and allowed about the devices of your end users:

● Devices types● Densities● Android versions● Vendors● Architectures

Gether the statistic

Device park

Google devices:● Uses original Android● First receives latest Factory Images● Easy to install previous versions of operating system

Google Nexus - Always new

Device parkUse cloud device storages

Performance

???

Viewing heap usage for a process

DDMS allows you to view how much heap memory a process is using. This information is useful in tracking heap usage at a certain point of time during the execution of your application.

To view heap usage for a process:

1. In the Devices tab, select the process that you want to see the heap information for.

2. Click the Update Heap button to enable heap information for the process.

3. In the Heap tab, click Cause GC to invoke garbage collection, which enables the

collection of heap data. When the operation completes, you will see a group of object

types and the memory that has been allocated for each type. You can click Cause GC

again to refresh the data.

4. Click on an object type in the list to see a bar graph that shows the number of objects

???

Viewing heap usage for a process

???

DDMS provides a feature to track objects that are being allocated to memory and to see which classes and threads are allocating the objects. This allows you to track, in real time, where objects are being allocated when you perform certain actions in your application. This information is valuable for assessing memory usage that can affect application performance.

To track memory allocation of objects:

1. In the Devices tab, select the process that you want to enable allocation tracking for.

2. In the Allocation Tracker tab, click the Start Tracking button to begin allocation

tracking. At this point, anything you do in your application will be tracked.

3. Click Get Allocations to see a list of objects that have been allocated since you clicked

on the Start Tracking button. You can click on Get Allocations again to append to the

list new objects that that have been allocated.

4. To stop tracking or to clear the data and start over, click the Stop Tracking button.

Tracking memory allocation of objects

???Tracking memory allocation of objects

???

Examining thread information

Threads tab in DDMS shows you the currently running threads for a selected process.

1. In the Devices tab, select the process that you want to examine the threads

for.

2. Click the Update Threads button.

3. In the Threads tab, you can view the thread information for the selected

process.

???Examining thread information

???

Method profiling is a means to track certain metrics about a method, such as number of calls, execution time, and time spent executing the method.

To start method profiling:

1. On the Devices tab, select the process that you want to enable method profiling for.

2. Click the Start Method Profiling button.

3. Interact with your application to start the methods that you want to profile.

4. Click the Stop Method Profiling button. DDMS stops profiling your application and

opens Traceview with the method profiling information that was collected between the

time you clicked on Start Method Profiling and Stop Method Profiling.

Starting method profiling

???Starting method profiling