利用 Appium + Robot Framework 實現跨平台 App 互動測試

利⽤用 Appium + Robot Framework!實現跨平台 App 互動測試Jeremy Kao ( / imsardine)

• Test cases involving multiple devices • A test script that works! • Essentials and long-term considerations • PyUIA - Python UI Automation • Q&A

Agenda

Test Cases Involving Multiple Devices

• Data synchronization • Social features

• Sharing • Following • Messaging

• …

KKBOX - Playlist Auto-Sync• Easy to manage and organize playlists on various devices. • More than one devices (generally with different platforms)

are involved.

The Test Case (iOS > Android)

1. On device A, create a new playlist.

2. On device B, check if the playlist created in step 1 appears.

A B

iOS Android

The Test Case (Android > iOS)

1. On device A, create a new playlist.

2. On device B, check if the playlist created in step 1 appears.

B A(the same test case)

iOS Android

A central/remote controller is needed.

?

A B

Automation for Apps

Test case Appium servers Android / iOS / Firefox OS

Appium Platforms Support

Native automation frameworks

• Extended Selenium WebDriver API • Various language bindings

(Source: Dev::Coder)

Multiple Devices Support - Multiple Servers

Driver A / B

*:4723

*:4725

Driver A / B

(one-to-one)

NOTE: Currently, each Appium server only serves one session/device at a time. To work around this, run multiple Appium servers. (#462)

iOS

Android

A (Pseudo) Test Script That Works!

Android > iOS

iOS > Android

Test scenario

Locate UI elements and interact with them (in different ways)

A (Pseudo) Test Script That Works!

Bad smell?May lead to too many if/else statements

Easy to read ?

Essentials and Long-term Considerations…

• Keyword-driven testing (KDT) support. • Reusable keywords. • Abstraction of layers. • Technology-independent

• Acceptance testing and A-TDD support. • Specified in business domain language/

terms (natural language), that are shared between stakeholders.

• Requirements as tests -> Executable • Data-driven testing (DDT) support. • RIDE - Robot Framework IDE.

Robot Framework and RIDE

Test Cases in Natural Language

Readability

Data-driven Support - Test Template

Arguments

Results - Reports

Results - Logs

Device logs collected during the execution of a keyword and a test case.

Logs generated by test code

Results - Embedded Screenshots

• Take screenshots automatically before (and after) keyword execution.

• Take screenshots on demand in the test code.

PageObj. (iOS)

PageObj. (Android)

Abstraction of LayersRequirements

Keywords

Keyword Impl.

Application Under Test

Testing Tools

Test Data

Python Prog.

User Keywords and Library Keywords

User Keywords

Library Keywords

Import Test Libraries

Application Library

Page/Screen Libraries

Application Library

A

B

Application Library

Open App <DEVICE_ID> [<ALIAS]

Switch Device <ALIAS>

Page/Screen Libraries - Modeling

Generally, one for each page/screen.

Services offered by this page/screen. (Page Object Pattern)

iOSAndroid

Library Keywords - High Level, Domain TermsiOSAndroid

Library Keywords - Scenario

Reusable keywords as bricks, from multiple keyword libraries.

iOSAndroid

PageObj. (iOS)

PageObj. (Android)

Abstraction of LayersRequirements

Keywords

Keyword Impl.

Application Under Test

Testing Tools

Test Data

Python Prog.

• Maintainability - less code duplications. • Readability - test code is about intention and scenarios. • Scalability - test code that scales without losing control.

Page Object Pattern / Modeling (POM)

Problem

Tests are brittle to (inevitable) changes in the UI.

Benefits

PageObj. (iOS)

PageObj. (Android)

Abstraction of LayersRequirements

Keywords

Keyword Impl.

Application Under Test

Testing Tools

Test Data

Python Prog.

A

B

Page Library Impl. in KKBOX

PyUIA

self._page_object points to CURRENT device. (delegation)

A/B = iOS | Android

Page Library Keywords

Keyword Impl.

PageObj. (iOS)

PageObj. (Android)

Abstraction of LayersRequirements

Keywords

Keyword Impl.

Application Under Test

Testing Tools

Test Data

Python Prog.

Page Objects Impl. in KKBOX (Android)

Page Library Keyword

Page Object Impl. (Android)

Page Objects Impl. in KKBOX (Android)

UI interaction and details

UI Mapping

PageObj. (iOS)

PageObj. (Android)

Abstraction of LayersRequirements

Keywords

Keyword Impl.

Application Under Test

Testing Tools

Test Data

Python Prog.

Page Objects Impl. in KKBOX (iOS)

UI Mapping

Implement the same interface.

PageObj. (iOS)

PageObj. (Android)

Abstraction of LayersRequirements

Keywords

Keyword Impl.

Application Under Test

Testing Tools

Test Data

Python Prog.

where PyUIA fits into

PyUIA (imsardine/pyuia)

Fork me on GitHub

$ pip install pyuia

Example: imsardine/pyuia-example-wordpress

A library/framework aiming to facilitate the implementation of UI test automation on various platforms, including mobile and desktop OSs.

Incubated by KKBOX SQA Team

PyUIA and Your App (WordPress)

PageObj. (iOS)

PageObj. (Android)

PyUIA and Your App (WordPress)Requirements

Keywords

Keyword Impl.

Application Under Test

Testing Tools

Test Data

Python Prog.

• WordPress • SignInScreen • MainScreen

• SignInScreen • MainScreen • …

• Appium • Robot Framework and RIDE • Keyword-driven testing • Page Object Pattern / Modeling • PyUIA - Python UI Automation • …

利⽤用 Appium + Robot Framework!實現跨平台 App 互動測試

PyUIA (imsardine/pyuia)Jeremy Kao ( / imsardine)

Google Test Automation Conference

Oct 28-29 Live Stream

• Appium • Robot Framework, RIDE • Keyword-driven testing - Wikipedia, the free encyclopedia • Page Object - Martin Fowler • Page Objects - Selenium • Acceptance Test-Driven Development With Robot Framework

(PDF)

References

Page Object Pattern / Modeling (POM)Pattern

• Model areas/fragments (on a given page/screen) that test code interacts with as objects.

• Public methods as services offered by the page/screen. (abstract, high-level, application/domain-specific)

• Assertion-free, except for asserting it is on the correct page/screen. The tests should be responsible for making assertions.

• Except for getter methods, return a page object representing the (navigation) destination.

• Encapsulate UI interaction, details, async (wait for).