The Mobile DevOps Company
Ville-Veikko HelppiHead of Demand Generation
Getting Started with XCTest and XCUITest for iOS App Testing
WEBINAR
XCTest & XCUITest
• The State of The Art in iOS App Testing – Frameworks, Tools & Methods
• The Basics of XCTest & XCUITest – And How to Get Started
• Alternative iOS Test Automation Frameworks – Pros and Cons of Each
• Demonstration• Q&A
Agenda
http://bitbar.com/testing/
More information about test automation frameworks for iOS:
bitbar.com/testing/
Public Cloud
• Device ‘Cloud’ built for internal testing use• Enterprise-grade testing infrastructure hosted
by the customer• Usually sits behind customer firewall and
connects to preproduction environments
• Private Device Cloud• Reserved and Dedicated devices• Hosted and Fully Managed by Bitbar• Devices chosen by and reserved exclusively for
customer
• Also known as Testdroid Cloud• On-demand devices (multi-tenant)• Mobile App testing on over 1000+ real
Android and iOS devices hosted by Bitbar
Private Cloud
PRODUCT UPDATE:
XCTest & XCUITest Support Available
On-Premise
XCTest/XCUITest Supported
• Integral framework in Xcode• Not a new framework: Xcode 5
introduced the first version of XCTest• Incrementally new
capabilities/features:o Xcode 6 – performance measuremento Xcode 7 – UI testing (XCUITest)
• Works with both Swift and Objective-C
XCTest & XCUITest
What is XCTest/XCUITest?
• Easy to learn, no additional installations or components required (to Xcode)
• Native iOS language support• Xcode Test Recorder (for XCUITest)• Integration with Xcode Server
(continuous integration) and Bots• Faster than many other popular
functional (and cross-platform) test frameworks
• Works with both Swift and Objective-C
XCTest
Pros of XCTest
• No cross-platform support• Limited programming language
support• Selectors may seem complex• Requires (always) a separate
test target• Signing can be ‘tricky’• Sometimes… well, flaky
XCTest
Cons of XCTest
The Status of iOS Test Automation Frameworks
• If you running a test script built for prior Xcode versions, all uia_* calls will raise an error with the latest Xcode
• When upgrading to Xcode 8 all your existing UIAutomation scripts will fail
Deprecated UI Automation
Migrating UIA -> XCTest?
The Basics of XCTest & XCUITest andHow to Get Started
New Ebook Available!
Download This Guide
• A test method is an instance method of a test class that begins with prefix “test”
• Tests are simply classes and methods and e.g. @property and helper methods can be added to the class
XCTest & XCUITest
Getting Started
-(void)testExample {XCTAssert(...);XCTAssertTrue(...);XCTAssertFalse(...);
}
• Tests are grouped into classes that are subclass from XCTestCase
XCTest & XCUITest
Writing Test Methods
class DemoTest: XCTestCase {
- (void)testExample { // This is an example of a functional test case. // Use XCTAssert and related functions} - (void)testPerformanceExample { // This is an example of a performance test case. [self measureBlock:^{ // Put the code you want to measure the time of here. }];}}
XCTest & XCUITest
XCTestCase Example - Swiftimport UIKitimport XCTest
@testable import LocalizationDemo
class LocalizationDemoTests: XCTestCase { override func setUp() { super.setUp() } override func tearDown() { super.tearDown() } func testMyModel() { // Example of a functional test case. var model = MyModel(name: "first", surname: "last"); XCTAssertEqual(model.a, "first"); XCTAssertNotEqual(model.b, "first"); } func testIfLocalizationWorks() { // This is an example of a functional test case. XCTAssert(true, "Pass") }}
• A performance test takes a block of code (that is measured) and runs it ten times, collecting the average execution time and the standard deviation for the runs
XCTest & XCUITest
Writing Performance Tests
- (void) testAdditionPerformance { [self measureBlock:^{ // set the initial state [calcViewController press:[calcView viewWithTag: 6]]; // iterate for 100000 cycles of adding 2 for (int i=0; i<100000; i++) { [calcViewController press:[calcView viewWithTag:13]]; [calcViewController press:[calcView viewWithTag: 2]]; [calcViewController press:[calcView viewWithTag:12]]; } }];}
XCUIApplication
• The XCUIApplication is basically a proxy for an app that can be launched and terminated
• User can tell the application to run in testing mode by defining app as a “Target Application” in Xcode target settings
Code Syntax (XCUITest)
// Objective-CXCUIApplication *app = [[XCUIApplication alloc] init];
// Swiftlet app = XCUIApplication()
XCUIElement
• XCUIElement is the actual UI element in an iOS application
• XCUIElement provides all the basics symbols and functions for UI element interactions
• Gestures with XCTest include clicking UI elements (normal, double, pressing), interacting with the screen (swipe, pinch, zoom, rotate etc.)
Code Syntax (XCUITest)// Click-based functionstap() doubleTap() twoFingerTap() tap(withNumberOfTaps: UInt, numberOfTouches: UInt) press(forDuration: TimeInterval) press(forDuration: TimeInterval, thenDragTo: XCUIElement)
// Generic UI interactionsswipeLeft() swipeRight() swipeUp()swipeDown()pinch(withScale: CGFloat, velocity: CGFloat)rotate(CGFloat, withVelocity: CGFloat)
XCUIElement
• XCUIElement is constructed using the actual user interface elements on the screen
• XCUIElement inherits from NSObject
• In order to perform any interaction (tap on this example) on a UI element the UI interactions are available for use.
Code Syntax (XCUITest)XCUIApplication *app = [[XCUIApplication alloc] init];
XCUIElement *masterNavigationBar = app.navigationBars[@"Master"];
XCUIElement *editButton = masterNavigationBar.buttons[@"Edit"];
// Objective-C[masterNavigationBar.staticTexts[@"Master"] tap];[editButton tap];
• Two ways to create IPA for test automation1. Working with Xcode2. Working from Command Line
• Test package can be built as APP file and zipped for cloud execution
Preparing IPA and Test Package for Simultaneous Device Runs
Methods to Create IPA and Test Package for Simultaneous Tests
• By archiving any build package will be compatible with an iOS device
• When your build is done and archiving has finished, select the build from Archive list and click “Export…”
Working on Xcode to Create IPA for Testing
1. Archive Your Build
• When the following window is shown, simply select “Save for Ad Hoc Deployment” and click Next.
Working on Xcode to Create IPA for Testing
2. Select The Right Method
• Use the same identify what you use in build settings for code signing.
• If your project and Xcode is properly configured you should see the following type of dialog proposing the first usable identifier:
Working on Xcode to Create IPA for Testing
3. Identify Yourself (and Your Build)
• It’s almost always recommended to include all possible device support for your app.
• If you want to reduce the size of your IPA you can shrink it by selecting to support only certain devices and OS versions
Working on Xcode to Create IPA for Testing
4. Select OS and Devices
• This can be done Product -> Build for -> Testing menu:
Working on Command Line to Create IPA for Testing
1. Select to Build for Testing
• Next, select your project in Project Navigator and right click to “Show it in Finder”:
Working on Command Line to Create IPA for Testing
2. Locate App File on HD
• After Finder dialog has been opened and files are shown, just highlight the .app file and right-click to see copying option, as follows:
Working on Command Line to Create IPA for Testing
3. Copy App File Properly
• After Finder dialog has been opened and files are shown, just highlight the .app file and right-click to see copying option, as follows:
Working on Command Line to Create IPA for Testing
4. Create IPA from CMD
$ mkdir /tmp/Payload$ cd /tmp/Payload$ cp -r /User/Path/Debug-iphoneos/LocalizationDemo.app . $ cd ..$ zip --symlinks -qr "LocalizationDemo.ipa" Payload$ ls -lrt LocalizationDemo.ipa-rw-r--r-- 1 username staff 0 Dec 16 12:42 LocalizationDemo.ipa
• If building for iPhone 5 or iPhone 5C (with ARMv7 32-bit processor) devices, additional step is needed before creating the build.
• Starting from Xcode 7, armv7s is no more part of the default $(ARCHS_STANDARD) and so should be added as a target build architecture
Creating an IPA for iOS App Testing
Good To Know!
Quick XCTest/XCUITest Comparison
Ebooks Available for…
XCTest/XCUITest
Calabash Appium
Top 3 iOS Test Automation Frameworks
XCTEST/XCUITEST
APPIUM CALABASH
CROSS-PLATFORM
No Yes Yes
IOS Yes Yes YesMOBILE WEB Yes Yes No
LANGUAGE Obj-C/Swift Almost any RubyTOOL FOR TEST
CREATIONXcode Appium.app CLI
(Human-Readable syntax)
COMMUNITY Apple Community Community
DemoXcode 6 / 7 / 8
Summary – Q&AMore information about mobile app testing,
mobile monitoring and mobile devops at
bitbar.com
Recommended
