A free webinar produced by the Technology Association of Oregon's Quality Assurance Forum Group. Presented by Trent Peterson, CEO and Co-Founder of Portland-based AppThwack, providing fast automated mobile device testing services for Android, iPhone, iPad, and mobile web apps. The recording to the webinar can be accessed here: http://youtu.be/r9Vm6ssSEU8
Citation preview
1. Testing for Android & iOS When, Where, and How to
Successfully Use Test Automation Trent Peterson | @tdpeterson |
[email protected]
2. Who? 3. AGENDA What is testing? Whats the problem? Why should
I care? When: Choosing our battles Where: Best Practices How: Tools
& Frameworks Q&A 4. INTRO TO TESTING 5. Types of tests
Functional Non-functional Integration Unit Regression Smoke
Usability Performance Stress Acceptance Load Sanity End-to-End
System Security And on and on 6. Testing distilled for this
discussionFunctional Testing Non-functional Testing 7. THE PROBLEM
8. Building things is hard. 9. WHY SHOULD I CARE? 10. So whats the
big deal?1,000,000+ apps in 1,000,000+ apps in2 crashes and 84%
will uninstall - Compuware as reported in TechCrunch 11. Weve been
here before. 12. CHOOSING OUR BATTLES 13. First, whats our
goal?Save time! BE REALISTIC Save money! Increase regularity of
testing Shrink QA! Free up resources for other testing efforts
Enable previously impossible tests AUTOMATE EVERYTHING ! 14.
Deciding what to automateHow often will I test this? Is this
something a human is good at? How much effort will automation
require? 15. Some examples AUTOMATEMANUAL Granular functionality
(unit tests) Repetitive tasks (update paths, navigation, etc.)
Performance UX Responsiveness and feel New functionality 16. Our
options (Machine) JVMEmulatorsReal Devices FAST Unit tests only*
Mocks Cheap Catch layout issues Simulate calls/SMS SLOW Illusion of
1:1 with real devices Exactly what end customers use Performance
data Fast Expensive: Purchase, maintai n, etc. 17. Our options
(Human) QACrowdEnd Users Pros Real feedback Know the product
Expensive Real feedback Might know product Expensive (usually) Slow
Dont do this. 18. BEST PRACTICES 19. No barriers between Dev &
QA 20. Automation is a software project 21. Automation is a tool,
not a solution 22. Automate with precision 23. Its all about data
24. Know your matrix 25. Abstraction is importantbut dont obsess
26. TOOLS and FRAMEWORKS 27. Tool: Recorders GOODBAD Fast
Non-developers can create tests Fragile Difficult to maintain
Non-developers can create testsBEST PRACTICE Treat as a fancy spy
28. Tool: UI Exerciser Monkey $ adb shell monkey p my.sweet.app v
1000 29. Framework: Android Testing SUMMARY JUnit/Java
Instrumentation provides hooks into Android SDK to control
Android-specific functionality Very basic functionality supported
The foundation for most Android testing. Learn it, use it, love it.
30. Framework: Robolectric SUMMARY JUnit Runs in JVM Mocks Android
functionality with shadow objects Super fast Not everything is
mocked (also, mocks) Great for staying sane while developing 31.
Framework: UI Automator SUMMARY JUnit/Java Extends testing
framework with UI selectors and manipulators Handle/control system
dialogs, etc 4.1+ only Excellent choice if you have the luxury of
supporting 4.1+. 32. Framework: Espresso SUMMARY JUnit/Java Extends
testing framework with UI selectors and manipulators Removes
boilerplate code (sleeps, etc) Very new and unstable If youre
feeling adventurous give it a try 33. Framework: Robotium SUMMARY
JUnit extension Runs on emulators and real devices Oriented towards
black-box testing and ensuring real-world outcomes Limited to the
app under test Great choice for developers going beyond basic unit
testing and testing UI. 34. Framework: Cucumber-based SUMMARY
Human-readable tests Runs on emulators and real devices
Cross-platform Complex scenarios require development
Behavior-driven design. Be careful! Great choice for simple flows
and teams with limited developer resources. 35. Framework: Appium
SUMMARY Supports many language adapters Runs on emulators and real
devices Cross-platform New, and not all WebDriver concepts map
intuitively to native apps Great choice for teams happy with
Selenium and expanding to native. 36. Framework: UI Automation
SUMMARY JavaScript Installs with Xcode Functions with other
instruments Has a recorder (Careful) Dev/maintenance of JS Good
choice to get basic tests up and running quickly. 37. Framework:
XCTest SUMMARY Objective C Installs with Xcode Poor documentation
Obvious choice for unit testing. 38. Framework: KIF SUMMARY
Objective C Enables functional test automation capabilities by
providing UI helpers Relies on undocumented APIs Great choice for
devs automating functional tests. 39. Tool:
SpoonSpoonInstrumentation Tests 40. Tool: BotsContinuous
Integration Execute tests, etc. 41. Frameworks: Commercial GOODBAD
Support Added features and integrations Longevity? Proprietary
Often heavy and difficult to maintain $$$ADVICE Evaluate very
carefully and acknowledge lock-in. 42. When choosing, askWhat are
we testing? What are our automation goals?Who will write the
automated tests? 43. SHAMELESS PLUG 44. AppThwack 100s of
non-rooted devices Results in minutes Built-in compatibility tests
Supports all popular automation frameworks Simple API and
integrations Focused, comprehensive reports 45. Q&A Trent
Peterson | @tdpeterson | [email protected]
