View
1.337
Download
1
Embed Size (px)
Citation preview
1
Introduction to Back End
Automation Testing
16 | 07 | 2016
Nguyen Vu HOANG
Hoang PHI
FPT Software
WHO we are
2
HOANG Nguyen PHI Hoang
Education and Certification Education and Certification
Master of Computer Sciences
ISTQB (Advanced Level Technical Test Analyst)
Scrum Master, Product Owner and ITIL (FL)
CEH, LPIC-1, Novell CLA, CCNA, MCSE, MCDBA, MCSA, MCSD.NET and MCAD.NET.
Bachelor of Sciences
ISTQB (FL)
IBM Certified (Plan, Estimation, Design and Negotiation)
Agenda
3
WHAT is back end testing
WHY Test Automation
Testing Problems and Solutions
WHAT we do
HOW we do
Measurable Results
Demo
WHAT is back end testing
4
Service layer (80%) – perform testing for services WITHOUT User Interface (UI), AND
WHAT in the scope
Enterprise Services at API layer.
RESTful Web Service architectural style.
SOAP Web Service used for general enterprise integration.
Connect-based middleware stack (i.e.: JMS – Java Message Service)
WHAT are tested…
No UI or Used by back-office operators Web Service, Database, File System, Web Application
Backend application (20%) –perform testing for application/system with UI which is used to monitor and control services.
Enterprise system with UI used by back-office operators.
WHAT is back end testing (Cont.)
5
Web Application(Service Monitor and Control)
Core Business Services
End-User Platforms
External Services or Apps
Client Platforms
Operators
Applications and Services Backend Web
Application can be
tested via user
interface.
Platforms can be
any if it follows
service protocols.
Delivered services
are NOT for
specific platform.
1
1
2
2
Not Available
3
Platforms are not
available when
testing.
3
3rd party Services DBMSXML
WHY Test Automation
6
Saves time and money
Increases test coverage
Improves accuracy and quality
Huge data verification achievable only by automation. (i.e. verify thousands of XML attributes)
Better coverage, more executions, higher number of test cycles.
Reliability – ensure correct test at protocol layer and eliminate human error.
Auditability – provide comprehensive results.
Regression testing – gain confidence that nothing is broken.
Reusability – same tests on different environmentsand configurations.
Reduce execution time.
Testing Problems and Solutions
7
Testing Problems High Level Solutions
Different technology platforms under
test (Operating System, Database, Web
Server and application protocols and
development).
High data rates and responsiveness
Defragmented testing result so it is
difficult to make decision.
Must align testing with agile
development methodologies.
Integrate automation testing to Continuous
Integration.
Select proper technology stack to centralize
the testing result from both manual and
automation teams
Design automation test framework to deal
with Shell, Web Service, Web Application,
gigabytes of XML Data on Windows and
Linux.
1
2
3
WHAT we do - Standardize CI/CD
8
Continuous Integration
Build Automation
Static Code QualityAnalysis
VersionControl
Collaboration Wikis, Forums
Manage Develop Build Test Deploy
Test Automation, Performance Testing
Code Review and Quality Analysis
Unit Testing
Cloud Deployment
Containerize
Improve the way we develop and test applications by standardizing Continuous Integration (CI) and
Continuous Deployment (CD) tool set.
Reduce test and deployment cycles through automation.
Issue TrackingTest / ReqManagement
Artifact Management
1
WHAT we do – Do Continuous Deployment
9
By saving time in deployment, testers will be able
to spend more time on feature testing.
Prevent mistakes in installation while there is lots
of configuration defined in release note.
New and old application/services are
deployed side by side to eliminate delay
impact on testing team in case testing is
blocked on the new version.
Testing team starts testing cycle on the new
version after smoke testing passed.
Ant executes test cases
Automation Test Engine
VIP Cut Over
Application1.0
Virtual IP
Application1.1
Dependent ComponentDB 1 DB 2
1
WHAT we do – Centralize Test Result
10
Full cycle from test design to test
execution and reporting.
LISA facilitates test result to
SpiraTeam.
Integration with different tools –
extensibility mechanisms to
support Web GUI and bash shell.
Simple maintenance – applied
keyword and data driven
automation, conventions and best
practices.
2
Reporting Database
Test Case ManagementRequirement Management
HOW we do – LISA as Core
11
Deliver end-to-end parallel app development solutions to help develop and test simultaneously [1].
Powerful at testing middleware using Service-Oriented Architecture and Database.
Allow high level of customization. Functionality can be extended through its SDK (software development kit) [2].
Enable in-process, bi-directional communication between the Java VM and the CLR (.NET)
Both desktop or server versions provide integrated environment to build all in a code-less manner.
3
HOW we do – Test Web Service and Database
12
Use Service Virtualization to
break dependency of testingfrom development schedule [2].
Use consistent approach to
design test cases for web
services.
Use powerful built-in
functionalities of LISA to
automate Web Services and
Database testing.
Extend LISA’s capabilities to
test JSON.
Client Server
Web Server Database Server
Development in progress
Business process analysis
Transaction analysis (loss and duplication)
Performance analysis
Path coverage analysis
Error Handling analysis
Profiling …
Service Mocking, Service Virtualized
Test Case Design
Approach
3
HOW we do – Consideration in Scripting
13
Identification
Implementation
Execution
Report
Assessment
3
HOW we do – Test Web GUI and Shell
14
Integrate Selenium into LISA so
that it can test Web GUI.
A wrapper layer is built on top of
core Selenium in order to provide
strong GUI testing capability for
LISA.
Integrate JCraft into LISA so that
it can test Shell script.
Use Awk/Nawk/Gawk [3]language at remote host instead
bash script to have fast text
processing with low memory
usage. Performance is 10+ times
faster than bash script.
JCraft
Extended Selenium (Java base)
Core Selenium
LISA
SSH
3
HOW we do – Test Big XML files
15
LISA
Xstream JAXB
Process large files (2GB to 3GB for each).
Traverse nodes to smaller chunk.
Cost less memory resource but takes time to read XML file.
Xstream
Process delta (small) data.
Binding Java-to-XML makes it easy to incorporate XML data and processing Java functions quickly.
Cost more memory resource but it’s fast.
JAXB
Combine Xstream and JAXB
3
~3 min: XML (~3GB) and XSD Schema.
~6 min: 2 XML files (~3GB per each)
Results
Measurable Results
16
Unified automation test approach for 60+ projects.
Automated tests that cannot be done by manual.
100 000+ executed tests
12 000+ automated tests
92% saving execution time
Application1(547TCs)
Manual4 MDs
Automated50min
97%
Application2(590TCs)
Manual23.5MDs
Automated90min
99%
Application3(211TCs)
Manual5.1MDs
Automated1 MD
80%
Application4(2000TCs)
Manual1755hrs
Automated155hrs
91%
Application5(221TCs)
Manual117hrs
Automated7 hrs
94%
Application6(30TCs)
Manual30hrs
Automated13min
99%
Tool Cost ($) Similar Tool
Spira 3-Users ($79.99 / month) Testlink (free)
LISA 5-Users (~$1,666 / month) SoapUI (free/license)
Tool price and similar tool
Demo
17
1) Trigger test from Jenkins
2) Call LISA script
3) Call Shell script
4) Call Selenium script
5) Update result to SpiraTeam
(Duration: 1.5 minutes)
References
18
[1]. Develop & Test (former named CA LISA), http://www.ca.com/us/products/develop-test.html
[2]. Documentation of DevTest Solutions - 8.0, https://docops.ca.com/devtest-solutions/8-0/en
[3]. Awk/Nawk/Gawk tutorial, http://www.grymoire.com/Unix/Awk.html
© 2016 HCMC Software Testing Club
THANK YOU