Upload
zavad
View
34
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Functional Testing with the Java Stack Test Runner . Disclaimer. This is a training not a presentation. Please be prepared to: Learn Ask questions Participate in the lab Prerequisites: Knowledge of Maven (required) Java Stack ALM Training (recommended). What we’ll cover…. - PowerPoint PPT Presentation
Citation preview
Functional Testingwith the
Java Stack Test Runner
DisclaimerThis is a training not a presentation.Please be prepared to:
Learn Ask questions Participate in the lab
Prerequisites: Knowledge of Maven (required) Java Stack ALM Training (recommended)
What we’ll cover…In this training, we will discuss
What is functional testing? How is it unique from other forms of testing? What is the Java Stack Test Runner? How does it integrate with Maven and
AnthillPro? How do I invoke Test Runner from my IDE? Where do I find my functional test results?
What we won’t cover…Topics that we will not have time to discuss are:
Testing frameworks and APIs• i.e. TestNG, Selenium, Spock, etc.Testing methodologiesWriting testsTest case management tools
First, some background…
Stages of TestingWhat When1. Unit Testing After code compilation
2. Integration testing After the build artifact is packaged.
3. Functional testing* After the build artifact is deployed and started.
4. Load testing After deployment to staging environment
5 End-user testing After code is “feature complete”.
* We will discuss functional testing and how it differs from unit and integration testing.
Unit Testing Scope• Narrow (single class or method)
Purpose• Validates small units of code
Responsible Party• Code developer
Failure:• Indicates code is broken• Typically fails the build
* Has direct dependencies on application code
Integration Testing Scope• Medium (spans two integrated systems)
Purpose• Validate code that integrates with another system.
(e.g. a database or web service) Responsible Party• Code developer.
Failure:• Indicates failure of systems to communicate as
expected.• Does not typically fail the build.
* Has direct dependencies on application code.
Functional Testing Scope• Broad (end-to-end feature testing)
Purpose• Validate functional requirements and/or user stories
Responsible Party• QA resource or feature developer
Failure:• Indicates failure to meet user story requirements• Results recorded in test case management and/or bug
tracker
* Has little or no direct dependencies on application code.
Functional Testingand
Application Life-cycle Management
(ALM)
ALM Build ToolsMaven
• Project definition• Dependency management• Plug-in configuration and
management• Build profiles
AnthillPro
• Manages build processes Creation of build artifact(s) Deployment to server
environment Functional test execution
• Maintains build life history• Provides ability to re-
execute a process on any build life
ALM Tools: Maven Life-cyclesThe default Maven build life-cycle (simplified):
compile test (unit tests) package integration-test (integration tests) install (to local Maven repository) deploy (to remote Maven
repository)
Where do we plug in functional testing?
ALM Tools: Maven Life-cyclesThe woes of having “one build to rule them all”:
Functional Testing ≠ Integration TestingConfuses build with server deploymentMust re-build to re-deploy or re-testComplicates build managementLess productive use of build agents
ALM Tools: Maven Life-cyclesThe Java Stack ALM build life-cycle (simplified):
alm-db (run database scripts) alm-deploy (deploy application to
server) alm-test (run functional tests) alm-promote (run all previous phases)
This life-cycle is available to the artifact produced by the “alm” module of a Java Stack project.
ALM Tools: Maven Life-cycleAdvantages of a second build life-cycle:
Clean separation of configuration:• One configuration for build• One configuration for deployment and/or functional
testing Separate processes:
• One process for building artifact• One process for deployment and/or functional testing
Simplified build management More productive use of build agents
ALM Tools: Test RunnerIn Stack 3.2, as part of its ALM tooling, the Java Stack introduced the “test-bundle” artifact as a part of its suite of testing plug-ins, collectively known as “Test Runner”:
Why another artifact?Why another tool?
ALM Tools: The Test BundleThe “test” jar
Not an executable jar Difficult to execute without
help from Maven. Relied on maven to supply
its dependencies. Difficult to execute tests
from past code revisions.
The “test-bundle” jar An executable jar Can be executed with or
without Maven. Comes packaged with all
its dependencies. Each test artifact
represents a snapshot of the current code revision.
ALM Tools: Test RunnerThe “failsafe:test” goal
Supports TestNG Supports test suites Supports environment
configuration Produces JUnit and TestNG
reports. Executes tests from the
target/classes folder
The “stack-test:functional-test” goal
Supports TestNG Supports test suites Supports environment
configuration Produces JUnit and TestNG
reports. Executes “test-bundle”
artifact.
ALM Tools: Functional Test GoalCommon configuration for stack-test:functional-test:• testBundle – groupId and artifactId of QA test bundle• testEnv – the test environment properties to load• skipFTs – whether to skip functional tests• suiteXmlResources – TestNG suites to load• includes – test class patterns (e.g. **/*FT.class)• systemPropertyVariables – custom system properties• argLine – JVM arguments (e.g. memory options, debugger,
etc.)
For the additional options, see:• http://code.lds.org/maven-sites/stack/modules/test/
ALM Tools: The Test BundleSimple to upgrade existing QA projects.1. Upgrade the project to Stack 3.2
Then change the QA pom’s packaging type from to
2. Add QA module as a dependency to the ALM module.
<packaging>jar</packaging> <packaging>test-bundle</packaging>
<dependency> <groupId>${project.groupId}</groupId> <artifactId>example-qa</artifactId> <version>${project.version}</version> <type>test-bundle</type></dependency>
ALM Tools: The Test Bundle3. Bind the functional-test goal to the alm-test phase:
<plugin> <groupId>org.lds.stack.test</groupId> <artifactId>stack-test-maven-plugin</artifactId> <executions> <execution> <id>test</id> <goals><goal>functional-test</goal></goals> <phase>alm-test</phase> <configuration> <testBundle> <groupId>org.lds.training.testrunner.lab1</groupId> <artifactId>testrunner-lab1-qa</artifactId> </testBundle> </configuration> </execution> </executions></plugin>
ALM Tools: The Test Bundle4. Modify build profiles to fit your needs<profile> <id>test</id> <properties> <skipFTs>false</skipFTs> <testEnv>test</testEnv> </properties></profile>
Lab #1A Simple QA Test Bundle Project
https://tech.lds.org/wiki/Functional_Testing_with_the_Java_Stack_Test_Runner
Lab #2Running Functional Tests
in AnthillProhttps://tech.lds.org/wiki/Functional_Testing_with_the_Java_Stack_Test_Runn
er
CreditsThe Apache Maven Project:http://maven.apache.org/Urban Code, for it’s AnthillPro documentation:http://www.urbancode.com/html/products/anthillpro/The Dark Lord Sauron, for the quote, “One ring to rule them all”. (The Lord of the Rings, by J.R.R. Tolkien)
<code snippet block>