Test Strategy Template Instructions_Automation

7/27/2019 Test Strategy Template Instructions_Automation

1/24

Test Automation Strategy

Author:


2/24

Master Test Plan Version 1.0- 10/14/2009

Page 2

Document Status: Final Template Version: 1.1

Version History

Changes to the Test Automation Strategy must follow the update process to ensure that the most

current version(s) are being used by all project team members. This section outlines how theseupdates to the Test Automation Strategy will be published and approved. The project teams

versioning standards should be adhered to for numbering convention.

Circumstances requiring approvals for changes to the Test Automation Strategy could include

any of the following:

Changes in automation scope or requirements resulting in corresponding updates to theAutomation Test Strategy

Risks, Constraints, Dependencies and Assumptions as and when identified as far asautomation is concerned.

Following any significant revision, the Test Automation Strategy will be distributed by email forreview and sign off by individuals identified as approvers in section 1.2. Voting button options

will include approve, approve with change(note in email response) orreject. Smaller or lesssignificant changes do not require project team approval.

Review Log

Version Date Section(s) Changed By Approval

0.1 Original Release QA Lead


3/24


Page 3


This Test Automation Strategy template reflects the standard processes and

procedures for the test automation-planning phase. The format of this template is

designed to facilitate the creation of a Test Automation Strategy by providing theessential sections that should be included in a comprehensive test automation plan,

and by suggesting subject matter for each section. The author should be familiarwith the Principal Financial Group development processes and related automation

testing procedures.

Text displayed in blue italics (style=InfoBlue) is included to provide guidance to

the author and should be deleted before publishing the document.

To customize automatic fields in Microsoft Word (which display a gray

background when selected), select File>Properties and replace the Title, Subject,

Project, and Version fields with the appropriate information for this document.After closing the dialog, automatic fields may be updated throughout the document

by selecting Edit>Select All (or Ctrl-A) and pressing F9, or simply click on the

field and press F9. This must be done separately for Headers and Footers. Alt-F9will toggle between displaying the field names and the field contents. See Word

help for more information on working with fields.

All sections of this Test Automation Strategy document are to be preserved to

maintain uniformity and indicate areas of deliberate omissions. The author shouldnot deliberately omitsections with the notation NA.

Each section of this template provides guiding information for the author and or

actual text that the author may use as examples of the content of each section.

Action Item: Any text in blue serves as a placeholder and/or sample data.Substitute it with real information or delete it.

Action Item: Please make sure to update all fields and tables in this document.(The key sequence is then )


4/24


Page 4


Table of Contents

1.0 INTRODUCTION ......................................................................................................................................... 6

1.1 PURPOSE...................................................................................................................................................... 6

1.2 TARGET AUDIENCE ..................................................................................................................................... 6

2.0 ESTABLISHING THE ASSIGNMENT ...................................................................................................... 7

2.1 IDENTIFY THE SCOPE ................................................................................................................................... 7

2.1.1 In scope .................................................................................................................................................. 7

2.1.2 Out of scope ........................................................................................................................................... 7

2.2 IDENTIFY THE PRECONDITIONS.................................................................................................................... 82.2.1 Dependencies/Constraints/Considerations ............................................................................................. 8

2.3 IDENTIFY THE ASSUMPTIONS....................................................................................................................... 9

2.4 IDENTIFY THE RISKS .................................................................................................................................. 10

3.0 DETERMINING THE STRATEGY ......................................................................................................... 13

3.1 TEST LEVELS............................................................................................................................................. 13

3.1.1 System Test .......................................................................................................................................... 13

3.1.2 Regression Testing .......... .......... ........... .......... ........... .......... ........... .......... ........... .......... ........... .......... .. 13

3.2 RISK TABLE ............................................................................................................................................... 13

4.0 HIGH LEVEL MILESTONES .................................................................................................................. 14

4.1 HIGH LEVEL MILESTONE ESTIMATED DATES ............................................................................................ 14

5.0 ENTRANCE / EXIT CRITERIA ............................................................................................................... 15

6.0 SUSPENSION CRITERIA ......................................................................................................................... 15

7.0 DEFINE THE TEST PRODUCTS ............................................................................................................ 16

7.1 TEST STRATEGY ........................................................................................................................................ 16

7.2 TEST SCRIPTS ............................................................................................................................................ 16

7.3 TEST DATA................................................................................................................................................ 167.4 PROGRESS REPORTS................................................................................................................................... 16

7.5 TEST EVALUATION REPORTS..................................................................................................................... 16

8.0 DEFINE THE ORGANIZATION ............................................................................................................. 17

8.1 ROLES &RESPONSIBILITIES ...................................................................................................................... 17

8.2 TRAINING NEEDS ....................................................................................................................................... 18


5/24


Page 5


9.0 COMMUNICATION PLAN ...................................................................................................................... 19

9.1 MEETINGS AND REPORTS ........................................................................................................................... 19

9.2 TELECONFERENCE INFORMATION .............................................................................................................. 21

9.3 WEB CONFERENCE INFORMATION............................................................................................................. 21

9.4 PROJECT DISTRIBUTION LISTS .................................................................................................................... 21

9.5 OUTLOOKPUBLIC FOLDERS ...................................................................................................................... 21

9.6 DOCUMENT STORAGE ............................................................................................................................... 21

9.7 QUERY LOG............................................................................................................................................... 21

10.0 DEFINE THE INFRASTRUCTURE ........................................................................................................ 22

10.1 APPLICATION SETUP IN SYSTEM TEST ENVIRONMENT .............................................................................. 22

10.2 TEST USERS SETUP.................................................................................................................................... 22

10.3 SYSTEM SETUP .......................................................................................................................................... 22

11.0 ORGANIZE TEST MANAGEMENT ....................................................................................................... 23

11.1 TEST PROCESS MANAGEMENT .................................................................................................................. 23

11.2 TEST CASE COVERAGE RATIOS .................................................................................................................. 24

11.3 INFRASTRUCTURE MANAGEMENT ............................................................................................................. 24

11.3.1 Meetings .......................................................................................................................................... 24

11.3.2 Environment, Tools and Workplaces .............................................................................................. 24

11.4 DEFECT MANAGEMENT............................................................................................................................. 24


6/24

Page 6


1.0 INTRODUCTION

This Test Automation Strategy communicates the approach to implementing the automation

testing effort using a test automation tool.

This document serves as both a communication tool for the automation team and a governingdocument for the test automation initiative. It will be maintained to reflect all significant

changes to the scope, approach, test methods or entry/exit criteria defined in this strategy. In

addition, the only authorized deviations from standard practice controls and deliverables are

those noted in the approved plan. All activities and their deliverables will be conducted inaccordance with standard policy and procedures and are subject to a compliance audit.

1.1 PURPOSE

The purpose of this Test Automation Strategy is to communicate to the project

stakeholders and to the testing team how the Strategy will be implemented and to

establish authorization to proceed with the test initiative using the referenced approach,

standards and controls. This is a governing document for the test activities therefore thedeliverables and key controls identified in this Strategy are subject to review for

compliance with standard policies and procedures.

1.2 TARGET AUDIENCE

Indicate who will approve document. Sign-off is only required for approvers.

The target audiences for this document are those that represent the following roles on thisproject:

Role Name Approver /

Reviewer

Sign-off

Date

QA LeadTest Manager(As per

the RUP template)

Test Analyst -Automation

Business Analyst


7/24

Page 7


2.0 ESTABLISHING THE ASSIGNMENT

2.1 IDENTIFY THE SCOPE

2.1.1 In scope

This Test Strategy for the testing initiativeincludes the following areas of scope:

In this section, identify the scope of automation testing activities planned. Includethe following topics:

Explain about the need/goals of automation for the project and how it would help

in overall testing.

What types of testing are covered under this automation strategy, such as

functional, regression, smoke or other tests?

Why a particular test tool is selected.

How the proof of concept was carried out, how much coverage is expected throughautomation, which types of test cases are to be automated.

Define dependencies, such as, impact of changes in development plan, features,

etc. Make recommendations on the time-line for automation.

Provide a high level design that is scalable and allows future addition of testscripts.

Provide a plan to manage and maintain the Automation scripts (post-delivery)

What components of the overall application are to be tested?

What is the relationship of the application to be automated to the overall system /

environment?

2.1.2 Out of scope

The following functions, systems, processes are specifically out of scope for the testing initiative:

In this section, identify specific items that are outside the scope of testing

What functional business areas will not be automated?

Mention that high complex functions will not be automated.

Mention that tests requiring large calculation or more user interference will not beautomated. Files containing multiple records would not be tested.

Mention if tests involving backend validations are not done

Mention that non repetitive Test Cases (They may be automated but is not feasible)

Mention that tests requiring longer time to automate but manual testing time would

be too less, would not be automated.


8/24

Page 8


Mention that tests involving Graphics might not be automated.

Mention that test cases involving testing of printing features can not be tested

Mention that test cases involving the Mozilla browser can not be tested

2.2 IDENTIFY THE PRECONDITIONS

2.2.1 Dependencies/Constraints/Considerations

List any dependencies/constraints/considerations that may impact the design,

development or implementation of automation testing; e.g., dependencies on other

organizations (such as DEV, Environment group, Users, and or SMEs) and orschedules, resource dependencies, etc. Some samples have been provided below.

Indicate other projects that may affect this project.

Description Type

Dependency, Constraint, Offshore

consideration, XBU consideration,

Vendor consideration, other

Impact/Consideration

Manual test cases not ready Dependency Automation may not start as

the framework maynot be

created without knowing allthe test cases that needs to be

automated.

Contractors contract expires12/01/2010

Constraint Resource availability

There is a company freezebeginning 12/15/2009

Constraint Time frame/ Schedule

PFG will be assessed a fine if

change does not occur by12/31/2009

Constraint Regulatory

SharePoint Access Offshore consideration Who will need access toSharePoint

Timing Issues related to cycle Offshore consideration Time difference

Communication across

business units

XBU consideration How will communication be

handled?

Determine which QualityCenter Domain & Project will

be used

XBU consideration Access/Security to QualityCenter Domain & Project

Defect Resolution Vendor consideration How will defects becommunicated to vendor and

how will vendor communicate

when defect has been


9/24

Page 9


resolved?

2.3 IDENTIFY THE ASSUMPTIONS

Identify the significant assumptions made in developing this plan, which if false,

may impact the design, development or implementation of testing

The Business Analysts for this project will be available to answer questionswhen necessary.

The environment will be available andstable.

Manual testing will be completed before automation begins. The results willbe available to the project team.

If there is a slippage in the project timeline or scope, automation test scriptcreation will also be impacted.

Existing production issues will remain unless specifically addressed by the

project. QA resources assigned to the project will have previous knowledge of

creating test scripts using the test automation tool.

All assigned QA resources must be given a background of the project andunderstand those aspects prior to the start of script creation.

Test execution will not start until signoffs are received for all QA artifacts thatrequire signoff.

All the security access needed will be given to the test automators prior to testscript creation.

Test cases will be written based on business and functional requirements andtechnical designs. If any of these documents are not completed, test scripts

will not be in accordance and there will be a risk of having gaps and rework.


10/24

Page 10


2.4 IDENTIFY THE RISKS

The table below outlines project or process risks to testing. These types of risk generally

affect the test teams ability to work efficiently or meet project milestones.

The content of the table below are examples only -

Estimates of project Impact and Probability of occurrence are listed in the table as

L=Low, M=Medium, or H=High. The Risk Measure is a calculation of[Impact x Probability] with a scale of 1-9. Values are assigned as follows:

L = 1

M = 2

H=3

Description Impact Probability

Risk

Measure Mitigation


11/24

Page 11


Description Impact Probability

Risk

Measure Mitigation

The completion of

automation scripts woulddepend on the Manual

Testing completion

H H 9 As and when the manual test case

creation and execution iscomplete, one can start

automation, need not wait for theall the test cases to be complete.

Certain test cases will not

be automated due to high

complexity. Henceautomation will be

incomplete for those area

H H 9 This might not be decided at the

time of planning but as the work

progresses in automation, one mayfind such a scenario which needs

to be discussed and addressed.

Requirements will changewhile the project is in

development due to

changing regulatoryrequirements, businesschanges, etc.

H H 9 Project Management will reviewany changes, assess impact, risk,

and cost and present these to the

steering committee for review.This will include date changediscussions.

Testers unfamiliar with newenvironments.

H H 9 PM will provide point of contactfor environments to the QA Lead.

If any environmental issues arise,

QA will raise any issues orconcerns immediately to the PM.

Security access and profile

timelines.

M L 2 PM has negotiated the dates for

Security definition

Testing resources assigned

to project are unable to

commit time for testing.

M M 4 Work with testing resources to set

up dedicated testing time. Work

with Project Manager to resolveany resourcing issues.

Delay in code delivery. H L 3 Additional QA effort will need tobe identified to overcome the

timeline gap. This will happen if

the timeline is not adjusted for the

delay.


12/24

Page 12



13/24

Page 13


3.0 DETERMINING THE STRATEGY

The test strategy is based on the product risk analysis, and the decision on which

quality characteristic must be tested per test level, and to what depth.

3.1 TEST LEVELS

The following test levels will be executed:

3.1.1 System Test

A system test is a test carried out by the test team, with the aim of

demonstrating that the developed system, or parts of the system, meets with thefunctional specifications and the technical design. (This includes regression

testing of existing functionality and interfaces with other systems.)

The automation scripts would be created as per the test cases and would be

executed end to end to complete the System Testing test cases.

3.1.2 Regression Testing

Regression Testing will be to demonstrate rerunning previously run scripts andchecking whether previously fixed faults have re-emerged.

3.2 RISK TABLE

Determine which test phase characteristics are tested in; System, Performance,

Regression. Rate them by High. Medium, Low

The manual test cases would cater to these characteristics but in case you want to

emphasize on the part creation of test scripts, you can enter the product riskanalysis and the decision on which quality characteristic the test scripts must be

created per test level, and to what depth. The decision totally depends on the

feasibility of creation of automation scripts and support of the test tool.

System Performance Regression

Connectivity

Continuity

Data

controllability

EffectivelyEfficiency

Flexibility

Functionality

Suitability of

Infrastructure

Maintainability

Manageability


14/24

Page 14


Performance

Portability

Reusability

Security

Suitability

TestabilityUser-

Friendliness

4.0 HIGH LEVEL MILESTONES

4.1 HIGH LEVEL MILESTONE ESTIMATED DATES

Below are high-level milestone as currently defined. For more detailed schedule

and milestone information reference the project Strategy on the project share drive.

Activity Estimated Start Date Estimated End Date

Create Test Automation Strategy

Create Automation Framework

Identifying the reusablecomponents and preparation

Creating the OR

Test Data Preparation

Script Creation

Review

Rework

Individual Execution

Batch Execution

Report Analysis

Downtime/Issue Log

Documentation


15/24

Page 15


5.0 ENTRANCE /EXIT CRITERIA

System Test (performed by automation Tester role)Criteria Description

Entry Criteria Coding is complete and unit integration testing has been reviewed and

accepted by the test organization

No outstanding critical defects open

The defined System Test environment is prepared and stable

System Test scripts have been written and reviewed for coverage and

completeness

System Test data is available for test execution

Change Request and Version Control process is in place

Exit Criteria Agreed upon coverage ratios have been met and all scripts have been

executed.

Application has been sufficiently tested to meet requirements (by

execution of scripts)

No outstanding critical defects remain open

Regression Test (performed by automation Tester roles)

Criteria Description

Entry Criteria Approval to deploy application has been received from business

Application has been deployed to production

Exit Criteria All test scripts have been executed

No outstanding critical defects remain open

6.0 SUSPENSION CRITERIA

This section will identify and address any situation where the automation test script

creation effort might have to be temporarily suspended.

6.1 Test environment becomes unavailable.

6.2 Test cases are blocked by a documented defect.


16/24

Page 16


7.0 DEFINE THE TEST PRODUCTSDefine the artifacts that will be produced for this project. Examples have been includedbelow.

7.1 TEST STRATEGY

System Test Plan

Regression Test Plan

7.2 TEST SCRIPTSTest scripts, test data and documentation and other xls, vbs files attached to the

script are housed in Quality center:

Domain=Project=

7.3 TEST DATADocumentation of data to be used for execution of test scripts, both in QC and in

the xls data sheet.

7.4 PROGRESS REPORTS

Test execution status reportsweekly reports of test execution status andapplication quality

7.5 TEST EVALUATION REPORTS

Issues after Release Advice

Gives insight into the way the test script creation and execution process hasgone

Provides results from the lessons learned session


17/24

Page 17


8.0 DEFINE THE ORGANIZATIONDefine the Roles & Responsibilities that will be utilized with this project. If you will notbe using a specific role, indicate Not Applicable.

8.1 ROLES &RESPONSIBILITIES

Role Assigned To Responsibilities

QA Lead/

AutomationAssignment Manager

Prepare Test Automation Strategy

Schedules and facilitates automation testrelated meetings

Provides oversight of automation relatedDefect Management process

Assists in mentoring and training of projecttesters when appropriate

Provides testing status updates throughout theautomation testing phase

Provides resource estimatesTesters/ Test

Automationdeveloper(as per the RUPtemplate)

Create test scripts

Execute test scripts within the timeframeallowed

Log defects and retest as necessary

Communicate status of testing to QA Lead

Assignment Manager

Partner with QA Lead to ensure successfulexecution and completion of automationtesting related activities on business unit

projects.

Business Owner/SME Work with Business Analyst to clarify

requirements. Collaborate with QA to get clarification while

creating test scripts in order to ensure propercoverage of all business requirements andfunctional specs., that are already dealt with inthe test case.

Help identify new areas for automationtesting, and define validation responsibilities.

Project Manager Ensure the project scope is finalized prior tothe team beginning test script creation and

execution.

Ensure adequate project timelines support thetime needed for test preparation andautomation script execution.

Assist the QA Lead, as a resource, inidentifying the reporting needs, specific to the

project.


18/24

Page 18


Test Environment

Support

Migrate code

Test environment issue resolution.

Configuring test environment.

Participate in Automation Testing StatusMeetings as needed.

8.2 TRAINING NEEDS

Describe what training, if any, is needed for the test team to come up to speed.

Examples may include automation tool training, Quality Center training orapplication knowledge.


19/24

Page 19


9.0 COMMUNICATION PLAN

9.1 MEETINGS AND REPORTS

Describe how often and in which way the automation test team will report withinand outside the team. Make sure this is in accordance with the customers wishes.

Describe how often and with whom a meeting shall take place

This section should address all forms of communication; meetings, reporting, etc.,list exceptions to the standards.

Event Objective Method

(Report

/Meeting)

Chair Attendees /

Audience

Frequency

Testing Kickoff Review timeline

and introduce theTest Automation

Strategy document

to QA resources.

QA Lead Project Manager

QA LeadAutomation

developer

Technical Lead

Project Sponsor

Once

Test Status

Review

Review general

health of

Automation Testing

Initiative.

Operational Issues

QA Lead Test team

QA Lead

Weekly

General

Project

Meeting(s)

Describe any

general project

meetings known at

the time of the Test

Strategy that

members of the test

team will be

attending.

Project

Manager

Technical Lead

Business AnalystProject Sponsor

QA Lead

Weekly

Weekly Status Shows status of

automation script

creation and

execution.

Report QA Lead Project Manager

QA Lead

Project Sponsor

Weekly

Defect Review Review status of

open defects.

Identify any newissues. Release

status.

QA Lead QA Lead

Project Manager

Technical Lead

Weekly or As

needed

Lessons

Learned

Discuss project

results and ID

lessons learned for

future

improvement.

Project

Manager

QA Lead

Project Manager

Technical Lead

Automation

developer

Once


20/24

Page 20


Business Analyst


21/24

Page 21


9.2 TELECONFERENCE INFORMATION

Update items in red with your specific information.

US & Canada: Dial-In Number: XXX-XXX-XXXXInternational: Dial-In Number: XXX-XXX-XXXX

Enter conference Code: XXX-XXX-XXXX#

Leader enter *; then PIN: XXXX#, then 1

System will ask you to state your namemay do so, or leave blank bypressing #

System will sound a tone then connect the call

9.3 WEB CONFERENCE INFORMATION

Update items in red with your specific information.Sign in by clicking on the URL keying all information below:

Meeting URL:https://www.livemeeting.com/cc/principal/joinMeeting ID: XXXXXX

Presenter Key: XXXXXX

Attendee Key: XXXXXXXX

9.4 PROJECT DISTRIBUTION LISTS

Include email distribution lists built for this project.

9.5 OUTLOOKPUBLIC FOLDERS

The following public folders will be used by this project for testing:

Add Outlook path here

9.6 DOCUMENT STORAGE

Indicate where your documentation is stored and provide the path or a link.

SharePoint

Project folder

9.7 QUERY LOGIf your project is not utilizing PGS resources, the Query Log is Not Applicable.

The Query Log within SharePoint will be utilized to communicate general

questions and answers regarding terminology, resources and knowledge gaps

encountered while creating automation test scripts.
https://www.livemeeting.com/cc/principal/joinhttps://www.livemeeting.com/cc/principal/joinhttps://www.livemeeting.com/cc/principal/joinhttps://www.livemeeting.com/cc/principal/join


22/24

Page 22


10.0 DEFINE THE INFRASTRUCTURE

10.1 APPLICATION SETUP IN SYSTEM TEST ENVIRONMENT

The system test environment needs to mimic the production environment. The

environment, including the software and data, must be a controlled environment.Development will be delivering new builds to the System environment on an agreed upon

schedule. When each new build is delivered to QA, the following application setup needs

to be completed:

Provide additional information as needed for your project.

10.2 TEST USERS SETUP

Specify any special user setup. Consider the needs for test accounts with various

access levels.

10.3 SYSTEM SETUP

The following applications must be controlled and available in the System environmentto support the System Testing effort.(This should include any hardware setup that needs

to be done for the automation tool to run smoothly.)

System Included in test? Notes

Yes No Note special data requirements, etc.

Yes No

Yes No

Yes No

Yes No

Yes No

Yes No

Yes No

Yes No


23/24

Page 23


11.0 ORGANIZE TEST MANAGEMENT

Describe the manner in which the management of the test process, infrastructure, test

products and defects is organized

11.1 TEST PROCESS MANAGEMENT

This should be in line with the manual test process management. The defects detected bythe execution of the automation scripts should be assigned and tracked. Some in-process,

production readiness metrics can be calculated from the data gathered in defect

management tool and the test management tool. Some of these real-time metrics help

predict whether the quality of the software will be acceptable by the scheduled release.Others provide status on the completeness of the testing phases, which can be compared

to the project plan. Typically, these metrics are calculated and reviewed as part of project

status.

Note: In the metrics below defects include defects only. Defects do not include anyenhancements that may be logged. Each test level may report the metrics for that level in

their weekly status.

Metric: Indicates/Reflects on:

# open defects by severity Status of the software towards production ready.

List of open defects used to evaluate the current status,

severity and priority of open defects.

# new defects found by week Whether the software is approaching production ready.

This is viewed graphically as a time series chart.

If defect arrival rate does not quickly decrease, teamleaders and management should promptly assess the

situation.

# new defects found by week by

severity

Whether the software is approaching production ready.

This is viewed graphically as a time series chart.

Critical and major defects should decrease very quickly.

# defects resolved by week Whether the software is approaching production ready.This is viewed graphically as a time series chart.

If defect resolution rate is not greater than the # of newdefects found by week, team leaders and management

should promptly assess the situation.

# test cases by execution status

pass/fail/no run

% test cases executed

Whether the software is converging on production

ready.

If the # test scripts passed per week does not quicklyincrease, team leaders and management should promptly


24/24

Page 24

Metric: Indicates/Reflects on:

assess the situation.

11.2 TEST CASE COVERAGE RATIOS

80% is the recommended overall test case coverage ratio for test automation. Thismay be adjusted up or down based on project risks, priorities and impacts.

The test coverage ratio is the comparison of the total number of test cases

automated to the total number of test cases written.

11.3 INFRASTRUCTURE MANAGEMENT

11.3.1 Meetings

The Automation Test Manager & QA Lead will meet on an agreedupon scheduled time to discuss what more needs to be automation and

its feasibility.

11.3.2 Environment, Tools and Workplaces

Describe any additional items that are specific to your project. Examples havebeen provided.

Test automation validation will occur in Text Pad and/or Excel asneeded

Test Environment Location:

Path/Logical location information needed to access Test Environment

Security required for access to the Test Environment

Script creation and execution will occur at automation testers desks

11.4 DEFECT MANAGEMENT

Defects will be tracked within the following Quality Center Domain & Project.

< QC Project>

Justify any deviations from this process.

Documents

Test Strategy Template Instructions_Automation