61
Software Engineering MCA-303 Mayank Gupta(Student of MCA) ABES Engineering College, Ghaziabad

software testing

Embed Size (px)

DESCRIPTION

 

Citation preview

Page 1: software testing

Software Engineering MCA-303

Mayank Gupta(Student of MCA)ABES Engineering College, Ghaziabad

Page 2: software testing

Software testing is the process of testing the software product.

Effective software testing is the delivery of higher quality software products , more satisfied users, lower maintenance costs, more accurate and reliable results.

However, ineffective testing will lead to opposite results, low quality products, unhappy users, increased maintenance cost s, unreliable and in accurate results.

Software Testing

Page 3: software testing

Testing Objectives

In an excellent book on software testing, Glen Myers states a number of rules that can serve well as testing objectives:

1.Testing is a process of executing a program with the intent of finding an error.

2. A good test case is one that has a high probability of finding an as-yet-undiscovered error.

3. A successful test is one that uncovers an as-yet-undiscovered error.

Page 4: software testing

• Software testing is process of executing software in a controlled manner.

• The objective of testing is to show incorrectness and testing is considered to success when error is detected (Myers 1979).

• Software testing involves operation of a system or application under controlled conditions and evaluating results.

• The controlled condition should include both normal and abnormal conditions.

Page 5: software testing

The goal of testing is to find bugs and fixed them as early as possible. But testing is potentially endless process.

We cannot test till all the defects are unearthed and removed -- it is simply impossible and expensive.

At some point, we have to stop testing and ship the software.

Page 6: software testing

Test case is a set of test data and test programs (test scripts) and their expected results. A test case validates one or more system requirements and generates a pass or fail.

Test case

Page 7: software testing

Test suite is a collection of test scenarios and / or test cases that are related or that may cooperate with each other. In other words Test suit contains set of test cases ,which are both related or unrelated

Test suite

Page 8: software testing

Test scenario is set of test cases that ensure that the business process flows are tested from end to end . They may be independent tests or a series of tests that follow each other , each dependent on the output of the previous one .

The terms “ test scenario “ and test case” are often synonymously.

Test Scenario

Page 9: software testing

Testing Principles

• All tests should be traceable to customer requirements.

• Tests should be planned long before testing begins.

• Testing should begin “in the small” and progress toward testing “in the large.”

• Exhaustive testing is not possible.

• To be most effective, testing should be conducted by an independent third party.

Page 10: software testing

Manual Testing

Test case generation with using Pen and paper

Automation Testing

WINRUNNER QTP

LOAD RUNNER SELINIUM

etc.

Page 11: software testing

Test Case ID

Description/ function

Test Prerequisite

I/O Action

Expected Values

Test Case No. Test Case purpose

Procedure Expected Result

Actual result

ID which can be referred to in other documents like ‘Traceability Matrix’, Root Cause Analysis of Defects etc.

What to test How to test What should happen

What actually happened?This column can be omitted when Defect Recording Tool is used.

Test Case generation Format-1

Test Case generation Format-1

Page 12: software testing

WHITE-BOX TESTING

White-box testing, sometimes called glass-box testing, is a test case design method that uses the control structure of the procedural design to derive test cases.

White box testing is based on knowledge of the internal logic of an application’s code. Tests are based on coverage of code statements, branches, paths and conditions.

Page 13: software testing

WHITE-BOX TESTINGUsing white-box testing methods, the software engineer can derive test cases that

(1) Guarantee that all independent paths within a module have been exercised at least once,

(2) exercise all logical decisions on their true and false sides,

(3) execute all loops at their boundaries and within their operational bounds, and

(4) exercise internal data structures to ensure their validity.

Page 14: software testing

Boundary value analysis

It is a test case design technique that complements equivalence partitioning.

Rather than selecting any element of an equivalence class, BVA leads to the selection of test cases at the "edges" of the class.

Rather than focusing only on input conditions, BVA derives test cases from the output domain as well

Page 15: software testing

Guidelines for BVA are similar in many respects to those provided for equivalence partitioning:

1. If an input condition specifies a range bounded by values a and b, test cases should be designed with values a and b and just above and just below a and b.

2. If an input condition specifies a number of values, test cases should be developed that exercise the minimum and maximum numbers. Values just above and below minimum and maximum are also tested.

Page 16: software testing

3. Apply guidelines 1 and 2 to output conditions. For example, assume that a temperature vs. pressure table is required as output from an engineering analysis program. Test cases should be designed to create an output report that produces the maximum (and minimum) allowable number of table entries.

4. If internal program data structures have prescribed boundaries (e.g., an array has a defined limit of 100 entries), be certain to design a test case to exercise the data structure at its boundary.

Page 17: software testing

Testing Strategies

Page 18: software testing

Software Testing Steps

Page 19: software testing

Unit testing

Unit testing focuses verification effort on the smallest unit of software design—the software component or module.

Using the component-level design description as a guide, important control paths are tested to uncover errors within the boundary of the module.

Page 20: software testing

Unit testing

The relative complexity of tests and uncovered errors is limited by the constrained scope established for unit testing.

The unit test is white-box oriented, and the step can be conducted in parallel for multiple components.

Page 21: software testing

Unit testing

Unit testing is the first level of dynamic testing and is first the responsibility of developers and then that of the test engineer .

Unit testing is performed after the expected test results are met or differences are explainable/acceptable.

Page 22: software testing
Page 23: software testing

User Acceptance Testing

User Acceptance testing occurs just before the software is released to the customer.

The end-users along with the developers perform the User Acceptance Testing with a certain set of test cases and typical scenarios.

Page 24: software testing

When custom software is built for one customer, a series of acceptance tests are conducted to enable the customer to validate all requirements.

Conducted by the enduser rather than software engineers, an acceptance test can range from an informal "test drive" to a planned and systematically executed series of tests.

In fact, acceptance testing can be conducted over a period of weeks or months, thereby uncovering cumulative errors that might degrade the system over time.

Page 25: software testing

Integration Testing

Integration testing is a systematic technique for constructing the program structure while at the same time conducting tests to uncover errors associated with interfacing.

The objective is to take unit tested components and build a program structure that has been dictated by design.

Usually, the following methods of Integration testing are followed:

1. Top-down Integration approach.2. Bottom-up Integration approach.

Page 26: software testing

Top-Down IntegrationTop-down integration testing is an incremental approach to construction of program structure. Modules are integrated by moving downward through the control hierarchy, beginning with the main control module.

Modules subordinate to the main control module are incorporated into the structure in either a depth-first or breadth-first manner.

Page 27: software testing

The Integration process is performed in a series of five steps:

1.The main control module is used as a test driver and stubs are substituted for all components directly subordinate to the main control module.

2.Depending on the integration approach selected subordinate stubs are replaced one at a time with actual components.

Page 28: software testing

3.Tests are conducted as each component is integrated.

4.On completion of each set of tests, stub is replaced with the real component.

5.Regression testing may be conducted to ensure that new errors have not been introduced.

Page 29: software testing

Bottom-Up Integration

Bottom-up integration testing begins construction and testing with atomic modules (i.e. components at the lowest levels in the program structure).

Because components are integrated from the bottom up, processing required for components subordinate to a given level is always available and the need for stubs is eliminated.

Page 30: software testing

A Bottom-up integration strategy may be implemented with the following steps

1. Low level components are combined into clusters that perform a specific software sub function.

2. A driver is written to coordinate test case input and output.

3. The cluster is tested.

Page 31: software testing

Performance testing

It is designed to test the run-time performance of software within the context of an integrated system.

Performance testing occurs throughout all steps in the testing process. Even at the unit level, the performance of an individual module may be assessed as white-box tests are conducted.

However, it is not until all system elements are fully integrated that the true performance of a system can be ascertained.

Page 32: software testing

The Designing Process

Page 33: software testing

Performance tests are often coupled with stress testing and usually require both hardware and software instrumentation. That is, it is often necessary to measure resource utilization (e.g., processor cycles) in an exacting fashion. External instrumentation can monitor execution intervals, log events (e.g., interrupts) as they occur, and sample machine states on a regular basis. By incrementing a system, the tester can uncover situations that lead to degradation and possible system failure.

Page 34: software testing

Each time a new module is added as part of integration testing, the software changes.New data flow paths are established, new I/O may occur, and new control logic is invoked.

These changes may cause problems with functions that previously worked flawlessly.

In the context of an integration test strategy, regression testing is the re-execution of some subset of tests that have already been conducted to ensure that changes have not propagated unintended side effects.

Regression Testing

Page 35: software testing

Regression testing is the activity that helps to ensure that changes (due to testing or for other reasons) do not introduce unintended behavior or additional errors.

Regression testing may be conducted manually, by re-executing a subset of all test cases or using automated capture/playback tools.

Capture/playback tools enable the software engineer to capture test cases and results for subsequent playback and comparison

Page 36: software testing

The regression test suite (the subset of tests to be executed) contains three differentclasses of test cases:

• A representative sample of tests that will exercise all software functions.

• Additional tests that focus on software functions that are likely to be affected by the change.

• Tests that focus on the software components that have been changed.

Page 37: software testing

As integration testing proceeds, the number of regression tests can grow quite large.

Therefore, the regression test suite should be designed to include only those tests that address one or more classes of errors in each of the major program functions.

It is impractical and inefficient to re-execute every test for every program function once a change has occurred.

Page 38: software testing

Smoke testing is an integration testing approach that is commonly used when “shrink wrapped” software products are being developed.

It is designed as a pacing mechanism for time-critical projects, allowing the software team to assess its project on a frequent basis.

In essence, the smoke testing approach encompasses the following activities:

Page 39: software testing

1. Software components that have been translated into code are integrated into a “build.” A build includes all data files, libraries, reusable modules, and engineered components that are required to implement one or more productfunctions.

2. A series of tests is designed to expose errors that will keep the build from properly performing its function. The intent should be to uncover “show stopper” errors that have the highest likelihood of throwing the software project behind schedule.

Page 40: software testing

3. The build is integrated with other builds and the entire product (in its current form) is smoke tested daily. The integration approach may be top down or bottom up.

Page 41: software testing

The smoke test should exercise the entire system from end to end.

It does not have to be exhaustive, but it should be capable of exposing major problems.

The smoke test should be thorough enough that if the build passes, you can assume that it is stable enough to be tested more thoroughly.

Smoke testing

Page 42: software testing

Black-box testing, also called behavioral testing, focuses on the functional requirements of the software.

That is, black-box testing enables the software engineer to derive sets of input conditions that will fully exercise all functional requirements for a program. Black-box testing is not an alternative to white-box techniques. Rather, it is a complementary approach that is likely to uncover a different class of errors than white-box methods.

Black-box testing

Page 43: software testing

Black-box testing attempts to find errors in the following categories:

(1)Incorrect or missing functions,

(2)interface errors,

(3)errors in data structures or external data base access,

(4) behavior or performance errors, and

(5) initialization and termination errors.

Page 44: software testing

Black-box testing purposely disregards control structure, attention is focused on the information domain. Tests are designed to answer the following questions:

• How is functional validity tested?

• How is system behavior and performance tested?

• What classes of input will make good test cases?

Page 45: software testing

• How are the boundaries of a data class isolated?

• What data rates and data volume can the system tolerate?

• What effect will specific combinations of data have on system operation?

Page 46: software testing

Black box is a test design method. Black box testing treats the system as a "black-box", so it doesn't explicitly use Knowledge of the internal structure.

Or in other words the Test engineer need not know the internal working of the “Black box”.It focuses on the functionality part of the module.

Page 47: software testing

Some people like to call black box testing as behavioral, functional, opaque-box, and closed-box.

While the term black box is most popularly use, many people prefer the terms "behavioral" and "structural" for black box and white box respectively.

Behavioral test design is slightly different from black-box test design because the use of internal knowledge isn't strictly forbidden, but it's still discouraged.

Page 48: software testing

Advantages of Black Box Testing• Tester can be non-technical.

• This testing is most likely to find those bugs as the user would find.

• Testing helps to identify the vagueness and contradiction in functional specifications. • Test cases can be designed as soon as the functional specifications are complete

Page 49: software testing

Disadvantages of Black Box Testing• Chances of having repetition of tests that are already done by programmer.

• The test inputs needs to be from large sample space.

• It is difficult to identify all possible inputs in limited testing time. So writing test cases is slow and difficult Chances of having unidentified paths during this testing

Page 50: software testing

Black-Box Testing

• Graph-Based Testing Methods• Equivalence Partitioning• Boundary Value Analysis• Comparison Testing• Orthogonal Array Testing

Page 51: software testing

•Basic Path Testing•Flow Graph Notation•Cyclomatic Complexity•Graph Matrices•Control Structure Testing

Condition TestingData Flow Testing

•Loop TestingSimple Loops Nested LoopsConcatenated LoopsUnstructured Loops

•Data Structure

White Box testing

Page 52: software testing

[ Functional Testing {User Interface Testing,

Validation testing, functionality testing}] [ Integration

{Intra-Modular Testing, Application and System Testing,

Integration Testing} ],

[ Performance Testing {Load Testing, Stress Testing, Volume Testing, Stability Testing}

],[ System Testing

{Usability Testing,Accessibility Testing,

Compatibility Testing, Portability Testing, Localization Testing, Security Testing(Web, Window)}

]

Structure of testing approach

Page 53: software testing

Alpha TestingA software prototype stage when the software is first available for run.

Here the software has the core functionalities in it but complete functionality is not aimed at. It would be able to accept inputs and give outputs. Usually the most used functionalities (parts of code) are developed more. The test is conducted at the developer’s site only.

Page 54: software testing

Aim

•is to identify any serious errors

•to judge if the indented functionalities are implemented

•to provide to the customer the feel of the software

Page 55: software testing

–Client uses the software at the developer’s environment.

–Software used in a controlled setting, with the developer always ready to fix bugs.

Page 56: software testing

Alpha testing is simulated or actual operational testing by potential users/customers or an independent test team at the developers’ site.

Alpha testing is often employed for off-the-shelf software as a form of internal acceptance testing, before the software goes to beta testing.

Alpha Tests are conducted at the developer’s site by some potential customers. These tests are conducted in a controlled environment.

Alpha testing may be started when formal testing process is near completion.

Page 57: software testing

Beta Testing

The Beta testing is conducted at one or more customer sites by the end-user of the software.

The beta test is a live application of the software in an environment that cannot be controlled by the developer.

Page 58: software testing

Beta Testing Objectives

Evaluate software technical content

Evaluate software ease of use

Evaluate user documentation draft

Identify errors

Report errors/findings

Page 59: software testing

– Conducted at client’s environment (developer is not present)

– Software gets a realistic workout in target environment

Beta Testing

Page 60: software testing

• Beta testing comes after alpha testing. Versions of the software, known as beta versions, are released to a limited audience outside of the company.

• The software is released to groups of people so that further testing can ensure the product has few faults or bugs.

• Sometimes, beta versions are made available to the open public to increase the feedback field to a maximal number of future users.

Page 61: software testing

• Beta Tests are conducted by the customers / end users at their sites.

• Unlike alpha testing, developer is not present here.

• Beta testing is conducted in a real environment that cannot be controlled by the developer