Software Software DevelopmentDevelopment

Chapter 4Chapter 4

Software Testing and ToolsSoftware Testing and Tools

TestingTesting

► Software testing involves a series of processes to verify that :Software testing involves a series of processes to verify that : The finished product meets the original specificationThe finished product meets the original specification The software is robustThe software is robust The software is reliableThe software is reliable

► Testing can only find errors in a program, not prove there are Testing can only find errors in a program, not prove there are nonenone

► Testing should be systematicTesting should be systematic► Test data needs to be planned and the test process needs to Test data needs to be planned and the test process needs to

record a test logrecord a test log► The actual test data selected depends on the test strategy The actual test data selected depends on the test strategy

usedused

Test StrategiesTest Strategies

► Recapping testing methods..Recapping testing methods..

Module TestingModule Testing, on all components, on all components Integration TestingIntegration Testing, where the components function as an entity, where the components function as an entity System TestingSystem Testing or or Black Box TestingBlack Box Testing Acceptance TestingAcceptance Testing or or Beta TestingBeta Testing, ensuring that the system is , ensuring that the system is

ready for operational useready for operational use

► Software testing is a complex and expensive issueSoftware testing is a complex and expensive issue

Test Approaches Test Approaches

► There are a number of different testing approaches:There are a number of different testing approaches:

• Testing ‘by scope’ starts at the component level then proceeds through each phase until the final system is tested as a whole

• Testing ‘by purpose’ entails the whole reason for testing. It makes sure the program is correct and runs to the original specification

• Testing ‘by life-cycle’ phase include the final software development test, acceptance testing where the software is sold to the client

By scope By purpose By life-cycle phase

Component testing Correctness testing Requirements testing

Module testing Performance testing Design testing

Integration testing Reliability testing Acceptance testing

System testing Security testing Maintenance testing

ComponentsComponents

► Components are sometimes called units Components are sometimes called units ► They are the building blocks of software applicationsThey are the building blocks of software applications► They are defined as pieces of software such as sub-routines or They are defined as pieces of software such as sub-routines or

procedures that can be compiled and executed on their ownprocedures that can be compiled and executed on their own► A component will be the work of a single programmer and they will also A component will be the work of a single programmer and they will also

test the code at a basic, test the code at a basic, White BoxWhite Box, level, level

Component TestingComponent Testing

► With With White BoxWhite Box testing component code is seen and it is the testing component code is seen and it is the workings of the code that are testedworkings of the code that are tested

► The component can be tested in The component can be tested in Static Static or or Dynamic Dynamic modemode In static mode the component does not require to be executed. A In static mode the component does not require to be executed. A

detailed catalogue of reviews and inspections is compiled for each detailed catalogue of reviews and inspections is compiled for each modulemodule► Inspections contain a precise framework for rigorously checking component Inspections contain a precise framework for rigorously checking component

documentation and codedocumentation and code In dynamic testing the component program is executed using test values In dynamic testing the component program is executed using test values

for normal, extreme and exceptional casesfor normal, extreme and exceptional cases

ModulesModules

► Modules are a collection of dependant components or procedures designed Modules are a collection of dependant components or procedures designed to be part of the main programto be part of the main program

► They are not complete programs in themselves, they can not run They are not complete programs in themselves, they can not run independently independently

► It often happens that a module becomes ready for testing before the main It often happens that a module becomes ready for testing before the main program. It is common practice to write a small program, called a program. It is common practice to write a small program, called a Driver Driver or or Test-Harness Test-Harness to run the module and supply it with test datato run the module and supply it with test data

► Driver programs are commercially availableDriver programs are commercially available

StubsStubs

► A stub is a module that has the appropriate interface but does not contain a A stub is a module that has the appropriate interface but does not contain a lot of codelot of code

► It might contain no more than a line of code to display what the module will It might contain no more than a line of code to display what the module will do when its completedo when its complete

► The system as a whole is laid out using stubs to ensure that the overall The system as a whole is laid out using stubs to ensure that the overall structure is correctstructure is correct

► Stubs are then converted into fully functional modulesStubs are then converted into fully functional modules

Module TestingModule Testing

► Modules are created and tested within their sub-systemsModules are created and tested within their sub-systems► When all modules have been tested the sub-system as a whole is tested then When all modules have been tested the sub-system as a whole is tested then

sub-systems are brought together and the whole program is testedsub-systems are brought together and the whole program is tested► At system level the type of testing is At system level the type of testing is Black Box Testing Black Box Testing ► Black box testing takes the program specification as the sole source of test Black box testing takes the program specification as the sole source of test

casescases► One advantage of this is that developers who test a system can be different One advantage of this is that developers who test a system can be different

from those who create itfrom those who create it

Module Test ProgressionModule Test Progression

► The testing process involves feedbackThe testing process involves feedback► Testing might indicate errors in a module which would need debugged Testing might indicate errors in a module which would need debugged

and tested againand tested again► The usual strategy is to test software twiceThe usual strategy is to test software twice

The module is first tested within the company, this is Alpha Testing

It is followed by Acceptance or Beta Testing

Alpha TestingAlpha Testing

► Alpha testing can be regarded as ‘Does the software work?’Alpha testing can be regarded as ‘Does the software work?’► This is the stage in the development cycle where the software This is the stage in the development cycle where the software

is first able to runis first able to run► This testing takes place on the software developers premisesThis testing takes place on the software developers premises► Alpha testing is done for 2 reasons;Alpha testing is done for 2 reasons;

To reassure the clients the software is workingTo reassure the clients the software is working To find errors that can only be found in operational useTo find errors that can only be found in operational use

► Alpha testing is performed on an early version of the code, Alpha testing is performed on an early version of the code, that might not have all intended functionality that might not have all intended functionality

► Alpha testing should be conducted with as much Alpha testing should be conducted with as much independence from the development team as possibleindependence from the development team as possible

Beta (Acceptance) TestingBeta (Acceptance) Testing

► This is for the client to say ‘yes we will pay for the software’This is for the client to say ‘yes we will pay for the software’► Beta testing is the highest level of testing in the software development Beta testing is the highest level of testing in the software development

cycle and is done prior to commercial releasecycle and is done prior to commercial release► It confirms that the program is as near to correct as possibleIt confirms that the program is as near to correct as possible► The software is tested by independent users who log and report back to The software is tested by independent users who log and report back to

the development team, this process might be iterative if errors have to the development team, this process might be iterative if errors have to be correctedbe corrected

► The functionality delivered in beta test releases might not be the final The functionality delivered in beta test releases might not be the final productproduct

DebuggingDebugging► A bug is a fault in a program that causes it to function abnormallyA bug is a fault in a program that causes it to function abnormally► Debugging is the process of locating and fixing errorsDebugging is the process of locating and fixing errors► Programs that help to do this are called debuggersPrograms that help to do this are called debuggers► Most software development tools offer debugging toolsMost software development tools offer debugging tools► Using a compiler the first task of the debugging tool is to link the object code Using a compiler the first task of the debugging tool is to link the object code

to source code to locate the errorto source code to locate the error► The part of the compiler responsible for this is the The part of the compiler responsible for this is the Link-LoaderLink-Loader

Debugging ToolsDebugging Tools

► With the debugging features activated a program will run With the debugging features activated a program will run slower, so it is only used when necessaryslower, so it is only used when necessary

► Debugging tools usually take the form of:Debugging tools usually take the form of: Dry RunsDry Runs Trace ToolsTrace Tools Watch StatementsWatch Statements Break PointsBreak Points

Dry RunDry Run

► Bugs can be difficult to locate and correct, the temptation is to tweak the Bugs can be difficult to locate and correct, the temptation is to tweak the code until it runs correctly, however this can lead to more errorscode until it runs correctly, however this can lead to more errors

► If a bug does not become apparent a dry run is conductedIf a bug does not become apparent a dry run is conducted► A dry run is based on the listing of the code that the programmer works A dry run is based on the listing of the code that the programmer works

through manually through manually ► A A Trace TableTrace Table is used to store and display program variables as the logic of is used to store and display program variables as the logic of

the algorithm unfolds, the actual output can then be checked against the the algorithm unfolds, the actual output can then be checked against the expectedexpected

► Dry runs are used for locating logic errors and is only practical for simple Dry runs are used for locating logic errors and is only practical for simple programsprograms

Pupil taskPupil task

Name and describe two methods of locating errors in a puzzle software.

AnswerAnswer

Set breakpoints (1) – used to stop execution of the program at predefined point (1)

Trace tables (1) – used to record the contents of a variable during manual execution (1)

Dry run (1) – stepping through each line of code manually recording variables (1)

Trace ToolsTrace Tools► Some programming languages have TRACE facilities as a debugging featureSome programming languages have TRACE facilities as a debugging feature► Tracing gives access to otherwise invisible information about a program Tracing gives access to otherwise invisible information about a program

execution allowing the programmer to step through the program line by line execution allowing the programmer to step through the program line by line and stop at points to examine variable contentand stop at points to examine variable content

► More enhanced tools allow investigation of memory locations and the More enhanced tools allow investigation of memory locations and the content of the stackcontent of the stack

► Programs that use large numbers of procedures use the stack to store all Programs that use large numbers of procedures use the stack to store all their procedure callstheir procedure calls

► Using trace a program will run slowerUsing trace a program will run slower

Program WatchProgram Watch► A watch takes an identifier and displays its values as the program A watch takes an identifier and displays its values as the program

progressesprogresses► The programmer steps through the code one line at a time and the The programmer steps through the code one line at a time and the

variable being traced is display on a watch screenvariable being traced is display on a watch screen► Many programmers prefer to put in watches in the form of output Many programmers prefer to put in watches in the form of output

statements that cause the value of the variable to be displayed at statements that cause the value of the variable to be displayed at points in a program where a bug exists points in a program where a bug exists

► A draw back of program watching is the number of lines of code A draw back of program watching is the number of lines of code increase as does programming timeincrease as does programming time

BreakpointsBreakpoints

► A break point is a marker set within the code of a program to halt program A break point is a marker set within the code of a program to halt program execution at a predefined spotexecution at a predefined spot

► The statement responsible will be highlighted and can be inspected while The statement responsible will be highlighted and can be inspected while the program is temporarily interruptedthe program is temporarily interrupted

► The program then continues to completion or until another breakpoint is The program then continues to completion or until another breakpoint is foundfound

► Breakpoints are often used with tracesBreakpoints are often used with traces

Pupil taskPupil task

The project team is ready to beta test the software.

(i) Give two reasons why beta testing is necessary.

(ii) Name the two groups involved at this stage and describe the role of each group.

AnswersAnswersi) • Checks software is reliable on the client’s computer.• Checks client is happy with software and allows increasedreliability of testing.• Allows client an opportunity to check that software meetsrequirements.• Allows programmers to observe the clients using thesoftware determining success of user interface etc.

(ii) • End users − provide feedback• Programming team − maintenance

CASE ToolsCASE Tools

► It refers to the collection of software programs that are designed to It refers to the collection of software programs that are designed to automate the development cycleautomate the development cycle

► The implementation of new systems requires many different complex tasks The implementation of new systems requires many different complex tasks to be organised and completed correctly and efficiently to be organised and completed correctly and efficiently

► Information generated has to be kept in synchronisationInformation generated has to be kept in synchronisation► The use of CASE tools eases the task of coordinating these activates from The use of CASE tools eases the task of coordinating these activates from

analysis to implementationanalysis to implementation

Computer Aided Software Engineering

Computer-Aided Software Engineering (CASE), in the field of Software Engineering is the scientific application of a set of tools and methods to a software system which is meant to result in high-quality, defect-free, and maintainable software products.It also refers to methods for the development of information systems together with automated tools that can be used in the software development process

Types of CASE ToolsTypes of CASE Tools

► Basic categories of CASE tools are available:Basic categories of CASE tools are available: Diagramming tools that represent data models according to system Diagramming tools that represent data models according to system

specifications. specifications. Screen and Report Generators for creating system specifications. Screen and Report Generators for creating system specifications. Data Dictionaries that contain a history of changes made to a system Data Dictionaries that contain a history of changes made to a system Code Generators to be able to generate code from data Diagrams Code Generators to be able to generate code from data Diagrams

themselves. themselves. Documentation generators that make the code more readable.Documentation generators that make the code more readable.

Development of CASEDevelopment of CASE

► Sine the early days of writing software there has been a need for Sine the early days of writing software there has been a need for automated tools to help the software developerautomated tools to help the software developer

► As computer became more powerful ad the software that ran on them As computer became more powerful ad the software that ran on them grew larger and more complex, power tools with increasing functionality grew larger and more complex, power tools with increasing functionality were requiredwere required

Categories of CASE ToolsCategories of CASE Tools

► CASE tools can be divided into two main groups CASE tools can be divided into two main groups Those that deal with the first three parts of the system development life Those that deal with the first three parts of the system development life

cycle (preliminary investigation, analysis, and design) and are referred to cycle (preliminary investigation, analysis, and design) and are referred to as as Front-End CASE ToolsFront-End CASE Tools or or Upper CASE ToolsUpper CASE Tools

Those that deal mainly with the implementation, testing and installation Those that deal mainly with the implementation, testing and installation are referred to as are referred to as Back-End CASE ToolsBack-End CASE Tools or or Lower CASE ToolsLower CASE Tools

Upper CASE ToolsUpper CASE Tools

► These are basically general-purpose analysis and design specification These are basically general-purpose analysis and design specification toolstools

► During the initial stages of system development, analysts have to During the initial stages of system development, analysts have to determine system requirements and analyse this information to design determine system requirements and analyse this information to design the most effective systemthe most effective system

► Computerised CASE tools allow for changes to be made automatically, Computerised CASE tools allow for changes to be made automatically, very quickly and accuratelyvery quickly and accurately

► Data Dictionaries Data Dictionaries are data processing systems products that are are data processing systems products that are specifically designed to hold, maintain and organise informationspecifically designed to hold, maintain and organise information

► They have facilities for producing a variety of reportsThey have facilities for producing a variety of reports► They allow developers to be more productiveThey allow developers to be more productive► The goal of a CASE tool is to refine the analysis and design stages to The goal of a CASE tool is to refine the analysis and design stages to

allow automated production of code, around 75% allow automated production of code, around 75%

Data DictionariesData Dictionaries

► A data dictionary is an automated tool for collecting and A data dictionary is an automated tool for collecting and organising detailed information about system componentsorganising detailed information about system components

► A data dictionaries main facility is to document data A data dictionaries main facility is to document data elements, records, programs, files, users etcelements, records, programs, files, users etc

► They also have the facility to cross-reference all system They also have the facility to cross-reference all system components and contain details of:components and contain details of: Systems environmentSystems environment Audit trailsAudit trails ReportsReports FormsForms FunctionsFunctions ProcessesProcesses

Lower CASE ToolsLower CASE Tools

► These focus on the architecture of the system and its implementation These focus on the architecture of the system and its implementation and maintenanceand maintenance

► These tools are effective in helping with the generation of the program These tools are effective in helping with the generation of the program code and are referred to as code and are referred to as Code GeneratorsCode Generators

► A code generator is a tool that enables automatic generation of code A code generator is a tool that enables automatic generation of code from the analysis and design specificationsfrom the analysis and design specifications

► Generating code this way ensures that all code is produced with identical Generating code this way ensures that all code is produced with identical naming conventionsnaming conventions

Code GeneratorsCode Generators

► Code generators produce a high quality of code that is easy to maintain and Code generators produce a high quality of code that is easy to maintain and is portableis portable

► They are able to interact with Upper CASE tools to aid in the development of They are able to interact with Upper CASE tools to aid in the development of codecode

Document GeneratorsDocument Generators

► A further enhancement to CASE tools is document generatorsA further enhancement to CASE tools is document generators► A document generator is a CASE tool that generates technical documentation A document generator is a CASE tool that generates technical documentation

from source code commentsfrom source code comments► The comments that are produced conform to a standard formatThe comments that are produced conform to a standard format

Object-Oriented CASEObject-Oriented CASE

► CASE tools are well supported in object-oriented CASE tools are well supported in object-oriented programming systems (OOPS)programming systems (OOPS)

► Earlier problems with OOPS focussed on models created for Earlier problems with OOPS focussed on models created for software development systems and their different notationssoftware development systems and their different notations

► This was overcome by the use of This was overcome by the use of UML, Unified Modelling UML, Unified Modelling LanguageLanguage, which is a new standard for producing diagrams , which is a new standard for producing diagrams and chartsand charts

► All current CASE tools have now adopted UML for specifying, All current CASE tools have now adopted UML for specifying, visualising and constructing software systemsvisualising and constructing software systems

UMLUML

► CASE tools offer automatic code generation from the UML CASE tools offer automatic code generation from the UML diagramdiagram

► This tool produces a framework for the code which contains This tool produces a framework for the code which contains objects and classes. An objects and classes. An Executable Prototype Executable Prototype is producedis produced This is an executable source code program obtained directly from This is an executable source code program obtained directly from

UMLUML

► One of the main features in UML is One of the main features in UML is Reverse Engineering Reverse Engineering where existing source code is reversed engineered into a set where existing source code is reversed engineered into a set of UML diagramsof UML diagrams

RUPRUP

► In conjunction with UML developers also employ RUPIn conjunction with UML developers also employ RUP► This is a development process to deliver best practices during each stage This is a development process to deliver best practices during each stage

of a projectof a project► Using RUP risks are lowered during all stagesUsing RUP risks are lowered during all stages

Rational Unified Process

Advantages of CASEAdvantages of CASE► Increase SpeedIncrease Speed

CASE tools provide automation and reduce the time to complete tasksCASE tools provide automation and reduce the time to complete tasks► Increased AccuracyIncreased Accuracy

CASE tools can provide on-going debugging and error checking which is vital for CASE tools can provide on-going debugging and error checking which is vital for early defect removalearly defect removal

► Reduced Costs and MaintenanceReduced Costs and Maintenance Overall system quality improves using CASE. There is also better documentation Overall system quality improves using CASE. There is also better documentation

so the net effort and cost involved is reducedso the net effort and cost involved is reduced► Better DocumentationBetter Documentation

Using CASE tools vast amounts of documentation are produced. An important Using CASE tools vast amounts of documentation are produced. An important aspect is aspect is CASE RepositoryCASE Repository, this is a developers database with information relating , this is a developers database with information relating to the projects stored. It can be regularly updatedto the projects stored. It can be regularly updated

► Better CommunicationBetter Communication Information generated from one tool can be passed to another, allowing for Information generated from one tool can be passed to another, allowing for

important information to be passed efficiently and effectivelyimportant information to be passed efficiently and effectively

Limitations Of CASE ToolsLimitations Of CASE Tools

Although CASE tools are becoming more popular user can beAlthough CASE tools are becoming more popular user can besubject to some limitations:subject to some limitations:

► ChoiceChoice Over 1000 CASE tools exist and choosing one is difficult, a major Over 1000 CASE tools exist and choosing one is difficult, a major

project in itselfproject in itself► CostsCosts

CASE tools are not cheap. Hardware, software, training and CASE tools are not cheap. Hardware, software, training and consulting are all factors in the total cost equationconsulting are all factors in the total cost equation

► TrainingTraining Users need time to learn the technology. A CASE consulting Users need time to learn the technology. A CASE consulting

industry has evolved to support uses of CASE toolsindustry has evolved to support uses of CASE tools

Pupil taskPupil task

Describe two benefits of using Computer-Aided Software Engineering (CASE) tools.

AnswerAnswer

• Computerised generation of graphical designs and information shared across the system would be monitored. • Automated data dictionaries. • Tools to track dependencies.

Do not allow compilers, editors, debuggers or other software development type answers