Basic Concept of Testing and Quality
Unit testingUNIT - 21Prepared By: Ravi J Khimani, CSE Department, SLTIET, Rajkot
conceptsFirst level of testingRefers to testing program units in isolationCommonly understood units are function, procedures or methodsOr we can say class in OOP is a unitA program unit is assumed to implement a well defined function providing a certain level of abstraction to the implementation of higher level functions. 2
conceptsIt is only natural to test the unit before it is integrated with other units. Thus, a program unit is tested in isolation, that is, in a stand-alone mannerDue to two reasonsErrors found during testing can be attributed to a specific unit so that it can be easily fixedDuring unit testing it is desirable to verify that each distinct execution of a program unit produces the expected result
conceptsIn terms of code details, a distinct execution refers to a distinct path in the unit. Ideally, all possibleor as much as possibledistinct executions are to be considered during unit testing. This requires careful selection of input data for each distinct execution4
conceptsUnit testing has a limited scope.Needs to verify, code works perfectly or not?a programmer needs to test a unit as follows:Execute every line of code. This is desirable because the programmer needs to know what happens when a line of code is executed. In the absence of such basic observations, surprises at a later stage can be expensive.Execute every predicate in the unit to evaluate them to true and false separately.Observe that the unit performs its intended function and ensure that it contains no known errors.5
conceptsIn spite of the above tests, there is no guarantee that a satisfactorily tested unit is functionally correct from a system wide perspective.means that some errors in a program unit can only be found later, when the unit is integrated with other units in the integration testing and system testing phases.It serves no purpose to integrate an erroneous unit with other units for the following reasons: (i) many of the subsequent tests will be a waste of resources (ii) nding the root causes of failures in an integrated system is more resource consuming.
conceptsUnit testing is performed by the programmer who writes the program unit because the programmer is intimately familiar with the internal details of the unit.The objective for the programmer is to be satised that the unit works as expected with no errorsUnit testing is conducted in two complementary phases:Static unit testingDynamic unit testing
conceptsStatic Unit Testing, programmer does not execute Unit, instead of code is examined all possible behavior.Possible issues can be solved by reviewing code in SUT.Dynamic Unit Testing, is totally execution based. After rounds of code reviewing, execution testing is conducted to minimize possible runtime errors.8
Static unit testingIt is undergo of philosophical belief of inspection and correction.Idea behind review is to find possible defects such that they can not be converted to errors further.In SUT, code review applied by two methods, INSPECTION and WALKTHROUGH.
Static unit testingInspection: It is a step-by-step peer group review of a work product, with each step checked against predetermined criteria.Walkthrough: It is a review where the author leads the team through a manual or simulated execution of the product using predefined scenarios.So, any of the method is a systematic approach to examine source code in detail.10
Static unit testingGoal: to assess the quality of product, not to assess the quality of process to develop the product.Its time consuming processNot perfect up to the level. The key to the success code review is Divide & Conquer.Means having an examiner inspect small parts of the unit in isolation, while making sure of the following:nothing is overlookedthe correctness of all examined parts of the module implies the correctness of the whole module. 11
Static unit testingMust assure the each step should simple enough.The objective of code review is to review the code, not to evaluate the author of the code. Therefore, code review must be planned and managed in a professional manner. There is a need for mutual respect, openness, trust, and sharing of expertise in the group. code review consists of six steps : readiness, preparation, examination, rework, validation, and exit. the process produces two types of documents, a change request (CR) and a report. 12
Static unit testing13
Step 1: Readiness14The author of the unit ensures that the unit under test is ready for review.A unit is said to be ready if it satises the following criteria:Completeness: All the code relating to the unit to be reviewed must be available.Minimal Functionality: The code must have been tested to some extent to make sure that it performs its basic functionalities.Readability: It is essential that the code is highly readable.Complexity: There is no need to schedule a group meeting to review straightforward code which can be easily reviewed by the programmerRequirements and Design Documents: Relevant documents must be available to the reviewer.
Step 1: Readiness15People involved in review process are informed for group meeting before two days.Review Group involves a number of peoples with their roles, which are as follows,Moderator: A review meeting is chaired by the moderator. The moderator is a trained individual who guides the pace of the review process.Author: This is the person who has written the code to be reviewed.Presenter: A presenter is someone other than the author of the code. It is the presenter who presents the authors code in the review meeting for the following reasons:
Step 1: Readiness16an additional software developer will understand the work within the software organization; if the original programmer leaves the company with a short notice, at least one other programmer in the company knows what is being done; The original programmer will have a good feeling about his or her work, if someone else appreciates their work. Record-keeper: The record keeper documents the problems found during the review process and the follow-up actions suggested. Reviewers: These are experts in the subject area of the code under review. Observers: These are people who want to learn about the code under review.
Step 2: Preparation17Before the meeting, each reviewer carefully reviews the work package. Each reviewer develops the following:List of Questions: A reviewer prepares a list of questions to be asked, if needed, of the author to clarify issues arising from his or her reading.Potential CR: A reviewer may make a formal request to make a change. These are called change requests rather than defect reports.These reports are not included in defect statistics related to the product.Suggested Improvement Opportunities: The reviewers may suggest how to fix the problems, if there are any
Step 3: examination18Includes following activities,The author makes a presentation of the procedural logic used in the code, the paths denoting major computations, and the dependency .The presenter reads the code line by line. The reviewers may raise questions if the code is seen to have defects.Problems are not resolved in the meeting.The record-keeper documents the change requests and the suggestions for fixing the problems
Step 3: examination19Includes following activities,The moderator ensures that the meeting remains focused on the review process. At the end of the meeting, a decision is taken regarding whether or not to call another meeting to further review the code.If the review process leads to extensive rework of the code or critical issues are identied in the process, then another meeting is generally convened..
Step 3: examination Change request20A CR includes the following details:Give a brief description of the issue or action item.Assign a priority level (major or minor) to a CR.Assign a person to follow up the issue. Since a CR documents a potential problem, there is a need for interaction between the authorSet a deadline for addressing a CR
Step 3: examination21
Step 4: RE-work22At the end of the meeting, the record keeper produces a summary of the meeting that includes the following information.A list of all the CRs, the dates by which those will be fixed, and the names of the persons responsible for validating the CRsA list of improvement opportunitiesThe minutes of the meeting (optional)A copy of the report is distributed to all the members of the review group.After the meeting, the author works on the CRs to x the problems. The author documents the improvements made to the code in the CRs.
Step 5: validation23The CRs are independently validated by the moderator or another person designated for this purpose.The validation process involves checking the modified code as documented in the CRs and ensuring that the suggested improvements have been implemented correctly.The revised and final version of the outcome of the review meeting is distributed to all the group members.
Step 6: exit24It is said to be complete if all of the following actions have been taken:Every line of code in the unit has been inspected.If too many defects are found in a module, the module is once again reviewed after corrections are applied by the author. As a rule of thumb, if more than 5% of the total lines of code are thought to be contentious, then a second review is scheduled.The author and the reviewers reach a consensus that when corrections have been applied the code will be potentially free of defects.All the CRs are documented and validated by the moderator or someone else. The authors follow-up actions are documented.A summary report of the meeting including the CRs is distributed to all the members of the review group.
Code review matrices25The effectiveness of static testing is limited by the ability of a reviewer to find defects in code by visual means.to observe its behaviours in response to a variety of inputs.It is important to collect measurement of data relevant to a review process.The Review process can be evaluated, made visible to the upper management as a testing strategy, and improved to be more effective.
Code review matrices26Collecting metrics during code review facilitates estimation of review time and resources for future projectsThe following metrics can be collected from a code review:Number of lines of code (LOC) reviewed per hourNumber of CRs generated per thousand lines of code (KLOC)Number of CRs generated per hourTotal number of CRs generated per projectTotal number of hours spent on code review per project
Defect prevention what is?27It is in the best interest of the programmers in particular and the company in general to reduce the number of CRs generated during code review.Because CRs - indications of potential problems - must be resolved.Addressing CRs - spending more resources and potentially delay in project.Therefore, it is essential to adopt the concept of defect prevention.
Defect prevention - guidelines28These guidelines focus on incorporating suitable mechanisms into the code:Build internal diagnostic tools, also known as instrumentation code, into the unitsProvides information about the internal states of the unitsallow programmers to realize built-in tracking and tracing mechanismsInstrumentation plays a passive role in dynamic unit testingUse standard controls to detect possible occurrences of error conditions.
Defect prevention - guidelines29Ensure that code exists for all return values, some of which may be invalid.Ensure that counter data fields and buffer overflow and underflow are appropriately handled.Provide error messages and help texts from a common source so that changes in the text do not cause inconsistency.Validate input data, such as the arguments, passed to a function.Use assertions (act of stating something) to detect impossible conditions, undened uses of data, and undesirable program behaviour.
Defect prevention use of assertion30Assertion should be routinely used to perform the following kinds of checks:Ensure that preconditions are satised before beginning to execute a unit. Ensure that the expected post conditions are true while exiting from the unitEnsure that the invariants hold. That is, check invariant states conditions which are expected not to change during the execution of a piece of code.Leave assertions in the code. You may deactivate them in the released version of codeFully document the assertions that appear to be unclear.
Defect prevention31After every major computation, reverse-compute the input(s) from the results in the code itself. Then compare the outcome with the actual inputs for correctness. In systems involving message passing, buffer management is an important internal activity. Develop a timer routine which counts down from a preset time until it either hits zero or is reset. Include a loop counter within each loop. If the loop is ever executed less than the minimum possible number of times or more than the maximum possible number of times, then invoke an exception handler routine.
Dynamic unit testing32Execution-based unit testing.this execution differs from ordinary execution in the following way:A unit under test is taken out of its actual execution environment.The actual execution environment is emulated by writing more codeThe outcome of such an execution is collected in a variety of ways, such as straightforward observation on a screen, logging on files, and software instrumentation of the code to reveal run time behaviourThe context of a unit test consists of two parts: a caller of the unitall the units called by that unit.
Dynamic unit testing - Structure33The caller unit is known as a test driver.all the emulations of the units called by the unit under test are called stubsThe test driver and the stubs are together called scaffolding
Dynamic unit testing - structure34Test DriverA test driver is a program that invokes the unit under test.input values received from the driver and returns a value to the driver. The driver compares the actual outcome with the expected outcome The test driver functions as the main unit in the execution process. StubA stub is a dummy subprogram that replaces a unit that is called by the unit under test. A stub performs two tasks. First, it shows an evidence that the stub was, in fact, called. Second, the stub returns a precompiled value to the caller so that the unit under test can continue its execution.
Dynamic unit testing - structure35A test driver and the stubs for a unit should be reusable and maintainable.For each unit, there should be one dedicated test driver and several stubs as requiredthe test driver should not depend on the external input data filesAny modification to the driver to accommodate changes in one of the units under test may have side effects in...