Unit Level Software Testing:

  • Published on
    24-May-2015

  • View
    551

  • Download
    2

Transcript

  • 1. Unit Level Software Testing:Extra Effort Upfront Saves Time and Boosts Safety by David S. BernazzaniIntertech Engineering Associates, Inc.as published in Todays Medical Developments,March, 2007 A unit test strategy thought about during the be developed to verify the accuracy of the software implementation. The idea is to test that small sections development phase will assist the developersof code (typical functions) work as intended and are in structuring its testability, as well as make reasonably robust. Once these individual code blocks the code more reliable during the remaining are tested, larger groups of those building blocks will phases of testing.hold up better during integration. The execution of a unit test should be as fully automated as possible and should be given strict pass/fail criteriaSystem-level, or black-box, testing verifies based on the intended design. Unit test configurationsAthat software correctly implements thesystem-level requirements and specifications. It requires no knowledge of the software design or often allow batching or scripting of tests to be run. TheCOMPANY PROFILE structural implementation. In contrast, unit-level testing is based on detailed knowledge of the architecturalIntertech Engineering Associates, Inc. and logical design, data structures, performance and timing specifications, and interface requirements.Address:100LowderBrookAvenue The result is software that does what it is supposed to Suite2500 Westwood,MA02090 do and that is far less likely to fail, to require costly rework, or to result in harm to patients. www.inea.com-(781)801-1100 Industry: (Electro)MedicalDevices Regulators recognize this. The FDA, in its General Services:Assessments Principles of Software Validation guidance document,Training recommends that device software should be challenged Consulting Hands-onEngineering with test cases based on its internal structure, and with test cases based on its external specifications. Skills: ProductDesign RiskManagement RequirementsEngineering What Is Unit Testing? ElectronicsDevelopment White-box testing is based on internal structure. UnitSoftwareDevelopment SoftwareVerificationandValidation testing is a form of whitebox testing that is done at orProduction/QualitySystemSoftwareValidation near the code level to ensure that the implementation matches the intended design. The design must be documented in sufficient detail so that tests can

2. scripts allows automated loading of the units to betesting tries to encompass too many functions at once.tested, feeding of test inputs, capture of test outputs, Tools, Techniques and Methodologiesand logging of pass/fail results. Unit tests should be written to verify that the code isThis allows the unit tests to be run quickly and the a thorough and accurate implementation of the designresults to be tabulated with great efficiency. There that is described in the design documentation. Theshould be little left open to interpretation by the tester. assumption that the design elements of the documentationUnit testing that requires tester interaction (such as have been traced back to the requirements is implicit.pressing buttons or stepping through the code with This assures that the design being tested is actually onea debugger) should only be used as a last resort, but that the user and other product stakeholders need. Themay be necessary for some areas (e.g. critical startup core algorithms, computations and path coverage shouldroutines that may be coded in assembly). A high level be tested for compliance with the design documentationof automation allows the developer to exercise the as much as possible.code after a build, but prior to release, for integrationor systems-level testing. This would not be practicalThe unit test strategy should be planned as part ofif the unit testing was by manual means. Frequentlythe development phase. Creating code that containsre-running the tests provides some confidence that a framework for unit testing is easier than trying tothe code was not broken in some major way with shoehorn unit testing in later. Finding errors during unitthe most recent changes. Stopping errors at this early testing is also more cost effective to the project, ratherrelease stage can save time as the code goes through than waiting for problems to be found in systems tests,the integration process. which take longer to develop, debug, run and process. The unit test framework is usually straightforward andWhat Is a Unit? requires the ability to isolate code, drive testing and reportFrom a regulatory perspective, the device manufacturer results. The framework code handles communicatingcan define a unit to be any grouping of source code. with the PC to download test input parameters and toHowever, the definition of a unit for unit testing upload test results. Output is often captured to a filegenerally should be done at the smallest, most practical for rapid analysis. An expected-results file is criticalcode level.for objectively determining if the unit has behaved asA unit can be defined as a single function, group of expected.related functions, or an entire file of functions to verifyThe results should include the output of the test invocation,the code is in relative isolation. The most traditionalas well as a log of each sub-unit that was invoked. Forapproach is to define a unit to be at the funtion levelobject-oriented systems, it is possible to test one objectensuring that each function behaves properly and the by using mock objects to intelligently fake the variousdesign was implemented correctly. In cases where there interfaces in the system, leaving the tested object free toare groups of related functions, testing can be done onrun under controlled circumstances. Sets of mock-objectthe set of functions and related data. For classes, oneinterfaces can be interchanged with real objects as partpossible decision is to apply unit tests to every public of integration testing.method, since the private methods will inherently be By controlling the inputs and checking the resultingcovered as part of the object-oriented hierarchy. By sequence of function calls and outputs, a unit can betesting at the higher levels and stubbing out fewer verified to work under a wide range of conditions. Bothinternal functions, code can easily be refactored by the in-range and out-of-range inputs should be tried, thisdeveloper, without wholesale changes to the unit tests includes minimum values, maximum values, boundarythat cover them. Care should be given not to test at too values and any corner cases that may occur as a resulthigh a level, it is possible to lose some control if the of errant software. Programmers often assume that theUnit-Level Software Testing: Extra Effort Upfront Saves Time and Boosts Safety pageas published in: Todays Medical Developments -March, 007Copyright 007 - Intertech Engineering Associates, Inc. 3. calling functions will never pass in incorrect parameters. handling of unexpected failures that crop up in the However, software systems are complex, and ensuringsoftware. Resolution of these findings results in that a function is robust and defensive often helps when software in which unexpected failures are properly other, more complicated areas of software cause invalidtrapped and corrected before the device fails and harms values to be propagated within the system. the patient. Several tool suites (JUnit, VBUnit, CPPUnit, Cantata++)Consequently, unit tests generally make individual can be used as the test framework. It is even possible units more defensive and less reliant on checks in other to create a custom framework for testing as part of code areas of software. If bugs in other areas of software development. The unit being tested must be compiledare introduced during later development, this defensive using the same compiler and toolset as the actual productposture will make an excellent safeguard. This tends to software.aid in the debugging process as well, since robust code Additionally, the testing should run on the actual hardwarewill be more likely trap with a meaningful error rtahter and processor to validate the compiled code. If leavingthan, say, hung code followed by a watchdog reset. unit test hooks in place is a problem, or if code spaceUnit testing, however, will generally not catch is tight, compilation switches can be used to switch the integration errors. The focus here is on the smaller tests off for actual releases. building blocks in the system to ensure that each doesits job properly. How these smaller pieces interact How Much Should Be Tested? is not likely to be found with unit testing, which is Generally, 80% of the errors are found in 20% of the why integration testing and systems-level testing is so code. Spend the most time on functions containingcritical to the entire validation process. key algorithms, safety, are complex or are likely to be defective. How do you know what is likly to beSummary defective? Large functions, complex functions, codeUnit testing is best thought about during development, with multiple branches, complex calculations or deep not afterwards. Creating a unit-test strategy will help layers of nesting are good targets. Functions written by the developers structure the code to be more easily less experienced or less skilled engineers should also betestable. This leads to more robust code and code considered for more focused testing. Straight-line codethat is generally less tightly coupled (i.e. unit more is unlikely to turn up much in the way of problems and independent and easier to unit test) than code written is not likely to be broken by changes to the code. Spend without unit testing in mind. Unit testing should be the most time computations, looping constructs and areas automated as much as possible so that regression is as with data structure access.simple as building and running the code. Unit testing typically finds problems with corner cases, This will save time in later phases of the development lack of input checking and computational problems, effort as the software iteration process occurs. For flaws that might not be easily detected at a systems level.best results, an independent person should review the Both expected and unexpected inputs should be tried, test and update for proper coverage. The FDA wants focusing especially on the boundary conditions that areindependent review and testing of the device software most likely to occur in the case of errant software. Coreas much as possible. Finally, dont be leery of the algorithms that drive the software are easily stressed unit-test process. Proper unit testing done early will when controlling the inputs as part of unit testing. make the code far more reliable as the software entersthe remaining phases of testing. A little more time up Unit Testing and Defensive Programming front will pay dividends down the road. Unit testing helps ensure that code is robust. It indirectly strengthens segments of code by exposing improperUnit-Level Software Testing: Extra Effort upfront Saves Time and Boosts Safetypageas published in: Todays Medical Developments - March, 007Copyright 007 - Intertech Engineering Associates, Inc. 4. ABOUT THE AUTHOR: David Vogel is the founder and presidentLeverage INTERTECHs expertise to:of Intertech Engineering Associates, Inc. Dr. Vogel was a participant in a jointAAMI/Reduce Project Risk FDA workgroup to develop a standardShorten Time to Market for Critical Device Software ValidationCut Development and Test Cost which was subsequently included in the IEC 62304 Software Lifecycle Assure Quality Products Standard. He was also a participant on the joint AAMI/FDA workgroup to develop a TechnicalInformation Report (TIR) for Medical Device SoftwareRisk Management. Currently, Dr. Vogel is a memberABOUT INTERTECH:of the AAMI/FDA workgroup developing a TIR onQuality System Software Validation.A frequent lecturer for workshops and seminars onIntertech Engineering Associates has beentopics related to medical device development andhelping medical device manufacturers bring theirvalidation, Dr. Vogel also is the author of numerousproducts to market since 1982. Through a distinctpublications and holds several patents. top-down service model, Intertech offers high-Dr. Vogel received a bachelors degree in electricallevel consulting and hands-on engineering. Byengineering from Massachusetts Institute of Technology. balancing technical expertise and practical businessHe earned a masters degree in biomedical engineering, aexperience, we support clients through all phases ofmasters degree in electrical and computer engineering, product development. While we do make your joband a doctorate in biomedical engineering from theeasier, Intertech exists not to replace but to partnerUniversity of Michigan. with clients to help balance the concerns of quality,time and cost.With considerable experience in FDA regulatory Intertech Service Offerings:compliance, our time-tested development process Risk Analysis and Management can anticipate and solve problems inexpensively Software Design and Developmenton the planning board rather than through costly Electronic Design and Developmentsolutions later in the development, test, or post- Requirements Development and Managementdeployment phases. By using deliberate processes, Documentation and Traceability Intertech ensures an improvement in quality and Verification and Validationcan build client expertise. Evaluations, Reviews, Inspections Planning Call us today for more information or a free Project Managementconsultation at 781.801.1100 Compliance Consulting and Training Manufacturing and Quality System Software Validation INTERTECH EngineeringAssociates,Inc. 100LowderBrookDriveSuite2500Westwood,MA02090USAwww.inea.com-info@inea.com-Tel:(781)801-1100-Fax:(781)801-1108

Recommended

View more >