Next Generation Java Testing. TestNG and Advanced Concepts

  • Published on

  • View

  • Download

Embed Size (px)


Book on Test NG Automation


<p>Next Generation Java Testing</p> <p>This page intentionally left blank</p> <p>Next Generation Java TestingTestNG and Advanced Concepts</p> <p>Cdric Beust Hani Suleiman</p> <p>Upper Saddle River, NJ Boston Indianapolis San Francisco New York Toronto Montreal London Munich Paris Madrid Capetown Sydney Tokyo Singapore Mexico City</p> <p>Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and the publisher was aware of a trademark claim, the designations have been printed with initial capital letters or in all capitals. The authors and publisher have taken care in the preparation of this book, but make no expressed or implied warranty of any kind and assume no responsibility for errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of the use of the information or programs contained herein. The publisher offers excellent discounts on this book when ordered in quantity for bulk purchases or special sales, which may include electronic versions and/or custom covers and content particular to your business, training goals, marketing focus, and branding interests. For more information, please contact: U.S. Corporate and Government Sales (800) 382-3419 For sales outside the United States please contact: International Sales</p> <p>Visit us on the Web: Library of Congress Cataloging-in-Publication Data Beust, Cdric. Next generation Java testing : TestNG and advanced concepts / Cdric Beust, Hani Suleiman. p. cm. Includes bibliographical references and index. ISBN 0-321-50310-4 (pbk. : alk. paper) 1. Java (Computer program language) 2. Computer softwareTesting. I. Suleiman, Hani. II. Title. QA76.73.J3B49 2007 005.13'3dc22 Copyright 2008 Pearson Education, Inc. All rights reserved. Printed in the United States of America. This publication is protected by copyright, and permission must be obtained from the publisher prior to any prohibited reproduction, storage in a retrieval system, or transmission in any form or by any means, electronic, mechanical, photocopying, recording, or likewise. For information regarding permissions, write to: Pearson Education, Inc. Rights and Contracts Department 501 Boylston Street, Suite 900 Boston, MA 02116 Fax: (617) 671-3447 ISBN-13: 978-0-321-50310-7 ISBN-10: 0-321-50310-4 Text printed in the United States on recycled paper at Courier in Stoughton, Massachusetts. First printing, October 2007 2007031939</p> <p>ContentsForeword Preface Acknowledgments About the Authors xiii xv xxi xxiii</p> <p>Chapter 1</p> <p>Getting StartedBeyond JUnit 3Stateful Classes Parameters Base Classes Exceptions Are Not That Exceptional Running Tests Real-World Testing Configuration Methods Dependencies Epiphanies</p> <p>13 3 4 4 4 5 6 6 6 7 7 8 8 9 10 16 17 17 18 20 21 21</p> <p>JUnit 4 Designing for TestabilityObject-Oriented Programming and Encapsulation The Design Patterns Revolution Identifying the Enemy Recommendations</p> <p>TestNGAnnotations Tests, Suites, and Configuration Annotations Groups testng.xml</p> <p>Conclusion</p> <p>Chapter 2</p> <p>Testing Design PatternsTesting for FailuresReporting Errors Runtime and Checked Exceptions</p> <p>2323 24 25</p> <p>v</p> <p>vi</p> <p>Contents</p> <p>Testing Whether Your Code Handles Failures Gracefully When Not to Use expectedExceptions testng-failed.xml</p> <p>Factories@Factory org.testng.ITest</p> <p>Data-Driven TestingParameters and Test Methods Passing Parameters with testng.xml Passing Parameters with @DataProvider Parameters for Data Providers The Method Parameter The ITestContext Parameter Lazy Data Providers Pros and Cons of Both Approaches Supplying the Data Data Provider or Factory? Tying It All Together</p> <p>Asynchronous Testing Testing Multithreaded CodeConcurrent Testing threadPoolSize, invocationCount, and timeOut Concurrent Running Turning on the Parallel Bit</p> <p>Performance TestingAlgorithm Complexity Testing Complexity</p> <p>Mocks and StubsMocks versus Stubs Designing for Mockability Mock Libraries Selecting the Right Strategy Mock Pitfalls</p> <p>Dependent TestingDependent Code Dependent Testing with TestNG Deciding Whether to Depend on Groups or on Methods Dependent Testing and Threads Failures of Configuration Methods</p> <p>Inheritance and Annotation ScopesThe Problem Pitfalls of Inheritance</p> <p>Test GroupsSyntax Groups and Runtime Running Groups Using Groups Effectively</p> <p>27 31 32 34 35 38 39 42 44 47 50 50 52 54 59 60 62 63 67 71 72 75 79 82 83 84 87 90 90 95 96 99 100 103 104 105 106 110 110 113 113 116 119 120 122 125 127</p> <p>Contents</p> <p>vii132 133 134 136 146 147 147 150</p> <p>Code CoverageA Coverage Example Coverage Metrics Coverage Tools Implementation Beware! A Guide to Successful Coverage</p> <p>Conclusion</p> <p>Chapter 3</p> <p>Enterprise TestingA Typical Enterprise ScenarioParticipants Testing Methodology Issues with the Current Approach</p> <p>153154 155 155 156 157 159 160 160 161 163 166 172 175 177 178 182 182 184 186 187 193 194</p> <p>A Concrete ExampleGoals Nongoals</p> <p>Test ImplementationTesting for Success Building Test Data Test Setup Issues Error Handling Emerging Unit Tests Coping with In-Container Components Putting It All Together</p> <p>Exploring the Competing Consumers PatternThe Pattern The Test</p> <p>The Role of RefactoringA Concrete Example An In-Container Approach</p> <p>Conclusion</p> <p>Chapter 4</p> <p>Java EE TestingIn-Container versus Out-of-Container Testing In-Container TestingCreating a Test Environment Identifying Tests Registering Tests Registering a Results Listener</p> <p>197198 200 200 201 203 204 207 207 209 210</p> <p>Java Naming and Directory Interface (JNDI)Understanding JNDIs Bootstrapping Springs SimpleNamingContextBuilder Avoiding JNDI</p> <p>viii</p> <p>Contents</p> <p>Java Database Connectivity (JDBC)c3p0 Commons DBCP Spring</p> <p>Java Transaction API (JTA)Java Open Transaction Manager (JOTM) Atomikos TransactionEssentials</p> <p>Java Messaging Service (JMS)Creating a Sender/Receiver Test Using ActiveMQ for Tests</p> <p>Java Persistence API (JPA)Configuring the Database Configuring the JPA Provider Writing the Test Simulating a Container Using Spring as the Container</p> <p>Enterprise Java Beans 3.0 (EJB3)Message-Driven Beans Session Beans Another Spring Container Disadvantages of a Full Container</p> <p>Java API for XML Web Services (JAX-WS)Recording Requests Setting Up the Test Environment Creating the Service Test XPath Testing Testing Remote Services</p> <p>ServletsIn-Container Testing Mock/Stub Objects Refactoring Embedded Container In-Memory Invocation</p> <p>XMLUsing dom4j Using XMLUnit</p> <p>Conclusion</p> <p>210 212 213 213 215 217 218 219 219 221 225 227 227 229 230 231 236 237 240 243 244 246 248 248 251 253 254 255 255 255 257 257 260 262 263 264 266</p> <p>Chapter 5</p> <p>IntegrationSpringSprings Test Package Features Test Class Hierarchy</p> <p>269270 271 272 280 280 281 282</p> <p>GuiceThe Issue with Spring Enter Guice A Typical Dependency Scenario</p> <p>Contents</p> <p>ix284 286 290 291 293 295 295 297 299 303 304 305 310 312 312 313 314 316 320 320 320 321 322</p> <p>The Object Factory Guice Configuration Guice-Based Test Grouping Test Dependencies Injecting Configuration</p> <p>DbUnitConfiguration Usage Verifying Results</p> <p>HtmlUnitConfiguration Usage</p> <p>Selenium Swing UI TestingTesting Approach Configuration Usage</p> <p>Tests for Painting Code Continuous IntegrationWhy Bother? CI Server Features TestNG Integration</p> <p>Conclusion</p> <p>Chapter 6</p> <p>Extending TestNGThe TestNG APIorg.testng.TestNG, ITestResult, ITestListener, ITestNGMethod A Concrete Example The XML API Synthetic XML Files</p> <p>325325 325 328 331 333 335 335 337 339 341 346 346 348 348 353 355 355 360 360</p> <p>BeanShellBeanShell Overview TestNG and BeanShell Interactive Execution</p> <p>Method Selectors Annotation TransformersAnnotation History Pros and Cons Using TestNG Annotation Transformers Possible Uses of Annotation Transformers</p> <p>ReportsDefault Reports The Reporter API The Report Plug-in API</p> <p>x</p> <p>Contents</p> <p>Writing Custom AnnotationsImplementation Testing</p> <p>Conclusion</p> <p>366 367 371 375</p> <p>Chapter 7</p> <p>DigressionsMotivation The TestNG Philosophy The Care and Feeding of Exceptions Stateful TestsImmutable State Mutable State</p> <p>377377 378 378 382 382 383 385 385 386 388 388 391 392 394 397 399</p> <p>The Pitfalls of Test-Driven DevelopmentTDD Promotes Microdesign over Macrodesign TDD Is Hard to Apply Extracting the Good from Test-Driven Development</p> <p>Testing Private Methods Testing versus Encapsulation The Power of Debuggers Logging Best Practices The Value of Time Conclusion</p> <p>Appendix A IDE IntegrationEclipseInstalling the Plug-in Verifying the Installation Creating a Launch Configuration Configuring Preferences Converting JUnit Tests</p> <p>401401 401 404 404 410 410 411 411 412 417 418 419</p> <p>IntelliJ IDEAInstalling the Plug-in Running Tests Running Shortcuts Viewing Test Results Running Plug-in Refactorings</p> <p>Appendix B TestNG JavadocsJDK 1.4 and JDK 5 Shortcut Syntax for JDK 5 Annotations</p> <p>421421 423</p> <p>Contents</p> <p>xi423 425 426 426 427 428 432</p> <p>Annotation Javadocs@DataProvider/ @Factory/@testng.factory @Parameters/@testng.parameters @Test/@testng.test The org.testng.TestNG Class</p> <p>The XML API</p> <p>Appendix C testng.xmlOverview Scopes XML Tags and and , , , and , , and and </p> <p>435436 437 437 437 440 441 442 443 444 446 446 447</p> <p>Appendix D Migrating from JUnitJUnitConverter From the Command Line From ant</p> <p>449449 449 452 453 453 454 455 456 457 458 461 463 463 463 467 469</p> <p>Integrated Development EnvironmentsEclipse IDEA</p> <p>Incremental Migration and JUnit Mode Converting JUnit CodeAssertions Running a Single Test Maintaining State between Invocations Suite-wide Initialization Class-wide Initialization The AllTests Pattern Testing Exceptions The Parameterized Test Case Pattern</p> <p>Index</p> <p>471</p> <p>This page intentionally left blank</p> <p>ForewordDoing the right thing is rarely easy. Most of us should probably eat better, exercise more, and spend more time with our families. But each day, when confronted with the choice of being good or doing the easy thing, we often choose the easy thing because, well, its easier. It is the same way with software testing. We all know that we should spend more effort on testing, that more testing will make our code more reliable and maintainable, that our users will thank us if we do, that it will help us better understand our own programs, but each day when we sit down to the computer and choose between writing more tests and writing more application code, it is very tempting to reach for the easy choice. Today, it is largely accepted that unit testing is the responsibility of developers, not QA, but this is a relatively recent development, one for which we can largely thank the JUnit testing framework. It is notable that JUnit had such an impact because theres not really very much to itits a simple framework, with not a lot of code. What enabled JUnit to change developer behavior where years of lecturing and guilt could not was that, for the first time, the pain of writing unit tests was reduced to a bearable level, making it practical for the merely responsible to include unit testing in our daily coding. Rather than make testing more desirable, which is not such an easy sell (eat those vegetables, theyre good for you!), JUnit simply made it easier to do the right thing. With all the righteousness of the newly converted, many developers proclaimed their zeal for testing, proudly calling themselves test-infected. This is all well and goodfew could argue that software developers were doing too much testing, so more is probably an improvementbut it is only the first step. Theres more to testing than unit tests, and if we expect developers to take this next step we must provide testing tools that reduce the pain of creating themand demand testability as a fundamental design requirement. If software engineering is ever to become a true engineering discipline, testing will form one of the critical pillars on which it will be built. (Perhaps one day, writing code without tests will be considered as professionally irresponsible as constructing a bridge without performing a structural analysis.)</p> <p>xiii</p> <p>xiv</p> <p>Foreword</p> <p>This book is dedicated to the notion that weve only just begun our relationship with responsible testing. The TestNG project aims to help developers take the next stepthe NG stands for next generationenabling broader and deeper test coverage that encompasses not only unit tests but also acceptance, functional, and integration tests. Among other useful features, it provides a rich mechanism for specifying and parameterizing test suites, encompassing concurrent testing and a flexible mechanism for decoupling test code from its data source. (And, as proof that TestNG is succeeding, a number of its features have been adopted in more recent versions of JUnit.) One challenge to more effective developer testing, no matter what tools are provided, is that writing effective tests requires different skills than writing effective code. But, like most skills, testing can be learned, and one of the best ways to learn is to watch how more experienced hands might do it. Throughout this book, Hani and Cdric share with you their favorite techniques for effectively testing Java applications and for designing applications and components for testability. (This last skilldesigning for testabilityis probably one of the most valuable lessons from this book. Designing code for testability forces you to think about the interactions and dependencies between components earlier, thereby encouraging you to build cleaner, more loosely coupled code.) Of course, the TestNG framework is used to illustrate these techniques, but even if you are not a TestNG user (and not interested in becoming one), the practical techniques presented here will help you to be a better tester and, in turn, a better engineer. Brian Goetz Senior Staff Engineer, Sun Microsystems</p> <p>PrefaceWere all in the business of software development. Code is written and then deployed. Once weve deployed the code, our customers will express pleasure or, depressingly often, displeasure. For the last few decades, much has been written about how to minimize this displeasure. We have countless languages, methodologies, tools, management techniques, and plain old-fashioned mythology to help address this issue. Some of these approaches are more effective than others. There has certainly been a renewed emphasis and focus on testing lately, along with the pleasures said testing would bring to developers and users alike. Much has been written extolling the virtues of testing. It can make your code better, faster, and lighter. It can add some sorely needed spice to the drudgery of coding. Its exciting and new (and therefore worth trying), not to mention the feeling of responsibility and accountability it imparts; theres something mysteriously satisfying about adding a feature...</p>