Next generation-java-testing.-test ng-and-advanced-concepts

  • Published on

  • View

  • Download

Embed Size (px)


Next Generation Java testing


<ul><li> 1. Next Generation Java Testing </li> <li> 2. This page intentionally left blank </li> <li> 3. Next Generation Java Testing TestNG and Advanced Concepts Cdric Beust Hani Suleiman Upper Saddle River, NJ Boston Indianapolis San Francisco New York Toronto Montreal London Munich Paris Madrid Capetown Sydney Tokyo Singapore Mexico City </li> <li> 4. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and the publisher was aware of a trademark claim, the designations have been printed with initial capital letters or in all capitals. The authors and publisher have taken care in the preparation of this book, but make no expressed or implied warranty of any kind and assume no responsibility for errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of the use of the information or programs contained herein. The publisher offers excellent discounts on this book when ordered in quantity for bulk purchases or special sales, which may include electronic versions and/or custom covers and content particular to your business, training goals, marketing focus, and branding interests. For more information, please contact: U.S. Corporate and Government Sales (800) 382-3419 For sales outside the United States please contact: International Sales Visit us on the Web: Library of Congress Cataloging-in-Publication Data Beust, Cdric. Next generation Java testing : TestNG and advanced concepts / Cdric Beust, Hani Suleiman. p. cm. Includes bibliographical references and index. ISBN 0-321-50310-4 (pbk. : alk. paper) 1. Java (Computer program language) 2. Computer softwareTesting. I. Suleiman, Hani. II. Title. QA76.73.J3B49 2007 005.13'3dc22 2007031939 Copyright 2008 Pearson Education, Inc. All rights reserved. Printed in the United States of America. This publication is protected by copyright, and permission must be obtained from the publisher prior to any prohibited reproduction, storage in a retrieval system, or transmission in any form or by any means, electronic, mechanical, photocopying, recording, or likewise. For information regarding permissions, write to: Pearson Education, Inc. Rights and Contracts Department 501 Boylston Street, Suite 900 Boston, MA 02116 Fax: (617) 671-3447 ISBN-13: 978-0-321-50310-7 ISBN-10: 0-321-50310-4 Text printed in the United States on recycled paper at Courier in Stoughton, Massachusetts. First printing, October 2007 </li> <li> 5. v Contents Foreword xiii Preface xv Acknowledgments xxi About the Authors xxiii Chapter 1 Getting Started 1 Beyond JUnit 3 3 Stateful Classes 3 Parameters 4 Base Classes 4 Exceptions Are Not That Exceptional 4 Running Tests 5 Real-World Testing 6 Configuration Methods 6 Dependencies 6 Epiphanies 7 JUnit 4 7 Designing for Testability 8 Object-Oriented Programming and Encapsulation 8 The Design Patterns Revolution 9 Identifying the Enemy 10 Recommendations 16 TestNG 17 Annotations 17 Tests, Suites, and Configuration Annotations 18 Groups 20 testng.xml 21 Conclusion 21 Chapter 2 Testing Design Patterns 23 Testing for Failures 23 Reporting Errors 24 Runtime and Checked Exceptions 25 </li> <li> 6. vi Contents Testing Whether Your Code Handles Failures Gracefully 27 When Not to Use expectedExceptions 31 testng-failed.xml 32 Factories 34 @Factory 35 org.testng.ITest 38 Data-Driven Testing 39 Parameters and Test Methods 42 Passing Parameters with testng.xml 44 Passing Parameters with @DataProvider 47 Parameters for Data Providers 50 The Method Parameter 50 The ITestContext Parameter 52 Lazy Data Providers 54 Pros and Cons of Both Approaches 59 Supplying the Data 60 Data Provider or Factory? 62 Tying It All Together 63 Asynchronous Testing 67 Testing Multithreaded Code 71 Concurrent Testing 72 threadPoolSize, invocationCount, and timeOut 75 Concurrent Running 79 Turning on the Parallel Bit 82 Performance Testing 83 Algorithm Complexity 84 Testing Complexity 87 Mocks and Stubs 90 Mocks versus Stubs 90 Designing for Mockability 95 Mock Libraries 96 Selecting the Right Strategy 99 Mock Pitfalls 100 Dependent Testing 103 Dependent Code 104 Dependent Testing with TestNG 105 Deciding Whether to Depend on Groups or on Methods 106 Dependent Testing and Threads 110 Failures of Configuration Methods 110 Inheritance and Annotation Scopes 113 The Problem 113 Pitfalls of Inheritance 116 Test Groups 119 Syntax 120 Groups and Runtime 122 Running Groups 125 Using Groups Effectively 127 </li> <li> 7. Contents vii Code Coverage 132 A Coverage Example 133 Coverage Metrics 134 Coverage Tools 136 Implementation 146 Beware! 147 A Guide to Successful Coverage 147 Conclusion 150 Chapter 3 Enterprise Testing 153 A Typical Enterprise Scenario 154 Participants 155 Testing Methodology 155 Issues with the Current Approach 156 A Concrete Example 157 Goals 159 Nongoals 160 Test Implementation 160 Testing for Success 161 Building Test Data 163 Test Setup Issues 166 Error Handling 172 Emerging Unit Tests 175 Coping with In-Container Components 177 Putting It All Together 178 Exploring the Competing Consumers Pattern 182 The Pattern 182 The Test 184 The Role of Refactoring 186 A Concrete Example 187 An In-Container Approach 193 Conclusion 194 Chapter 4 Java EE Testing 197 In-Container versus Out-of-Container Testing 198 In-Container Testing 200 Creating a Test Environment 200 Identifying Tests 201 Registering Tests 203 Registering a Results Listener 204 Java Naming and Directory Interface (JNDI) 207 Understanding JNDIs Bootstrapping 207 Springs SimpleNamingContextBuilder 209 Avoiding JNDI 210 </li> <li> 8. viii Contents Java Database Connectivity (JDBC) 210 c3p0 212 Commons DBCP 213 Spring 213 Java Transaction API (JTA) 215 Java Open Transaction Manager (JOTM) 217 Atomikos TransactionEssentials 218 Java Messaging Service (JMS) 219 Creating a Sender/Receiver Test 219 Using ActiveMQ for Tests 221 Java Persistence API (JPA) 225 Configuring the Database 227 Configuring the JPA Provider 227 Writing the Test 229 Simulating a Container 230 Using Spring as the Container 231 Enterprise Java Beans 3.0 (EJB3) 236 Message-Driven Beans 237 Session Beans 240 Another Spring Container 243 Disadvantages of a Full Container 244 Java API for XML Web Services (JAX-WS) 246 Recording Requests 248 Setting Up the Test Environment 248 Creating the Service Test 251 XPath Testing 253 Testing Remote Services 254 Servlets 255 In-Container Testing 255 Mock/Stub Objects 255 Refactoring 257 Embedded Container 257 In-Memory Invocation 260 XML 262 Using dom4j 263 Using XMLUnit 264 Conclusion 266 Chapter 5 Integration 269 Spring 270 Springs Test Package Features 271 Test Class Hierarchy 272 Guice 280 The Issue with Spring 280 Enter Guice 281 A Typical Dependency Scenario 282 </li> <li> 9. Contents ix The Object Factory 284 Guice Configuration 286 Guice-Based Test 290 Grouping Test Dependencies 291 Injecting Configuration 293 DbUnit 295 Configuration 295 Usage 297 Verifying Results 299 HtmlUnit 303 Configuration 304 Usage 305 Selenium 310 Swing UI Testing 312 Testing Approach 312 Configuration 313 Usage 314 Tests for Painting Code 316 Continuous Integration 320 Why Bother? 320 CI Server Features 320 TestNG Integration 321 Conclusion 322 Chapter 6 Extending TestNG 325 The TestNG API 325 org.testng.TestNG, ITestResult, ITestListener, ITestNGMethod 325 A Concrete Example 328 The XML API 331 Synthetic XML Files 333 BeanShell 335 BeanShell Overview 335 TestNG and BeanShell 337 Interactive Execution 339 Method Selectors 341 Annotation Transformers 346 Annotation History 346 Pros and Cons 348 Using TestNG Annotation Transformers 348 Possible Uses of Annotation Transformers 353 Reports 355 Default Reports 355 The Reporter API 360 The Report Plug-in API 360 </li> <li> 10. x Contents Writing Custom Annotations 366 Implementation 367 Testing 371 Conclusion 375 Chapter 7 Digressions 377 Motivation 377 The TestNG Philosophy 378 The Care and Feeding of Exceptions 378 Stateful Tests 382 Immutable State 382 Mutable State 383 The Pitfalls of Test-Driven Development 385 TDD Promotes Microdesign over Macrodesign 385 TDD Is Hard to Apply 386 Extracting the Good from Test-Driven Development 388 Testing Private Methods 388 Testing versus Encapsulation 391 The Power of Debuggers 392 Logging Best Practices 394 The Value of Time 397 Conclusion 399 Appendix A IDE Integration 401 Eclipse 401 Installing the Plug-in 401 Verifying the Installation 404 Creating a Launch Configuration 404 Configuring Preferences 410 Converting JUnit Tests 410 IntelliJ IDEA 411 Installing the Plug-in 411 Running Tests 412 Running Shortcuts 417 Viewing Test Results 418 Running Plug-in Refactorings 419 Appendix B TestNG Javadocs 421 JDK 1.4 and JDK 5 421 Shortcut Syntax for JDK 5 Annotations 423 </li> <li> 11. Contents xi Annotation Javadocs 423 @DataProvider/ 425 @Factory/@testng.factory 426 @Parameters/@testng.parameters 426 @Test/@testng.test 427 The org.testng.TestNG Class 428 The XML API 432 Appendix C testng.xml 435 Overview 436 Scopes 437 XML Tags 437 437 and 440 441 and 442 , , , and </li> <li> 12. This page intentionally left blank </li> <li> 13. xiii Foreword Doing the right thing is rarely easy. Most of us should probably eat better, exercise more, and spend more time with our families. But each day, when confronted with the choice of being good or doing the easy thing, we often choose the easy thing because, well, its easier. It is the same way with software testing. We all know that we should spend more effort on testing, that more testing will make our code more reliable and maintainable, that our users will thank us if we do, that it will help us better understand our own programs, but each day when we sit down to the computer and choose between writing more tests and writing more application code, it is very tempting to reach for the easy choice. Today, it is largely accepted that unit testing is the responsibility of developers, not QA, but this is a relatively recent development, one for which we can largely thank the JUnit testing framework. It is notable that JUnit had such an impact because theres not really very much to itits a simple framework, with not a lot of code. What enabled JUnit to change developer behavior where years of lecturing and guilt could not was that, for the first time, the pain of writing unit tests was reduced to a bearable level, making it practical for the merely responsible to include unit testing in our daily coding. Rather than make testing more desirable, which is not such an easy sell (eat those vegetables, theyre good for you!), JUnit simply made it easier to do the right thing. With all the righteousness of the newly converted, many developers proclaimed their zeal for testing, proudly calling themselves test-infected. This is all well and goodfew could argue that software developers were doing too much testing, so more is probably an improvementbut it is only the first step. Theres more to testing than unit tests, and if we expect devel- opers to take this next step we must provide testing tools that reduce the pain of creating themand demand testability as a fundamental design require- ment. If software engineering is ever to become a true engineering discipline, testing will form one of the critical pillars on which it will be built. (Perhaps one day, writing code without tests will be considered as professionally irre- sponsible as constructing a bridge without performing a structural analysis.) </li> <li> 14. xiv Foreword This book is dedicated to the notion that weve only just begun our rela- tionship with responsible testing. The TestNG project aims to help develop- ers take the next stepthe NG stands for next generationenabling broader and deeper test coverage that encompasses not only unit tests but also acceptance, functional, and integration tests. Among other useful fea- tures, it provides a rich mechanism for specifying and parameterizing test suites, encompassing concurrent testing and a flexible mechanism for decoupling test code from its data source. (And, as proof that TestNG is suc- ceeding, a number of its features have been adopted in more recent ver- sions of JUnit.) One challenge to more effective developer testing, no matter what tools are provided, is that writing effective tests requires different skills than writ- ing effective code. But, like most skills, testing can be learned, and one of the best ways to learn is to watch how more experienced hands might do it. Throughout this book, Hani and Cdric share with you their favorite tech- niques for effectively testing Java applications and for designing applications and components for testability. (This last skilldesigning for testabilityis probably one of the most valuable lessons from this book. Designing code for testability forces you to think about the interactions and dependencies between components earlier, thereby encouraging you to build cleaner, more loosely coupled code.) Of course, the TestNG framework is used to illustrate these techniques, but even if you are not a Te...</li></ul>