• Published on

  • View

  • Download

Embed Size (px)




<ul><li> 1. May/June 2009 Volume 2 SOFTWARE TESTINGTesting strategies for complex environments: Agile, SOA CHAPTER 1A SOA VIEW OF TESTING CHAPTER 2AGILE TESTING STRATEGIES: EVOLVING WITH THE PRODUCT </li></ul><p> 2. EDITORS LETTER1 Testing in the new worlds of SOA and agile p UNLESS YOU LIVE in a sci-fi novel,Karen N. Johnson in Agile testing a theres one rule of thumb for any new strategies: Evolving with the product.EDITORS LETTERplace you go: some things are differ- Agile is a more collaborative process ent; some are the same. Usually, suc- and calls for seizing iterations as a cess, fun or survival in that new place chance to evolve test ideas, Johnson a depends upon how well you handlewrites. Fundamental testing tasks, likeCHAPTER 1 A SOA VIEWthe differences. In this issue of exploratory and investigative testing, OF TESTINGSearchSoftwareQuality.coms Soft- stay the same, but the productivity of ware Testing E-Zine, the new places completing those jobs can increase. are a service-oriented architecture Johnson and Kelly discuss the finer a (SOA) and an agile software develop-points of testing in agile and SOA,CHAPTER 2 AGILE TESTING ment environment. respectively, in this issues articles. STRATEGIES: EVOLVING Examining the ins and outs of soft-They also describe revelations theyve WITH THE PRODUCT ware testing in SOA environments in had while working in those environ- An SOA view of testing, consultantments and best practices theyve Mike Kelly focuses on the subtle dif-learned and continue to use. ferences. Whats the same is thatThe theme of connectivity runs testers here must start with thethrough these discussions of SOA and basics. In SOA, the basics are connec-agile testing, as the former focuses on tivity, basic capacity, and authoriza-system connectivity and the latter on tion and authenticationnot that dif- human collaboration. Both approach- ferent from other environments. The es, when done well, can help develop- level of complexity of data models, ment and IT organizations achieve however, is very different from that of lower costs and better software. I other architectures and requires some new methods of testing. JAN STAFFORDWhile agile development requires Executive Editor different testing methods than waterfall model, those differences lie as much in human behavior as in technology, according to consultant 2 SOFTWARE TESTING MAY/JUNE 2009 3. CHAPTER 11 A SOA view of testingThe complexity of a service-oriented architecture cancomplicate testing and make choosing and implementingthe right testing tools more difficult. BY MICHAEL KELLY aEDITORS LETTERp SERVICE-ORIENTED architecturestest. This means people testing SOA (SOAs) are stealthily becoming ubiq-need to be comfortable with varied uitous. Large and small enterprises and changing technology, need to a leverage them to integrate wide understand the various models forCHAPTER 1 A SOA VIEWarrays of disparate systems into aSOA, and should be familiar with OF TESTING cohesive whole. Most companies andthe risks common to SOA. project teams that implement SOA do so to reduce the cost, risk and difficul- a ty of replacing legacy systems, acquir- WHY SERVICE-ORIENTEDCHAPTER 2 AGILE TESTING STRATEGIES: ing new businesses or extending the ARCHITECTURES? EVOLVINGlife of existing systems. Large companies typically turn to WITH THE PRODUCTTesting a SOA requires a solid SOA because their existing systems understanding of the SOA design and cant change fast enough to keep up underlying technology. Yet, SOAs with the business. Because business complexity makes figuring out the operations dont exist in discrete or functional purpose of the SOA diffi-finite units, there are a lot of depend- cult and choosing and implementingencies built into existing systems. the right testing tools and techniquesSOA is an attempt to separate the to use with it more difficult.system from the business operationsSOA is not simply Web supports. This allows companies to According to iTKO (the makers of theincrementally build or decommission TechTarget award-winning SOA test-systems and minimizes the impact of ing tool LISA), nine out of 10 Webchanges in one system to changes in services also involve some other type another. of technology. In addition, they state Implementing a SOA removes that most testing for SOA isnt donethe requirement of direct integration, at the user interface level. Its donewhich results in business operations using either specialized tools or asthat are easier to understand. That part of an integration or end-to-endtranslates into a more testable system 3 SOFTWARE TESTING MAY/JUNE 2009 4. overall. Over time, this makes itdards such as application-oriented easier to support a broader technolo-networking (AON), WS-Security, gy base, which in turn makes it easier SAML, WS-Trust, WS-SecurityPolicy, to acquire and integrate new systems,or other custom security schemes for extend the life of existing systemsdata security as well as authorization and replace legacy systems.and authentication.There are a number of different model architectures for implementing ILoad balancing: SOAs are often SOA, including publish and subscript designed to spread work between (often called pub/sub), requestresources to optimize resource and reply, and synchronous versusutilization, availability, reliability asynchronous. In addition, they canand performance. a be implemented in a broad rangeEDITORS LETTERof technologies, including message-I Translation: As data moves oriented middleware (MOM), through a SOA, its often converted simple HTTP posts with DNS routing,or changed based on business rules a Web services, MQ series, JMS (or or reference data.CHAPTER 1 A SOA VIEWsome other type of queuing system) OF TESTINGand even files passed in batch ILogging: For monitoring and processes. auditing, SOAs often implementIf youre testing a SOA you needmultiple forms of logging. a to understand the implementationCHAPTER 2 AGILE TESTING model and technologies involved, I Notification: Based on the model STRATEGIES: EVOLVINGbecause these typically combine to of SOA implemented, different notifi- WITH THE PRODUCT give you some common features of cations may happen at different a SOA. These features often become times. a focal point for your testing. They commonly include:IAdapters: Adapters (both customand commercial) provide APIs to I Transformation and mapping:access data and common functions As data moves through a SOA, itswithin a SOA. often transformed between data formats and mapped to various standard or custom data schemes. FIGURING OUT WHAT TO TESTThe first thing you need to do when I Routing: The path informationyou start figuring what to test for your takes as it moves through a SOASOA is to develop (or start develop- is often based on business rules ing) your test strategy. When Im embedded at different levels offaced with a new project and Im not the architecture.sure where to start, I normally pullout the Satisfice Heuristic Test Strate- I Security: SOAs often use stan- gy Model and start there. Each sec- 4 SOFTWARE TESTING MAY/JUNE 2009 5. Feeling theAgile Rush?7 Test FasterWith TestComplete The Easiest7 TestComplete Ever! Agile development moves fast, too fast for manual testing alone. In an agile environment you need automated testing tools to succeed. Check out TestComplete from AutomatedQA. TestComplete lets anyone automate tests for the widest range of technologies like Web, Ajax, Windows, .NET, Java, Flash, Flex and Silverlight. TestComplete has won four Jolt awards, been voted Best Testing ToolScript-Free Test Creation and yet it's still a ordable.Simple to learn and easy to extendTestComplete's extensive feature set and Download and Customize ExtensionsPre-built and custom add-ons make great price has long made it the choice ofcomplex tests point and click easy expert testers. Now version 7 adds script-free test creation and a simpli edUnmatched Technology Support user interface so new testers can getNew technology? No problem! Rapid productive fast. TestComplete 7's easy support for the latest software automated testing lets your entire QA teamPrice and Performance Leader test more, test faster and ship on time with Unsurpassed features and a low price con dence. to make you smile. Start Testing In Minutes!Download TestComplete now and starttesting today. DownloadTestCompleteFREEAutomatedQA 236-7900 6. tion of the Satisfice Heuristic TestI Project environment: What fac- Strategy Model contains things youll tors will be critical to your success need to consider as you think about and the success of the in-house team your testing: as you take over this work? How will you take in new work, structure your I Test techniques: What types ofrelease schedules, or move code testing techniques will you be doing? between teams, phases or environ- What would you need to supportments? What are the business drivers those types of testing in terms ofto moving to a SOA and how will people, processes, tools, environ-those come into play while youre ments or data? Are there specific testing? things you need to test (like security, a transformation and mapping, or loadRecognize that as you think ofEDITORS LETTERbalancing) that may require special-these questions, its a matrix of con- ized techniques?cerns. A decision (or lack of decision) in each of these categories affects the a I Product elements: What productscope of the decisions in the otherCHAPTER 1 A SOA VIEWelements will you be covering in your three. Therefore, you will most likely OF TESTINGtesting? Whats your scope? To what find yourself approaching the problem extent will different data elements befrom different perspectives at differ- covered in each stage? Which busi-ent times. a ness rules? Will you be testing theOnce you have an understandingCHAPTER 2 AGILE TESTING target implementation environment?of what features youll be testing and STRATEGIES: EVOLVINGHow much testing will you be doingyou know which quality criteria youll WITH THE PRODUCT around timing issues? How will yoube focused on, youre ready to dig in. measure your coverage against those Dont be surprised if in many ways elements, track it and manage it from your testing looks like it does on a a documentation and configuration non-SOA project. SOA isnt magic, management perspective? and it doesnt change things all that much. However, Ive found that there I Quality criteria: What types of are some subtle differences in where risks will you be looking for while test- my focus is when Im working on a ing? Will you focus more on the busi- SOA project. ness problem being solved or the risks related to the implementation model or technology being used? Will youCOVER THE BASICS AS SOON look at performance at each level?(AND AS OFTEN) AS YOU CAN What about security? How will you The first area to focus on is typically need to build environments, integrate connectivity. Establishing a successful with service providers, or find tools round-trip test is a big step early in a to do all the different types of testingSOA implementation. The first time youll need to do?connecting is often the most difficult. 6 SOFTWARE TESTING MAY/JUNE 2009 7. Once you have that connectivity,development and test environments dont lose it. Build out regression had different security controls than tests for basic connectivity that you production, which led to confusion can run on a regular basis. While a and rework once we deployed and SOA evolves, it is easy to break con- tried to run using an untested permis- nectivity without knowing it. If yousions scheme. If you use LDAP in pro- duction, use it when testing. If youre handling authorization in your SOAP As data moves request, do it consistently through through a SOA itsyour testing, even if you think its easier to disable it while doing translated, transformed,functional testing. a reformatted andEDITORS LETTERprocessed. That FOCUS ON THE DATA means youll have Service-oriented architectures a a lot of tests focusedarent just cleaner interfaces betweenCHAPTER 1 A SOA VIEW on testing, they are also complex data OF TESTINGmodels. As data moves through a SOA its translated, transformed, break it, youll want to find out asreformatted and processed. That a soon as possible so its easier tomeans youll have a lot of testsCHAPTER 2 AGILE TESTING debug any issues. focused on testing data. These tests STRATEGIES: EVOLVING Next you may want to look at basic typically make up the bulk of the WITH THE PRODUCT capacity. Capacity at this level can be regression test beds Ive seen on anything from basic storage to con- projects implementing a SOA. nection pooling to service throughput. Tests focused around testing data Ive been on a couple of projects in an SOA typically happen at the where these elements werent looked component level and leverage stubs at until the end, only to find that weand harnesses to allow for testing as had to requisition new hardware atclose as possible to where the change the last minute or end up spendingtakes place. This simplifies debugging days changing configuration settingsand allows for more granular tracking on network devices and Web servers. of test coverage (based on a schema, Figure out what you think youll need mapping document or set of business early on, and run tests to confirmrules). SOA teams are constantly those numbers are accurate andthinking about regression testing for consistent throughout the project.these tests. Small changes can haveIn addition, dont put off testing for a large impact on the data, so these authorization and authentication untiltests become critical to refactoring you get into the target environment.or when adding new features. Ive worked on several teams where For many teams, these tests can 7 SOFTWARE TESTING MAY/JUNE 2009 8. be part of the continuous integration agers and testers alike. That means process. While building these regres- faster communication, clearer sion test beds can be very easy, once requirements and better tests. At you have libraries of XML test casessome level, every SOA implementa- lying around, maintaining them can be tion needs to think about perform- very time consuming. That makes a ance, availability, reliability and teams unit testing strategy a critical capacity. Usage models help focus the part of the overall testing strategy. testing dialogue on those topics (as What will be tested at the unit level,opposed to just focusing on function- component level, integration level andality and data). end-to-end? Starting as close to the When testing for SOA performance, code as possible, automating there, I focus on speed, load, concurrency a and running those tests as often as and latency. My availability and relia-EDITORS LETTERpossible is often the best choice for bility scenarios for SOA often focus data-focused testing. on failover, application and infrastruc- ture stress, fault toler...</p>