Embed Size (px)
Citation preview
Testing SOA ArchitecturesEnsuring Quality In Next-Generation Applications
Lee Copeland, Software Quality EngineeringConsultant
John Michelsen, iTKO, Inc.Chief Scientist and Co-Founder
Thank you for joining us. We’ll be starting shortly.
Today’s Presenters
Lee CopelandConsultantSoftware Quality Engineering
John MichelsenChief Scientist & Co-FounderiTKO, Inc.
Service Oriented Architecture
Promises
© SQE 2006
• Many of today’s applications are tightly coupled collections of data and function.
• Reuse, redesign, and replacement of component parts can be very difficult.
The World As It Is Today
© SQE 2006
FunctionE
FunctionD
Database FunctionC
FunctionB1
FunctionA
UserInterface
FunctionB2
Database
Application A
Innovations
• A key enabler of innovation in the IT industry is the myriad of available choices in hardware, software, and process technologies.
© SQE 2006
Effects Of Emerging Technologies
• But, these new technologies emerge so rapidly that often only one application is built with a specific set of technologies.
© SQE 2006
Application A
Application B Application C
Incompatible Applications
• And so, over time, we build a set of successful applications that are incompatible with each other.
© SQE 2006
Can’t reuse functionality Can’t share data
Can’t communicate
The Concept Of SOA
• Service Oriented Architecture (SOA) is a holistic approach to implement business systems across a loosely coupled set of technologies.
• Rather than continuing to build monolithic applications …
© SQE 2006
The Concept Of SOA
• SOA is “a way of designing and implementing enterprise applications that deals with the intercommunication of loosely coupled, coarse grained (business level), reusable artifacts (services) that are accessed through well-defined, platform independent, interface contracts."
- Steve Wilkes
© SQE 2006
SOA Application Design
© SQE 2006
Function A Function C
Function D
Function B
Function E
User InterfaceApplication A
Business LevelFunction
Component (Service)
The Concept Of SOA
• In fact, these services can from many sources and be distributed over heterogeneous environments.
© SQE 2006
Developed Purchased
Purchased
3rd PartyShared
Developed
Service Oriented Architecture
TestingChallenges
© SQE 2006
Testing Challenges
• Since SOA applications are composed of loosely coupled, business-level services, distributed over a network, we must test the application from end-to-end.
• But, before we begin end-to-end testing, we should test the individual services.
© SQE 2006
Testing SOA Services
• SOA services should be tested in the areas of:
– Functionality
– API (Application Programming Interface)
Parameter selection
Combinations
Sequencing
© SQE 2006
Functionality(Service)
API
Testing Functionality
• In a real sense, the overall functionality of SOA applications is easier to test.
• That’s because (but only if) we thoroughly test the application’s components (services) before we assemble them to create the application.
• Starting with lower-defect components means a smoother testing process BUT …
© SQE 2006
Testing Functionality
• SOA applications generally have an increased number of:
– APIs (one for each service)
– Communication paths between those services
• There is now an increased level of integration testing that must be performed.
© SQE 2006
API Parameter Selection
• Consider a function
determineEmploymentStatus (int age)
that is to implement the following rules:
© SQE 2006
Age 0-16 Don’t hireAge 16-18 Part timeAge 18-55 Full timeAge 55 or > Fire ‘em
0 16 18 55
What would you think about these test points?
20, 23, 25, 27, 30, 32,37, 38, 40, 42, 50, 51
API Parameter Selection
• Services should be tested directly through the API to avoid GUI “filtering”.
© SQE 2006
Age
determineEmploymentStatus (DWORD age)
API Parameter Combinations
© SQE 2006
• Now let’s consider the function
doSomethingReallyUseful(parm1, parm2, parm3)
where:
parm1 has 6 equivalence classes
parm2 has 6 equivalence classes
parm3 has 6 equivalence classes
Choosing all combinations of parameters yields 216 different test cases – more than I want to do.
All-Pairs Testing
• An alternative to all-combinations testing is all-pairs testing in which pairs of parameters are combined for testing.
• The reasoning behind all-pairs testing is this: the simplest bugs in a program are generally triggered by a single input parameter. The next most common category of bugs are those dependent on interactions between pairs of parameters, which can be discovered through all-pairs testing.
© SQE 2006
A B C
All-Pairs Testing
• All-pairs testing can result in a significant reduction in the number of test cases required but still provide excellent defect find rates.
© SQE 2006
5-20% of test cases
70-90% of defects
Service Invocation Sequencing
• SOA services are stateless. This means that they know nothing about functions that have executed before, and they require nothing of functions that will execute after.
• But real applications have state, that is, they must guarantee that services are executed in a well-defined order.
• A classic example is the Input-Process-Output sequence of most applications.
© SQE 2006
Service Invocation Sequencing
• Well designed applications remember their state, allow valid transitions, and prevent invalid ones.
• State-transition diagrams are a valuable tool in documenting the required temporal relationships.
© SQE 2006
Service Invocation Sequencing
• Test cases are then derived from these state-transition diagrams.
• Test cases should verify all services have been exercised, all transitions have been followed, and, if possible, all paths have been tested.
© SQE 2006
Additional Testing Challenges
• Let’s now consider:
– Publish, Find, and Bind
– Security
– Interoperability
– Performance
© SQE 2006
Publish, Find, And Bind
• Service providers must “advertise” their existence to brokers (automated catalogs).
• Service requestors (your SOA application) must find providers and connect (bind) to them to use their services.
• Service brokers must acceptregistrations and then provideinformation through searchfunctions.
© SQE 2006
Publish, Find, And Bind
• You didn’t have to test anything like this before because all the application’s functionality was there, right inside the application.
• Are services your organization is providing able to properly register themselves?
• Can your SOA application find and properly bind with services?
© SQE 2006
Security
© SQE 2006
• An SOA application is a collection of independent services, collaborating to provide valuable functionality.
• “Valuable” often suggests the need for security, for the authentication of users before access.
• Consider an application in which each service requires a different authentication approach and enforces different security policies – it’s a design, development, and testing nightmare.
Security
• Your organization needs a centralized SOA security management approach.
• Various strategies include:
– Ignore the problem
– Hide within a private network
– Whip up a home-brewed solution
– Buy this functionality from an experienced vendor
and it all has to be tested.
© SQE 2006
Interoperability
• Do you remember how Java’s
“Write once, run anywhere.”quickly became
“Write once, test everywhere.”
because of incompatible implementations?
• Well, we’re seeing the same thing in the SOA world. In fact, it’s so problematic that the Web Services Interoperability Organization (WS-I) was created to deal with the problems.
© SQE 2006
Interoperability
• Remember, just because it looks good in the glossy brochure
• And in the vendor’s demo,
• That doesn’t mean it will work for you.
© SQE 2006
Performance
• All this loosely-coupled, platform-independent stuff is not free.
• Major performance problems are often due to:
– Layer upon layer; abstraction upon abstraction
– Small services with large overhead
– Large services that are undersupported by hardware
– Services distributed across a networkwith its associated latency
© SQE 2006
It’s performance testing Jim,
but not as we know it.
Performance
• You’ll need a tool that provides an integrated look across servers, networks, and layers.
© SQE 2006
Testing Acquired Services
• What is the trustworthiness of acquired services?
– Functionality
– APIs
– Interface definition
– Publish, find, and bind
– Security
– Interoperability
– Performance
© SQE 2006
Or Are You
Testing Acquired Services
• How well did your vendor test? How do you know?
• “Classical” Acceptance Testing is required here (with all the work that it entails).
© SQE 2006
Summary Of Testing Challenges
• Today we’ve discussed the promises and testing challenges in SOA applications:
– Functionality
– APIs
– Interface definition
– Publish, Find, and Bind
– Security
– Interoperability
– Performance
© SQE 2006
Thanks
• Thanks for joining with us today.
• If I can be of assistance, or if you’d just like to chat, please contact me at
• And now, I’m pleased to introduce John Michelsen …
© SQE 2006
