Embed Size (px)
Citation preview
SOA: What‘s in it for testing?
Graham Bath
2
SOA*: What’s in it for Testing?Contents.
What is SOA?
Why do businesses use SOA?
What does SOA mean for testing?
Key messages
*SOA = Service Oriented Architecture
3
What is SOA? The basics.
SOA is a system architecture which is based on the use of services.
To understand SOA we first need to understand services
Services, in general:
provide businesses with pre-defined “ready to go“ functionality
can be used via the Internet (Web-Services)
may also be defined for access via local networks
4
Services (technically):
are reusable software entities.
share a formal contract (interface).
are loosely coupled.
are discoverable.
are location transparent.
Understanding Services
5
Web-Services communicate via standardized protocols Web Service Definition Language (WSDL) Simple Object Access Protocol (SOAP)
Frameworks are commonly used for building Web-Services Microsoft’s .NET Sun’s Java 2 Enterprise Edition (J2EE)
SOA: The technology.
6
SOA: What’s in it for Testing?Contents.
What is SOA?
Why do businesses use SOA?
What does SOA mean for testing?
Key messages
7
More flexibility in constructing applications (services = building blocks)
Applications can be constructed quickly (slot the building blocks together)
Costs are reduced by a more agile development process
Easy integration with legacy systems
Web Services: No need to develop own software
Web Services: No ownership and maintenance issues
SOA: The promise.
8
Applications which use SOA are “Ready to go“
Development of systems using SOA is “Kids Stuff“
Attitudes towards testing:
Extreme position: “Who needs it!“
Common position: “Just some basic functional testing needed“
SOA: The expectations of business owners.
9
Achieving the right balance.
Based on the promises of SOA, theexpectations of business ownersare entirely reasonable.
But are they realistic?
Testing has a job to do here…
A balance needs to be achieved between the high expectations created by SOA and the real quality risks to implemented systems.
10
SOA: What’s in it for Testing?Contents.
What is SOA?
Why do businesses use SOA?
What does SOA mean for testing?
Key messages
11
Overview of the principal risk areas
Performance & Scalability
Reliability
Interoperability
Security
Maintainability
12
Answer: We will never know unless we test!
User
“here it is“Questions: Are all these components configured for optimal performance? How many messages need to be passed to get the data I want? Are all the servers scaled correctly for my application
db server
“give me the data“
Performance & Scalability.
13
Risks: Performance & Scalability.
Risk drivers Web services are communication intensive “Excess baggage“- communications protocols need a lot of it Messages need to be encrypted / decrypted for security Dynamically finding services via directories is time-intensive Middleware is needed to maintain a platform-neutral concept Synchronized handshake can mean lots of waiting around SOA design: has it taken performance into account? (e.g. load-balancing)
What this means for testing Design reviews need to focus on the overall architecture (if there is one) Performance tests are a “must”
1. Response times may be longer than your user will accept. 2. The load on individual “shared” servers cannot easily be predicted in advance.3. Easy to build systems which are unusable.
14
Risks: Reliability.1. The reliability of the entire system is determined by the reliability of the
“weakest link in the chain“ 2. Finding that weakest link can be especially difficult for complex SOAs3. Potential for unstable systems4. Potential for data loss
What this means for testing: The reliability of the individual services is one of the principal risks where
web-services developed by third parties are used. We need to enquire about service level requirements for reliability
(e.g. Mean Time Between Failure) for the business application Technical and management reviews of the service management concept
(“Service-Governance“) are needed We must determine whether service failures are correctly handled
(service switching, monitoring, controlled reduction of functionality) We must determine whether backup strategies are in place and that the
procedures used have been tested. Performing failover and backup/restore tests may be complex
Who actually owns the data we may want backed up and restored? How can we simulate service failures? How can the tests be orchestrated?
15
Risks: Interoperability.1. Technical interoperability may not be possible due if incompatible protocols are
used by collaborating services (not all services may be communicating via WSDL)*.
2. Services may not “understand” each other at the functional (semantic) level due to unpublished add-ons and new versions introduced by Service Providers.
3. Developers may need to create adapters to permit interoperability. 4. Some of the principal benefits of SOA (re-use, easy construction, flexibility) may
be lost.5. The cost of maintaining Service-based systems increases
Automation
What this means for testing: Technical reviews need to verify that services can cooperate as intended Need to perform “Alive“-Tests frequently Need to perform “Smoke“-Tests frequently Compliance testing becomes a significant testing activity
*Note: The Web Services-Interoperability Organization (WS-I) has been set up
to minimize technical interoperability risks.
16
1. The Internet: Security risk Nr.1!2. Security threats can spread between Web Services quickly.3. Do we know who we are connecting to? 4. Do we have confidence that the Services we are connecting to are secure?
Risks: Security.
What this means for testing: Testing needs to provide confidence regarding the authentification of
Services. Any use of Web Services must consider the various security threats
presented by using the Internet (e.g. input buffer overflow) Independent security evaluations (static and dynamic) may be necessary. SOA Governance must establish a security policy (which Services may be
used and what proof of security must be made available).
17
Risks: Maintainability
1. Common view: “Maintenance* - that‘s someone else‘s responsibility”2. Failures may be difficult to reproduce, analyze and locate3. The “testability” of Service-based systems can be technically and (above all)
organizationally complex4. Requirement to repeat testing frequently5. Overall risk of high “costs of ownership”
* Maintainability consists of
four attributes according
to ISO 9126:- Analyzability- Testability- Changeability- Stability
What this means for testing: Automation of test case execution
(e.g. “live” tests). A common defect management must be set up
and made available to all participants. Establish a full-time defect manager with system-wide scope Acknowledge that not all contributors may want,
need or be able to use a common system.
18
SOA: What’s in it for Testing?Contents.
What is SOA?
Why do businesses use SOA?
What does SOA mean for testing?
Key messages
19
Key message:Use a life-cycle approach to testing SOA systems.
Testing a SOA system is a continuous activitywhich applies just as much to the operational phase of the SOA application as the development phase.
This requires a consistent and system-widetest and release management.
An SOA Governance policy needs to beset up to support these management functions.
The SOA Governance policy should beimplemented using tools, e.g.
HP, Service Test Management
“Service-Governance” plug-in for the HP Quality Center test management tool.
20
Key Message:Testing costs have become even more significant.
Development costs may be relatively low, depending on the amount of software re-use involved.
In relation to development, the cost of testing for systems using SOA may be relatively high,depending on the risks to be mitigated.
Business owners and project leaders need to be aware of this shift in relative costs andadjust existing management practices.
Project approvals based on costs and benefitsneed to focus more on testing costs and quality risks.
Change management needs first to askthe question “what would this change cost to test?”
Testing effort estimations based ona percentage of development costs are unlikely to be accurate.
21
Some tests may simply be too expensive to perform dynamically
Well conducted technical reviews always have been a cost-effective way to identify and mitigaterisks – this is particularly the case for SOA systems.
Benefit for all stakeholders (development, business, users, test)
The architecture is the principal area to be checked.
Can business objectives be achieved with SOA? Is a SO-Architecture technically feasible? Are high-risk quality attributes addressed? Are existing (legacy) applications suitable for definition as a Service? Can we expect an acceptable return on investment?
Key message:Reviews are of critical importance.
22
Key message:Test management can be a significant overhead. With SOA we are often faced with more complex
test management
With multiple suppliers a single, coherent managementstructure for the entire system may simply not exist.
Who is responsible for fixing defects? Are my points of contact defined and stable?
Many of the essential testing tasks are more complexto organize:
Who owns data? Is it consistent? How can we orchestrate the tests?
Establishing status (obsolescence, versioning, availability)
Establishing SOA Governance
23
Concluding message: It’s testing’s job to make the SOA risks visible. Are SOA systems really “ready to go”
straight from the developer’s workstation?
As we have seen, there are many different areas of risk which may significantly affect the quality of the operational system.
Businesses might be expecting too muchfrom SOA, in particular the cost of achievingand maintaining an acceptable level of quality.
It’s the role of testing to make these risks visible to all stakeholders and propose testing measures to reduce them.
25
Further reading
In English Rami Jaamour, „Ensuring SOA ROI“, Testing Experience,
December 2008
In German W. Beinhauer, M.Herr, A. Schmidt, „SOA für agile Unternehmen“,
2008, Symposion Publishing GmbH, ISBN 978-3-939707-14-1
26
Services are reusable software entities. Regardless of whether immediate reuse opportunities exist, services are designed to support potential reuse.
Services share a formal contract (interface) that defines the terms of information exchange; and that’s all. The actual implementation is invisible and irrelevant to service requestors.
Services are loosely coupled. They must be designed to interact easily with other services without needing modifications or state-dependent information.
Services are discoverable. They should allow their descriptions to be remotely discovered and understood by humans and service users who may be able to make use of the services’ logic.
Services are location transparent. Service requestors do not have to explicitly access a service on a specific server. Services can move from one location to another without affecting the requestors.
Understanding Services
