PDQ, SOA, SLASOA Performance is a Critical Success Factor

Ralph Decker

Head of Performance Serviceline

7th May 2008

Agenda

» What is SOA

» Service Level Agreements (SLA’s) for SOA

» Key to SOA Performance Testing

» SOA Performance Testing Challenges

» Overcoming Challenges

» Methodology and Approach» Discovery

» Test Planning

» Automation and Tools

» Test Execution

» Monitoring

» Measurement and Analysis

» Summary

What is SOA

» Service Oriented Architecture» Architectural paradigm (pattern/model)» Variety of heterogeneous systems (dissimilar)» Different locations and owners» Web Services?

» SOA provides benefits in four basic categories:» Reduces expensive integration» Allows for more asset reuse» Increased business agility» Reduces business risk

SLAs for SOA

» A service-level agreement (SLA) is a formal contract between a service provider and a consumer » Service availability

» Performance

» Traffic levels

» Messages / queries per hour / minute / second

» Response time

» Rejected transactions

» Errors

» …

Poor Response Time

Missed SLA

Noncompliance with industry and government regulations

Key to SOA Performance Testing

» The key to successful performance testing in general require:» Understanding the application and the infrastructure

» Understanding the user/stakeholders of the application

» Generate accurate anticipated volumes of traffic

» Investigate the impact of the traffic on the application and systems under test

» SOA Adds Complexity to Performance Testing» Wide range of technologies

» Many different applications and usages

» Different hardware / infrastructure

» Knowledge of the application and the technologies

» Replicating traffic patterns

SOA Performance Testing Challenges

» SOA being a distributed environment, finding right skill sets who possess in-depth knowledge of involved platforms, applications, databases and any middleware.

» Assign appropriately skilled team of performance engineers to the test effort with knowledge of the systems to monitor and analyze the impact of testing

» An in-depth understanding of the service is required for adequate testing and evaluation

» A significant increase in testing activities and test assets (performance testing suites that include sophisticated harnesses and stubs) will be required at a service level

» Predicting the future usage of services to assist with performance, load, stress, scalability

» Test strategy differs from traditional testing and generally has to encompass many internal political factors e.g. ownership and responsibility

Overcoming Challenges

» To simplify performance testing for SOA applications break them down into the smallest components possible:

» Individual Service» Systems» Databases» Technology» Protocols» Messaging» Functionality

» Evaluate and analyze the performance of individual services based on components

Methodology and Approach

» Discovery: Narrow the testing event to the smallest element/service and understand the transaction, application service and the environment/systems

» Test Plan: Document the testing approach and the expected deliverables

» Automation: Develop automation to replicate transactions» Test Execution: Conduct testing generating traffic increasing the

traffic to pre-defined levels» Monitoring: Monitor the response time for the requests sent under

varying traffic levels and the impact of the traffic on the application and infrastructure under varying traffic levels

» Measurement and Analysis: Analyze the traffic patters to with the traffic/load

Discovery

» Identify the services and components to be included as part of the testing efforts

» Identify lower level and/or external services called by the services under test

» Identify the infrastructure hosting the services to be tested» Identify the messages sent to and received from the service that will be

automated for generating load» Narrow the “functionality” to a subset that will be used for the testing» Define data that will be used and how it will be validated» Define pass/fail criteria (usually response time or transactions per second)» Identify skills required for the testing effort» Determine SLA or load/transactions for the testing» Prioritize testing

Test Planning

» Test Plan - Document the Following:» Services and sub services

» Infrastructure

» Transaction

» Automation

» Data

» SLA requirements (load/transactions)

» Pass / fail criteria

Automation and Tools

» Automation» Subset of the functionality of the service

» Prepare data and mechanism to validate

» Tools to generate the transaction

» Proprietary SOA» Custom Test Harness

» Enterprise Tools» Loadrunner 9.1» Parasoft SOAtest» Green Hat GH Tester» Borland SilkPerformer SOA

» OpenSource Tools

Test Execution

» Load Testing (up to defined SLA transactions per second or other)» Stress Testing (up to service failure)» Volume Testing (introducing large amounts of data into system)» Reliability Testing (high levels of load over long periods of time)

Monitoring

» Load Size (transactions per second/other)» Throughput» Response Time» Hardware» OS» Disk» Web (for Web Services)» Application specific counters» Database

Measurement and Analysis

» Size of message» Response time» Throughput under load» CPU memory disk» Application specific counters

Summary

» NEED 6 BULLET SUMMARY HERE

Questions

Thank You