© 2006, iTKO, Inc. | All rights reserved. Achieving 99.99% Quality in SOA Applications – The “Headless” Dilemma John Michelsen StickyMinds Webinar November

© 2006, iTKO, Inc. | All rights reserved.

Achieving 99.99% Qualityin SOA Applications –

The “Headless” Dilemma

John Michelsen

StickyMinds WebinarNovember 7, 2006

Agenda

SOA development and QA trends

Strategy/best practices - the Three C’s

“Headless” Testing Fundamentals

LISA Intro, and LISA WS-Testing Offer

Who are we?

1999 . 2003 . 2005 . 2006 .2001.

1999 – iTKO Incorporates (Privately Held)

2001 – LISA under development – (4) Patents Pending

2003 – LISA 1.0 Released to Market

2004 – LISA 2.0

2006 - LISA 3 & 3.5

A History of Q

uality Leadersh

ip

2004 .

• Entered the market to deliver an SOA testing solution, LISA

• Current version 3.5 (2006)

• Key value propositions:• Increase Test Coverage• Lower Cost per Test• Faster Discovery & Resolution

Everyone should own quality™.

Mainframe Client Based SOA/Composite Apps

1980 . . . 1985 . . . 1990 . . . 1995 . . . 2000 . . . 2005 . . . 2010 . . . 2015

3270/CICS data stream testing

User-Interface oriented testing

Component, heterogeneous, distributed testing

Platform Evolution Causes Testing Evolution

The Promise of SOA

• Reduce integration cost• Through loosely-coupled forms of integration and industry standards

• Increase asset reuse• Build new business workflows from existing Services to form

composite applications

• Increase business agility• Better control business process definition and management to meet

customer needs

• Reduce business risk• Governance, compliance, and risk reduction through increased

business visibility and agility

Server (database)

BrowserUIs

Standardenterprise apps

ClientUIs

…before SOA

You had a standard delivery platform.And a standard infrastructure.

Transaction service

Datawarehouse

BI tools

Core app

PartnerReseller

Customer buyer

Business rules

Order management

Outsourcedsupplier

Transactionprovider

PartnerReseller

Customerbuyer

Your extendedenterprise

Division 1

Division 2

SOA promises a composite workflow

SOA was supposed to simplify extending workflows.

Transaction service

Datawarehouse

BI tools

Your AppWorkflow

Channel Partner

RMI objects

Customer company

CRM Web App

BusinessRules

Outsourced firm

Legacy App

SOAP objects

Web Services

.NET OrderingService

ContentDatabaseLegacy App

Web interface

Messagingservice

File System

Legacy Data

Division2

FinancialsMainframe

Division 1 Your Company

ESB

But heterogeneous technology is complex

Standards and components are still evolving

Composite apps have multiple owners

It’s Continuously Changing

Why A New Strategy?

• Few changes at the database level, but multiplies issues downstream

• For every middle tier change, there are hundreds of front-end impacts possible at the UI layer

• Limitless front end options• Any client, multiple customers• Mix of technologies –

XML/RPC, Swing, AWT, JS, DOM, now AJAX

• Services consume business logic

Database/mainframe= 1 change

UI level=1000x

Middle-tierlevel= 100x

Best Practices: “Three C’s” of SOA Testing

Complete Testing• Every heterogeneous layer of

architecture• Test UI verify in system of record• Reuse functional test for

performance testing

Collaborative Testing• Test early before UIs are created• Not just dev, business analysts and

QA should verify processes

Continuously Test• Regress on existing functionality• Add testing of new services to

existing testing workflows

Complete >single component all technologies

Continuous >Phased tests constant testing

Collaborative >Dev/QA test silos everyone tests, every phase

Data / Applications

Process and services layer

Complete: SOA Testing across every layer

Presentation layer

Integration layer

Web UI

RMI

DatabaseLegacy App File System

ASP.NET

MQ

Custom API

Swing

J2EE

SOAP/XML

CORBA

ESB

BPM .NET WS

AdditionalApps Now& Future



CORBA App

Typical Trading System Example

OrderingSwing App

ES

B

ProcessingWeb App

BusinessProcessAppA

BusinessProcess AppB


Database

DB

RMI

Data is correlated in here.

1

9

App Server

EJB, WS,POJO

Collaborative: SOA Requires Agile

QA expands dev tests and create functional and load

tests

Complete Test CoverageLower Cost per TestFaster Discovery & Resolution

Developers unit test and jump-start QA

Business Analysts outline a business process and ‘not implemented’ test

cases

Production test their implementation and report

issues to support

Support rapidly debug issues and communicates appropriately

Collaborative Testing in an SOA environment

BPM Validation

Orchestration Tests

Service Validation

Component Tests

Service 3

Dev/QE

Order Acquisition

Check InvCheck Credit

Place Order

QA/BA

Place OrderService 1 Service 3

Service 2

QE QA Con

tinuo

us S

OA

Tes

t Pla

tform

Continuous: SOA is never done

• SOA development is constantly evolving

• Project-based development had a “test phase”

DatabaseCreate

componentsCreate UI

AcceptanceTest

DeployAppUnit test Integration

New service

Database Create UI

Database

Create UI

Create UI

New service

Servicechanged

New service

Serviceretired

ESB

Continuous: Minimize Unintended Consequences

APP 1

Database

Registry MDM Transactions

InternalService

LegacyApp

PartnerService

APP 2

???

???

?

?

NEW

ESB

Continuous: Minimize Unintended Consequences

APP 1

Database

Registry MDM Transactions

InternalService

LegacyApp

PartnerService

APP 2

A data service is currently failing

due to a change in the internal

service

ContinuousTesting

Continuous: SOA at 5-9’s

Business Investment

Ava

ila

bil

ity

100%99.999%systemuptime Physical

Infrastructure

Sca

lab

ilit

y

100%

99.999%applicationavailability

Inte

gri

ty100%

?????%functionalintegrity

ApplicationInfrastructure

BusinessApplications

Fault Tolerant HWDiesel PowerRedundant Bandwidth

App ServersTransaction MonitorsClustered Services

SOA-aware Dev/QA ProcessComprehensive Testing ToolsContinuous Testing Service

Availability to the business is the product of all three

Continuous Levels: Build & Deploy

Component/Service Teams

Development

QA/QE

Pre-production

Production

product component product component

Dev build

test bed

Staging

Production

Dev build

test bed

Con

tinuo

us S

OA

Tes

t Pla

tform

Deployment Level

BuildLevel

So how do we defeat the “Headless” dilemma?

Requirements are King -- On-line Music Scenario

• Use Case: (UC1) Customer with existing account does a search for ‘Rush’, selects the Grace Under Pressure tour DVD and uses the 1-click purchase feature to buy.

• Business: (BR1) We need a flexible mechanism to change these criteria at our whim. After discussion: for now partners get 20% of net proceeds on their product sold on the first $100,000 / mo, 10% thereafter.

• PSR: (PSR1) Search can’t take any longer than 3 seconds for a user on a typical broadband connection.

• Policy: Punt for now. This is security, interop standards, corp dev standards, etc.

• Functional: (FR1) Search must include results from CDs, DVDs, Books, and Memorabilia, with CDs on the top (UC1).

• Technical: (TR1) Create a search service that aggregates CD, DVD, Book, and Mem data (FR2). (TR2) Search service execution must be under 2 seconds (PSR1).

Service Level Guidelines

Service Validation

Component Tests

Service 3

Dev/QE

• UC & BR: Validation mechanism

• PSR: Load/performance test at the service level

• FR: Likely 80% of your requirements can be nailed right here

• TR: Dev test at the component level

UC=Use Case, BR=Business Requirements, FR=Functional Requirements, TR=Technical Requirements

Orchestration Level Testing Guidelines

• UC & BR: Not usually

• PSR: Often the best way to prove PSR

• FR: Invoke a behavior, validate outcomes

• TR: Simulate unstable/unavailable services

Orchestration Tests

Place OrderService 1 Service 3

Service 2

QE QA


Composite/BPM Level Testing Guidelines

• UC: Primary focus, performs a cause that we validate at the service/component level

• BR: Many times same as UC, sometimes not relevant

• PSR: A few PSR scenarios proved at this level, but continuous testing is the key here

• FR & TR: Hopefully very fewBPM Validation

Order Acquisition

Check InvCheck Credit

Place Order

QA/BA


Typical SOA Test Map -- Dramatically Simplified

Content

Presentation Layer

Content Mgmnt

Offers & Specials

Search OrderMgmnt

CommissionRules Svc

CD DVDBooks Mem

CRM System

Media Delivery Billing

Media Repo

AP

CRMFacade

Batch Feeds

UC1

UC1

UC1

UC1: Search & Buy


Content

Presentation Layer

Content Mgmnt

Offers & Specials

Search OrderMgmnt

CommissionRules Svc

CD DVDBooks Mem

CRM System


Media Repo

AP

CRMFacade

Batch Feeds

BR1*

BR1

BR1: Commissions


Content

Presentation Layer

Content Mgmnt

Offers & Specials

Search OrderMgmnt

CommissionRules Svc

CD DVDBooks Mem

CRM System


Media Repo

AP

CRMFacade

Batch Feeds

TR2*

PSR1PSR1: 3 sec search


Content

Presentation Layer

Content Mgmnt

Offers & Specials

Search OrderMgmnt

CommissionRules Svc

CD DVDBooks Mem

CRM System


Media Repo

AP

CRMFacade

Batch Feeds

FR1: Search is aggregate, CDs top results

FR1


Content

Presentation Layer

Content Mgmnt

Offers & Specials

Search OrderMgmnt

CommissionRules Svc

CD DVDBooks Mem

CRM System


Media Repo

AP

CRMFacade

Batch Feeds

TR1 & 2

TR2: 2 sec search

TR1: Create search on 4 repos

TR1

The Three Dimensions of Quality maturity

Complete >single component all technologies

Continuous >Phased tests constant testing

Collaborative >Dev/QA test silos everyone tests, every phase

LISA meets SOA Testing Challenges

• LISA is Complete - Test heterogeneous SOA• LISA can test every tier of an application, regardless of location• LISA can test every service technology with one tool, one test• LISA’s framework approach creates bridges to legacy services

that means ‘you can get there from here’

• LISA is Collaborative – Everyone owns quality• LISA shines at testing all components before UI even exists, as

well as integrating with the team’s ALM lifecycle and process• Unit, regression, system, load, and monitoring in one tool• Leverages diverse skill sets in Dev, QA/QE, and BA roles

• LISA is Continuous – Enables Agile SOA• Tests constantly evolving, interdependent systems during and

after deployment• Traps for the “what-if” unintended system-wide consequences of

making changes or corrections to components.

LISA WS-Testing $5,000 Off for next 50k users

• LISA WS-Testing Edition is THE tool for Web Services testing

• All our web services goodies are in there:• WSDL-based, point and click• WS-I Interop testing• Functional testing and service simulation• UI and Command Line execution

Thank you for your interest!

? ? ?

• iTKO Web Site Resources• URL: http://www.itko.com or http://www.ws-testing.com• White papers• Support FAQs, Tech Notes, and Forums (once registered)• Download WS-Testing or start a larger evaluation

• More from me• See above• Blog: http://itko.blogspot.com• Email: Start with [email protected]; I’m [email protected]

Documents

© 2006, iTKO, Inc. | All rights reserved. Achieving 99.99% Quality in SOA Applications – The “Headless” Dilemma John Michelsen StickyMinds Webinar November