Implementing SOA using ESB: beyond hype

CSE@UNSW

Implementing SOA using ESB: beyond hype

Abdelkarim ErradiTrung Nguyen Kien

September 2004

Agenda

Service Oriented Architecture (SOA)Enterprise Service Bus (ESB)ESB Architecture and ComponentsESB design patterns ESB case study

Agenda

Service Oriented Architecture (SOA)Enterprise Service Bus (ESB)ESB Architecture and ComponentsESB design patterns ESB case study

Why should we care?

“By 2008, SOA will be a prevailing software engineering practice, ending the 40-year domination of monolithic software architecture (0.7 probability)”

Service Orientation is a New Computing Paradigm

Not as a new name for APIComponent

A genuine set of concepts with which we can construct new kinds of software

This is as significant if not more than Object Orientation

In particular SO forces us to think about enabling the same piece of code to be leveraged

by large numbers of consumers in unforeseen context

So… what is a service?"

There really are just two types of servicesMessage Producers

• Act and add stuff to messages

Message Consumers• Take stuff from messages and act on it

Messages flow through "pipelines"Pipeline is a sequence of servicesMessages grow and shrink on the way

Technology Agnostic Interaction

An approach to logic partitioning that maximizes the re-use of application-neutral services.

SOA Drivers

Outsou

rcing

Outsou

rcing

Desire

for

Desire

for

incre

ased

increa

sed

Reuse

Reuse

Business

BusinessProcess

ProcessAutomation

Automation

B2B Integration

B2B Integration

Constructing software in the web era (J2EE, .Net, …)

App ServerApp Server

Client Client

DB

CCI CCI CCI

ERP CRM

requestresponse

Model

ERPEAIEAI

b2b

Internet

CCI: Client Communication Interface

Controller

View

A Component now Becomes a Service Running Outside the Consumer Boundaries

DB

CCI CCI CCI

ERP CRM

Service Service Service

Registry

11register

ConsumerConsumer

SOAP SOAP SOAP

XML XML XML

33 invoke22

Discover and/or Bind

Policies

SOA is …

technologyproduct protocolstandard

Is notIs not

Set of policiespracticesframeworksarchitectural patterns

IsIs

SOA is … (Cont’d)

SOA enables application functionality to be provided and consumed as sets of services Services can be invoked, published and discoveredare abstracted away from the implementation using a single, standards-based form of interface.

Services vs. Components and Objects

SimilaritiesLike classes and components, services have one or more interfacesPublisher and consumer agree on the interface

DifferencesSOA is about schemas, not object typesSOA talks about messages, not method calls

RelationServices are built using classes and components

SOA Tenets : PEACE

Policy-based compatibility Explicitness of boundaries

We should opt-in to make our code accessible

AutonomyIndependent deployment, versioning and security

Contract ExchangeShare contracts and schemas not classes or objects

Clemens VastersClemens VastersCTO, newtelligence AGCTO, newtelligence AG

Don BoxDon BoxArchitect, Architect, Microsoft Corp.Microsoft Corp.

Applications as Fiefdoms

Application are like fiefdomsThey protect their citizens

StateData

They don’t trust foreigners -> every alien is subject to authentication and inspectionThe gate (service interface) in the only entrance into the fiefdomTransactions and security implications

From Components to (Web) Services

Requires a client library

Client / ServerExtendableStateless

FastSmall to medium granularity

Loose coupling via Message exchanges Policies

Peer-to-peerComposableContext independent

Some overheadMedium to coarse granularity

Shift To A Service-Oriented Architecture

Function orientedBuild to lastProlonged development cycles

FromFrom ToToCoordination oriented Build to changeIncrementally built and deployed

Application silosTightly coupledObject orientedKnown implementation

Enterprise solutionsLoosely coupledMessage orientedAbstraction

Source: Microsoft (Modified)

Achieving Loose Coupling

Loose coupling – reduction of dependencies between components that communicate with one another in order to allow them to operate and evolve independentlyRequires:

Coarse-grained communicationSupport for message evolutionSupport for asynchronous messages

RequesterRequester Runtime

ReceiverRuntime

Receiver

Requester Server Receiver Server

RequesterStore

ReceiverStore

Agenda

Service Oriented Architecture (SOA)Enterprise Service Bus (ESB)ESB Architecture and ComponentsESB design patterns ESB implementationsESB case study

What ESB?

ESB = MOM++ESB – combines MOM, Web services, transformation and routing intelligence Standards-based integration: all WS ‘standards’ work toward providing secure, reliable messaging workflows'Any to Any’ (A2A)Facilitate large-scale implementation of the SOA principles with suitable service levels and manageability.

Why ESB?

InflexibleTightly-coupled(Location & implementation aware)

Synchronous (RPC, availability dependent)

Fine-grained (Method level, development-time binding)

Many connections and data formatsNot scalableExtremely difficult to manage

Overcome Point-to-Point integration problems

Hub and Spoke PatternPoint to Point Hub & Spoke

Hub and spoke organizing principlesHub and spoke organizing principles

1. Don’t connect anything directly to

anything

2. Applications are autonomous and share

no databases directly

3. Knowledge of interconnections removed

from source and targets and moved to

the hub

BenefitsBenefits

1. Operational simplification

2. Adaptation to change

3. Reuse leverage

Enterprise Service Bus (ESB)

Enterprise messagingReliable, secure interactions across the extended enterpriseDistributed deployment architecture for high scalability

XML as native data type for document exchangeIntelligent routing of business transactions

Itinerary, content and rule-based routing Transformation of business data between applications

Service container end-pointsWeb services, JCA and Application Server supportUnified management and monitoring of entire services network

ESB Characteristics

XML oriented MessagingTransformationIntelligent routing services

Basic connectivity (Web Services, JCA Adapters, JMS)Service-oriented architectureSupport for highly distributed deploymentsManageabilityRobustnessScalability and PerformanceSecurityBreadth of connectivityDevelopment / Deployment toolset

ESB = MOM++

Web Services Broker

Multiprotocol ESB Web Services Controller

Web Services Application Manager

Actional AmberPoint Oblix (Confluent) Hewlett-Packard/

Talking Blocks Infravio Itellix

Computer Associates (Adjoin)

Hewlett-Packard/ Openview

Reactivity Service Integrity Westbridge

Fiorano Software's ESB IBM's Services Integration Bus (a

future product) IONA Technologies' Artix Kenamea's Web Messaging Platform KnowNow's Event Routing Platform Microsoft's Indigo (a future product) PolarLake's JIntegrator Software AG's EntireX Sonic Software's ESB SpiritSoft's Spiritwave WebV2's Process Coupler

Blue Titan's Network Director

Cape Clear's 4 Server Digital Evolution's DE

Management Server Flamenco Networks Primordial's Web

Services Network Systinet's Web

Services Bus webMethods' Fabric

Enterprise Service Bus

Enterprise Systems Management

Development

IntegrationManagement

Web Services Middleware

Managing Web Services: A New Vendor Taxonomy

Aspects of the Enterprise Service Bus - IBM

Enterprise Service Bus

POLICY

WSDL Described Services

Uncluttered Business logic

CustomizedRouting

Service Selection

DataLogging

FormatTranslation

Mediations

Queues Pub/Sub Req/Rep Assured Secure AvailableComms patterns and QoS

WBI Adaptors

MQ SOAP/HTTP JMS CEI .NETWide connectivity

Enterprise applications Enterprise data

Data Access ServicesApplication Access Services

IBM Software Offerings

Monitoring Services

IBM’s Business Integration Reference Architecture

Model, design, development, test tools

Common Runtime Infrastructure

WebSphere BI Modeler

WebSphere BI Monitor

Web Services Gateway WebSphere BI Event/Message BrokerWebSphere MQ

WebSphere BI Adapters DB2 Information Integrator Classic

WebSphere Studio

DB2 Information Integrator

WebSphere Business

Integration Server

WebSphere Business

Integration Connect

WebSphere Application Server

Enterprise Service Bus

ProcessServices

Community Integration Services

Application Services

Information Services

WebSphere Portal Server

User Interaction Services

WSDLWSDL

HTTPHTTP

HTTP-SHTTP-S

ActiveXActiveX

C/C++C/C++

JavaJava

SonicXQSonicXQDistributed Processing Framework

TransformationContent-Based

RoutingItinerary

Mgmt

JCAAdapterToolkit

WebServicesSupport

Distributed Management Framework

Distributed Services Framework / API

SonicMQSonicMQ

Dynamic Routing Architecture (DRA)

ParallelClustering

ActiveRouting

ConnectionMgmt

Explorer (GUI Administration)

End-to-End Security

Pub/Sub

Pointto

Point Bri

dg

es

EJB/J2EEEJB/J2EE

JDBCJDBC

3rd-PartyJCA

Adapters

3rd-PartyJCA

Adapters

MQSeriesMQSeries

TIBCOTIBCO

JMSJMS

FTP/SMTPFTP/SMTP

P4GLP4GL

SOAPSOAP

The SonicXQ Platform

Agenda

Service Oriented Architecture (SOA)Enterprise Service Bus (ESB)

ESB Architecture and ComponentsESB design patterns ESB implementationsESB case study

Is there any concrete architecture?

There’re different views of ESB implementation but general ideas are same.Let’s look at the implementation from few vendors.

Cape Clear

Cape Clear Business Integeration SuiteCape Clear StudioCape Clear ManagerCape Clear ServerCape Clear Data Interchange

IBM

WebSphere MQWebSphere Business Integration Message BrokerWebSphere MQ EveryplaceWebSphere Application Server

SpiritSoft

SpiritWare JMS BusSpiritWare Integration Server

Fiorano ESB

Convergence of Ideas

Built-in MOMQoSMultiple transport protocols (e.g.: HTTP, SMTP, JMS, MSMQ, etc)

Web servicesCommon interfacesUbiquitous

Transformation servicesInteroperabilityIntegration

Intelligent routingCommunication

Agenda

Service Oriented Architecture (SOA)Enterprise Service Bus (ESB)ESB Architecture and Components

ESB design patterns ESB implementationsESB case study

Agenda

Service Oriented Architecture (SOA)Enterprise Service Bus (ESB)ESB Architecture and ComponentsESB design patterns ESB implementations

ESB case study

Case studies

Supply Chain Management (SCM)International Investment Bank (IIB)

SCM

B2C situationCustomers review catalogues, and place orders onlineRetailer system orders from warehouses

B2B situationNo stock availableReplenishment order sent to manufacturers

SCM – ESB solution

Broker variation patternEncapsulate service invocations behind a single request.

SCM – ESB solution cont.

Choose appropriate product mapping based on:

Available productsProduct capabilities

IIB

Typical bank architectureProprietary specific applicationsComplex and expensive to maintain legacy solutions

IIB – ESB solution

ESB/JMS implementationSpiritWave Integration Server bridging among ESBs

Summary

SOA is an evolutionary thingNo radically new thinkingConsolidates good things

The Message is the Message!Think "message" instead of "call"Asynchronous messagingLoose coupling

ESB can helpIntegration HubService Invocation and Execution FrameworkService Locator

SOA is still far ahead of us

We still need to shape what SOA meansMore standards are requiredBPM and SOA will complement each other

We need lots of work to achieve and deliver the SOA value and go beyond “toy” applications of SOA

At best we are capable of delivering web services todayWe need ESB!

Standards “convergence” is now of primary importance

Recommendations

Technology alone is not enough!

Design with loose coupling in mindDecoupling is an investment in future change

The important issues for interoperability of autonomous computing system are around messaging and defining the interactions

Schema, protocol, meta-data, and versioning are essential to success

Resources

MSDN SOA Resourceshttp://msdn.microsoft.com/architecture/soa

Bishop, S., Hopkins, A., Milinski, S., Nott, C., Robinson, R., Adams, J., Verschueren, P. & Acharya, A. IBM Redbook 2004, Patterns: Implementing an SOA using an Enterprise Service Bus. Available: http://www.redbooks.ibm.com/redbooks/pdfs/sg246346.pdfCraggs, S., 'Best-of-breeds ESBs: Identifying best-of-breed characteristics in Enterprise Services Buses (ESBs)', EAI Industry Consortium, June 2003Sonic Software 2004, 'Distributed Service-Oriented Architecture: Delivering Standards-based Integration, the Advantages of an ESB'.

Q & A ?