LiquidHub Lunch & Learn Demystifying SOA: Concepts & Best Practices

our business revolves around you

LiquidHub Lunch & Learn Demystifying SOA: Concepts & Best

Practices

Prepared by: Ray Bordogna

Partner & Chief Strategy OfficerLiquidHub, Inc.

Presented: Tuesday, February 26, 2008

2

Executive Briefing Outline

About LiquidHub, Inc.Section I:

The Business ProcessSection II:

The IT EnvironmentSection III:

SOA Introductory ConceptsSection IV:

SOA Implementation ConsiderationsSection V:

SOA Proof of Concept (POC)

3

About LiquidHub, Inc.

LiquidHub is a systems integrator & technology consultancy focused on enabling the Agile Enterprise through our Strategy, Applications, Data, and Infrastructure solutions and an engagement lifecycle of planning, execution, and management.

Specific solutions include Enterprise & SOA, Enterprise Integration, Enterprise Portals, Content Management, and scalable Applications and Security Infrastructures.

With offices in Philadelphia, Boston, and Hyderabad, India, our more than 475 associates serve clients in Life Sciences, Financial Services & other key industries globally, at our sites or theirs.

LiquidHub’s Enterprise Services Transformation Roadmap (ESTR) helps organizations plan for technology simplicity and reusability, providing a roadmap to the Agile Enterprise. ESTR provides our clients with a clear process for evaluating business needs, identifying existing technology & process assets, and planning the implementation and integration of new technologies in a way that ensures technology reuse and lower total cost of ownership.

#339 in 2005 List

4

Enterprise Services Architecture

The LiquidHub Enterprise Services Architecture (ESA) Framework

Enterprise Architecture Service Orientation

Frameworks (e.g., Zachman)

Planning Approaches (e.g., EAP)

Reference Models (e.g., TOGAF)

Governance Models (e.g., TOGAF)

ESA Reference Model

ESTR: (Integrated View)

Enterprise Planning Methodology

Solution Methodology (Project)

integrates

Services Definition:

1. Loosely Coupled

- Abstracted

- Platform Independent

2. Designed & Built for Agility

3. Articulating

4. Meaningful to the Service Consumer

5. Contract-based

6. Standards-based

7. Discoverable

Service-Oriented Principles:

1. Federated

2. Traceable

3. Aligned with Business & IT

4. Evolutionary

5. Managed

6. Application Neutral

5

The LiquidHub Enterprise Architecture Framework

Business Architecture

Business Strategy

Application (Services) Architecture

Data Architecture

Infrastructure Architecture

CAPABILITY 1

Resource Architecture

Financial Architecture

Business IT

Business IT

Device 2

Device 13

Expense: $XX Capital $YY Development: $XX Maintenance: $YY

DB 1

DB 12

Process 1 Process 6

Process 12

Service 1 Service 3 Service 6 Service 7

What markets do we compete in and

how are we different?

What processes support our capabilities?

What applications (services) enable our processes?

What data support our business?

What is the foundation of our

business?

What is our optimal resource

allocation?

What is our optimal investment

portfolio?

6

TheAgile

Enterprise

YourCurrent

Enterprise

ProgramManagement

PlanGovernance

Model

Business Architecture

Model

Business Service Domain Model

Composite ApplicationsOrchestration & Choreography

BusinessServicesDesign

SDLCMethodologyRefinement

ProjectPortfolio

Management

BusinessProcess

Optimization

BusinessServices

Implementation

BusinessSolutionDeliveryRoadmap

TechnologyServices

Reference Model

InformationArchitecture & Taxonomy

BusinessProcessModel

EnterpriseData Model/Semantic

Model

MetadataRepository

Design

ApplicationIntegration

Platforms Blueprint

SecurityArchitecture

Network Services

Architecture

NetworkInfrastructure

Services

ApplicationManagement

Services

Application (SOA)

Platforms

Federated Data Source

and Data Services

The LiquidHub Enterprise Services Transformation Roadmap (ESTR)

Program Management

Enterprise Business Services

Information Management

Technology Shared Services

Phases

WorkStreams

Envision(Strategy and Plan)

Engineer(Architect/Design)

Execute(Develop/Implement)

7

The LiquidHub ESA Reference Model (Portfolio of Services)


Enterprise Presentation Services

Information Services

Infrastructure Services

Enterprise Application Services

Enterprise Platforms

Network Backbone & Topology

Routing & Security Architecture

Storage ArchitectureNetwork Resource ManagementFile and Print Access

Services

File and Print AccessServices

Security & Access Management

DirectoryServices

Messaging & Calendaring

Mobile, Wireless & Telephony

Development & Deployment

MonitoringServices

Integration PlatformsIntegration Platforms

Application PlatformsApplication Platforms

Client ServicesClient Services

Personalization ServicesPersonalization Services

Business Process IntegrationBusiness Process Integration

Core Application ServicesCore Application Services

Business IntelligenceBusiness Intelligence

Data InfrastructureData Infrastructure

Data Access ServicesData Access Services

Digital Asset ManagementDigital Asset Management

Business Applications

Business Process Services Information Assets

CustomerCustomer

OrdersOrders

FinanceFinance

Business Services

Domain 1

Business Services

Domain 2

Enterprise Packaged Applications

Business Architecture & Business Process Models



Section I

The Business Process

9

An Enterprise can be Viewed as a Collection of Processes

An Enterprise

Inputs Outputs

Resources

10

In fact, the essence of strategy revolves around processes

The business process movement has many parents, but none has been more influential than Michael E. Porter, who has given us the ideas that dominate today's thinking about strategy and the role of processes in achieving competitive advantage.

Competitors, Porter argued, would always try to copy your innovations and "best practices." What they couldn't easily copy was a well integrated Value Chain in which every activity fit together to achieve a well thought out strategy. As Porter explained; "The essence of strategy is choosing to perform activities differently than rivals do.“

Porter suggests that smart senior executives think in terms of processes. In effect, one strategic goal of the organization should be to create value chains and processes that are unique and that fit together to give the organization a clear competitive advantage that is difficult for rivals to duplicate.

Adapted from “What is Strategy,” HBR, Porter 1996

Limited Passenger

Service

Frequent & Reliable

Departures

Lean, Productive Ground &

Gate Crews

High Aircraft Utilization

Very Low Ticket Prices

Short, Point to Point Routes

to 2nd Airports

11

But, what about the capability to Introduce and/or Change processes?

Enterprise A Enterprise A*

How fast?

How much $?

The ability to implement new processes and change existing processes in a fast, cost-effective manner facilitates competitive advantage and is the essence of

‘agility.’


Section II

The IT Environment

13

Issue #1: The n(n-1) Integration Complexity

Consider the n(n-1) integration problem. Many organizations face integration problems of some sort; perhaps because of a corporate merger, a new business alliance or just the need to interconnect existing systems. If n application systems must be directly interconnected, the process will produce n(n-1) interfaces. Note that each arrowhead represents an interface.

This set of 5 applications requires 20 interfaces; adding a 6th application would require 10 new interfaces.

And to further increase complexity, one must modify code in each of the existing applications to include the new interfaces, generating substantial testing costs.

To reduce this cost and complexity, a solution that produces the minimum interfaces is required.

Application 1

Application 2

Application 3

Application 4 (.NET)

Application 5 (J2EE)

Systems (n) # of Interfaces: n(n-1)

1 0

2 2

3 6

4 12

5 20

6 30

14

Issue #2: Redundant & Non-Reusable Programming

Over time, many enterprise application portfolios have increased in redundancy due to business combinations and business unit independence. As a result, many organizations deal with redundant applications – or applications with functions that can’t easily be reused.

Commonly, in decentralized organizations, business unit independence hinders any coordinated effort to create reusable functional assets or services. Collectively, this redundancy increases both cost and time to market to deploy new products or services, because changes have to be made in each application or system that is affected. This lack of reuse ultimately requires more resources – and often more time – to deliver new applications.

Application 1 Application 2

Application Portfolio

1 2

Application 3 Application 4

1 1

11

2


Section III

SOA Introductory Concepts

16

Service-Oriented Architecture (SOA) is a multi-purpose buzzword

A service-oriented architecture is a collection of services that communicate with each other. The services are self-contained and do not depend on the context or state of the other service. They work within a distributed systems architecture. Source: DMReview.com

SOA is an architectural style whose goal is to achieve loose coupling among interacting software agents. A service is a unit of work done by a service provider to achieve desired end results for a service consumer. Both provider and consumer are roles played by software agents on behalf of their owners. Source: XML.com

A service-oriented architecture is essentially a collection of services. These services communicate with each other. The communication can involve either simple data passing or it could involve two or more services coordinating some activity. Some means of connecting services to each other is needed. Source: service-architecture.com

In computing, the term Service-Oriented Architecture (SOA) expresses a perspective of software architecture that defines the use of services to support the requirements of software users. In a SOA environment, nodes on a network[1] make resources available to other participants in the network as independent services that the participants access in a standardized way. Most definitions of SOA identify the use of Web services (i.e. using SOAP or REST) in its implementation. However, one can implement SOA using any service-based technology. Source: http://en.wikipedia.org/wiki/Service-oriented_architecture

Is it an Approach?

Is it a Framework?

Is it an Application

Architecture?

Is it a Reference Model?

SOA refers to the re-engineering of IT systems and development that makes use of reusable chunks of software, aligned to business processes.Source: Diagonal IntegratorsSOA is one of the most popular architectural paradigms today, but without any standardized reference model. It is an architecture that provides seamless Enterprise Information Integration between loosely coupled distributed applications or services over the network. Source: ASP Alliance

Service-oriented architecture (SOA) is an evolution of distributed computing based on the request/reply design paradigm for synchronous and asynchronous applications. Source: Javaworld.com

Service-oriented architecture (SOA) is a design methodology aimed at maximizing the reuse of application-neutral services to increase IT adaptability and efficiency. Source: http://dev2dev.bea.com/soa/

SOA is an architectural style for building software applications that use services available in a network such as the web. It promotes loose coupling between software components so that they can be reused. Applications in SOA are built based on services. Source: Sun.com

Is it an Enterprise

Architecture?

Is it Software Architecture?

17

Basic Concept: The Service

ServiceContent

Service Interface

Definition of a Service: A service is a unit of logic expressed in software that is designed for re-use by other software elements in different execution contexts.

Service content provides or encapsulates logic such as a business process, a technical feature, a stateless computation, etc…

Provides service identification, definition of parameters, and conventions concerning passing the service results back to the consumer

18

Fundamental SOA consists of services, descriptions & messages

Process Step

sub-process

PROCESS

service

service

service

Services Encapsulate Logic

In SOA, units of logic are known as services.

To retain their independence, services encapsulate logic within a distinct context. This context can be specific to a business task or some other logical grouping.

The concern addressed by a service can be large or small. Therefore, the size and scope of the logic represented by the service can vary.

A collective is composed of several services.

Service A Service B

service description for Service

B

Self-governing message

1 2 3

1

Service Encapsulation

Service Composition

Services Relate Through Service Descriptions

In SOA, services are aware of each other through the use of service descriptions.

A service description establishes the name of the service and the data expected and returned by the service.

The manner is which services use service descriptions results in a relationship classified as loosely coupled.

2

Services Communicate Through Messaging

Messages are “independent units of communication” which need to be outfitted with enough intelligence to self-govern their parts of processing logic.

Similar to services, messages must be autonomous since after a service sends a message on its way, it loses control of what happens to the message thereafter.

3

19

The Nature of a Service

A service can be a simple business capability:

getStockQuote getCustomerAddress checkCreditRating

A service can be a complex business transaction:

openAccount: verifyCustomerIdentity createCustomerAccount

commitInventory sellCoveredOption scheduleDelivery

A service can be a system service: logMessageIn authenticateUser

This may seem like an artificial distinction of services.

The distinction is made to help quantify the level of granularity.

Services may be low-level (i.e., fine-grained) or complex high-level (coarse-grained) functions and there are tradeoffs in:

Performance Flexibility Maintainability Reuse

The level of granularity is a statement of a service’s functional richness.

Adapted from “Migrating to a Service-Oriented Architecture,” IBM, 2005

20

“Granularity”

Adapted from “ SOA: Principles of Service Design,” Erl, 2007

Consolidate Invoices

Get Invoice

Get Header

Retrieve PO Records

Perform Calculations

View Client History

The term “granularity” is most commonly used to communicate the level of (or absence of) detail associated with some aspect of software program design.

Within a service, different forms of granularity exist, all of which can be impacted by how service-orientation design principles are applied.

Functional Granularity refers to the potential breadth of the service’s functional scope. For example, “Consolidate Invoices” is a coarse grained service.

A fine-grained service will have less work to do than a coarse grained

service.

Data Granularity refers to the quantity of data a service needs to exchange in order to carry out its function. There has been a tendency for services implemented as Web Services to exchange document-centric messages – messages containing entire information sets or business documents. Constraint Granularity refers to the amount of detail with which a particular constraint is expressed. The schema or data model representing the structure of the information being exchanged by a service can define a series of specific validation constraints (data type, data length, data format, allowed values, etc…) for a give value and would represent a fine-grained constraint as opposed to a coarse-grained level of constraint granularity that would permit a range of values with no predefined length or formal restrictions.

21

These Design Issues Require (Service-Oriented) Principles

Service A Service B


B


How should services be designed?

How should service descriptions be designed?

How should the relationship between services be defined?

How should messages be designed?

22

The Service-Orientation Design Paradigm & Principles

a design paradigm is an approach to designing solution logic.

when building distributed solution logic, design approaches revolve around a software engineering theory known as the "separation of concerns” which states that a larger problem is more effectively solved when decomposed into a set of smaller problems or concerns. This gives us the option of partitioning solution logic into capabilities, each designed to solve an individual concern. Related capabilities can be grouped into units of solution logic.

The fundamental benefit to solving problems this way is that a number of the solution logic units can be designed to solve immediate concerns while still remaining agnostic to the greater problem. This provides the constant opportunity for us to reutilize the capabilities within those units to solve other problems as well.

What distinguishes service-orientation is the manner in which it carries out the separation of concerns and how it shapes the individual units of solution logic. Applying service-orientation to a meaningful extent results in solution logic that can be safely classified as "service-oriented" and units that qualify as "services." To understand exactly what that means requires an appreciation of the strategic goals of service-oriented computing combined with knowledge of the following service-orientation design principles:

1.Standardized Service Contract

2.Service Loose Coupling

3.Service Abstraction

4.Service Reusability

5.Service Autonomy

6.Service Statelessness

7.Service Composability

8.Service Discoverability

Service Oriented Design Principles:


23

Principle #1: Standardized Service Contract

"Services within the same service inventory are in compliance with the same contract design standards." Services express their purpose and capabilities via a service contract. The Standardized Service Contract design principle is perhaps the most fundamental part of service-orientation in that it essentially requires that specific considerations be taken into account when designing a service’s public technical interface and assessing the nature and quantity of content that will be published as part of a service’s official contract. A great deal of emphasis is placed on specific aspects of contract design, including the manner in which services express functionality, how data types and data models are defined, and how policies are asserted and attached. There is a constant focus on ensuring that service contracts are both optimized, appropriately granular, and standardized to ensure that the endpoints established by services are consistent, reliable, and governable.


Figure: Using Web service contract documents (WSDL, XML schema, and WS-Policy definitions) as an example, this illustration highlights the areas that are typically affected by the application of this design principle.

<definitions>

<types>

<message>

<parts>

<portType>

<Policy>

<ExactlyOne>

<All>

assertions…

<schema>

<element>

<complex type>

WSDL WS PolicyXML

Schema

24

Key Concept: The Service Contract

WSDL

Definition: A Service Contract establishes the terms of engagement, providing technical constraints and requirements as well as any semantic information the service owner wishes to make public

Figure: A Web Service can be comprised of the following service description documents: WSDL Definition, XML Schema Definition and WS Policy Description.

The term “technical service contract” is used simply to refer to service description documents that are programmatically consumed at runtime. Note that SLAs and other human consumable service description documents can also be part of the service.

XML Schema

WS Policy

Technical Web Service Contract

Service Contract

SLA

Historical Context: In the past, technical contracts have commonly been represented by a form of technical interface known as the API. The Interface Definition Language (IDL) and Abstract Syntax Notation (ASN.1) were frequently used to express technical contracts for remote invocation frameworks such as those based on Remote Procedure Calls (RPC).

Web Services: established a non-proprietary distributed communications framework that introduced the Web Services Description Language (WSDL) as the core part of the service contract. The XML schema language is used to define the data model for messages exchanged via Web services and the WS-Policy language facilitates policy assertion definition and attachment to various parts of the WSDL

25

Principle #2: Service Loose Coupling

"Service contracts impose low consumer coupling requirements and are themselves decoupled from their surrounding environment." Coupling refers to a connection or relationship between two things. A measure of coupling is comparable to a level of dependency. This principle advocates the creation of a specific type of relationship within and outside of service boundaries, with a constant emphasis on reducing (“loosening”) dependencies between the service contract, its implementation, and its service consumers. The principle of Service Loose Coupling promotes the independent design and evolution of a service’s logic and implementation while still guaranteeing baseline interoperability with consumers that have come to rely on the service’s capabilities. There are numerous types of coupling involved in the design of a service, each of which can impact the content and granularity of its contract. Achieving the appropriate level of coupling requires that practical considerations be balanced against various service design preferences.

26

Principle #3: Service Abstraction

Service contracts only contain essential information and information about services is limited to what is published in service contracts. Abstraction ties into many aspects of service-orientation. On a fundamental level, this principle emphasizes the need to hide as much of the underlying details of a service as possible. Doing so directly enables and preserves the previously described loosely coupled relationship. Service Abstraction also plays a significant role in the positioning and design of service compositions. Various forms of meta data come into the picture when assessing appropriate abstraction levels. The extent of abstraction applied can affect service contract granularity and can further influence the ultimate cost and effort of governing the service.

27

Principle #4: Service Reusability

Services contain and express agnostic logic and can be positioned as reusable enterprise resources. Reuse is strongly emphasized within service-orientation; so much so, that it becomes a core part of typical service analysis and design processes, and also forms the basis for key service models. The advent of mature, non-proprietary service technology has provided the opportunity to maximize the reuse potential of multi-purpose logic on an unprecedented level. The principle of Service Reusability emphasizes the positioning of services as enterprise resources with agnostic functional contexts. Numerous design considerations are raised to ensure that individual service capabilities are appropriately defined in relation to an agnostic service context, and to guarantee that they can facilitate the necessary reuse requirements.

28

Principle #5: Service Autonomy

"Services exercise a high level of control over their underlying runtime execution environment." For services to carry out their capabilities consistently and reliably, their underlying solution logic needs to have a significant degree of control over its environment and resources. The principle of Service Autonomy supports the extent to which other design principles can be effectively realized in real world production environments by fostering design characteristics that increase a service’s reliability and behavioral predictability. This principle raises various issues that pertain to the design of service logic as well as the service’s actual implementation environment. Isolation levels and service normalization considerations are taken into account to achieve a suitable measure of autonomy, especially for reusable services that are frequently shared.

29

Principle #6: Service Statelessness

"Services minimize resource consumption by deferring the management of state information when necessary." The management of excessive state information can compromise the availability of a service and undermine its scalability potential. Services are therefore ideally designed to remain stateful only when required. Applying the principle of Service Statelessness requires that measures of realistically attainable statelessness be assessed, based on the adequacy of the surrounding technology architecture to provide state management delegation and deferral options.

30

Principle #7: Service Discoverability

"Services are supplemented with communicative meta data by which they can be effectively discovered and interpreted." For services to be positioned as IT assets with repeatable ROI they need to be easily identified and understood when opportunities for reuse present themselves. The service design therefore needs to take the “communications quality” of the service and its individual capabilities into account, regardless of whether a discovery mechanism (such as a service registry) is an immediate part of the environment.

31

Principle #8: Service Composability

"Services are effective composition participants, regardless of the size and complexity of the composition." As the sophistication of service-oriented solutions continues to grow, so does the complexity of underlying service composition configurations. The ability to effectively compose services is a critical requirement for achieving some of the most fundamental goals of service-oriented computing. Complex service compositions place demands on service design that need to be anticipated to avoid massive retro-fitting efforts. Services are expected to be capable of participating as effective composition members, regardless of whether they need to be immediately enlisted in a composition. The principle of Service Composability addresses this requirement by ensuring that a variety of considerations are taken into account.

32

Summary of Service-Orientation Principles

Loose Coupling: Services maintain a relationship that minimizes dependencies and only requires that

they retain an awareness of each other vs. Tight-Coupling: the functionality of each component heavily depends on the functionality

implemented by other components. Often such components cannot be used independently of the overall system. That is the design is component-based, but the components are not stand alone.

Service Contract: Services adhere to a communications agreement, as defined collectively by one or

more service descriptions and related documents Autonomy:

Services have control over the logic they encapsulate Abstraction:

Beyond what is described in the service contract, services hide logic from the outside world

Reusability: Logic is divided into services with the intention of promoting reuse

Statelessness: Services minimize retaining information specific to an activity

Discoverability: Services are designed to be outwardly descriptive so that they can be found and

assessed via available discovery mechanisms Composability:

Collections of services can be coordinated and assembled to form composite services

33

Service-Orientation & the Enterprise

The collective logic (or processes) that defines and drives the enterprise is an ever-evolving entity constantly changing in response to external & internal influences

From an IT perspective, this enterprise logic can be divided into 2 important halves: business logic and application logic

Business Logic: is structured into processes that express business requirements

Application Logic: is an automated implementation of business logic organized into technology solutions

Services establish an abstraction layer wedged between traditional business & application layers.

Services are developed & deployed in proprietary environments, wherein they are individually responsible for the encapsulation of specific application logic.

Bu

sin

ess

Pro

cess

La

yer

Serv

ices

Layer

Ap

plic

ati

on

Layer

Application A (.NET)

Application B (J2EE)

Application C (Legacy)

Business Logic

Application Logic

34

Business Processes & Services

Business Process Orchestration

BusinessServices

TechnicalServices

Get customer

details

“Open account for customer”

Locateaccount

type

Add account tocustomer

Locatecustomer

record

Checkcustomer

status

Presentation – user interface

Create Customer-Accountrecord

Lookupaccount

typetable

Retrieveaccountdetails

Business Process

Coarse Grained

FineGrained

Service Orchestration(Process

Orchestration)

Adapted from ANZ Banking Group Australia

35

SOA & Web Services

SOA can be implemented without Web services, and Web services can be used for non-SOA (e.g. RPC) interactions. However, Web services delivers key standards for implementing SOA.

The WS-* family scales to meet integration challenges intra-enterprise (enterprise application integration [EAI]) and inter-enterprise (business to business [B2B]).

XML is an ideal candidate for loosely coupled inter-application data sharing. XML is not self-describing, but XML Schema can be be used to constrain message layout and content.

RPC interactions Binary XML

Services architecture Service contract Message based Service directory Protocol independent Coarse grained & document centric

Web services specs WSDL SOAP & XML UDDI HTTP Doc literal binding

Process orchestration (BPEL)

Web services“An Implementation”

SOA“The architecture”

Web Services is the stack of standard web technologies required at both consumer and provider ends to implement the pipe for shipping XML messages between them.

You don't have SOA until you build/buy services and compose them to implement business functionality.

36

In theory, SOA does not equal Web Services but is there another model?

Web Services

UDDI

WSDLSOAP

serves as a registry for

is accessed using enables discovery of

is accessed using describes

is used to invoke operations defined by

Enterprise Services

?

??

serves as a registry for

is accessed using enables discovery of

is accessed using describes

is used to invoke operations defined by

Figure 1. Web Services Model

Figure 2. ? Model

SOA is often discussed in conjunction with Web services, but

the 2 are not synonymous. In theory, SOA does not require Web

services, and simply implementing Web services does not result in an

SOA. However, Web services are the 1st standard for service-

oriented computing to gain the near-universal vendor support. By

standardizing essential elements of SOA, Web services remove the

risk of having to be on a particular technology (e.g., CORBA).

37

1st and 2nd Generation SOA

ServiceRequestor

ServiceProvider

ServiceRegistryFIND:

Discover & retrieve WSDL

BIND & EXECUTE

PUBLISH

1st generation SOA, mostly inspired by the initial set of web services, defined SOA as an architecture modeled around 3 basic components:

service requester

service provider

service registry

1st generation web services standards:

WSDL described the service

SOAP provided the messaging format

UDDI provided the standardized service registry format

2nd generation SOA added:

WS-* specifications (extension)

WS-BPEL supported the interest in applying service-orientation concepts to the world of business analysis.

38

Contemporary Service Oriented Architecture Reference Model

Business Process

Modeling

SOA Repository

SOA App Testing

SOA Governanc

e

Corporate Network

Software DevelopmentModeling Tools

Programming ToolsApp Servers

DBMSCM & Lifecycle Tools

Testing Tools

Enterprise Service Bus

Presentation Services Portal Services Device Services Messaging Services Security Services Translation Services

Data Services Data Federation ETL Metadata Management Data Archiving Backup & Recovery

Infrastructure Services Identity & Authentication Message Management System Management Security Management Resilience & Recovery Provisioning Federation Services

Workflow Engine

SOA Registry

Service Broker

SOA Supervisor

Package Apps

In House Apps

Adapter

Adapter

Collab’nApps

Adapter

BI Apps & Services

Adapter

Source: Hurwitz Group, 2006

39

2 Key Advantages of SOA: easier logic evolution & state management

Service A Service B


B


SOA establishes a loosely-coupled relationship between units of processing encapsulated as services. This allows the logic within each service boundary to be updated and evolved independently of existing service requestors, as long as the original service contract is preserved.

SOA promotes the standardization of XML data representation throughout solution environments. Further, service statelessness is emphasized by deferring state management to the message level. This maximizes reuse, availability and scalability of service logic but also provides a standardized state management approach.

logic

logic

logic

40

Comparing Previous Architectures to Web Services

Compared to Web Services, CORBA solutions: Are nearly as capable for cross-platform and cross-language development Are harder to understand because CORBA relies on IDL to translate data Can handle higher transaction loads because they keep a persistent connection

Compared to Web Services, RMI solutions: Lock you into a purely Java solution on both the client and server Can be somewhat more difficult to deploy because of network port considerations Can handle higher transaction loads because RMI keeps a persistent connection between

clients and servers at the expense of servicing fewer clients per server In a Java-only trusted environment, RMI will perform faster than XML-based Web Services

because of the reduced work in getting the data into a wire-friendly format Compared to Web Services, DCOM solutions:

Are nearly as capable for Microsoft cross-language development but lock you into Microsoft.

Compared to Web Services, CGI solutions: Are more difficult to find because of no directory service Are more difficult to write clients for without a well-documented service-specific API to rely

on Are more difficult to interact with because there’s no accepted data interchange format.

Compared to Web Services, Servlet solutions: Can only be written in Java Lack a service directory Lack an interface specification explaining how to communicate with them; no accepted

data interchange format

41

Distributed Internet Architecture vs. Service Oriented Architecture

Category Distributed Internet Architecture Service-Oriented Architecture

Application Logic

Tightly Coupled: at design time, the expected interaction components will have with others is taken into account – so much so that actual references to other physical components can be embedded in code. It is efficient in that little processing is wasted in trying to locate a required component but the embedded coupling leads to a tightly coupled component network. Parameter-based Data Exchange: components provide methods that, once invoked, send and receive parameter data. Re-use: emphasized but rarely achieved due to tight-coupling.

Loosely Coupled: SOA still reply on components but the use of Web Services (i.e., standardized interface and open communications framework) supports a composition model, allowing individual services to participate in aggregate assemblies. Document-Style Data Exchange: web services communicate with SOAP messages which rely primarily on document-style messages which are structured to be as self-sufficient (meta information, processing instructions and policy rules) as possible which results in less individual transactions. Re-use: fosters re-use and cross-application interoperability by promoting the creation of solution-agnostic services.

Application Processing

Inter-Component Communication: DIA promotes the use of proprietary communication protocols such as DCOM and vendor implementations of CORBA for remote data exchange supports the creation of stateful and stateless components that interact with synchronous data exchanges (asynchronous supported by some platforms but rarely used).

Inter-Service Communication: Message-based communication that involves the serialization, transmission an de-serialization of SOAP messages containing XML payloads. (Despite improved parsers and hardware accelerators, SOAP still lags RPC communication.) Document and message modeling conventions and the strategic placement of validation logic are important factors that shape the transport layer of SOA.Although synchronous communication is supported, asynchronous patterns are encouraged, as they help minimize communication. Further supporting the statelessness of services are various context management options that can be employed, such as WS-Coordination and WS-BPEL

Technology There is no governance of the technologies used by DIAs (e.g., components, server-side scripts, raw web technologies such as HTML and HTTP)

XML data representation and the Web Services technology platform

Security

Traditional delegation and impersonation approaches as well as HTTP encryption

WS-Security Framework: emphasize the placement of security logic onto the messaging level. SOAP messages provide header blocks in which security logic can be stored which helps to preserve individual autonomy and loose coupling between services as well as the extent to which a service can remain fully stateless.

Administration

Maintaining component-based applications involves keeping track of individual component instances, tracing local and remote communication problems, monitoring server resource demands and standard DBA tasks. DIA introduces the Web Server as the official first point of contact for clients so is must therefore be designed for scalability. RPC-based data exchange generally requires a response from an initiating component and an exception handling routine is employed.

Require additional runtime administration since problems with messaging frameworks (especially when working with asynchronous exchange patterns) can more easily go undetected than with RPC-based data exchanges since so many variations exist as to how messages can be interchanged. WS-* extensions for messaging frameworks have yet to reach maturity UDDI helps with resource management and service description


Section IV

SOA Implementation Considerations

43

Where are We? An SOA Maturity Model

Technical Applications

Application Integration

Business Process Improvement

Business Services

Federated BusinessCollaborative services creating

dynamic, collaborative business relationships

Services directly implement business service capability

Service as a process creates modular units of business process

Service increases loose coupling

and separation of concerns

Data integration,client neutrality,

shared internal services

Adapted from CBDi

44

What’s Our Approach? Top-Down, Bottom-Up, Middle-out, Hybrid?

“Top down” and “bottom up” considerations need to be balanced.

Technical Ownership

ExecutiveBusinessOwnership

Funding

SOADesign and

Development Skills

Repository

Proof of Concepts Simple Web

Services

SelectSOA tools

TechnologyEnablers

Governance Business Architecture

• Principles• Patterns• Architecture• Skills

Top

dow

nB

otto

m u

p

• Measurement• Management• Rewards

46

ESTR Guidelines: Tips for Smarter Service Design

No. Tip Description

01

Design services to be shared

The value of a service is magnified by the value of the relationships enabled. Being shared does not mean the code is shared; the service is shared. One of the important advantages of a services-based model is that the provider of a service is not concerned with the consumer's platform and the consumer does not have to install and maintain software. Services enable the acquisition of new functionality without having to deploy and maintain new applications.

02

Services have a clear purpose

The business value of services to consumers of those services must be clear and unambiguous. To maximize the value of services, it is necessary to understand the core competencies your organization provides to others. When this business value can be articulated clearly, it defines the requirements for services that are useful to others in your value chain.

03

Services are discoverable and support introspection

To share services, the producer of a service must be able to publish it in a form the consumer of the service can find and bind to dynamically. The consumer must be able to discern how to use the service without having to talk person to person to the producer of the service. The conversation on how to use the service is ideally machine to machine.

04

Services are designed to be loosely coupled

Services are intended live in a loosely coupled environment and should use other services to perform common clearly defined tasks (for example, authentication or reporting). The value of services is magnified by their reuse and further magnified by their ability to be combined with other services to create new services. As services are typically owned by multiple entities, they need to be loosely coupled to allow each one to change and evolve independently of the others.

05

Services plug into a framework

Once a service has been discovered, it needs a framework that provides other common services and loose coupling. While services may be created without an SOA, they need an SOA to operate in. SOAs by their nature are federated, as they need to interact in a loosely coupled manner.

06

A service has a well-defined use policy/contract

It is important to realize that in a services model both the consumer and the producer of a service need the ability to set use polices. The consumer and producer of a service define policies around security, availability, reliability, and error and exception handling.

work_stream: Business Services

The coarse- and fine-grained trade-off is a matter of latency and usability. At the end of the day, you should move to a good SOA, exposing the right services that do the right things, and not be as concerned about granularity. Services that implement fine-grained interfaces and that are meant for local invocation will work well. Moreover, services that implement fine-grained interfaces, that are meant for distributed invocation, and that are on a low-latency network will also work well.

47

Common Tangible Benefits of SOA

Improved Integration (and intrinsic interoperability) Because of the vendor-neutral communications framework established by Web

Services driven SOA, the potential exists for enterprises to implement highly standardized service descriptions and message structures

State of Federation, where previously isolated environment now can interoperate without requiring expensive and fragile point-to-point solutions

Inherent Reuse Building services to be inherently reusable results in a moderately increased

development effort and requires design standards. Subsequently leveraging reuse within services lowers the cost and building service-

oriented solutions Streamlined Architectures & Solutions

The concept of composition can lead to highly optimized automation environments: Assembly of service collections into aggregate services The WS* platform is based on the principle of composability

Leveraging the Legacy Investment Industry-wide acceptance of the Web Services technology set has spawned a large

adapter market Best of Breed Alternatives

Because SOA establishes a vendor-neutral communications framework, IT is not longer restricted to a single proprietary development and/or middleware platform

Organizational Agility How building blocks can be realized and maintained within existing financial and

cultural constraints ultimately determines the agility of the organization as a whole

Many of the benefits promised by SOA do not manifest themselves until the use of service-oriented principles becomes established within an enterprise. As a result, there are few short-term benefits

48

Common Pitfalls of Adopting SOA

Building service-oriented architectures like traditional distributed architectures Problems:

proliferation of RPC-style service descriptions (leading to increased volumes of fine-grained message exchanges)

inhibiting the adoption of features provided by WS-* specifications Further entrenchment of synchronous communication patterns Creation of hybrid or non-standardized services

Not creating a transition plan Migration needs to happen at the technical, process and organization levels to avoid

ad-hoc adoption Not standardizing SOA

Like any other architecture, SOA requires the creation and enforcement of design standards

Failing to create an XML foundation architecture SOA requires standardizing how core XML technologies are used to represent, validate

and process corporate data Failing to account for SOA performance requirements

As message-based communication increases, processing latency can be an issue Lack of proper SOA security model

Secure Sockets Layer (SSL) is not the technology of choice for SOA; the need for message-level security implies that the WS-Security Framework is optimal

Failure to understand standards organizations Web Services Interoperability (WS-I): Basic Profile and Basic Security Profile

49

Performance Issues

Message-based communication in SOAs can, in fact, increase performance requirements when compared to RPC-style communication within traditional distributed architecture XML processing-related performance challenges

Web services security measures, such as encryption and digital signing, add new layers of processing to both the senders and recipients of messages

Need to test the message processing capabilities of your environments Stress testing the vendor supplied processors (for XML, XSLT, SOAP, etc..) Considering alternative processors, accelerators or other types of technology:

XML-binary Optimized Packaging (XOP) SOA Message Transmission Optimization Mechanism (MTOM)

Performance is one of the reasons why: Coarse-grained service interfaces and asynchronous messaging are emphasized when

building Web Services

50

Best Practice: Create a Services Blueprint for Transformation

CustomerSegments

Channels

Retirement Client

Trust Client

Brokerage Client

Kiosk/POSKiosk/POS

WebWeb

Call Center/IVRCall Center/IVR

Fax

Client

Mobile

Endowment Client

Defined Contribution

Client

Defined Benefit Client

Customer Relationship Management

Record KeepingFund Accounting

Analysis & Product

Development

Marketing

• Analyze/Understand Client

• Perform Market Research/Analysis

• Create/Modify Products/Services

• Educate Client• Prepare

Communications

Cash Management

• Manage Cash

Pricing• Price/Value

Investments• Correct Pricing Error

Securities Accounting

• Conduct Securities Lending

• Perform Clearing & Settlement

Process Information Requests

• Generate Reports• Generate Statements• Generate Confirm/

Notifications

Process Client Transactions

• Manage Assets• Manage Account Balance• Administer Installment

Payment• Rebalance Assets• Calculate Benefits• Convert New Plan• Bill Fees• Process Credit/Margin• Calculate Annuity

Payments• Administer Annuitization

Payment• Manage Loans• Manage Trust Assets• Manage Trust Income &

Disbursements• Process Corporate Actions• Withhold Taxes• Purge/Archive Records• Manage Client Payments• Prepare Excess Refund• Prepare Pass-Through

Dividend• Manage Brokerage Orders

Advisory ServicesFinancial Planning

• Create Financial Plan• Manage Financial Plan

Investment ManagementInvestment Strategies Management

• Manage Portfolios• Replicate Indexes• Manage Order Routing and Execution• Monitor Performance

Account ManagementManage Accounts

• Setup/Maintain Person

• Setup/Maintain Account

• Setup/Maintain DC/DB Plan

• Setup/Maintain Trust

Prepare Client Transactions

• Prepare Purchase• Prepare Redemption• Prepare Exchange• Prepare Buy• Prepare Sell• Exercise Options• Prepare Contribution• Prepare Withdrawal• Prepare Rollover• Perform Adjustments• Administer Installment Setup• Administer Annuitization Setup• Administer Loan• Prepare Asset/ Account Transfer• Prepare Plan Level Transfer• Prepare 1035 Exchange• Terminate Person• Prepare Death Claim

Manage Information Requests

• Provide Information

• Provide Personalized Performance DataPerformance Analysis

• Product/Service Success Analytics

Service Development

• R&D• Product Lifecycle

Customer Service

• Call Center Services• Customer Lifecycle• Customer Segment

Management

Sales Force Automation

• Campaign Management

• Contact Management• Sales Goal

Performance Management/Dashboard

Management & Operations• Account• Reconciliation

• Reconcile Checks• Reconcile Client

Accounts• Reconcile

Transfer Agency Accounts

• Reconcile Custody Bank Accounts

• Reconcile Omnibus Accounts

Compliance

• Monitor Investment Compliance

• Monitor Client Compliance

• Audit Dividend & Capital Gain Disbursements

Client Control Reporting

• Process As-of Transactions

• Provide Tax Services

Money Movement

• Move Money

Inventory Management

• Manage Literature Inventory

• Fulfill Literature Requests

Financial Management

• Perform Corporate Budgeting

• Provide Executive Information

• Execute Monthly/Yearly Financials

• Perform AP/AR• Issue Payroll• Track Assets• Prepare

Compliance Reporting

HumanResources

• Hire/Retain/Release Employees

• Provide Career Management

• Provide Work/Life Initiatives

• Prepare Compliance Reporting

Paper

Partner & Supplier

Interaction

Brokers

Trust Accounting

External Advisors

Transaction Clearing-houses

51

Best Practice: Create an ESA Reference Model


Enterprise Presentation Services

Information Services

Infrastructure Services

Enterprise Application Services

Enterprise Platforms

Network Backbone & Topology

Routing & Security Architecture

Storage ArchitectureNetwork Resource ManagementFile and Print Access

Services

File and Print AccessServices

Security & Access Management

DirectoryServices

Messaging & Calendaring

Mobile, Wireless & Telephony

Development & Deployment

MonitoringServices

Integration PlatformsIntegration Platforms

Application PlatformsApplication Platforms

Client ServicesClient Services

Personalization ServicesPersonalization Services

Business Process IntegrationBusiness Process Integration

Core Application ServicesCore Application Services

Business IntelligenceBusiness Intelligence

Data InfrastructureData Infrastructure

Data Access ServicesData Access Services

Digital Asset ManagementDigital Asset Management

Business Applications

Business Process Services Information Assets

CustomerCustomer

OrdersOrders

FinanceFinance

Business Services

Domain 1

Business Services

Domain 2

Enterprise Packaged Applications

Business Architecture & Business Process Models


52

Best Practice: Create & Publish an SOA Standards Stack

Layer Standard

Generic vocabulary UBL

Knowledge definition UML

Choreography BPEL

Presentation WSRP

Service invocation SOAP, WSIF

Security WS-Security – Liberty profile supports this (including SAML and XACML)

Service description WSDL

Schema of the syntax XML Schema, RelaxNG, DTD, ASN.1, RDF Schema, IFX, LIXI

Document syntax XML, EDI, IIOP, BER encoding

Messaging envelope SOAP, S/MIME, ebMS, WS-Reliability

XML transformation XSLT

Data queries XPATH, XQUERY, XBRL

Consu

mer

Pro

vid

er

Single standard desirable

Multiple standards acceptable

53

Best Practice: Define Registry/Repository Use Cases

RegistryDeveloper

Service Consumer

System Admin

Business Owner

Enterprise Architect

Project PM

Publish Service

Update Service

Deprecate Service

Delete Service

Discover Service Retrieve Service

Publish Polcies Approves Services

Performs Validation

Perform Cataloging

Perform Versioning

Store Artifacts

Notifications

54

Considerations when Implementing Service-Oriented Architecture

Service identification: What is a service? What is the business functionality to be

provided by a given service? What is the optimal granularity of the

service? Service domain definition:

How should services be grouped together into logical domains?

Service packaging: How will existing functionality within, say,

legacy mainframe systems be re-engineered or wrapped into reusable services?

Service orchestration: How are composite services orchestrated?

Service location: Where should a service be located within the

enterprise? Service routing:

How are requests from service consumers routed to the appropriate service and/or service domain?

Service Publication and Discovery: How should the services be catalogued so

that business and IT users can identify and possible re-use the services in their applications and processes?

Service governance: How will the enterprise exercise governance

processes to administer and maintain services?

What standards will be defined for messaging, choreography, etc., that can be adopted consistently within the organization.

Service Development & Deployment : What modeling, design, development &

deployment platforms to be used? Service Integration Testing :

What is the testing strategy for loosely coupled components?

What testing harnesses or tools will be used? Service Performance :

How to ensure throughput, availability, reliability?

Service Versioning & Release Management:

What are the procedures for service versioning, release management & migration


Section VI

SOA POC: Insurance Claim Processing

56

SOA POC: Business Scenario

1. A customer enters a claim via the website / portal2. The entry is validated against a customer information system (service)3. A valid entry is registered – customer has valid policy (service)4. Customer receives acknowledgement of claim registration (service)5. Claim checked against insurance policy (coverage, amount, provider)6. Claim approved or rejected or requires clarification7. Customer notified of approval or rejection (notification service)8. If Approve or Reject, process to logical end9. Send payment advice to Accounts Payable10.Update customer claim processed (service)11.Notify customer

--------------------------------------------------------------- Customer can query status of claim (service)

57

SOA POC: Business Process Swim Lane Diagram

Invalid Provider/Subscriber Data

Medical Claim ProcessA

pp

rova

l &P

aym

en

tN

otific

atio

nD

ata

Po

licy

& C

laim

Re

co

ncila

itio

n

Ve

rific

atio

n/

Va

lida

tion

/R

eg

istr

atio

nC

lien

t/U

ser

ClaimVerification

Service

VerifyOK?

Invalid/Duplicate Claim

ClaimValidationService

ValidationOK?Yes

No

YesClaim

Registration

Claim Acknowledgement

No

Yes

ExtractEach ClaimLine Item

Verified & Validated Claim

Assess LineItem

coveragewith policy

Line Itemcovered?

Compute line itemcoverage based on

deductible & maximum

Compute line itemcoverage = 0

UpdateTotal Claim

Amount

Yes

No

CheckClaim

Amount

Amount >10000

Total Computed Claim

InitiateFunds

TransferApprove /Adust /Deny

Approved/Adjusted

No

Yes

Yes

PrepareDenial

NotificationDenied

PrepareApproval

Notifcation

UpdateClaimStatus

NotifyCustomer

NotifyAccounts &

Audit

ClaimsDB

PolicyDB

Provider/Subscriber

DB

Cla

im I

nfo

Pro

vid

er/

Subscriber

Info

Data Access Framework

PolicyHolder

DB

AddressDB

Notification Systems

Valid

Cla

im I

nfo

Polic

y &

Cla

im I

nfo

Updated Claim Info

Claim Inquiry (Acknowledgement No.)

Claim Info (Response)

58

SOA POC: Business Process Model (Level 0)

59

SOA POC: Business Process Model (Level 1)

60

SOA POC: Process Choreography

61

SOA POC: Service Interface Definition (WSDL)

62

SOA POC: Schema Definition

63

SOA POC: Service Interaction Diagram

Client

MedicalClaim UIProcess

Submit aClaim Process

Claim VerificationProcess

AcknowledgAcknowledg

Duplicate ClaimResponse


ClaimApproval Process

Claim

Register Claim Request

Acknowledg

Claim



SubscriberVerifyRequest

SubscriberVerifyResponse

GetClaimsRequest

ClaimsArray

ProviderVerifyRequest

ProviderVerifyResponse

GetPolicyRequest

Policy

GetPolicyHistoryRequest

PolicyHistory

Claim

DuplicateClaim Verification

Process

Get AllClaims Service

SubscriberVerification Service

ProviderVerification Service

Claim ReconciliationProcess Get

PolicyService

GetPolicyHistoryService

Claim RegistrationService

Claim Reconcilaition Response

Claim Reconcilaition request

GetManual Approval

Task

Manual Approval RequestRequest

ResponseManual Approval Response

AccountsPayableProcess

Claim PayRequest

Pay Response

Notification Request

FundsTransferService

NotificationService

PayableResponse

ClaimInquiry

ProcessStatusInquiryService

InquireAClaimRequest

InquireAClaimResponseResponse

Request

Medical Claim Process - Process and Service Interaction Diagram

Claim

64

Example: SOA Implementation Project Approach

Phase Activity Role Artifacts

Requirements Gathering & Analysis

Current & Future state problem definition

BA Requirements

Functional description of requirements

BA Use Cases

Business Process Modeling BA Business Process Model, Event Based Sequence Diagrams (“Swim-Lane” Diagrams) ( )

Business Process Design

Detailed Business Process Modeling & Simulation

BA / Architect Final Business Process Model, Event Based Sequence Diagrams (“Swim-Lane” Diagrams), Functional Specs

Business Process Technical Design

Business Process Choreography Design

Architect/Developer BPEL, Data Model, UI Model, Process Test Use Cases , Functional/Technical Specs (WSDL)

Business Process Implementation

Coarse-Grained Service Identification, Design & Development

Architect/Developer Process & Service Interaction Diagrams, Revised Data Model, Service Test Use Cases, Functional/Technical Specs (WSDL)

Service Implementation

Fine-Grained Services Identification, Design & Development

Architect/Developer Class Diagrams, Final-Data Model, Service Test Use Cases (WSDL)

65

SOA POC: High Level Technical Architecture

66

SOA POC: Technology Landscape

WebSphere MQ

SOAP/HTTP, SOAP/JMS, EJB

Broker (WBIMB)

Process Choreographer (WebSphere Server Foundation)

Web Services (Java, .NET, Other)

Message flowsWMQ Applications

WBI Modeler & Monitors


Additional Resources

68

LiquidHub’s SOA Foundations Course for Managers: Concepts, Strategy and Technology

Lesson 1: IntroductionContemporary Business and IT Challenges

Enterprise Architecture ConceptsService Oriented Architecture Concepts

SOA Benefits Lesson 2: The Evolution of SOA

SOA vs. Past ArchitecturesE.g., SOA vs. Client Server Architecture

Lesson 3: The Road To SOA SOA Enterprise StrategySOA Enterprise Governance

Lesson 4: The SOA Delivery LifecycleSOA SDLC PhasesSOA TemplatesSOA Project Roles

Lesson 5: SOA Project ManagementResource PlanningMigration PlanningElevation Planning Integration Planning

Lesson 6: Web Services FundamentalsXML OverviewSOAP OverviewUDDI OverviewWS* Overview

Lesson 7: Key SOA TechnologiesBusiness Process Management Suite

Enterprise Services BusRepository / Registry

Lesson 8: Case Studies SOA Strategy: HBR Peachtree Healthcare SOA Implementation: LH IBM wS

Documents

LiquidHub Lunch & Learn Demystifying SOA: Concepts & Best Practices