© 2008 Netcompany A/S Netcompany A/S · Grønningen 19 · 1270 København K · Tlf. 70131440 Challenges in process driven Service Oriented Architecture Presentation

© 2008 Netcompany A/S Netcompany A/S · Grønningen 19 · 1270 København K · Tlf. 70131440

Challenges in process driven Service Oriented Architecture

Presentation at the IT University

11th of February 2008

Jack Ekman, Netcompany


Agenda• Introduction to SOA

• SOA: Service infrastructure at a major pension company– Project overview

• Paradigm shift in SOA - From application to process driven development– SOA Fundamentals– Quick introduction to Business Process Management (BPM)

• Challenges and solutions in a SOA implementation

• When is SOA a success– Which business advantages can we achieve by using SOA?

• Discussion


Paradigm shift in SOA:

1) SOA is ultimately about business processes over applications

2) SOA initiatives should be driven by business needs more than technical needs

3) SOA must be anchored both in the technical and the business organization.

From Application Orientation to Business Process Orientation


What is a Business Process?

Examples: • Create new customer• Create Order

”A business process or business method is a collection of interrelated tasks, which solve a particular issue. “ [Wikipedia]


What is the goal of SOA business wise?

• Increase business Agility

• Enable Business Activity Monitoring (BAM)

• Automation and streamlining of business processes

• Leveraging of existing application assets

• Reuse Business assets as reusable services


What is the goal of SOA architecture wise?

• Loosely coupling of systems• Standards based interface• Simplified communication model for applications


Messaging

Routing

TrackingAudit Log

Monitoring

Error Handling

Quality of Service

Transfor-mations

Configuration

Service Registry

Transaction Management

File Transfer

External Services

Adapters BPM engine

Security

SOA foundation

Integration benefits

Business Benefits

Business Agility

Automation and streamlining of business processes

Leveraging of existing application assets

Business assets as reusable services

Loosely coupling of systems

Standards based interface

Business Activity Monitoring

Basic SOA foundation

Point to Point integration

SI as a Business Integration Platform

SI todaySI today

Elements in a Service Oriented Architecture

Simplified communication model for applications







• Discussion


The Story behind the project

- Started in 2003

- Motivated by

- The need to change and replace existing systems

- The need to be able to support new business processes

- The need to optimize existing processes

- The need to clearify existing and upcoming architecture

The Story goes…


– The customer plus 5 consultant companies (Netcompany, IBM, CSC, PA Consulting, 7N)

– Ca. 250 people, 150 of them consultants.

– 14 parallel development projects

– 21 applications (3 of them are completely new)

– The business driver is the development of a new pension system

Project overview


System architecture 1/2

– SOA solution based on Custom built Enterprise Service Bus

– Technical platform is Java / J2EE / BEA Weblogic

– Heavy use of Business Process Management (BPM)

– Common domain model / data model described in XSD architecture

– One of the biggest implemented solutions in Europe


System architecture 2/2

Technologies:

• J2EE

• Dependency injection: Spring-framework

• Persistent: Hibernate

• Unit test: JUnit

• Build/Configuration management: ParaBuild & Maven 2

• Version control of code and configuration: CVS & ClearCase


.

.

.

.

The development process 1/2

Process team

The Business

Test

Service Infrastructure

Applications

teams


The development process 2/2

• Process driven development in iterations

• Business processes are defined and analyzed by a process team

• Processes are used as a starting point for– Services– Business Process Management (BPM) processes– Test– Coordination of logic between applications


What is a business process in PFA Terms?Create/Convert scheme

CC

AM

EA

IAS

PE

CIA

LIS

TK

R

Select scheme to convert

Scheme rules set

Create scheme in AIA

Create scheme

Manual mapping of schemes

List partial schemes to be

converted

Enter schemes/contract rules

The start eventProblem: where to describe the starting conditions?

The result of the process

The roles or subject

Verbs and objects

The activitiesProblem: where to describe the activities or needed information to carry out the activity?

TimelineProject model definition: Time always moves from left to right


Who are using the business processes?

ProcessesService Infrastructure (SI)

Data

Test

Testing Test scenariosUse casesStart and end conditionsRoles Manual

Workflow

Organizationchange

Business impactRoles -> subjects


Consequences of using a process driven approach

• The processes are used to identify:– Services – Requirements for business data in the services – Dataflow between applications (BPM logic)

• Furthermore, processes are used to:– Plan tests– Plan organizational changes– For formal documentation of business processes


Consequences of using a process driven approach

The first important message

SOA is not better than the quality of the business processes







• Discussion







• Discussion


Messaging

Routing

TrackingAudit Log

Monitoring

Error Handling

Quality of Service

Transfor-mations

Configuration

Service Registry

Transaction Management

File Transfer

External Services

Adapters BPM engine

Security

SOA foundation

Integration benefits

Business Benefits

Business Agility

Automation and streamlining of business processes


Business assets as reusable services

Loosely coupling of systems

Standards based interface

Business Activity Monitoring

Basic SOA foundation

Point to Point integration

SI as a Business Integration Platform

SI todaySI today

Elements in a Service Oriented Architecture

Simplified communication model for applications


So why do we need SOA anyway?

• Integration without a Enterprise Service Bus

• Integration with Enterprise Service Bus

• Moving towards a SOA


Direct Integration

A

B

Integration without ESB (1/5)

The applications must agree on:

• Protocol

• Message format

• How to find each other


Direct Integration

A C

B



Direct Integration

A

D

C

B



Direct Integration

A

D

C

B


JMSUPIC

FTPSOAP

FTP

HTTP


Direct Integration

A

D

C

B


JMSUPIC

FTPSOAP

FTP

HTTP

Spaghetti!


SOA using an Enterprise Service Bus (ESB)

Integration with ESB

A

DC

B

ESB

JMSSOAP

FTP RMI

F

E• Applications use one

way (the ESB) to communicate

• Easy to add new applications

• Complexity is in the ESB

• Flexibility via adapters

• Loose coupling between systems

• Integration based on standards


SKAT

A real life ESB

Service Infrastruktur

BPM IS

MainframeUnitLink

Papyrus

SHAFT

FICOCRM

OS

Portal

F & P

Mægler

SOAPRMI HTTP

SOAPRMI FTP

Fil

UPIC

SOAP JMS

HTTP Legacy

SKATMainframeUnitLink SHAFTOS F & P


So what is a service [Conceptual level]?

Copied from the real world ie. A ”Fix my teeth” Dentist service A ”Change oil on my car” Mechanics service A ”Create a Bank Account” Bank Service

One formal and quite technical definition"a mechanism to enable access to one or more capabilities, where the access is provided using a prescribed interface and is exercised consistent with constraints and policies as specified by the service description." [OASIS]


So what is a service [Technical level]?

Format a Service Infrastruktur

App A App BAdapter AdapterTransformation

Format A => Format B

Format b







• Discussion


Example: Get Contract List implemented without BPM

Portal

System 1 System 2 System 3

New system X


Example: Get Contract List implemented in BPM

Portal

BPM

AIA Mainframe UL

Advantages:

• Other systems can easily reuse the functionality

• Less maintenance for the Portal, for instance when introducing a new insurance system

• Easy reuse of process logic


BPM can be used on several levels in a SOA

• Technical processes– Transforming a file to a sql call in a database

• Orchestration of sub processes across systems– Managing very long process flows

• End-to-end business processes– Create new customer in several systems– Manage contract changes


The practical solution: Make the code graphical!

• Her, et screendump fra WLI


SKAT

BPM is a natural part of a SOA

Service Infrastruktur

BPM IS

KRUnitLink

Papyrus

OS

Portal

F & P

Mægler

BPM







• Discussion


Challenges in process driven SOA development 1/2

• IT and business are closer to each other. This increases the need for a detailed level of understanding of the business processes.

• First we focus on the business processes, then we focus on the services


Challenges in process driven SOA development 2/2

• Which level of details should the processes have?

• Which granularity should the services have?

• Are there Pro’s and Con’s of a common data model?

• How do we handle service lifecycle management?


How detailed should the processes be? 1/4

• Who is responsible for adding information to the processes?


Service granularity 2/3 How much should a service cover?



Time

Am

oun

t o

f w

ork

Analysis Phase è Design PhaseBusiness analysis Implementation phase Test phase

Business drive

Governance board Gate 1 Conceptual business process

Governance board Gate 2Detailed Business Process

Governance board Gate 3 Detailed Business process

with Technical Details

Implementation delivery

Technical drive of implemnetation

Business drive with technical assistance

Technical drive with business assistance

Drawing by: Jack Ekman – [email protected] – ver 1 – 14-08-2007

Business drive

Combined business and technical drive

Technical drive

Process development procedure

The Grey Area



Assertion: Business people thinks in 2D and technical people thinks in 3D. Both ways of thinking are needed.

• Experiences from our process work

– Business people thinks in “Sunshine” versions (everything always go well) and perceive processes as processes.

– Technical people thinks in “rain”-versions (takes error situations into account) and business process networks (i.e. the processes are a part of a larger business flow). They are not capable of seeing, which steps, that has to be taken to carry out a business process.

• Solution: Levels with different degree of details



A general service

A specific service pr. business

requirement

1 1000+Number of services

Services are customized to fit within certain areas

of business needs




Challenges with a common data model 1/2

What happens when a common data model is changed?

Portal AIA-CP Taxpool3

AIA-CPBPMKR

Domain model

XSD

XSD

XSDXSD

XSD

Business service1

Business service2

Business service3

XSD XSD XSD

Service call1

Service call2

Service call3


Challenges with a common data model 2/2

• Potentially many impacts and derived impacts

• Great need for coordination and deployments of changes

• Potentially a great need for regression test

• Solution: Versioning of services– However, this increases the complexity of

the solution


Lifecycle management of services 1/2• When is it needed?

– Design time governance

• What services do we have?

• Which ones can be reused?

– Run time governance

• Which applications uses which services?

• Which services are the most used

– Change governance

• What will be impacted, if a service is changed

• What happens if we discontinue a service


Lifecycle management of services 2/2

• Solution: Create a service repository– Contains data about all services and service calls – Can be used to lookup services– Can be used to make impact analyses– Can be used for service version control

• But is it sufficient?– Where can security policies be managed?– Where can access control to services be managed?– Where should other types of service related metadata be

managed?– How can consistency among business processes, services,

security policies etc. be ensured?


Conclusions

• SOA requires at least one paradigm shift (From application to process)

• SOA brings business and technique closer together, thus creating new possibilities and newpotential value.

• SOA requires a sliding shift from business to technical considerations as the development process progress

• The SOA development contains several stakeholders. Coordination and communication is essential between development projects.

• Reuse must be supported by the use of a service repository in order to have a central place to manage services and other SOA aspects.

• There is a need for a SOA architect (Solution Architect), who has the grand overview and has an organizational strength to get decision carried out


The results from using SOA 1/2:• SOA is business driven:

– Business services are central– A SOA is developed and extended based on demand

from the business– Higher degree of automation– More flexible organization


The results from using SOA 2/2:

• Integration based on standards– Introduction of new services are primarily configuration and new adapters

• Better support to the whole organization– Messages are never lost– Overview of the services via the service repository

• There is a need for better support of the’SOA-process’– Enterprise repository







• Discussion


When is SOA a success?

– Which parameters defines when a SOA is a success?

– And how can we measure them?


Flashback - What is the goal of SOA business wise?

• Increase business agility• Enable Business Activity Monitoring (BAM)• Automation and streamlining of business processes• Leveraging of existing application assets• Reuse Business assets as reusable services


Increase Business Agility

Success goal: Improved organizational and technical readiness for change

– How can you measure this?– New products can be adopted and be adopted faster?

– (Could there be something missing here?)


Business Activity Monitoring (BAM or BI)Business processes can be monitored in runtime.

Questions like these can be answered: – How long time does each part of the process take? – Where is the biggest load in the business right now?– Where is the business right now?

Success criteria Real time information about data flows in the organization

Measuring criteria Services used as Key Performance Indicators (KPIs) for business critical services


Atomization and streamlining of business processes 1/2

Existing business processes are optimized and automated

– Success criteria: business processes takes less time and with fewer resources

– Measuring criteria : Production time and production resources


Atomization and streamlining of business processes 2/2

Business processes are more precise scoped and specified– Success criteria: A clear specification of all business

processes, so they can be changed when needed– Measuring criteria : Time used to change and/or create a new

business process

Process logic is centralized in the BPM. Not spread in several applications– Success criteria: Easier and cheaper to change process logic– Measuring criteria: Development time and development

resources



• Simplified communication between systems makes it easier to integrate systems.

– Create a common domain model and thereby a common terminology across the applications

– Integration takes place through standard adapters – Measuring criteria : Development time and development resources

• Integration to existing business applications takes place through standard adapters, to ensure simplified communication

– Success criteria : easier integration– Measuring criteria: Development time and development resources


Discussion?

Documents

© 2008 Netcompany A/S Netcompany A/S · Grønningen 19 · 1270 København K · Tlf. 70131440 Challenges in process driven Service Oriented Architecture Presentation