WEB SERVICES

AND

SERVICE ORIENTED ARCHITECTURE

N E T C O M PA N Y L E C T U R E S E R I E S

2 0 11

P E T E R F R I I S A N D B R I A N M U N C K A N D E R S E N

• Approx. 270 consultants– M.Sc. or Ph.D. typically from DTU, DIKU, ITU, ÅU, AAU– Excel academically

• Offices in Copenhagen (HQ), Aarhus and Aalborg

• We deliver business-critical IT solutions– Systems integration and SOA– Portals– Business applications

• Our customers include– Financials Nordea, Saxo Bank, Letpension, PFA pension, FORCA– Government SKAT, Økonomistyrelsen– Industry Carlsberg, Dong Energy, Pandora– Commerce Novozymes, Danmarks Apotekerforening– Media JP/Politikens Hus, Aller– Membership organizations IDA, Dansk Erhverv– Telco Telenor, TDC

WHO WE ARE

WHAT YOU’LL LEARN

SOA Concepts

Describing a Service

Designing Services

Security

Managing Change

Governance

The Enterprise Service Bus

Business Process Management

READY?

S E R V I C E O R I E N T E D A R C H I T E C T U R E

1

S O A C O N C E P T S

C o n t r a c t - b a s e d

L o o s e c o u p l i n g

A b s t r a c t i o n Reusabi l i ty

AUTONOMYD i scoverabi l i ty

S TAT E L E S S N E S S

FIRST

SOA isn’t a technology.

SECOND

SOA isn’t new.

SOA rests on centuries old principles.

…yes, centuries!

R E C E P T I O N

F I N A N C E

A C C O U N T I N G A R C H I V E

S A L E S

A D M I N I S T R A T I O N

B U S I N E S S S E R V I C E S

R E C E P T I O N

F I N A N C E

A C C O U N T I N G A R C H I V E

S A L E S

A D M I N I S T R A T I O N

B U S I N E S S S E R V I C E S

ADMINISTRATION

SALES

ACCOUNTING

ARCHIVE

FINANCE

RECEPTION

D E P E N D E N C I E S

SALES ACCOUNTING

Open Account

D E P E N D E N C I E S

ADMINISTRATION

SALES

ACCOUNTING

ARCHIVE

FINANCE

RECEPTION

S E R V I C E D E S I G N P R I N C I P L E S

S E R V I C E D E S I G N P R I N C I P L E S

S O U N D B U S I N E S S P R I N C I P L E S

S O A

ADMINISTRATION

SALES

ACCOUNTING

ARCHIVE

FINANCE

RECEPTION

MODEL

THIRD

SOA isn’t a dogma

THIRD

SOA isn’t a dogmashouldn’t be

Service orientation is a PARADIGM that frames what you do.

Service-oriented architecture (SOA) is a TYPE OF ARCHITECTURE that results from applying service orientation.

S O A M A N I F E S T O

””

www.soa-manifesto.org

S O A M A N I F E S T O

www.soa-manifesto.org

P R I O R I T I E S

Business value over technical strategyStrategic goals over project-specific benefits Intrinsic interoperability over custom integrationShared services over specific-purpose implementationsFlexibility over optimizationEvolutionary refinement over pursuit of initial perfection

G U I D I N G P R I N C I P L E S

Respect the social and power structure of the organization. Recognize that SOA ultimately demands change on many levels. The scope of SOA adoption can vary. Keep efforts manageable and within meaningful boundaries. Products and standards alone will neither give you SOA nor apply the service orientation paradigm for you. SOA can be realized through a variety of technologies and standards. Establish a uniform set of enterprise standards and policies …

D E F I N I T I O N

2

D E S C R I B I N G A S E R V I C E

Design by contract• Definition of contract• “Fully spelled out contract”• Implicit references

Real world• Example of contract• Incomplete information• Resulting code

“Key points”• Aim for adequate specification• Document decisions• Make references explicit

...a client who satisfies all the constraints listed is entitled to the benefits. This is the No Hidden Clause rule.The No Hidden Clauses principle does not prevent us from including references, implicit or explicit, to rules not physical part of the contract.

Computer, Vol. 25, No. 10. (1992), pp. 40-51.

A contract is a common agreement with benefits for all participants.

The Beatles’ first contract

A more recent contract

What is a contract?

A real world example

Domain model from customer

Service description from customer

XML type definition

Java service implementation stub

Complete vs loose contract

A complete contractParts of the contract differ in degree of detail/specificity. From more to less detail:

• Security model, infrastructure• Input / output complex types• Simpler xml types

• Normal behavior• Restrictions between optional elements• Expected business exceptions• Extreme behavior (e.g. no elements

found)

• Runtime exceptions (e.g. time out, full disk)

• Transactional integrity• Concurrency (even on www)

… is not always a possibilityTherefore, take great care in documenting the decisions made during implementation:

• Record changes, and the motivation for these

• Beware of other usages of type

• Detailed behavior in service description.• Restrictions between optional elements• Reuse exception types across services• Agree on common pattern for services

• Wrap in reusable exception types• Part of system architecture not service• Concurrency (even on www)

3

D E S I G N I N G S E R V I C E S

Design deliverables• Systems, Domains and Services• Global Logical Domain Model• Operations and Processes• Physical Document Types

Real world• Internal and external services• Reuse and coupling• CRUD and Process services

“Key points”• Decouple internal and external

services on the physical level• Aim for reusable services• Services are different

Domain models and services

• Model your data tables directly from communication formats

• Don’t share physical types between internal and external services. Transformations in one system is easier than changing two systems.

• Loose coupling of systems

• Provide a global logic domain model as part of the contract, but leave external physical communication types flexible.

• Be aware that reuse of types between services is strong coupling

• Remember that services are used together. Output will be used as input by others.

Do’s Don'ts

Simple and more complex services

Keeping results in memory is a problem with large result sets. Consider streaming services.

Two separate systems for vehicles and license plates and a proxy service for an external system with information about people.

Simple services for Create, Read, Update and Delete, but what about more complex services?

FindVehiclesWithPlateMatching(pattern) {FindPlates(pattern)For each plate

result +=ReadVehicle(VIN)return result;

}

RegisterVechicleToPersonAndAssignPlate(VIN, person) {UpdateVehicle // ad ownerGetNextAvailablePlateUpdatePlate // add VIN

}

NotifyPoliceAboutExpiredPlates(startDate, endDate) {GetAllExpiredPlates(startDate, endDate);Foreach Plate

FindVehicle(VIN)FindPersion(PersonID)result += (Vehicle, Plate, Person)

}

Issues with concurrency and transactional integrity

“Batch job as service” with join over web service per element and non-domain information (address).

CRUD, Find or Process service?

• Filtering on fields and values• Avoid joining over services• Behavior for not found and large

result sets.

Not all services are born alike!

Searching for entities via services

• On create, return assigned id• Optional/required might differ in

create and read.• Update by sending changes or

overwrite whole entity.• Versioning on update can provide

optimistic locking• Keep old versions on update and

delete operations

Create, Read, Update and Delete• Sequences and available data

(order of creation)• Avoid joining over services• Consider asynchronous patterns

e.g. message queues• Candidates for BPM

Processes in services

Bulk updates• Avoid joining over services• Let update services take a list of

elements to update

4

S E C U R I T Y

CASE: A PENSION COMPANY

Member Portal

CRM Service Pension Service Finance Service

Contact dataCommunications Pension policies Funds

PBS

T R U S T E D S U B S Y S T E M

Member Portal

Pension Service

Portal Identity

Member Identity

Member Portal

Pension Service

Member Identity

Member Identity

I M P E R S O N A T I O N

CRM Service Pension Service Finance Service

Security Model Security Model Security Model

Identity Identity Identity

S I L O - B A S E D S E C U R I T Y

SILOS

ARE A

SOA ANTI PATTERN

CRM Service Pension Service Finance Service

Claims Claims ClaimsEnterprise

SecurityModel

S I L OC L A I M S - B A S E D S E C U R I T Y

C L A I M S - B A S E D S E C U R I T Y

Member Portal

Pension Service

Claims

ClaimsIdentity Provider

EnterpriseSecurityModel

Security Service

Login

AFTER THE BREAK:

M A N A G I N G C H A N G E

G O V E R N A N C E

T H E E N T E R P R I S E S E R V I C E B U S

B U S I N E S S P R O C E S S M A N A G E M E N T

5

M A N A G I N G C H A N G E

ALL TOO COMMON SYSTEM LANDSCAPE

…THIS JUST IN:

AS THE BUSINESS CHANGES

SO DOES ITS SERVICES!

DEPENDENCIES

CRM Service

Member Portal

Reporting

”I need an additional field on the Member objectin three weeks”

”We’re busy for 2 months implementingthat other stuff you wanted!”

VERSIONING

CRM ServiceVersion 1.0

Member Portal

Reporting

CRM ServiceVersion 1.1

6

G O V E R N A N C E

G O V E R N A N C E I S A B O U T

TRANSPARENCY

and

CONTROL

T R A N S PA R E N C Y• Understanding your data• Understanding your services and their dependencies• Understanding your business processes

(is prerequisite to)

C O N T R O L• Enforcing your policies• Managing change• Successfully connecting IT with Business Strategy

T R A N S PA R E N C Y:

UNDERSTANDING YOUR DOMAIN

CRM PENSION FINANCE

CRM PENSION FINANCE

Member

T R A N S PA R E N C Y:

UNDERSTANDING YOUR DOMAIN

E N T E R P R I S E D ATA M O D E L

Forklaring:

= Informationsobjekt

= Relation

= ”Er-en-slags”-relation

IO

Ordning Police

1..11..*

Manifesteres i

Medlem

Pensionsselskab 1..1

1..1

Ejes af

1..*

1..*

Tegnes af

Tilbud

1..1

1..*

Giver løfte om

Forsikrings-hændelse

1..1

1..*Er involveret i

Kunde

1..*

1..*

Er serviceselskab for

Indbetaling

Helbreds-information

Begunstiget

Pensionsoversigt

Sag

Sags-ekspedition

1..1

0..*

Indeholder

Indbetaler

Medlemskonto

Indberetning

Afgifts-indberetning

AndenOrdning

Bestilling

Arbejdsgiver

Notifikation

AndetSelskab

Medarbejder

Aktuaranalyse

Aktuar-beregningsgrundlag

Forsikringsteknik-grundlagRisikorapport

Aktuarensberetning/rapport Fremregnings-

rapport

Problem-rapport

Dødsfalds-Dokumentation

Investerings-parametre

Finans-notifikation

Fonds-aktiver

Benchmark-data

Faktura

Bidrags-afstemning

Ordre

Marketing-kampagne

IntranetIndhold

Stilling

Medarbejder-gode

Medarbejder-kompetence

Medarbejder-lønning

Ansættelses-brev

Fraværs-registrering

Medarbejder-udgift

Leder-handlingsplan

Leder-måling

Henvendelse

Skifte-modtager

FeatureEjendoms-

data

Overenskomst

Udbetaling

Budget

Regnskab

StrategiHandlings-

planMarkeds-

information

Arbejdsgiver-konto

Afgifts-konto

Økonomi-rapport

Hensættelses-rapport

1..*

1..1

Definerer

1..*

1..1

Indeholdes i

1..1

1..*

Afgiver

1..1

1..*

Sendes til

1..1

1..*

Udløser

1..1

1..*

Registrerer historie om

1..1

1..*

Afgør antagelse af 1..1

1..*

Informerer om1..1

1..1

Giver dækning for

1..1

1..*Indbetaler

1..1

1..*Giver indbetaling til

1..1

1..*

Afgiver

1..1

1..*

Retter

1..1

1..*

Er ansat i

1..1

1..*

Besidder

1..1

1..*

Benytter

1..1

1..*

Afholder

1..1

1..*

Modtager

1..1

1..*

Lægges for

1..1

1..*

Evaluerer

1..1

1..1

Gælder for

1..1

1..*

Registrerer

1..1

1..*

Får leveret

Pensions-administrations-

produkt

1..1

1..*

Definerer parametre for

1..1

1..*Danner basis for beregning til

1..11..*Notificerer om 1..1

1..*

Fastsætter niveauer for1..11..*

Sammenlignes med

1..1

1..*Kontrollerer

1..1

1..1

Indbetales fra

1..1

1..*Udløser betaling til

Pensionsmodtager

1..*

1..*

Sikrer dækning for

1..1

1..*

Serviceres af

1..1

1..*

Link1

E N T E R P R I S E D ATA M O D E L

CRM Service

Pension Service

Finance Service

Enterprise Data Model

T R A N S PA R E N C Y:

UNDERSTANDING YOUR SERVICES

S E RV I C E R E P O S I TO RY

• PROVIDERS• Service contracts

• CONSUMERS• Dependencies

CONTROL

Example: Organization changes

Dev Team 1

Dev Team 2

SOA - ESB

Review – architecture teamDev Team 1

Dev Team 2

EAI integration

Integration team

Dev Team 1

Dev Team 2

Database integration

Corba integration

Ad hoc integration EAI SOA

7

T H E E N T E R P R I S E S E R V I C E B U S

ENTERPRISE SERVICE BUS

Format a

Service Consumer

Service Provider

Format b

Consumer ProviderESBTransformation

Format A => Format B

Routin

g

Routin

g

Common ESB functionality

• Routing• Addressing• Transformation

• Activity monitoring• Policy enforcement• Logging

ENTERPRISE SERVICE BUS

Member Portal

CRM Service Pension Service Finance Service

Enterprise Service Bus

ENTERPRISE SERVICE BUS

Member Portal

CRM Service Pension Service Finance Service

Enterprise Service Bus – Intranet Segment

Enterprise Service Bus – DMZ Segment

DMZ Zone

Intranet Zone

MESSAGE EXCHANGE PATTERNS

Consumer

Provider

S Y N C H R O N O U S A S Y N C H R O N O U S

Consumer

Provider

Consumer

Provider

Request/Response Request/Response One-way

AN EFFECIENT SOA

IS

ASYNCHRONOUS

Synchronous = Poor scalability

Member Portal

CRM Service Pension Service Finance Service

Enterprise Service Bus

8

B U S I N E S S P R O C E S S M A N A G E M E N T

Business process• Workflows and queries• Automated processes• Rules• Related services

Real world• An example• Manual process steps• Automated process steps

“Key points”• Aim for reusable services• More business issues than technical

issues

The customer wants to• Reduce some manual activities over time• Optimize processes

• => Less waiting time for the members• Simplify the processes• Minimize bottlenecks• Evaluate performance to spot problematic workflows

The current workflow in the new system• Many activities are given by current laws• Manual activities exist in the system, given complex

decisions or necessary individual evaluation• Loose contract of communication formats (phone

conversation, letters)• Decisions based on human judgment• Hints about existing bottlenecks• Possible improvements can be identified

A real world example

Business Process on Services• Focus on main responsibility: Answer request.• Manual activities eliminated or wrapped in processes.• Some reusable services: Receive event service.

What if the customer wants to…• Change rules for archiving?• Use another document archive?• Change how events are handled in “new system”?• Include more parameters in the event sent between

systems?

T H E F U T U R E O F S O A

2011Common practice?

REST vs SOAPREpresentational State Transfer- a client/service pattern

• No standard for metadata (WADL to come?)• Operations and processes other than HTTP Verbs

tends to be messyGET /VechicleType/Matching?brand=O*

• Security is limited to HTTP/HTTPS. Difficult to encrypt some parts of message only.

• Really good for representing public data hierarchies.GET /VechicleType/Opel/Kadett/?year=1983

• Can be very easy to implement, but depends on message format (html/txt, json and xml).

Simple Object Access Protocol- a protocol for structured data and remote procedure calls

• Rich metadata standardization in WSDL• HTTP verbs can be used as operations.• Support for more transports of same operation

• Many standardized security policies for authentication, signatures and encryption.

• Possible to route messages with partially encrypted content.

• More natural in a Business Process context

• Can be difficult because of platform specific differences in protocol implementation

REST over SOAP is a possibility…

Cloud Computing

• Why Cloud computing?– Cost-efficient handling of uneven usage scenarios– Outsourcing of hosting

• Cloud computing and SOA– Architecture considerations

• Security• off vs. on-premise data and services

– SaaS– PaaS

S O A

I S H E R E T O

STAY!

w w w . n e t c o m p a n y . c o m