Modern Enterprise Integration Strategies

Agenda

• About US• The enterprise integration landscape• Enterprise application integration servers• Enterprise service bus• Integration platform as a service• New trends in enterprise integration

The Enterprise Integration Landscape

Types of Integration Platforms

Early 2000s

Enterprise Application Integration Platforms

2005+

Enterprise Service Bus

2011+

Integration Platform as a Service

2014+

Microservices Architecture

Enterprise Application Integration Platforms

Goals

• Based on the traditional enterprise integration patterns architecture • Enable integration between different enterprise line of business

systems• Facilitate long running processes• Support for B2B standards like EDI or Rosettanet

Technological Foundation

• Orchestration engine• Business rules platform• Business activity monitoring • B2B interfaces

Strengths

• Implementing long running processes • Supporting B2B processes • Rapid integration with line of business systems

Challenges

• Expensive to implement • Lack of extensibility • Lack of support for modern architecture styles like microservices or

cloud computing

Platforms

• Microsoft BizTalk Server• Tibco Businessworks• Oracle SOA suite• IBM WebSphere Process Server• Software AG • …….

Enterprise Architect Standpoint

• Implementing enterprise app integration server is an expensive endeavor both technologically and financially

• Require on-premise infrastructure• Development using proprietary workflow and orchestration tools• Training • Challenge hiring talent

EAI Example: BizTalk Server

BizTalk Server Capabilities

Management and

Operations

RFID Platform

Business Rule Framework

Business to Business

Integration Business Activity

Monitoring

Messaging

Orchestration

Tools

AppFabric Connect

Integration

BizTalk Server Integration Scenario

14

SuppliersApplication

11

33

22

44

Inventory Application

Enterprise Resource Planning (ERP)Application

Microsoft BizTalk Server 2010

IT Pros and Developers

Manage and

Operate

Design and Implement

RFI

55

EDI

Messaging

Messaging Assign SupplierPolicy

Filter TagsPolicy

Re-stock OrdersOrchestration

Read Shipment RFID Tags

Up-to-date KPIs in BAM

Business Users

Messaging

15

BizTalk Server Runtime Architecture

www.devscope.net

Receive Port Orchestration

XML EDI or Flat File

XML EDI or Flat File

Send Port

SendAdapter

SendAdapter

SendPipeline

SendPipeline

MessageBox

MappingMapping

TO: NWTraders (Flat file format)TO: NWTraders (Flat file format)

FROM: Fabrikam (XML format)FROM: Fabrikam (XML format)

MappingMapping

FROM: Contoso (Flat file format)FROM: Contoso (Flat file format)

TO: Fabrikam (XML format)TO: Fabrikam (XML format)

ReceiveLocation

ReceiveAdapterReceiveAdapter

ReceivePipelineReceivePipeline

Enterprise Service Bus

Goals

• Provide integration capabilities in service oriented solutions• Enable a lighter, messaging-centric model to integrate services in a

SOA architecture

Technological Foundation

• Messaging channels• Services• Topics, queues• Routing rules

Strengths

• Lightweight messaging between services• Abstract infrastructure capabilities such as routing and transformation

in a SOA solution • Enable traditional messaging patterns such as persistent messaging,

reliable messaging, retransmissions, multi-cast, uni-cast, publish-subscribe etc

Challenges

• Lack of B2B support• Scalability challenges with centralized messaging models• Fundamentally based on .NET and J2EE programming models• Lack of support for modern architecture styles such as APIs, micro-

services etc

Platforms

• MuleSoft• WSO2• BizTalk ESB Toolkit• Oracle ESB• WebSphere ESB

ESB Example: Mule ESB

O N N

ESBFrameworks

Apache ServiceMix

Sun (Oracle) OpenE

SB JBossESB

Fuse ES

B

Mule ESB

Messaging Framework

•Routing Connectivity (HTTP, JMS, FTP)

• Cloud connect (Salesforce, Facebook, Twitt er)

Web Services (SOAP, REST)

Flow

Message source

Message process

or

Componen

t

Transforme

r Filter

Router

Endpoint

Subflow –Private Flow

Message

Properties

Variables

Payload

Attachment

s

3 <mule xmlns:jdbc="http:// www.mulesoft .org/ schema/mule/jdbc " xmlns:sftp="http:// www.mulesoft .org/ schema/mule/ sftp " xmlns:mu lexml="http:// www.muLesoft .org/ schema/mul

4 http://ww1uulesoft .org/ schema/mule/xml http://ww11.mulesoft .org/ schema/mule/xmL/ current/mule-xmL.xsd5 http:// www.mulesof t.org/ schema/mule/ sf tp http:// www.mulesof t.org/ schema/mule/ sf tp/ current/mule- sf tp.xsd6 http://www .mulesof t.org/ schema/mule/ jdbc http://www .mulesof t.org/ schema/mule/ jdbc/ current/mule-jdbc .xsd7 http:// www.springframework.org / schema/ beans http:// www.springframework.org / schema/ beans/ spring-beans-current.xsd8 http:// www.mulesoft .org/ schema/mule/ core http:// www.mulesoft .org/ schema/mule/ core/ current/mule.xsd9 http:/ /111111.mulesoft .org/ schema/mule/ scripting http:/ /111111.mulesoft .org/ schema/mule/ scripting/ current/mule- scripting .xsd ">

10 <flow name="VcuPreApprSFTP " doc:name="VcuPreApprSFTP ">11 <jdbc: inbound -endpoint queryKey="GetPADataForVCU" queryTimeout="-1" pollingFrequency="600000" connector-ref="NySQL"

doc :name="GetPADataForVCU">12 <jdbc :query key="GetPADataForVCU" value="call sp_GetPreApprovedinfo(173,$ {last.approved.hours}) "></jdbc :query>13 </jdbc :inbound -endpoint>14 <logger message="####Pay load #(message.pay load)" level="INFO" doc:name="Logger"></logger>15 <choice doc:name="Choice">16 <when expression="pay load( 'resultsetl '] .size()==@">17 <processor-chain >18 <logger message="No Data Found for VCU PA" level="INFO" doc:name="No Data Found"></logger>19 </processor-chain>20 </when>21 <othemise>22 <processor-chain>23 <component class="com.chromeri ver.mule.export .preappro ve.component .VCUPreApproveDataCompoent"

doc :name="VCUPreApproveDataComponent" ><Icomponent24 <mulexml :object-to-xml-t ransformer doc:name="Object to XNL "></mulexml :object-to-xml-t ransformer>25 <sftp:outbound-endpoint exchange-pattern="one-way " outputPattern="PreAppr_#{function:dates tamp:yyyyf Yl.ddHHmmss] .xml"

host="${ftp.panam. host} " po26 </processor-chain>27 </othemise>28 </choice>29 <rollback-exception-st rategy doc :name="Rollback Exception Strategy"></rollback-exception-st rategy>30 </flow>313233

</mule>

flo VcuPreApprSFTP @::'

- - - - - - - - - - ---- - - - - - - - - - - -- - - - - - - - - - - --- - - - - - - - - -

Processor Chain @::'

f lGetPADataFor

VCU

Logger

Choice

No Data Found

Deffault·- - - - - - -:------------------

Processor Cha in @::'

I l l ) aObject to XML SFTP toVCU

Folder

I IVCUPreApproveDataComponent

Filter:

Select

'aEndpoint

s• Hf t i'lpl'. w -•

CS)IMA

P

Q JMS

HTIP

->SOA

P

- --

>

• -> ·Je

Eli Scopesfill Components

custom-interceptor

WsMatterTransformers<X>mlByte-

AT.1t f u "Se1ializable

mlByte Array to String

mlEmail to String

( ? ,Filters( ? .Flow Control

Error Handling

(e Cloud Connectors

Iii!Amazon SQS

G D CMIS

0 Facebook

M essage Flow

Global Elements

Configuration XM L

Enterprise Architect Standpoint

• Implementing a ESB platform should be part of a broader SOA strategy

• An ESB implementation will require professional services and training

Integration Platform as a Service

Goals

• Deliver integration in a cloud computing model • Enable seamless integration with SaaS platforms • Leverage cloud infrastructure to run and scale integration solutions• Facilitate cloud-to-on-premise integration models

Technological Foundation

• Adapters• Messaging channels• Services• Topics, queues• Routing rules • On-Premise agents

Strengths

• Deep integration with modern SaaS platforms• Based on simpler architectures than its predecessors

Challenges

• Lack of support for complex orchestrations and choreographies• Not designed for pure on-premise integration scenarios• Security, regulatory and compliance challenges

Platforms

• Azure BizTalk Services• MuleSoft CloudHub• IBM Castiron Live• Dell Boomi

iPaaS Example: Azure BizTalk Services

Azure BizTalk Services

Extensible PlatformCustomized solutions for you scenarios

Managed Service Focus on business challenges

Config-Driven ExperienceLower time to market & cost of development

Cloud-Scale ESBCost effective, scalable infrastructure

Bridges in BizTalk Services

Basic building block for building your integration platform

A Bridge is a single message processing unit with 3 parts: Sources: From where the messages originatePipeline: Which processes the messages [Flat file, XML, Pass-through]Destinations: Where the messages are sent to

TransformationGraphical UI tool to define your map

OOB Support for more than 35 operations• Supports custom XSLT and inline

C# scripts

Test the map locally and validate the output as well

PropertyAny interesting data in your message• Defined in the Enrich stage of a pipeline• Can be tracked as part of message processing

Property source could be from:• Transport headers/properties• A part of the message (specified using XPath)• An external data lookup• System properties

Use in Filter Rules/Actions for routing or in Maps

Supported destinations auto-convert properties into meta data

Routing

Rule based routing• Message delivered to first matching

destination

Rules use the same SQL92 syntax used in Service Bus Topic/Subscription Rules

Route Actions may be used to write destination specific headers

Hybrid Connectivity

Cloud Application Bridge

FTP/SFTP

HTTP

WCF

Blob

Service Bus

Database ERPBizTalk Adapter Service

Server Explorer(Visual Studio)

PowerShell CmdLets

Management Service REST API

Lob Relay (Service Host)Lob TargetLob Target

Enterprise Architect Standpoint

• Implementing an iPaaS should be part of a broader cloud strategy• iPaaS should be focused on on-premise to cloud integration scenarios

Modern Integration Strategies

Some Thoughts

• The industry has evolved from centralized to federated models• Cloud infrastructures are called to play an important role in

integration solutions• SOA is being replaced with new architecture models based on APIs

and Microservices• Modern integration strategies should take into consideration mobile

and IOT solutions

Microservices

Principles Of Microservices

Modelled Around Business Domain

Culture Of Automation

Hide Implementation Details

Decentralise All The Things

Isolate Failure

Deploy Independently

Highly Observable

DUMB-PIPES, SMART ENDPOINTS

Magical Mystery Bus

Principles of Microservices Integration • Focus on messaging brokering capabilities• Routing• Transformation• Protocol adaptation• Integration infrastructure optimized fro working with hundreds of

endpoints• Highly dynamic, continuously changing in

API-Oriented Architecture

In the beginning….

Data

Application

Proliferation

ApplicationData

CRM

DataERP

Data Risk

Application

Applications Become Services

Service interface

Data

Service implementation

SOAP, JMS, XML…

Services in the Network

Controlled Interconnection and Re-use

Data

CRM

DataERP

Data Risk

A Service Bus

Data

CRM

DataERP

Data Risk

The Service or Message Bus

OSS

Other Corporate

SVs

BSS

EnterpriseOrchestration

BPM

BRMS

Application

REMOTE DATA CENTERDeployment Option C

REMOTE DATA CENTERDeployment Option B

REMOTE DATA

CENTERDeployment

Option A

Application

Application

Application

ApplicationOrchestration

BPM

BRMS

DOMAIN ESB

Platform

Application

Application

ApplicationOrchestration

BPM

BRMS

DOMAIN ESB

Platform

Application

ExternalMashups

ExternalMashups

Hosted Services(e.g., google,

facebook)

BPM

BRMSJava

Service DeliveryOrchestration

Identity & AccessManagement(Common)

MAIN ESB – A

API Gateway Internal Mashups(e.g., web 2.0)

ESB Gateway

InvocationModule

APIDirectory

ExternalUsers

MAIN ESB – B MAIN ESB – CORPORATE SVCs

Big Data

Profile DB

Service Events Mgmt,Processing & Continuous

IntelligenceOther

Modules

We’ve All Been Here

Changing Constraints

2005: Yahoo Maps API

2005: Flickr API

2007: First iPhone & SDK

Service Orientation? Meet the Smartphone

?

SOAStrict contractsBasic ConnectivityFew, known developers,

Strictly managedSlow, unreliable,

segmented networks

APIsLoose contractsMinimal “stack”Self-service1000s of developersO(10^4) users, externalAlways connected apps

Different Sets of Requirements

Changes in Technology

Why OAuth and not WS-Security?

Why JSON and not XML?

Why the focus on HTTP?

Why add Developer on-boarding?

Why add Analytics?

The metaphor gets us to think differently.

Rapid iteration

Programmability from anything

Loose documentation

YAGNI

APIs as Design Metaphor

Thanks