IBM Software Group Enterprise - cs.columbia.edudff/bpm2006.pdf · and Tracking Customer System Shipping Manufacturing Assembly ... displayed on this diagram) G02 - General Ledger

®

IBM Software Group

© 2006 IBM Corporationbpm2006 17-Sep-06

Enterprise –Business Process Management/Business Performance Management:Architecture, Technology, Standards

Donald F. Ferguson IBM Fellow, SWG Chief Architect, [email protected]

IBM Software Group Enterprise BPM: Architecture, Technology, Standards

2bpm2006 17-Sep-06

© 2006 IBM Corporation

AgendaIntroduction

DisclaimerA sample business problemContext and major trends

Business Process/Performance ManagementModel, and why more than “process.”Assemble and Customize; EDA and SOADeployManage/Monitor

Summary, Discussion and Challenges,and “A Grand Challenge!”


3bpm2006 17-Sep-06


AgendaIntroduction





4bpm2006 17-Sep-06


A Sample Business Problem – HW PO/Supply Chain

CustomerPOs for

PCs, SAN, …

Manual Entryand Tracking

CustomerSystem

Shipping

Manufacturing

Assembly

Warehouse/Staging

Warehouse


5bpm2006 17-Sep-06


E01-EDI

Data Warehouse(Interfaces to and from the

Data Warehouse are notdisplayed on this diagram)

G02 - GeneralLedger

A05 - AP

S01 - SalesCorrections

I01 POReceiving

I03 Return toVendor

I06 WarehouseManagement

MainframePC/NT apps Unix apps3rd Party Interface

S06 - Credit App

P15 EES EmployeeChange Notice

OTHER APPS - PCAP - Collections/Credit

TM - Credit Card DB

ACCTS REC APPS - PC990CORBad Debt

Beneficial FeesBeneficial Reconcile

JEAXFJEBFAJEBKAJEDVAJESOAJEVSAJEVSFNSF

TeleCredit Fees

INVENTORY CONTROL APPS - PCCode Alarm

Debit ReceivingsDevo Sales

Display InventoryIn Home

JunkoutsMerchandise Withdrawal

Promo CreditsRTV Accrual

ShrinkAP Research - Inv CntrlAP Research-Addl Rpts

Book to Perpetual InventoryClose Out Reporting

Computer Intelligence DataCount Corrections

Cross Ref for VCB DnldsDamage Write OffDebit Receivings

DFI Vendor DatabaseDisplay Inventory ReconcileDisplay Inventory Reporting

INVENTORY CONTROL APPS - PCDPI/CPI

IC BatchingInventory Adj/Count CorrectInventory Control Reports

Inventory LevelsInventory Roll

Merchandise WithdrawalOpen ReceivingsPI Count Results

PI Time Results from InvPrice Protection

Sales Flash ReportingShrink Reporting

SKU Gross MarginSKU Shrink Level Detail

USMVCB Downloads

Journal Entry Tool Kit

Scorecard - HR

L02-ResourceScheduling(Campbell)

P09 - P17Cyborg

M02 - Millennium

M03 - Millennium 3.0

Banks - ACH and Pos toPay

Cobra

B01 - StockStatus

S03-Polling

P14 On-line NewHire Entry

CTS

Plan Administrators(401K, PCS, Life,

Unicare, SolomonSmith Barney)

D01 Post LoadBilling

I04 HomeDeliveries

I02 -Transfers

Arthur Planning

I07 PurchaseOrder

I12 EntertainmentSoftware

I05Inventory Info

E13E3 Interface

S04 - Sales Posting

V01-Price ManagementSystem

I10 Cycle PhysicalInventory

I55 SKUInformation

K02Customer Repair

Tracking I35 Early WarningSystem

B02 MerchandiseAnalysis

I13- AutoReplenishment

U18 - CTO

Intercept

I09 Cycle Counts

E02-EmployeePurchase

Texlon 3.5

ACH

Stock Options

I17 Customer PerceivedIn-Stock

U16-Texlon

SiteSeer

C02 - CapitalProjects

F06 - FixedAssets

US Bank ReconFile

Star Repair

EDICoordinator

Mesa Data

NEW SoundscanNPD Group

AIG Warranty Guard

Resumix

Optika

Store BudgetReporting

P16 - Tally Sheet

Cash Receipts/Credit

S05 - HouseCharges

Ad Expense

L01-PromoAnalysis

V02-PriceMarketingSupport

BMP - Busperformance Mngt

StoreScorecard

I11 PriceTesting

Valley Media

P09Bonus/HR

I15 Hand ScanApps

Roadshow

POS

S08 - VertexSalesTax

A04 - CustRefund Chks

Equifax

ICMS Credit

CellularRollover

S09 - DigitalSatelliteSystem

NPD,SoundScan

Sterling VANMailbox (Value)

I18SKU Rep

X92-X96Host to AS400

Communication

S02 -Layaways

Washington,RGIS,

Ntl Bus Systems

V04-SignSystem

I14 Count CorrectionsNARM

P01-EmployeeMasterfile

I06 - CustomerOrder

FrickCo

UAR - Universal AccountReconciliation

DepositoryBanks

S07 - CellPhones

S11 - ISPTracking

AAS

Fringe PO

Cash Over/Short

L60 MDFCoop SKU Selection

Tool

SKUPerformance

SupplierCompliance

1

I35 - CEIASIS

Misc Accounting/Finance Apps - PC/NTCOBA (Corp office Budget Assistant)

PCBS(Profit Center Budget System)Merchandising Budget

AIMSMerch Mngr Approval

Batch ForcastingAd Measurement

AIMS Admin

AIMSReportingAd

Launcher

V03- MktReactions

SpecSource

CTO2.Bestbuy.com

RebateTransfer

SignSystem

CopyWriter'sWorkspace

ELTPowerSuite

StoreMonitor

AIS Calendar

Stores & Mrkts

Due Dates

Smart Plus

InsertionsOrders

BudgetAnalysis Tool

Print CostingInvoice App

AIS Reports

BroadcastFilter

Smart PlusLauncher

GeneralMaintenance

Printer PO

PrinterMaintenance

VendorMaintenance

Vendor Setup

Connect 3

Connect 3Reports

Connect 3PDF Transfe

Spec SourceSKU Tracking

S20-SalesPolling

Prodigy

PSP

In-HomeRepair

WarrantyBillingSystem

Process Servers(Imaging)

Prepared by Michelle Mills

Example: Complexity is Forcing Change

Actual Application Architecture for Consumer Electronics Company


6bpm2006 17-Sep-06


Some Challenges and TrendsChallenges

Seams between “people activities,” “information integrated” and “automated SOA activities”Choice may change in a solution over time

Reusable solution templates with customization/configuration.Policy and rulesInformation –

Processes are a mix of “documents,” “people” and choreography.Rich information model – PO, Customer, … …

Legacy integrationFederated/decentralized control and goals

Some trendsSOA and Web services, obviously.Improved but fragmented formal modeling standardsComing together of IT processes and business processes (MUWS)Coherent models for EDA and {BPM, EAI}Domain standardsGovernance.


7bpm2006 17-Sep-06


Business Process and Performance Management:Model – Assemble – Deploy – Manage/Monitor

Gather requirementsModel & SimulateDesignA “better napkin”

DiscoverConstruct & TestCompose

Integrate peopleIntegrate processesManage and integrate information

Manage applications & servicesManage identity & complianceMonitor business metrics

Financial transparencyBusiness/IT alignmentProcess controlCharge backWho saw what and did what for whom?


8bpm2006 17-Sep-06


AgendaIntroduction





9bpm2006 17-Sep-06


ModelBPM is more than process

Business Objects, Content Model (e.g. scans)Business and object stateOrganization (people)InteractionPolicyKey Performance Indicators (KPIs)/Observation, EventsBusiness Vocabulary

Standards matterRuntime interoperabilityFederated toolsPortabilityMonitoring and reportingEvolution, Substitutability

Currently a bit of a messUMLE-RWSDL, BPEL, …BPMNSBVR… …

Why Model?Slow, time consuming, … …Real programmers “extreme” program

Well, everyone modelsWhite boards, PowerPoint, …Napkins, crayons, … …

Why model formally?Precise notationProgrammers don’t guessReliable hand-off (“structure," code here”)Traceable and change management

Model in “business user”terms and concepts, notXSD or com.payroll. …

Standards are more than runtime protocols

Design TimeWhat does company A’s tools give to B’s?How does the caller know invocation seqs?What CA’s does the service support?

PortabilityFlexible placement over disparate productchoicesDecouple BPM from infrastructure evolution


10bpm2006 17-Sep-06


and more…abc…DB2

IBM ContentManager

Oraclexyz…

Heterogeneous Applications & Information

InsightIn-line, Real-timeDashboardsTools & Applications

Information as a Service(Information Virtualization)

Data & Content

MasterData

InsightfulRelationships Master Data, Entity

Analytics, Decision Portals, Exec Dashboards,Industry Data Models

Extracted or Real-Time

Standards based: e.g., XQuery, JSR170, JDBC,

Web Services...

Information as a Service –An Example of Going Beyond Process


11bpm2006 17-Sep-06


Why Use Information Services in SOA?

ReceiveAccount

Info

Customer Exists?

Create InfoReceiveAccount

Info

Customer Exists?

Create Info

Why use Information Server/Services?EfficiencySophisticated clean/link/… … functionsSimplifies process designReuse in other processes, as well asapplications, portal, … …


12bpm2006 17-Sep-06


AgendaIntroduction





13bpm2006 17-Sep-06


Service Components

Prolog

Types

Messages

Bindings

PolicyControl Descriptors

Deployment Descriptors

OperationsPort Types

JMS/MQHTTPIIOP… …

Impl.

Stub

• Required Interfaces• Events• Policy Points• … …

ESB

Container

“Resources”

“Abstract Process”“Business State”

BPELABO

CICS TPJAVASQL… …


14bpm2006 17-Sep-06


Service Components “Web services describes the outsides. How do you implement a service?

How do you compose services?”Encapsulate Components for Reuse; All look the same from outsideComponents may be wired together and aggregated via flowBusiness Objects are the data flowing on wires between ComponentsEnable type, role and skills specific tools.

Interface:How to call

this component


this component

Reference:What this

components calls

Reference:What this

components callsJava

Interface

JavaInterface

WSDLPort Type

WSDLPort Type

JavaInterface

JavaInterface

WSDLPort Type

WSDLPort Type

Component

UniformRepresentation of

EncapsulatedImplementation

BusinessProcess

BusinessProcess Business

State Machine

BusinessState Machine Business

Rule

BusinessRule Human

Task

HumanTask Interface

Map

InterfaceMap SelectorSelector JavaJava SQL

XQuery

SQLXQuery

Policy


15bpm2006 17-Sep-06


Service Components “Web services describes the outsides. How do you implement a service? How do you

compose services?”Encapsulate Components for Reuse; All look the same from outsideComponents may be wired together and aggregated via flowBusiness Objects are the data flowing on wires between ComponentsEnable type, role and skills specific tools.


this component


this component

Reference:What this

components calls

Reference:What this

components callsJava

Interface

JavaInterface

WSDLPort Type

WSDLPort Type

JavaInterface

JavaInterface

WSDLPort Type

WSDLPort Type

Component

UniformRepresentation of

EncapsulatedImplementation

BusinessProcess

BusinessProcess Business

State Machine

BusinessState Machine Business

Rule

BusinessRule Human

Task

HumanTask Interface

Map

InterfaceMap SelectorSelector JavaJava SQL

XQuery

SQLXQuery

Policy


16bpm2006 17-Sep-06


A Simple Example and Some Concepts

public class QuoteRequest {String ticker;Date when;

}

Public class quoteResponse {String ticker;float value;float sharesTraded;

}

/* Pragma This;Pragma That;*/SELECT

ticker,value, activityFROM

StockQuotesWHERE

ticker == QuoteRequest.tickerINTO

quoteResponse.tickerquoteResponse.value,quiteResponse.sharesTraded;

Deployment Tools

SDOs

Something a DB duderecognizes

Interface(s)

Generated CodeInterpreted Metadata

Data Svc.

Deployment Package

Author Toolsor

Text Editor


17bpm2006 17-Sep-06


Assembling Services – Modules and Subsystems

get Approved

Approved/Denied

If Approved thenSend letter offering gold

If NOT ApprovedSend letter offering Credit counseling service

UDDI, Service RegistryProject, SCCS

ad hoc exchange,… …

Simple wiring metaphor and tools.

Unaware of “how” the component works.

May “drill into” a specificcomponent’s details


18bpm2006 17-Sep-06


Services Oriented Architecture and MediationsCommerce Service

Needs a CCService

But it’s not quite right. Darn

Credit Card Service

Found a CC Service

Mediation

TransformRoute

AugmentSide Effect

Policy SelectionMatching


19bpm2006 17-Sep-06


SCA, SOA and Event Driven Architectures

--------------------------------------------------------------------------Wiring is a simple model

butis limited

-------------------------------------------------------------------------- --------------------------------------------------------------------------

TransparentMediation

Implements

--------------------------------------------------------------------------

Emit Event

--------------------------------------------------------------------------

SubscriptionFilter

Topics

FilterAnalyzeRules


20bpm2006 17-Sep-06


Customizing Services – A “Design Pattern”

Control DescriptorsDeployment Descriptors

Impl

Prolog

Types

Messages

Bindings

OperationsPort Types

Required “Services”document POVs

Simple rule templatesJava, JavaScript

Rule EnginesDecision TableDecision Tree

… …

POV = Impl. Model for calling a service,e.g. JAX RPC Stub, BPEL Invoke

Bind using• Date/Time• Application Version• Message Predicate•… …

Web Service Binding (Find)

Mediate

Call

Just another “service”


21bpm2006 17-Sep-06


Services and ComponentsThere is a set of standards (emerging) for formally representing a service component’s behavior

WSDL, XSDBPEL, UMLWS-PolicyWS-ResourceFramework

There is an emerging, extensible set of service component kinds that provide a natural mapping for model elements

Process, Business State Machine, SelectorEliminates the “miracle happens here” modelHigh level, portable implementations emerging

Support for very dynamicStructural compositionBehavioral compositionConfiguration/customization

Bridge to existing skill sets.This “BPM” solution is a set of

documents enabling a “wiki like” BPM evolution.


22bpm2006 17-Sep-06


Some PerspectiveHaven’t we heard this before? OO, RPC, MDP, … …?

This time we really mean it.Can’t you take a joke?

There are some differencesXML is language neutral; previous approaches implied a language model.WSDL and XML are more forgiving of changesSupports RPC and message/document approaches from beginningCommon type model for applications/servers, message systemsand DBsBuilds on Internet protocols already deployed for “Web browsing.”Uniform model for events/pub-sub, message routing and RPCMore focus on logical behavior – WSDL, Policy, etc.SOA component model derives from business modeling, making MDD simpler and eliminating “spooky transformations!”


23bpm2006 17-Sep-06


AgendaIntroduction





24bpm2006 17-Sep-06


A Patch Management – Implementing the Process

Questions:How best to implement the process and make it operational?What tools should be used for sequencing work between people?What tools for automating particular activities in the process?

Release Mgmt

Test the change

ApplyPatch

VerifyPatch

DocumentImpacts

Open other RFCs

Package the change

Change Mgmt

Open RFC

CategorizeRFC

Assess Impacts

Review &ApproveChange

CloseRFC

DeployChange

SelectServers

QueryTPM

QueryDBs

JoinResults

SelectSubset

SelectChange Window

ScheduleChange

DeployPatch

CheckResults(verify)Change

UpdateConfig

DB

Rollback ManualPatching

Done?

CheckPolicy

(Business) Process Engine

Need to have:Logical Process implemented

in executable workflows User Interfaces for process stepsAdapters to Management AppsDocumentation to customize steps

Task 2

Task 4

Task 3

Task 1 Task 6

Task 5Process Instance

Infrastructure Mgmt tools automate some process steps

Human review and approval

steps

Task 2

Task 4

Task 3

Task 1 Task 6


Infrastructure Mgmt tools automate some process steps

Human review and approval

steps

SOA Infrastructure


25bpm2006 17-Sep-06


Building IT Process FlowsModel Assemble

Deploy

Provide portal console and run process using

Process Choreographer

(WPC)

Task 2

Task 4

Task 3

Task 1 Task 6


Infrastructure Mgmt tools automate process steps

Human review and

approval steps


26bpm2006 17-Sep-06


The Role of the Container

The Impl.

Security HeaderReliable Messaging HeaderAtomic Transaction Header

SOAP Message

double deposit(Message m) {checkForDuplicate(m.seqNo);registerForTransaction(m.context);isCAValid(m);checkSignature(m);updatePerformanceInfo();

balance += m.amount;

// … …updatePerformanceInfo();

}

This is fragile,changes over time,

complex for business programmers,error prone,

etc.

Policy Declarations


27bpm2006 17-Sep-06


The Role of the Container

The Impl.

Security HeaderReliable Messaging HeaderAtomic Transaction Header

SOAP Message

Wrapper

Container

Before After

Security

Reliability

Transactions

Ack.Retransmit

Check CertificateChallenge

etc.

Container is a set of policy driven functions.Interceptor pattern for business logic and “stubs.”Before and After factoring of code.


28bpm2006 17-Sep-06


4 Principles Of Application Management ConfidenceA repeatable approach to sense and respond to performance problems within the composite application infrastructure.

Measure Response Time

TraceTransactions

MediateServices &

Enforce Policies

Monitor & Adjust Resources


29bpm2006 17-Sep-06


A Typical Management/Monitor Portal

Topology View:aggregate interactions among services.

Topology View:aggregate interactions among services.

Sequence Diagram:Shows exact sequence of messages over time

Sequence Diagram:Shows exact sequence of messages over time

Statistics View:A table view of the raw data collected by the monitoring agent at each interception point

Statistics View:A table view of the raw data collected by the monitoring agent at each interception point

Content View:Shows content of a

SOAP message

Content View:Shows content of a

SOAP message


30bpm2006 17-Sep-06


Business Process/Performance Manage/Monitor

SOA infrastructure monitoringDynamic and policy driven“Transparent” to application logicSense – Respond – Act via MUWS

Observation/Performance ModelingObservation points are part of modelingDefine scorecard view of Key Performance Indicators

SOA event infrastructure and event databaseEnables

Dashboard, monitoring, management.Ability to intervene in deployed processes

Set situational triggers and notificationsDynamically respond to these alerts

Supporting continuous process improvementMonitor in-flight business processesMake process modifications based upon real-time data sent back to the Modeler for simulations


31bpm2006 17-Sep-06


Deploy, Manage and MonitorSystem and application management is a business process

Complex, multi-step process with compensation for change managementPatches, OS/middleware upgrades, application enhancementsAutomated steps, manual approvalCompensation, recovery, retry

Common Management (Information) DatabaseEvents, Sense/Respond, Monitor, Act, History, reports

Using “standard” BPM technologyEnables existing skills (e.g. no arcane SM language and APIs)First class tool support

The line between systems/application management and BPM is an illusion

Why are PO submissions failing? IT error or process design?Impact analysisThink about “customer on-boarding.”

Some updates to CRM, account system, etc.Also calls to security for UID, ACLs, … …


32bpm2006 17-Sep-06


AgendaIntroduction





33bpm2006 17-Sep-06


Summary and DirectionsBusiness Process/Performance Modeling/Management

Is a broad space with many “sub-models.”We are only beginning to

Bring them into a coherent wholeWithout overwhelming business professionals and IT professionals

We must drive standards and convergence. Standards are broader than runtime formats and protocols.

Tool federationFlexible function placementEvolution

There are many open areasGovernanceWeb 2.0Explosion of casual programmers in the workplaceModeling support for Recipes, Patterns and TemplatesService/component identification, factoring, good “size,” … …


34bpm2006 17-Sep-06


A Grand ChallengeThere is good and improving models for Process, Information and State,

EventsComponentsUse casesComponent collaboration… …

Policy is where we are the most “broken”Examples

“All POs over $10,000 must be approved by regional sales manager.”“An employee cannot close a customer complaint that he created or marked complete.”“Business class is authorized for flight more than 8 hours or overnight.”

We typically write these down in text.Programmers read the text and write code.

Can we do better?Many domains have nascent business vocabularies, e.g. law, dentistryWe write specs with nascent “grammars,” e.g. MUST, CANNOT, RECOMMENDDocuments are often simple combinations of policy

Nested lists (Decision Trees)Tables/Forms (Decision Tables)Hyperlinks (Decision Flows)

Can we improve hand-offs and traceability for rules and codes through structured language, business vocabularies


35bpm2006 17-Sep-06



36bpm2006 17-Sep-06


Some ClarificationAn Asset is, well an Asset. Can be anything

Word document, Powerpoint PresentationHandy code that I keep lying aroundExcel spreadsheet for costing a project… …

A Pattern is a recurring solution for solving a problemPatterns for eBusiness (http://www-106.ibm.com/developerworks/patterns/)Enterprise Integration Patterns (http://www.eaipatterns.com/)J2EE Patterns (http://corej2eepatterns.com/index.htm)… …Read the book and start typing!

A Template is a Pattern (or sub-pattern) thatHas associated metadataComes with a design time control (Wizard)Uses code generation or “data driven behavior” to convert to an instance.

A Recipe is an directed graph of Templates, with composite controlsWhich arcs to followMetadata flows through the graph as you follow the recipeSubsets, augments, modifies the constituent patterns.

A Solution Template isA complete solution, with install imagesWell-defined POVs for tailoring the elements and wizards

We should be morehelpful than this

This is my terminology.We are trying to come to a

simple, common terminology.


37bpm2006 17-Sep-06


Pattern Authoring in Rational Software Architect1. Create Plugin Project with Patterns Templates

2. Create the skeleton of the pattern. Define parameters for the pattern and constraints on the parameters

3. Add your custom code using the RSA Pattern Framework. It provides many extension points

6. Publish Asset to a RAS repository5. Export the pattern as a RAS asset. Classify your pattern to facilitate searching

4. Test pattern by launching a new runtime workbench


38bpm2006 17-Sep-06


Example of applying a Pattern1. Locate and Import the Asset from a RAS repository 2. Select the Pattern from Pattern Explorer

4. Run a Transformation to convert models into Artifacts (code, scripts, docs)

3. Specify Pattern Parameters


39bpm2006 17-Sep-06


Some PrinciplesMinimize concepts and rely on patterns

Pub-sub is a SOA patternMediation is a style of serviceBusiness rules are a way to implement services … …

There are very few new thingsFocus less on code reuse and more on pattern reuseThere is no difference between pattern, code, developer toolBasic building blocks (e.g. SCA) enable flexible patterns

BenefitsProductivityReduce risk, more predictable projects, technical community, …

There are two tiers of programmerArchitecture/Pattern/Template providerTemplate/Pattern user

InstantiateComposeConfigure

Code for customization


40bpm2006 17-Sep-06


To DoGovernanceITSMHaven’t we heard this before?Web 2.0/ESRIService Identification and FactoringComponent Business Model and abstract modelsEveryone is a “casual programmer”Patterns