View
31
Download
2
Category
Preview:
DESCRIPTION
IBM Support for FIA (FIXML) Messaging Enterprise Service Bus View of Standards The Evolution of Messaging for Financial Services March 2005. Jim Glenister IBM Consultant, WebSphere Software. WMQ and FIXML Standards for Financial Services. FIA to spearhead related efforts:. - PowerPoint PPT Presentation
Citation preview
®
IBM Software Group
© 2005 IBM Corporation
IBM Support for FIA (FIXML) Messaging Enterprise Service Bus View of Standards
The Evolution of Messaging for Financial Services
March 2005
Jim GlenisterIBM Consultant, WebSphere Software
IBM Software Group | WebSphere software
2
WMQ and FIXML Standards for Financial Services
POST TRADE VMU
- TRADE DATA (NOE) - CLEAR & SETTLEMENT
TRADING & REPORTING
MKT DATA ORDER REPORTING EXECUTION
MARKET DATA
RETRANSMIT REQUEST
VALUE ADDINGMKT INFORMATION PROVIDERS
PRIMARY FINANCIAL MARKETS SYSTEMS (EXCHANGES, NASD)
SECURITIES FIRM OFFICES
RETRANSMITTED DATA
LAST SALEBID - ASK
TRADES
SECURITIES FIRM
ORDER + TRADE PROCESSING
TRADESP + S
CONFIRMSMARGIN CALLS
ORDERS EXECUTIONS
ENTERPRISEMESSAGING
FINANCE INDUSTRY SHARED ENTERPRISE SERVICE BUS
ORDERS +INQUIRIES
REPORTS +CONFIRMS
MDS INQUIRIES
MKT INFO
TRADES COMPARISON
POSITION
CLIENT DIRECT MESSAGING
ORDERS +INQUIRIES
REPORTS +CONFIRMS
COLLATERALPOSITION
NET SETTLEMENTPOSITION
STP SERVICE PROVIDERCUSTODY + CLEARING
(OCC, DTCC, Other) Future SystemsT+1 (T+0) , STP
ORDERS EXECUTIONS
SECURITIES FIRM ACCOUNTINGCLEARANCE
SETTLEMENT
SCOPE OF FIXMLMESSAGING STANDARDS
INVESTMENT MANAGER
ACCESS GW
PRIVATE CLIENTACCESS GW
IBM Software Group | WebSphere software
3
FIA to spearhead related efforts:
Message Descriptors
Standardize on the usage of Message Descriptors
Put and Get
Guidelines for setting fields and Options in an MQ-Put
Guidelines for setting fields and Options in an MQ-Get
Queue Definition
Queue Depth based on volume
Maximum Message Length
Message Data Formats
Standardize on the use of ASCII for FIXML messages
Dead Letter Queues
Processing messages from Dead Letter Queues
Channels Exit Routines
Hardware requirements / Software requirements
Establishing standards for message transmission
Exchanges and Firms to work with FPL to define a FIXML messaging API for re-transmission of messages
MessageDB
Origin
Destination
Date/Time
Sequence Nbr
Type
IBM Software Group | WebSphere software
4
How WMQ (MQSeries) Works – FIA Standards
Message Queue Manager
Q1MessageQueuingService
MessageChannel
Agent
Requesting Application
PUT MSG
MQSeriesInterface
Message Queue Manager
MessageQueuingService
MessageChannel
Agent
Serving Application
GET MSG
MQSeriesInterface
MQSeriesInterface
Q1 Q2
XMIT Q Network
Serving Application
GET MSG
MSG MSG
DLQ
IBM Software Group | WebSphere software
5
WMQ Queue Manager Structure – FIA Standards
QueueManagerKernel
Command Queue
Event Queues
Data Conversion Interface
Security Enabling Interface
Name Service Interface
PluggableMessageChannelAgents
PluggableTriggerMonitors
Channels
Queues
Performance
ExitsDLQ
IBM Software Group | WebSphere software
6
Messaging Service Environment Characteristics
Multiple Platforms (zOS/OS390, UNIX , Windows NT/2000, AS/400, Tandem) "Store and Forward" of Messages Assured Delivery (Once, and only once) of Message Objects Persistent or Non-Persistent Messages (Designated by message) Priority Levels on Messages Purge of "Killer" Messages that Repeatedly Cause Failures Message Expiry (Time-to-live for a given message) 24 x 7 Continuous Operation Transactional Integrity as a Resource Manager and Recovery Coordinator Failover and Automated Recovery of Messaging Access to Business Services High Volume Performance with Peak Load Handling Mechanisms Flexible Message Resource Naming (Alias names that reflect business service ) Security (Protected message resources, data privacy via encryption, intrusion protection) Automated Management Controls (Programmed commands, monitoring, event alerts) Feedback: Confirmation and Notification Reports Message Logging (Optional redundant logging) Large Objects ( --> 100MB)
IBM Software Group | WebSphere software
7
Messaging Application Characteristics
Multiple Platforms (zOS/OS390, UNIX , Windows XP/2000, AS/400, Tandem) Consistent APIs in Multiple Programming Languages (Java, C, Cobol, Others) "Business Service Name" Transparently Mapped to Message Destinations Transparent Placement of Messages on Local or Remote Queues Support for Synchronous and Asynchronous Style Shared or Exclusive Queue Usage Message Arrival Triggers Initiation of Server Processes Multiple Servers Concurrent Access to Queues with Preservation of Message Integrity Correlation of Requests and Corresponding Replies Selective "Get" of a Specific Message from the Message Queue Non-destructive Browsing of Message Queues Selective Commit/Rollback (Syncpoint) Controls for Unit of Work Integrity Support for XML Messages Applications Have No Communications Network Responsibility or Awareness Multiple Programming Models Supported by Messaging
Client-Server One-way Event Push (Asynchronous "Fire-and-Forget" Datagram) Publisher-Subscriber Object Oriented Design (e.g., JMS Message Driven Beans) Multi-step Workflow Processing
IBM Software Group | WebSphere software
8
Messaging Architecture – Point-to-Point Patterns
Sv
Sv
Sv
Sv
Sv
Sv
Sv
Rq
Rq
Rq
Rq
Rq
Rq
Rq
Tp
MESSAGEHUB
Peer Applications 'One-way Event Push'
Peer Applications 'Requests-Replies'
Peer Applications 'Third-Party-Reply'
Peer Applications 'M x N via Mesh'
Peer Applications 'M x N via Hub'
SvRq M
Peer Applications 'One-way Event Push'(with Confirmation)
IBM Software Group | WebSphere software
9
Messaging Architecture - Publish and Subscribe Patterns
Ms
Sb
Sb
Sb
Sb
Sb
Sb
Sb
Sb
Sb
Sb
Ms
B
Sb
Sb
Sb
Pb
Pm
Ps
Ps
Fs
Publish to Subscribers'Broadcast Datagram'
Publish to Subscribers'Multicast Datagram'
Publish to Subscribers'Selective by Topic'
Publish to Subscribers'Selective by Content via Subscriber Filters'
IBM Software Group | WebSphere software
10
WMQ: Design for Performance
Exploitation of WMQ Capabilities Priority scheduling of messages and processes Thresholds on queues Load-responsive multiple server instantiation (e.g., Triggers) Alias queue name = service name (can inhibit G/P, change priorities) Message channel and queue activity (persistent, non-persistent) Logical unit of work (syncpoint) controls (MQCMIT, MQBACK) Batch transmissions across channels
Efficiencies for Throughput and Fast Response Long running tasks (eliminates startup/shutdown transients) Persistent-connection channels between queue managers Parallel processes (e.g., Logging can be asynchronous via MQ) 'Fast Path' techniques
Resources Allocated to Meet Critical Peaks (Chaos Control) Deferred processing of non-critical requests Deferred processing of secondary activities MQSeries Log on separate, high speed devices Page sets and buffer pools
IBM Software Group | WebSphere software
11
WMQ High Availability - Cluster Groups
Transparent Access to QMs and Queues in a Cluster Workload Balancing via WLM Exit Failover to Surviving Members of Cluster Group Application Affinity (Requester-Server) if Required Administrative and Operational Support (e.g., Q Definitions, Suspend/Resume)
REQUEST
REPLYREPORT
REQUEST
REPLYREPORT
REQUESTER 1
REQUESTER n
REQUEST
REPLYREPORT
REQUEST
REPLYREPORT
REQUEST
REPLYREPORT
SERVER 1
SERVER 2
SERVER n
MESSAGE HUB SITE 1
MESSAGE HUB SITE 2
MESSAGE HUB CLUSTER GROUP SERVER CLUSTER GROUP
ROUTINGLOGGINGFORMATTINGASSURED DELIVERYRECOVERYREPORTING
ROUTINGLOGGINGFORMATTINGASSURED DELIVERYRECOVERYREPORTING
QMR1
QMRn
QMS2
QMSn
QMS1 SERVERAPPLICATIONREQUESTER
APPLICATION
QMH1
QMH2REQUESTERAPPLICATION
1 2
43
SERVERAPPLICATION
SERVERAPPLICATION
IBM Software Group | WebSphere software
12
WMQ High Availability - z/OS V5.3 Shared Queue Groups
CF
Shared DB2
Shared Qs
WAS, CICS TS1
APPLICATIONS
zOS1
WMQ QM1DB21
Channel
WLM ARM RRS CS
WAS, CICS TS2
APPLICATIONS
zOS2
DB22WMQ QM2
Channel
WLM ARM RRS CS
SYSPLEX
Persistent shared messages kept in CF list structuresDefined as Shared Queue GroupsMessages logged and recoverable by MQBackup dataset is the log datasetPhysical message size still < 63KBPeer Recovery - Transactional recovery in concert with CS/390, CICS TS, DB2
Message Grouping supportAddresses 63KB shared message size limit in 5.2
SSL supportEnables MQ to MQ security
24x7 supportRemove the need for OS/390 IPLs with new MQ release
IBM Software Group | WebSphere software
13
Message Flow Standards via WMQ (MQSeries) and ESB
FIXML ClientBusiness Transaction
Gateway Service API
FIXML InterfaceApplication Services
Message Queuing API
Messaging Services
Network Transport
BUSINESS SERVICE INTERFACEDIRECTORY NAME SERVICESMESSAGE LOGGING + ARCHIVINGSECURITY SERVICESEXCEPTION HANDLINGCONTENT BASED ROUTINGDATA TRANSFORMATIONRETRIEVAL (RESEND + RETRANSMIT)PUBLISH - SUBSCRIBE SERVICES
MESSAGE QUEUEINGASSURED DELIVERYPRIORITY DELIVERYCONFIGURATION DEFINITIONS (Qs)WMQ NETWORK DEFINITIONS (CLUSTERS)ROUTE AROUND FAILURES RECOVERYPROCESS TRIGGERINGEVENT MONITORINGTRANSACTIONAL INTEGRITYSECURE MESSAGE TRANSPORT
FIXML Service Provider
Message Queuing API
Messaging Services
Network Transport
WMQ TCP/IP (FIXML) Networks
WMQ FIXML Utility Interfaces Messaging Management and Monitoring Services
FIXML-MQ STANDARD SERVICES
Business Service API
WMQ (MQSeries)
WMQ Gateway
FIXML InterfaceApplication Services
WBI Message Broker
WMQ (MQSeries)MSG Q
MSG Q
IBM Software Group | WebSphere software
14
WMQ and WBI Family Product Detail and Directions
Graham Oakes, IBM
IBM Software Group | WebSphere software
15
Summary
IBM appreciates the opportunity to work with FIA to evolve the FIXML over WMQ standards needed for high performance and reliable communication.
The objective of an ESB approach is to transcend the WMQ transport layer with value add services that reduce the cost of developing and managing disparate interfaces. An FIA sponsored ESB can enable sharing of standards and components that implement them: A communications “architecture” that enables software applications that run
on different platforms and devices or which are written in different programming languages or which use different programming models or which use different data representations
to reliably communicate with no disruption to existing applications or interfaces.
IBM has the most comprehensive WMQ and ESB skills, experience and offerings that we can share with the FIA membership. The formation of a working group and named interfaces to pursue the standards is recommended as the next step.
IBM Software Group | WebSphere software
16
FIA Standards Working Group Focus and Structure
FIA Enterprise Service Bus
BUSINESS SERVICES API
DIRECTORYSERVICES
SECURITYDATA
CONVERT
MESSAGELOGGING
RetransmissionArchive
EDUCATION+
TRAINING
MESSAGING NETWORKOPERATIONS
+SUPPORT
SYSTEMADMINISTRATION
+MANAGEMENT
LEGACYACCESS
MESSAGINGGATEWAYS
EXTERNALACCESS
APPLICATIONDESIGN
APPLICATIONSTANDARDS
MSG FORMATSTANDARDS
NAMINGCONVENTIONS
AVAILABILITYDESIGN
PERFORMANCEDESIGN +TUNING
SECURITYSTANDARDS
MQ NETWORKDESIGN
ARCHITECTURE+
DESIGN
BUILD+
TEST
INTEGRATION+
ROLLOUT
RULES BASEDROUTING
BUSINESSAPPLICATION
BUSINESSRECOVERYSERVICES
MAINTENANCE+
SUPPORT
MIGRATIONPLANNING
SYSTEM MGTINTEGRATION
APPLICATION DEVELOPMENT TOOLS + FACILITIES
REQUIREMENTSDEFINITION
WORKGROUPACCESS
WORKFLOWMANAGEMENT
PUBLISHSUBSCRIBE
DATATRANSFORM
WMQ (MQSeries)
WBI Message Broker
WBI Server (Flows)
WMQ Platforms z/OS
UNIX (Solaris, AIX, Linux, HP-UX)Windows XP/2003
TPFTandem
COMPLEXEVENTS
EVENTPROCESSING
FIA WMQ Standards
IBM Software Group | WebSphere software
17
IBM WMQ and WBI Components of FIA (FIXML) ESB
FIXML Client Access Adapter
WBI Business Integration Services
WBI Message Broker - FIXML Gateway and Broker Services
BusinessEvent Rules
+Routing Logic
BusinessProcessManager
ServiceRegistry
RecoveryServices
MessageTransformation
Message Flow Management
Rules Based Routing
Transaction Standards
Data BaseStandards
RequesterInterface
WMQ Basic Infrastructure Services
Security
Standard Messages
LoggingServices
EventMonitor
ClientProfiles
ServiceMonitoring
+Recovery
Management
System Administration
+Resource
Management
Message Transformation
ServerInterface
Message Transformation
FIXML Service Provider Connector
Process State Management
APIServices
BusinessProcess
FlowRules
Business Process Interfaces(APIs, Objects, Adapters)
Message Formatting
Data Enrichment Services
Bridges and Adapters
FIXML Data(State, UDDI, Metadata)
Transaction State Management
ExceptionHandling
TriggerMonitor
Configuration Management
Channel Session State
Publish Subscribe
AlertNotification
ResendRetransmit
QueueManagement
Message Repositories
Data Transform
ServiceLocator
LogicalSession State
Recommended