Embed Size (px)
Citation preview
1 05/02/2023
InterConnect2017
HHI-2124Operational and Business Monitoring with IBM Integration Bus
Sanjay Nagchowdhury
IBM Integration [email protected]
2
Agenda• Operational Monitoring
• Message flow accounting and statistics• Resource monitoring• Activity logging• Workload management• Demo
• Business Monitoring• Message flow monitoring• Record and replay• Business transaction monitoring• Demo
• Summary• Q&A
3
Message flow accounting and performance monitoring
• How many messages has my message flow processed, and how fast did my message flow process them?
IIB 1
4
Message flow accounting and statistics
• Off by default; administrators can enable message flow statistics at the message flow level, or for a set of message flows (for example all message flows in an application).
• Data is collected at runtime as messages are processed:
• Data collection is low impact – up to 5% overhead.
• Data can then be published at regular intervals and viewed using IIB tooling, or external applications.
• No message flow changes are required to use message flow statistics.
5
Message flow accounting and statisticsUsing the WebUI in IBM Integration Bus v10:– Control statistics at all levels– Easily view and compare flows– Easily view and compare nodes– View all statistics metrics available for each flow– View historical flow data
Multiple metrics recorded for each message flow:Number of messages processed.Number of messages committed and backed out.CPU time and elapsed time spent processing messages.CPU time and elapsed time spent waiting for input.Min/average/max sizes of messages processed.Number of threads processing messages.… and more!
6
Message flow accounting and statistics
• Additional detail levels can also be enabled to give further insight into how a message flow is performing:
• Node level:• Identify that a lot of CPU time is being spent in an ESQL Compute node, or a lot of
elapsed time is being spent waiting for an HTTP request.
• Terminal level:• How many times did the CICS Request node propagate to the ‘timeout’ terminal instead
of the ‘output’ terminal?
7
Message flow accounting and statistics
• Data can be published at two separate intervals:• Snapshot publications:
• Published every 20 seconds, with data in the publication covering a 20 second interval. This is not configurable.
• Useful for analyzing the performance of a message flow as it is processing messages, for example to identify hotspots or increases in routing down error paths in a message flow.
• Archive publications:• Published every 60 minutes (and this is configurable!).• Useful for recording and analyzing the use of a message flow; for example, you might
use archive publications to charge clients for their use of that message flow.
8
• Data can be published to multiple sources:• User trace log:
• Collected data is written to user trace, even if user tracing is disabled.• Can be collected via mqsireadlog/mqsiformatlog commands.
• IBM MQ topics:• $SYS/Broker/integrationNodeName/StatisticsAccounting/recordType/integrationServerName/messageFlowName
• MQTT topics:• IBM/IntegrationBus/integrationNodeName/StatisticsAccounting/integrationServerName/messageFlowName
• SMF (System Management Facility) on z/OS
Message flow accounting and statistics
• Data can be published in multiple different formats:• BIP messages (BIP2380-BIP2383) for user trace.• Type 117 records for SMF on z/OS.• XML and JSON messages for IBM MQ or MQTT topics• New in 10.0.0.7, you have the option to collect statistics in CSV files.
9
• Collected data can be associated by the message flow developer with a specified identifier; an accounting origin.
• Example accounting origins might be an account number, a client ID, or a department ID.
• Collected data is then published individually for each accounting origin, making it easy to collate statistics for the specified identifier.
Message flow accounting and statistics
10
Visualizing statistics data from multiple Integration Nodes using a Grafana dashboard
Message flow accounting and statistics
• New C plugin API in 10.0.0.5 which allows developers to write C Plugins to access statistics data and publish to StatsD (statistics gathering deaemon)
• Data can be published to Graphite (a time-series database)
• View the data in a Grafana dashboard.https://developer.ibm.com/integration/blog/2016/08/18/visualizing-ibm-integration-bus-statistics-data-using-grafana/
Data from multiple Integration Nodes
and Integration Servers
11
Resource monitoring
• What resources is my integration server using, and how much, or how often, is my integration server using those resources?
12
Resource statistics
• Off by default; Administrators can enable resources statistics at the integration server level.
• Data is collected at runtime as messages are processed:
• Data collection is low impact – up to 3% overhead.
• Data can then be published at regular intervals and viewed using IIB tooling, or external applications.
• No message flow changes are required to use resource statistics.
13
Resource statistics
Multiple resources can be monitored using resource statistics:
• Java Virtual Machine - heap usage and number of GC cycles.
• MQTT – how many messages are received or sent?
• Parsers – how many parsers are used, how many bytes parsed?
• Global Cache – how many reads from and writes to the cache?
• Files – how many files or bytes are read/written?
• … and more!
14
Resource statistics
• Data is published every 20 seconds.
• Data can be published to two sources:
• IBM MQ topics:• $SYS/Broker/integrationNodeName/
ResourceStatistics/integrationServerName
• MQTT topics:• IBM/IntegrationBus/integrationNodeName/
ResourceStatistics/integrationServerName
• Collected data can also be retrieved programmatically in Java:
15
Resource statistics
• Statistics data can be viewed in real time using the IIB web UI:
16
Activity logging
• What did my message flows do, and when did they do it?
• Activity logging records the actions by a message flow.• Activity logging is always enabled.
– It’s very lightweight and it’s very fast!• Activity log entries are frequent, short, and to the point
17
Activity logging
• Activity log entries are BIP messages.• Each activity log entry has:
• A timestamp• A BIP message and the inserts for that message• A thread ID
• Each activity log entry can also have a set of tags – key/value pairs that provide further context and help correlate entries:
• RM=MQTT|GlobalCache|File• connectionUrl=tcp://localhost:11883• Topic=Customers/5/Orders/1• DATASOURCE=MYDB
• 12074,I,"2016-10-19 15:02:48.000163",986484,"BIP12074I: Executing a database statement for 'MBRECORD'.","RM=ODBC,DATASOURCE=MBRECORD,ODBCOPERATION=EXECUTE,MSGFLOW=gen.MonitoringDemo,NODE=getAllRecords (Implementation).Retrieve records from database,",MBRECORD,
Message NumberSeverity
TimestampThread Id
Formatted Message
TagsMessage
Insert
18
Activity logging
• Activity log can be retrieved programmatically in Java, or through the use of a new REST API introduced in V10.0.0.4.
• Activity log entries are stored in a circular buffer in the memory of the integration server that created the entry.
• Entries can be written to disk by defining an ActivityLog configurable service – the configurable service takes care of log file management and rotation.
• Entries are written in CSV format.• Can choose whether or not to format BIP messages.• Can specify maximum log file size, and number of log files.
19
Activity logging
20
Activity logging
• As of V10.0.0.4, activity log can be viewed in the IIB web UI:
21
You can group your Integration Nodes and Integration Servers into
groups. For example: Dev, QA, Stage, Production
Usage data can be seen for a group of Integration Servers or an
individual Integration Server.
IBM Cloud Product Insights is a Bluemix service that is part of IBM Connect to Cloud.
Choose between:CPU time, Number of active CPUsResident Set Size
22
This spike shows that a sudden increase in error messages occurred.
You can zoom in and see precise BIP messages that were generated in this
timeslot.
Run mqsichangebluemixreporting to configure a connection to the Logmet service in Bluemix and send it logging information.
View logs from multiple Integration Nodes in a single customizable
dashboad.
23
Workload management policy
• I want to be notified if my message flows are exceeding a certain messages per second threshold!
“beep beep, too many messages!”
24
Workload management policy offers dynamic control over:
• Message rate limits.• Additional instances.• Commit count & commit
interval.• Automatic restart in case of
unresponsive message flow.
Workload management policy
25
Workload management policy can be applied to a message flow using the Web UI.
Takes affect immediately without requiring the Integration Server to be stopped and re-started.
Workload management policy
26
• Workload management policy can also send publications when a message rate is exceeded:
– Publication is an XML message over MQ or MQTT.
– Publication sent when the message rate exceeds the defined threshold.
– Publication sent when message rate falls below the threshold again.
• Publications can be consumed and used to notify an administrator.
Workload management policy
27
Demo
IIB
REST Clients
REST Clients
28
Links
• Message flow statistics and accounting dataMessage flow statistics and accounting data
• Resource statisticsResource statistics
• Activity log overviewActivity log overview
• Workload managementWorkload management
• Accounting and Statistics Use CasesAccounting and Statistics Use Cases
29
Events and business transaction monitoring
Message Flow Monitoring
Out TerminalEvent Source
In TerminalEvent SourceIn Terminal
Event Source
30
31
Input nodes have Transaction event sources in addition to
terminal event sources.
The event name can be a literal value, or can be
extracted from the message payload using an expression.
Use an Xpath expression to indicate if the event should
be emitted or not.
Data can be specified from the header, message body
or environment.
Define an event
32
Event Correlation
A local transaction id is automatically provided if
none is specified.
Parent transaction id can be used by a child message flow.
Global Correlation Id is used by Business Transaction
Monitoring.
33
Event Unit of Work
Message flow:Event is emitted when the
message flow transaction is committed.
None:Individual events are emitted
immediately regardless of whether the message flow transaction is
committed or rolled back.
Independent:Group of events are emitted
together regardless of whether the message flow transaction is
committed or rolled back.
34
Monitoring Event Format
EventPointData gives you information where the
event came from.
ApplicationData can come from the message
headers or body
35
Details about where the event was emitted
from.
applicationData can point to an area of the message which is of
interest.
Monitoring Event Example
36
topicRoot/Monitoring/integrationServerName/messageFlowName
Subscribing to monitoring event topics
Monitoring events can be published using MQ or using the built-in MQTT pub/sub broker.
To receive event messages from your message flows, subscribe to the following topic:
For an MQTT pub/sub broker, the topic root is IBM/IntegrationBus.
For an MQ pub/sub broker, the topic root is $SYS/Broker.
Event emitted from the MQ Input node in the flow.
Event emitted from the MQ Output node in the flow.
37
Record and ReplayMessages can be
recorded to a database for audit purposes.
Message flow nodes that are configured to emit events are recorded.
Messages can be viewed using the webui or using
the REST api.
Messages can be replayed to a MQ Queue
38
Configuring Record and Replay
3939
Business Transaction Monitoring A business transaction is a set of flows seen as a unit of function from business
perspective
What happened to order number 1234?
Which order requests have not completed?
How many orders have failed?
Mortgage Processing Flows:• ValidateData,• AssessCustomer,• ApproveLoan,• Reject Loan
Holiday Booking Flows:• CheckOrder, • BookFlight, • BookCar, • BookHotel,• DiscountAndPayment,• Holiday Confirmation
Customer order Flows:• ValidateOrder,• GoldCustomer,• RegularCustomer, • Complete
4040
Business Transaction Monitoring – conceptual model
Business Transaction Definition – a configuration which defines a set of flows that are used by a business transaction and a set of monitoring events which are flagged as business events.
4343
Business Transaction Monitoring – conceptual model
44
Business Transaction Monitoring – conceptual model (Notes)
In this example, a Business Transaction Definition has been created with 6 events with these roles:
Event Role
Order placed Start
Confirmed Progress
Dispatched Progress
Delivery attempted Progress
Delivery successful End
Shipment missing Failure
An instance of a business transaction is recorded and each instance can be in a different state (In progress, Ended, Failed or Inconsistent).
45
1. Develop flow and define monitoring events for the flows including global correlation ID.
2. Deploy flows to an Integration Node with a MQ Queue Manager.
3. Select a database (DB2 or Oracle), create the necessary tables (running server\ddl\db2\DataCaptureSchema.sql and BusinessDataCaptureSchema.sql) and define the datasource using mqsisetdbparms).
Business Transaction Monitoring – Workflow part 1
46
4. Create a business transaction definition in the WebUI by selecting the flows and flagging monitoring events.
5. Configure BTM for this Integration Node by selecting the datasource where the events are captured.
6. View the business transactions that were recorded and their events.
Business Transaction Monitoring – Workflow part 2
4747
Business Transaction Monitoring – Define, Configure, View
1) Create a Business Transaction
definition in Web UI
– Add the flows to the BTD – Choose the events that are the start, end or failure of the transaction– Choose the progress events that are part of the transaction
2) A default policy is used to record the BTM
Transactions. All you need to do is to specify
the name of the database to use.
3) View the business transactions that
were recorded and see their state. View
the individual events which contributed to
the business transaction.
48
MQ aspects for BTM
• When a Business Transaction Definition is created, a selection of monitoring events are flagged as business events.
• An MQ subscription is created for each topic that is used by these flagged events.
$SYS/Broker/intNodeName/Monitoring/intServerName/flow_name
• Monitoring events are configured on message flow nodes
• An MQ message is published to an MQ Topic when a flow node processes a message
• The relevant subscription picks up the event and puts it to a destination queue. <queueName>SYSTEM.BROKER.DC.RECORD</queueName>
• Messages are read from this queue and recorded into the database.
49
You can see what is in the default policy for BTM by running this command:
mqsireportpolicy btm_node -t DataCapture -l default
MQ aspects for BTM (Notes)
Values used by the policy can be changed using mqsichangepolicy :
mqsichangepolicy integrationNodeName -t DataCapture -l default -f default.xml
50
Database aspects for BTM
Stores state for eachBusiness Transaction:
• In Progress• Ended• Failed• Inconsistent
Entries that are in the WMB_BUSTRANS table for business transactions must have corresponding entries in the WMB_MSGS table.
The state of a business transaction is calculated depending on the events that have been received for that business transaction.
53
Demo!
5454 IBM Confidential - IBM Integration Bus V10 Beta Program
Define a business transaction definition
55
Define a business transaction definition
56
Define a business transaction definition
New BTD added
Add flows to your BTD
57
Define a business transaction definition
Select a flow to add to your BTD
58
Define a business transaction definition
Select an application to add to your BTD
59
Define a business transaction definition
Flows are added to BTD.Green Ticks indicate Global IDs for correlation have been defined in events on flow.
Warning indicates that BTD is not fully configured yet. Business events need to be defined in the BTD.
60
Define a business transaction definition
Message Flow nodes which have monitoring events are indicated. These events can be configured as business events.
61
Define a business transaction definition
Configure events asBusiness events. Assign a role for each configured event. One of: Start, Progress, End, Failure
62
Define a business transaction definition
After Start and End business events have been configured, the Business Transaction has a green tick shown.
63
Configure business transaction recording details
You must define the Data Source Name and ensure message flow monitoring is enabled so that business transactions can be recorded.
64
Monitor business transactions
Recorded business transactions are shown in the View.
65
Monitor business transactions
Click on Filter to filter the results
You can filter on Transaction ID and/or Status to see a reduced set of transactions.
66
Monitor business transactions
Reduced set of results are shown in view after applying the filter.
67
Monitor business transactions
Click on a business transaction to see the individual business events which are shown below.
These are the business events for the selected business transaction.
68
Monitor business transactions
Details for each business event can be viewed, including the payload and flow details.
69
Links
Articles:
Business Transaction Monitoring – Why, what, how
Business Transaction Monitoring in IIB
Advanced usages of Business Transaction Monitoring
Archiving Business Transaction Monitoring Data
Business Transaction Monitoring vs Record and Replay
Business Transaction Monitoring Performance Considerations
Video:
Business Transaction Monitoring Demo
70
Summary
Operational Monitoring• The message flow statistics and accounting data collection is highly configurable. This allows
you to customize data collection depending on your monitoring requirements. • Statistics can be gathered and visualized in Grafana or Kibana dashboards for multiple
Integration Nodes and Integration Servers.• Resource statistics are collected by an integration node to record performance and operating
details of resources such as JVM, ODBC, Parsers.• Activity logging provide immediate, basic information about what is happening in your message
flows, and how they are interacting with external resources. • Workload management allows you to monitor and adjust the speed that messages are
processed. You can define a notification threshold for individual message flows deployed.
Business Transaction Monitoring• Using Business Transaction Monitoring, you are able to track the life cycle of your business
events across your lines of business.• You can view which business transactions end successfully, which are in progress or which
have failed• You can track specific business transactions using a global transaction identifier which can
help you find what happened to a lost order or why a particular business transaction has taken longer to complete than normal.
IIB Sessions at Interconnect 2017
71
Session Who Time
2110A What's New in IBM Integration Bus BT Monday 16:15 – 17:00
2141A IBM Integration Bus Futures and Strategy (Inner Circle only) BT Tuesday 11:30 – 12:15
2158A Technical Introduction to IBM Integration Bus GG Tuesday 13:30 – 14:15
2118A Developing Integrations for IBM Integration Bus on Cloud GG Tuesday 14:30 – 15:15
2144A IBM Integration Bus Customer Roundtable BT Tuesday 15:45 – 16:30
2121A Docker and IBM Integration Bus GG Wednesday 09:00 – 09:45
2151A Effective Administration of IBM Integration Bus SN Wednesday 10:15 – 11:00
2144B IBM Integration Bus Customer Roundtable BT Wednesday 16:15 – 17:00
2124A Operational and Business Monitoring with IBM Integration Bus SN Thursday 09:30 – 10:15
2111A IBM Integration Bus and REST APIs SN Thursday 10:30 – 11:15
2166 IBM Integration Bus Version 10 Hands-On Scheduled Lab GG+SN Monday 13:00 – 14:45
9402 IBM Integration Bus Version 10 Hands-On Open Lab None Any Open Lab Session
In case powerpoint isn’t your thing …
72
https://developer.ibm.com/integration Lots of Blog entries, regular updates and links to product demo
videos! All our recent enablement material is on youtube
IIB and Kibana dashboards https://youtu.be/sCPrT2dHKSs
Running IIB in Bluemix Container Service https://youtu.be/ybGOiPZO3sY
IIB and Kibana dashboards https://youtu.be/sCPrT2dHKSs
IIB and Hybrid Connect https://youtu.be/gWbxIooq3_g
IIB and LDAP https://youtu.be/HrqY9MyfzNs
IIB LoopBack Request node https://youtu.be/rUK_OQ5-Anw
Using IIB to integrate with MongoDB and Cloudant https://youtu.be/Is1pphngUlM
Using IIB for REST, Graphical Mapping & Salesforce: https://youtu.be/XIK6QvNSHdY
IIB, Kafka and Twilio SMS: https://youtu.be/7mCQ_cfGGtU
Using Kafka with IIB https://youtu.be/kYv0crxL86Y
Consuming REST APIs using the IIB REST Request node https://youtu.be/C_6gPlrCHZQ
Easy demo of an IIB App Connect node https://youtu.be/StwPbOiFKzk
73 05/02/2023
Notices and disclaimersCopyright © 2017 by International Business Machines Corporation (IBM). No part of this document may be reproduced or transmitted in any form without written permission from IBM.
U.S. Government Users Restricted Rights — use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM.
Information in these presentations (including information relating to products that have not yet been announced by IBM) has been reviewed for accuracy as of the date of initial publication and could include unintentional technical or typographical errors. IBM shall have no responsibility to update this information. This document is distributed “as is” without any warranty, either express or implied. In no event shall IBM be liable for any damage arising from the use of this information, including but not limited to, loss of data, business interruption, loss of profit or loss of opportunity. IBM products and services are warranted according to the terms and conditions of the agreements under which they are provided.
IBM products are manufactured from new parts or new and used parts. In some cases, a product may not be new and may have been previously installed. Regardless, our warranty terms apply.”
Any statements regarding IBM's future direction, intent or product plans are subject to change or withdrawal without notice.
Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples are presented
as illustrations of how those customers have used IBM products and the results they may have achieved. Actual performance, cost, savings or other results in other operating environments may vary.
References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in which IBM operates or does business.
Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not necessarily reflect the views of IBM. All materials and discussions are provided for informational purposes only, and are neither intended to, nor shall constitute legal or other guidance or advice to any individual participant or their specific situation.
It is the customer’s responsibility to insure its own compliance with legal requirements and to obtain advice of competent legal counsel as to the identification and interpretation of any relevant laws and regulatory requirements that may affect the customer’s business and any actions the customer may need to take to comply with such laws. IBM does not provide legal advice or represent or warrant that its services or products will ensure that the customer is in compliance with any law.
74 05/02/2023
Notices and disclaimers continuedInformation concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. IBM does not warrant the quality of any third-party products, or the ability of any such third-party products to interoperate with IBM’s products. IBM expressly disclaims all warranties, expressed or implied, including but not limited to, the implied warranties of merchantability and fitness for a particular, purpose.
The provision of the information contained herein is not intended to, and does not, grant any right or license under any IBM patents, copyrights, trademarks or other intellectual property right.
IBM, the IBM logo, ibm.com, Aspera®, Bluemix, Blueworks Live, CICS, Clearcase, Cognos®, DOORS®, Emptoris®, Enterprise Document Management System™, FASP®, FileNet®, Global Business Services®,Global Technology Services®, IBM ExperienceOne™, IBM SmartCloud®, IBM Social Business®, Information on Demand, ILOG, Maximo®, MQIntegrator®, MQSeries®, Netcool®, OMEGAMON, OpenPower, PureAnalytics™, PureApplication®, pureCluster™, PureCoverage®, PureData®, PureExperience®, PureFlex®, pureQuery®, pureScale®, PureSystems®, QRadar®, Rational®, Rhapsody®, Smarter Commerce®, SoDA, SPSS, Sterling Commerce®, StoredIQ, Tealeaf®, Tivoli® Trusteer®, Unica®, urban{code}®, Watson, WebSphere®, Worklight®, X-Force® and System z® Z/OS, are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml.
75 05/02/2023
InterConnect2017
