© 2014 IBM Corporation

Connecting IBM MessageSight

to the Enterprise (IOT-1900) Andrew Schofield

Chief Architect, IBM MessageSight

Please Note

IBM’s statements regarding its plans, directions, and intent are subject to change

or withdrawal without notice at IBM’s sole discretion.

Information regarding potential future products is intended to outline our general

product direction and it should not be relied on in making a purchasing decision.

The information mentioned regarding potential future products is not a

commitment, promise, or legal obligation to deliver any material, code or

functionality. Information about potential future products may not be incorporated

into any contract. The development, release, and timing of any future features or

functionality described for our products remains at our sole discretion.

Performance is based on measurements and projections using standard IBM

benchmarks in a controlled environment. The actual throughput or performance

that any user will experience will vary depending upon many factors, including

considerations such as the amount of multiprogramming in the user’s job stream,

the I/O configuration, the storage configuration, and the workload processed.

Therefore, no assurance can be given that an individual user will achieve results

similar to those stated here.

IBM MessageSight

3

• A purpose-built messaging appliance

• The gateway to the Internet of Things for the enterprise

• Fast, lightweight, secure, reliable messaging for Mobile

Connecting IBM MessageSight to the Enterprise

4

MessageSight [Primary]

MessageSight [Standby]

Mobile

Andriod

Mobile

iOS

Mobile

Tablet

MQTT

MQTT

MQTT over

websockets

MQTT over

websockets

MQTT

MQTT

MQTT

DMZ

Sensor

(Embedded C)

Sensor

Sensor

JEE Server

(WAS)

JEE Server

(WAS)

Resource

Adapter

JMS Java

Application

MQTT

MQ

C

Application

IBM MQ

System Admin

Browser

Internet Intranet

Connecting to IBM MQ

Extend the reach of your MQ infrastructure

Plug directly into your enterprise with built-in MQ connectivity

Securely extends existing enterprise messaging infrastructures

Accelerate massive fan-out message delivery to huge numbers of devices

Reliable bi-directional messaging enabling intelligent decisions based on real-time events

6

Real-time big data

IBM MessageSight

IBM MQ

Massive-scale

concurrent connectivity

for machine-to-machine

and mobile use cases

Setting up MQ connectivity

MQ connectivity is built in to IBM MessageSight

Very simple to set up

Queue manager connection

Defines how to connect to a queue manager • Queue manager name

• Connection name – host name, port

• Channel name – server-connection channel

• SSL cipher specification (optional)

Destination mapping rule

Defines source and target of messages • Rule type – topic-to-topic, topic-to-queue, …

• Queue manager connection – one or more

• Maximum messages to buffer for transmission

• Retained messages?

7

How to connect to MQ

Queue manager connection

Source and target of messages

Destination mapping rule

Destination mapping rule Destination mapping rules are uni-directional

• Can map from MessageSight to MQ, or from MQ to MessageSight

8

Topic-to-topic Topic subtree-to-queue

Queue-to-topic Topic subtree-to-topic subtree

PriceUpdate PriceUpdate AllDiagLogs

Diagnostics

Car123 Car345

ServiceDue

Car123 Car345

Service

Car123 Car345

Updates

PatchQueue

Pattern 1: Fan-out broadcast Same data sent to all devices subscribed to common topic

• Business example: reference data, price list, catalog

9

Topic-to-topic mapping

destTopic sourceTopic

Publish “sourceTopic”

Subscribe “destTopic”

Pattern 2: Fan-out notification Publisher publishes to per-device topic

• Business example: commands sent to a device, e.g. “sync now”

10

Subtree-to-subtree mapping source

device1 device2 device3

dest

device1 device2 device3

Publish “source/deviceID”

Subscribe “dest/deviceID”

Pattern 3: Fan-in notification Each device publishes to one or per-device MessageSight topic

• Business example: sensors

11

Topic-to-topic mapping

sourceTopic destTopic

Subtree-to-subtree mapping source

device1 device2

dest

device1 device2

Publish “sourceTopic”

Subscribe “destTopic”

Publish “source/deviceID”

Subscribe “dest/#”

Pattern 4: Fan-in request-reply Device publishes request to common topic and awaits response

• Business example: device polling control center for updated info

12

Subtree-to-subtree mapping

requestDest requestSrc

Publish “requestSrc”

Subscribe “requestDest”

Topic-to-topic mapping

replySrc

device1 device2

replyDest

device1 device2

Publish “replySrc/deviceID”

Subscribe “replyDest/deviceID”

Request

message

contains

routing

information for

reply

Pattern 5: Fan-out request-reply Publisher publishes to per-device topic and awaits response

• Business example: control center interrogating device

13

Subtree-to-subtree mapping

requestSrc

device1 device2

requestDest

device1 device2

Publish “requestSrc/deviceID”

Subscribe “requestDest/deviceID”

Subtree-to-subtree mapping

replyDest

device1 device2

replySrc

device1 device2

Publish “replySrc/deviceID”

Subscribe “replyDest/#”

Request

message

contains

routing

information for

reply

Retained messages

Retained messages can be forwarded to or from MQ

You choose administratively whether messages are published

as retained on the target system

• Affects all messages published, regardless of initial publish

Restrictions:

• Destination-mapping rule can only be associated with 1 QM

• Target cannot be a queue

14

Securing the connection to MQ

Secure the connection between IBM MessageSight and MQ

with SSL/TLS

• Create a key repository in the same way as you do when using

SSL/TLS with MQ

• Upload the key repository on to the MessageSight appliance

• Specify the SSL/TLS cipher specification in the queue manager

connections

• You can choose between self-signed certificates and CA

certificates

15

Authorization in MQ

Authorization of IBM MessageSight clients takes place on the appliance • Define connection and messaging policies in the normal way

MQ connectivity connects securely to MQ using a single user ID • Obtain a user ID on the queue manager’s system

• Create a channel authentication record permitting IBM MessageSight to connect to the queue manager, specifying the user ID above

• Grant authorization to these objects so MQ connectivity can work – Queue manager

– SYSTEM.DEFAULT.LOCAL.QUEUE

– SYSTEM.DEFAULT.MODEL.QUEUE

– SYSTEM.ADMIN.COMMAND.QUEUE

– SYSTEM.IMA.* dynamic queues

• Grant authorization to the objects needed by the destination mapping rules – Define a TOPIC object and grant authority to it, or

– Define a QUEUE and grant authority to it

16

High availability

For IBM MessageSight

MQ connectivity works just the same on a single appliance or a

high-availability pair of appliances

• All configuration and in-flight message information is replicated

automatically

For IBM MQ

IBM MessageSight can connect to multi-instance queue

managers or queue managers in HA clusters

• Comma-separated connection name is supported by queue

manager connections

17

Performance considerations

IBM MessageSight is fast, very fast

• Most other middleware simply cannot keep up

• At millions of messages a second, you need very fast applications

• For highest throughput, connect the applications directly to MessageSight

When using MessageSight with MQ, you need to make sure MQ can cope with the message rate

• Use multiple queue manager connections when message rates are higher

• To cope with fluctuations in message rate, the number of messages buffered for transmission to MQ might occasionally get high

Carefully choose the scenario you want to perform

• Send from MQ to MessageSight to fan messages out to numerous devices

• Connect large numbers of devices to MessageSight and then send to MQ

• Send a subset of messages from MessageSight to MQ

18

Connecting to

IBM Integration Bus

Connecting to IBM Integration Bus

Two new patterns for integrating IBM MessageSight with back-

end systems

• Patterns encapsulate integration behavior for quick time-to-value

Alternatively, you can use JMS or MQTT nodes

• More skills required, but more flexible

20

Inbound Event Filter Pattern

MessageSight routes events into Integration Bus via JMS

It narrows down events using a filter and sends to the back-end

21

Many connected

devices

Outbound Event Notification Pattern

Integration Bus receives events from a back-end system

MessageSight receives them using JMS and fans out to devices

22

Many connected

devices

Connecting using JMS and MQTT nodes

JMS

• IBM MessageSight is used just like any other JMS provider

• Download and install the MessageSight JMS client JAR

• Use the regular JMS nodes: JMSInput, JMSOutput, JMSReply, …

MQTT

• MQTT Client Connector for IBM Integration Bus is on GitHub:

– https://github.com/ot4i/mqtt-client-connector

• Built on Paho Java client

• MqttPublish and MqttSubscribe nodes

• Still under active development

JMS is more mature, MQTT is a little faster

23

Connecting to

Application Servers

JCA Resource Adapter

IBM MessageSight v1.1 introduced a Resource Adapter

• Comes as part of the IBM MessageSight Client Pack

• Supported in WebSphere Application Server v8.0 or later

Outbound communication

• An application starts a connection to IBM MessageSight, and

then sends JMS messages to JMS destinations, and received

JMS messages from JMS destinations in a synchronous

manner

Inbound communication

• JMS messages arriving at a JMS destination are delivered to a

message-driven bean (MDB), which processes the messages

asynchronously

25

Configuring for outbound communication

Use J2C connection factories and administered destinations

• Resources > Resource Adapters > J2C connection factories

Properties:

• clientId – required to use non-shared durable subscriptions

• protocol – tcp or tcps

• server/port – connection information for the server

• transactionSupportLevel – XA, local or no transaction

Also the connection pool settings have a big impact on

performance http://pic.dhe.ibm.com/infocenter/wasinfo/v8r5/index.jsp?topic=/com.ibm.webspher

e.nd.multiplatform.doc/ae/udat_conpoolset.html

26

Configuration for inbound communication

JMS messages received by Message-Driven Beans

Use J2C activation specification

• Resources > Resource Adapters > J2C activation specifications

Properties:

• clientId – required to use non-shared durable subscriptions

• destination/destinationType – queue or topic name

• maxDeliveryFailures – # of failures before endpoint paused

• protocol – tcp or tcps

• server/port – connection information for the server

27

Configuration for sharing and concurrency

With appropriate settings you can get:

• Concurrent delivery of messages in a single server

• Horizontal scaling of message delivery in a WAS cluster

• Workload balancing of messages with a shared subscription

Properties:

• clientId – required to use non-shared durable subscriptions

• clientMessageCache – consumer’s message cache size

• concurrentConsumers – max # of consumers per connection

• subscriptionDurability – durable or nondurable

• subscriptionName – name of the subscription

• subscriptionShared – shared or nonshared

28

Scenario 1: Workload balancing in one server?

29

Message-Driven Bean

Message-Driven Bean

Message-Driven Bean

Subscription

Subscription

Subscription

Application Server

Each MDB gets its own copy of every message

• This isn’t workload balancing

Scenario 1: Workload balancing in one server!

30

Message-Driven Bean

Message-Driven Bean

Message-Driven Bean

Shared Subscription

Application Server

Use a shared subscription to attach multiple consumers to the

same subscription

subscriptionShared = “shared” subscriptionName = “mySub” concurrentConsumers = n

Scenario 2: Workload balancing in a cluster

31

Shared Subscription

Message-Driven Bean

Message-Driven Bean

Message-Driven Bean

Application Server

Application Server

Application Server Define activation specification

at cluster scope

Do not set ClientID

Can have >1 MDB per server

Scenario 3: Message delivery with high availability

32

Message-Driven Bean

Message-Driven Bean

Message-Driven Bean

Shared Subscription

Application Server

Give server information for primary and standby in configuration

Connection will switch when MessageSight fails over

server = primaryIP,standbyIP port = n subscriptionDurability = durable

Standby

Primary

Connecting to

IBM InfoSphere Streams

Big data meets big connectivity

34

Smarter Decisions

Powerful analytics

IBM MessageSight

IBM InfoSphere Streams

Smarter actions Real-time data

Internet Scale device connectivity

Use case: Real-time analytics

35

car/1

car/2

car/<ID> IBM InfoSphere

Streams (decisioning, analytics)

car/<ID>

alert/car/<ID>

Control Center

app Driver app(s)

…

{lat, lon, heading, speed, airbag, wipers}

car/#

event/#

cars

alert/car/<ID>

Connected Cars

IBM MessageSight

Emergency Services IBM Worklight

publish subscribe MQTT

Connecting to IBM InfoSphere Streams

InfoSphere Streams V3.2 introduced support for MQTT

Messaging Toolkit includes 2 MQTT operators:

• MQTTSink

– Publishes messages to an MQTT server

– Creates a message for every tuple it receives on its input port

– Either configured topic name, or take it from the input tuple

• MQTTSource

– Subscribes to topics and receives messages from an MQTT server

– Can specify list of topics and QoS for multiple subscriptions

– Message format can be “bin” (->parsed) or “block” (->blob)

Recommend using V3.2.1

Questions?

Come and hear about MessageSight at Impact 2014

Monday

4:00-5:00 PM

IOT-1899 Introduction to IBM

MessageSight

Palazzo F

Tuesday

10:30-11:30 AM

AMC-3134 Meet the Experts: IBM

MessageSight

San Polo 3501 B

Tuesday

2:15-3:15 PM

IOT-1900 Connecting IBM

MessageSight to the

Enterprise

Palazzo F

Wednesday

10:30-11:30 AM

MMA-1921

Hands-on Lab

Building an Event-

driven Mobile

Application with IBM

Worklight & IBM

MessageSight

Murano 3301 B

Wednesday

1:00-3:15 PM

IOT-1920

Hands-on Lab

IBM MessageSight &

Internet of Things

Cloud

Murano 3305

Thursday

2:15-3:15 PM

IOT-1912 Amaze Customers

with Dynamic, Event-

driven Mobile

Applications using

WebSockets and Other

Protocols

Marcello 4405

38

We Value Your Feedback

Don’t forget to submit your Impact session and speaker

feedback! Your feedback is very important to us – we use it to

continually improve the conference.

Use the Conference Mobile App or the online Agenda Builder to

quickly submit your survey

• Navigate to “Surveys” to see a view of surveys for sessions

you’ve attended

39

Thank You

Legal Disclaimer

• © IBM Corporation 2014. All Rights Reserved.

• The information contained in this publication is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained

in this publication, it is provided AS IS without warranty of any kind, express or implied. In addition, this information is based on IBM’s current product plans and strategy, which are

subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothing

contained in this publication is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and

conditions of the applicable license agreement governing the use of IBM software.

• References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or

capabilities referenced in this presentation may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to

future product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by

you will result in any specific sales, revenue growth or other results.

• If the text contains performance statistics or references to benchmarks, insert the following language; otherwise delete:

Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will

experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage

configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.

• If the text includes any customer examples, please confirm we have prior written approval from such customer and insert the following language; otherwise delete:

All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs

and performance characteristics may vary by customer.

• Please review text for proper trademark attribution of IBM products. At first use, each product name must be the full name and include appropriate trademark symbols (e.g., IBM

Lotus® Sametime® Unyte™). Subsequent references can drop “IBM” but should include the proper branding (e.g., Lotus Sametime Gateway, or WebSphere Application Server).

Please refer to http://www.ibm.com/legal/copytrade.shtml for guidance on which trademarks require the ® or ™ symbol. Do not use abbreviations for IBM product names in your

presentation. All product names must be used as adjectives rather than nouns. Please list all of the trademarks that you use in your presentation as follows; delete any not included in

your presentation. IBM, the IBM logo, Lotus, Lotus Notes, Notes, Domino, Quickr, Sametime, WebSphere, UC2, PartnerWorld and Lotusphere are trademarks of International

Business Machines Corporation in the United States, other countries, or both. Unyte is a trademark of WebDialogs, Inc., in the United States, other countries, or both.

• If you reference Adobe® in the text, please mark the first use and include the following; otherwise delete:

Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries.

• If you reference Java™ in the text, please mark the first use and include the following; otherwise delete:

Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.

• If you reference Microsoft® and/or Windows® in the text, please mark the first use and include the following, as applicable; otherwise delete:

Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both.

• If you reference Intel® and/or any of the following Intel products in the text, please mark the first use and include those that you use as follows; otherwise delete:

Intel, Intel Centrino, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and

other countries.

• If you reference UNIX® in the text, please mark the first use and include the following; otherwise delete:

UNIX is a registered trademark of The Open Group in the United States and other countries.

• If you reference Linux® in your presentation, please mark the first use and include the following; otherwise delete:

Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of

others.

• If the text/graphics include screenshots, no actual IBM employee names may be used (even your own), if your screenshots include fictitious company names (e.g., Renovations, Zeta

Bank, Acme) please update and insert the following; otherwise delete: All references to [insert fictitious company name] refer to a fictitious company and are used for illustration

purposes only.