MESSAGING ARCHITECTURES WITH NOSQL
DATABASES AS MESSAGE STORES
JavaOne 2012 Conference (#JavaOne)
Srini Penchikala (@srinip)
10.4.12
GOALS OF THIS PRESENTATION
NoSQL databases to store messages with focus on EIP
Patterns
Target Audience: Integration Architects and Developers
Redis, Oracle NoSQL, and MongoDB databases
Demos (Java application)
Not a NoSQL or Messaging intro presentation
Format:
50 min presentation & demos
10 min for Q&A
2
WHO AM I
Software Architect
Co-author of Spring Roo in Action book
Editor at InfoQ.com
Background:
Financial services industry
Regulatory compliance (PCI & FISMA)
J2EE architecture model
Scrum for software development
Agile (v./and) Architecture 3
QUICK POLL
Who is using EIP Patterns?
Spring Integration?
NoSQL?
Spring Data?
4
TODAY’S DISCUSSION
Enterprise Integration Patterns
NoSQL databases
NoSQL as message stores
MongoDB / Redis / Oracle NoSQL Database
Spring Data
Spring Integration framework
Sample Java application
Best Practices
Conclusions 5
MESSAGING
Advantages
Limitations
Message Loss
Performance
Security
Availability
Design Considerations*
6
*Main focus of this presentation
MESSAGING PATTERNS
Message Store
Claim Check
Queue Channel
Aggregator
Resequencer
Control Bus*
Wire Tap*
7
URL: http://www.eaipatterns.com
*System monitoring and management
MESSAGE STORE
Store Messages in a persistent
store (e.g. Database)
Why Message Store?
Performance
Availability
8
http://www.eaipatterns.com/MessageStore.html
MESSAGE STORE USE CASES
Message buffering
Aggregator
Resequencer
Queue Channel
Message loss prevention
Claim Check
9
CLAIM CHECK
Store message data in a
persistent store and pass a
Claim Check to subsequent
components.
These components can use
Claim Check to retrieve the
stored information.
10
http://www.eaipatterns.com/StoreInLibrary.html
PERSISTENCE
Key Value Data Store
Redis
Oracle NoSQL
Document Database
MongoDB
GemFire
JDBC
In-Memory
11
NOSQL DATABASES
Redis
Redis-based MessageStore
Publish-Subscribe Messaging adapters
Oracle NoSQL Database
Distributed key-value database
ACID transactions
Administration via Web console or CLI
Use Cases: Online Display Advertising & Mobile Social Gaming
MongoDB
Document database
12
http://www.redis.io, http://www.mongodb.org
http://www.oracle.com/technetwork/products/nosqldb/overview/index.html
SPRING INTEGRATION
Extension of the Spring framework
Implements Enterprise Integration Patterns (EIP)
Lightweight messaging
Integration with external systems via declarative adapters
Separation of concerns
13
*http://www.springsource.org/spring-integration
SPRING DATA
Spring applications that use new data access technologies
Non-relational databases
Map-reduce frameworks
Current Implementations:
Big Data (Apache Hadoop)
Data-Grid (GemFire)
Key Value Stores (Redis)
Document Stores (MongoDB)
Graph Databases (Neo4j)
Column Stores (Hbase)
14
*http://www.springsource.org/spring-data
SPRING ROO
Rapid Java application development
Nice integration for Spring Integration and Spring Data
frameworks
15
*http://www.springsource.org/spring-data
MESSAGE STORE “STRATEGY”
Based on Strategy design pattern
MessageStore strategy
Used by QueueChannel and ClaimCheck patterns
MessageGroupStore strategy
Used by Aggregator and Resequencer patterns
Implementations
SimpleMessageStore (In-memory, Map-based)
JdbcMessageStore (uses a relational database via JDBC)
NoSQL Data Store
16
MESSAGE STORES
17
Message Store
Simple (In-Memory)* NoSQL JDBC
Redis* MongoDB* Oracle NoSQL* GemFire
*Covered in the sample application demos
SAMPLE APPLICATION
Oracle NoSQL Database
Redis
Spring Integration
Spring Data
JUnit
18
*Code will be posted on Github
DEMO APPLICATION - USE CASE 1 (CLAIM CHECK)
19
Message Header
Enricher
Message
Store
(In-Memory)
Claim
Check (In)
Queue
Channel Transformer
Claim
Check (Out)
Message
Store
(MongoDB)
MESSAGE STORE DEMO 1
In Memory Message Store
MongoDB
20
DEMO APPLICATION - USE CASE 2 (AGGREGATOR)
21
Message Aggregator
Message
Store
(Redis)
Queue
Channel Transformer
Message
Store (Oracle
NoSQL)
ARCHITECTURE DIAGRAM
22
Source:
http://static.springsource.org/spring-integration/docs/latest-ga/reference/html/samples.html
MESSAGE STORE DEMO 2
Redis Demo
23
CUSTOM MESSAGE STORE IMPLEMENTATION
MessageStore interface
CustomOracleNoSqlMessageStore class
24
MESSAGE STORE DEMO 3
Oracle NoSQL Message Store
25
MESSAGE STORE LIMITATIONS
When using JdbcMessageStore, only Serializable data is
persisted by default
Different instance of some Spring Bean is returned upon
deserialization for headers that represent certain types of
data
Temporary Reply Channel issue
26
CONCLUSIONS
Message Persistence
Message Store pattern
Claim Check
NoSQL and Messaging
“One Size Fits All” fits nothing
27
RESOURCES - BOOKS
28
Enterprise Integration Patterns
(http://eaipatterns.com) Spring Integration in Action
(http://www.manning.com/fisher)
Spring Roo in Action
(http://www.manning.com/rimple)
RESOURCES
Enterprise Integration Patterns (http://www.eaipatterns.com)
Spring Integration (http://static.springsource.org/spring-integration/reference/html)
Spring Integration Support for Redis (http://static.springsource.org/spring-integration/reference/html/redis.html)
Oracle NoSQL Database (http://www.oracle.com/technetwork/products/nosqldb/overview/index.html)
http://blog.codecentric.de/en/2012/04/spring-data-redis/
Message Store (http://www.eaipatterns.com/MessageStore.html)
Redis (http://redis.io/topics/pubsub)
MongoDB
Spring Data
Spring Data Redis
Sample Application on Github 29
THANK YOU
Contact Information
http://www.infoq.com/author/Srini-Penchikala
@srinip
http://srinip2007.blogspot.com
Spring Roo in Action Book
Questions?
30
BONUS SLIDES
Bonus Slides
31
MESSAGING GATEWAYS
JMS
AMQP
File
FTP/SFTP
HTTP (REST)
WS (SOAP/POX)
JDBC
MongoDB
Redis 32
CONTROL BUS
Administer a messaging
system distributed across
multiple platforms
Uses same messaging
mechanism used by
application data, but uses
separate channels to
transmit data for the
management of message
flow components
33
http://www.eaipatterns.com/StoreInLibrary.html
WIRE TAP
Inspects messages that
travel on a point-to-point
channel
Recipient List into the
channel that publishes
each incoming message to
the main channel and a
secondary channel
34
http://www.eaipatterns.com/WireTap.html
XML CONFIGURATION OF MESSAGE STORE
<bean id="redisMessageStore"
class="org.springframework.integration.redis.store.RedisMess
ageStore">
<constructor-arg ref="redisConnectionFactory"/>
</bean>
<int:channel id="somePersistentQueueChannel">
<int:queue message-store="redisMessageStore"/> <int:channel>
<int:aggregator input-channel="inputChannel" output-
channel="outputChannel" message-store="redisMessageStore"/>
35
MONITORING
Control Bus
JMX
36
MONITORING SCREENSHOT
Screenshot
37
MONITORING (DEMO 4)
DEMO
38