Red Hat Open Day JBoss Fuse

RED HATFUSE TRAINING

JBoss Fuse and Camel

3

Keywords

● Apache Camel

● Enterprise Integration Patterns

● Components, Routes, Endpoints, Messages, Processors

4

Apache Camel

• Powerful Integration Framework– Based on known Enterprise Integration Patterns– Framework do the heavy lifting– You can focus on business problem– Not "reinventing the wheel"

• Use Case : a really simple way to do integration– Started as a Sub-project of ActiveMQ in March 2007– 36 committers (15 work for Red Hat)– 80-100k artifact downloads a month– 120k website views a month– 1000 user mailing list posts per month– 130 + Components and growing– Most widely used integration framework on the planet

5

Enterprise Integration Patterns

http://www.enterpriseintegrationpatterns.com/

6

Camel implements most EIP Patterns

http://camel.apache.org/enterprise-integration-patterns.html

• Messaging systems

• Messaging Channels

• Message Routing

• Message transformation

• Message Endpoints

Message router Message translator

Dead Letter Channel Message Bus

Recipient List Splitter Aggregator

Content Enricher Content Filter

Event Driven Consumer Polling Consumer

7

Example : complex deployment

Source : JBoss Fuse success story – Major Retail Pharmacy Chain

8

JBoss Fuse Architecture :routes, components, endpoints

From : “Camel in Action”, by Claus Ibsen

9

Camel Context

• CamelContext is the container of many Camel Services• Instance of the Camel runtime


10

What is a Camel Component?

• Components are the main extension point in Camel• Components are used to manage everything in Camel :

◾ Data Transport◾ Data Formatting◾ ... customized behaviour

• From a programming point of view they :◾ Have a name◾ Act as a Factory of endpoints

11

What is a Camel Processor?

• Core Camel concept : node capable of using, creating, or modifying an incoming exchange

• During routing exchanges flow from one processor to another

public interface Processor {

/**

* Processes the message exchange

*

* @param exchange the message exchange

* @throws Exception if an internal processing error has occurred.

*/

void process(Exchange exchange) throws Exception;

}

12

EIP Patterns and Processors

• Implement the actions of the EIP between the endpoints

• Perform actions on the message in the Route (e.g modify, use, create, enrich ...)

• Processors can be linked in a pipeline flow

• Processor examples :

Splitter Message Filter

Aggregator Message Router

Content Enricher

13

What is a Camel route?

• A Route is the step-by-step movement of a message: ◾ From a “listening” endpoint in the role of consumer◾ Through a processing component - enterprise integration pattern,

processor, interceptor, etc. (optional)◾ To a target endpoint in the role of producer

• May involve any number of processing components that modify the original message and/or redirect it◾ Use expressions and predicates

• An application developer specifies routes using:◾ Spring configuration files◾ Java Domain Specific Language (DSL)

14

Routes ....

• By decoupling clients from servers, and producers from consumers, routes can

◾ Decide dynamically what server a client will invoke◾ Provide a flexible way to add extra processing◾ Allow for clients and servers to be developed independently◾ Allow for clients of servers to be stubbed out (using mocks)

for testing purposes◾ Foster better design practices by connecting disparate

systems that do one thing well◾ Enhance features and functionality of some systems (such

as message brokers and ESBs)

• Each route in Camel has a unique identifier that’s used for logging, debugging, monitoring, and starting and stopping routes.

15

Expressions and Predicates

• Expressions return a selection of nodes for processing◾ XPath is used for parsing XML payloads

• Predicates are conditional expressions◾ Used to add logical branching to routes

• Camel supports other expression languages◾ XQuery, Simple, Bean, OGNL, JavaScript, Groovy, Ruby ...

16

What is a Camel Endpoint?

• An endpoint is something that can create or receive messages, for example, an FTP server, a Web Service, or a JMS broker

• Camel allows you to specifiy endpoints using simple URIs◾ ftp://john@localhost/ftp?password=nhoj◾ activemq:queue:MyQueue◾ timer://myTimer?period=2000

• Consumer Endpoint, source at beginning of Route, 'from'

• Producer Endpoint, sink at end of Route, 'to'

17

Endpoint Syntax

• Endpoint URI format:scheme:remainingURI[?options]

• Where:• scheme identifies the component

– Scheme is the registered service name for activating a component in Camel– Triggers dynamic loading of the component

– Maps to a specific instance of org.apache.camel.Component

• options is a list of name/value pairsoption1=value1&option2=value2&option3=value3

from("ftp://john@localhost/ftp?password=nhoj")

.to("xslt:MyTransform.xslt")

.to("activemq:queue:MyQueue")

Consumer endpoint

Producerendpoint

Parameters

18

Endpoint Functionality

• Endpoints are either message source or message sink

• Two roles :◾ Consumer

– Appears at the beginning of a routing rule (for example, in a from() command or in a <from ...> element, depending on the DSL used)

– Receives messages from an external source and creates a message exchange object, which the routing rule processes.

◾ Producer– Appears at the end of a routing rule (for example, in a to() command or in a

<to ...> element, depending on the DSL used)

– Sends the current message wrapped in the message exchange object to an external target destination.

19

Camel Components in practice

Use Case : Receive orders from ActiveMQ queue and based on the type of message forward to appropriate queue (ActiveMQ or Websphere MQ)

20

Example 1 of 3

from newOrderchoicewhen isWidget to widget

otherwise to gadget

21

Example 2 of 3

from(newOrder).choice().when(isWidget).to(widget).otherwise.to(gadget)

22

Example 3 of 3

Endpoint newOrder = endpoint("activemq:queue:newOrder");Predicate isWidget = xpath("/order/product = 'widget'");Endpoint widget = endpoint("activemq:queue:widget");Endpoint gadget = endpoint("wmq:queue:gadget");

from(newOrder).choice().when(isWidget).to(widget).otherwise.to(gadget)

23

Example : Java DSL

import org.apache.camel.Endpoint;import org.apache.camel.Predicate;import org.apache.camel.builder.RouteBuilder;

public class MyRoute extends RouteBuilder {

public void configure() throws Exception {

Endpoint newOrder = endpoint("activemq:queue:newOrder"); Predicate isWidget = xpath("/order/product = 'widget'"); Endpoint widget = endpoint("activemq:queue:widget"); Endpoint gadget = endpoint("activemq:queue:gadget");

from(newOrder) .choice() .when(isWidget).to(widget) .otherwise().to(gadget) .end(); }}

24

Example : XML DSL

<route><from uri="activemq:queue:newOrder"/><choice>

<when><xpath>/order/product = 'widget'</xpath><to uri="activemq:queue:widget"/>

</when><otherwise>

<to uri="wmq:queue:gadget"/></otherwise>

</choice></route>

25

Example : change the endpoint

<route><from uri="file:inbox/orders?delete=true"/><choice>

<when><xpath>/order/product = 'widget'</xpath><to uri="activemq:queue:widget"/>

</when><otherwise>

<to uri="wmq:queue:gadget"/></otherwise>

</choice></route>

parametersuse file instead

26

DSL : Java or XML ?

● Java

+ More options for custom development, embedded solutions in legacy applications/frameworks

– Developer needs to take control of instantiation and route lifecycles

● XML

+ Configuration over coding, which is simple, and means you can see your route and its resources / configuration, all in one place

– More coarse-grained and verbose

27

Consumers and Producers

• Message consumers and producers are created from endpoints

◾ Some endpoint technologies can create producers and consumers, others support the creation of either a producer or a consumer

◾ Examples :

from("timer://myTimer?period=2000")

.setBody().simple("Current time is ${header.firedTime}")

.to("stream:out");

from("ftp://john@localhost/ftp?password=nhoj")

.to("xslt:MyTransform.xslt")

.to("activemq:queue:MyQueue")

28

ESB and routes

• ESB creates a pipeline of processors◾ Passes incoming message to processor◾ Sends output from one processor to the next in the chain

29

ESB and messages

• ESB sends a Message Exchange along the pipeline◾ Passes incoming message to processor◾ Must be explicitly transformed, if needed

30

Camel Message Exchange• Message Container during routing• Two Properties:

• InOnly (fire & forget - e.g. JMS message)

• InOut (request-response - e.g. HTTP requests)

Exchange

InMessage

Headers

Attachments

Body

Exchange ID MEP

Exception Properties

OutMessage

Headers

Attachments

Body

31

Camel Message

• Basic structure for moving data over a Route

Sender ReceiverMessage

Message

Headers

Attachments

Body(payload)

32

Camel Message Lifecycle

Sender ReceiverMessage

33

Camel Message Lifecycle

• In messages are modified by each processor in the route

◾ If a processor sets the Out message, Camel automatically copies the Out message into the In message and resets the Out message to be null

34

End of Pipeline

• When processing is finished Camel returns the ME to the consumer

• The consumer can :◾ Send a response message ◾ Commit a transaction◾ Handle and error◾ Free any resource in use

35

Camel Components

120+ Components (supported)

36

Camel Components

120+ Components (supported)

37

Data Formats

Serialization String JSon Castor JAXB

XStream XmlBeans SOAP XmlJson CSV

EDI Flatpack DataFormat

HL7 DataFormat

Gzip Zip

PGP Crypto XMLSecurity RSS Syslog

TidyMarkup Custom

• Transform format (e.g. CSV to XML)

• Transform data type (e.g. Java.lang.String to javax.jms.TextMessage)

• Use built-in components (e.g. XSLT), use components (e.g. Enricher), built-in Data formats, templates, custom

38

Extending Camel

● Using Components


39

Extending Camel : create custom components• Use Maven to create a skeleton component

• Define endpoint name• Implement endpoint class to create

Consumer or Producer or both

• Implement Consumer and Producer classes

JBoss Fuse Camel :advanced topics

41

Data transformation overview

● Data Format & Data Type◾ Data format example : message body from .csv to .xml

◾ Data type example : message body from java.lang.String to javax.jms.TextMessage

◾ Data transformations can be combined


42

Data Transformation

Data transformation in routes Explicitly enforce transformation in the route using the Message Translator or the Content Enricher EIPs. This gives you the power to do data mapping using regular Java code.

Data transformation using components

Camel provides a range of components for transformation, such as the XSLT component for XML transformation.

Data transformation using data formats

Data formats are Camel transformers that come in pairs to transform data back and forth between well-known formats

Data transformation using templates

Camel provides a range of components for transforming using templates, such as Apache Velocity.

Data type transformation using Camel’s type-convertermechanism

Camel has an elaborate type-converter mechanism that activates on demand. This is convenient when you need to convert from common types such as java.lang.Integer to java.lang.String or even from java.io.File to java.lang.String.

Message transformation in component adapters

Extend Data transformation capability via custom transformers

43

Message Translator & Content Enricher


44

Message Translator

● 3 methods supported◾ Using Processor◾ Using beans◾ Using <transform>

Route

from("quartz://report?cron=0+0+6+*+*+?").to("http://riders.com/orders/cmd=received&date=yesterday").bean(new OrderToCsvBean()).to("file://riders/orders?fileName=report${header.Date}.csv");


45

Message Translator

public class OrderToCsvBean {public static String map(String custom) {

String id = custom.substring(0, 9);String customerId = custom.substring(10, 19);String date = custom.substring(20, 29);String items = custom.substring(30);String[] itemIds = items.split("@");

StringBuilder csv = new StringBuilder();csv.append(id.trim());csv.append(",").append(date.trim());csv.append(",").append(customerId.trim());

for (String item : itemIds) {csv.append(",").append(item.trim());

}return csv.toString();

}}

No Camel dependency!Message Body mapped

automatically tomethod signature

Message Body transformedInto java.lang.String


46

Content Enricher

from("file://repo/orders?fileName=theFile.csv").process(new Processor() { public void process(Exchange exchange) { Message in = exchange.getIn(); in.setBody(in.getBody(String.class) +

"World!"); }}).to("mock:result");

Access Message Header

Modify Message Header

Use Mock component From : “Camel in Action”, by Claus Ibsen

smb://repo/orders%3FfileName=theFile.csv

47

Using Beans

● Camel does not mandate to learn a specific component model

● Camel uses the ServiceActivator EIP to invoke POJOsinside a route, implementedby the Bean Component

● A sample route would look like:

from("quartz://report?cron=0+0+6+*+*+?").to("http://riders.com/orders/cmd=received&date=yesterday").bean(new OrderToCsvBean()).to("file://riders/orders?fileName=report${header.Date}.csv");

Bean componentPOJO


48

Bean invocation

● Camel works as a Service Activator, Camel resolves the bean to call at runtime


49

Error handlers

● Camel support two concepts of error :◾ Irrecoverable error (e.g. SQLException)◾ Recoverable error

● In Camel a Recoverable error is represented by a Java Throwable or Exception, accessible from org.apache.camel.Exchange

● An Irrecoverable error is represented by a message with a fault flag that can be set or accessed from org.apache.camel.Exchange

● These definitions allow Camel to decide what to do with an error

50

Testing

● Developers can perform full unit testing of integration flows without having to deploy and listen on physical transports:◾ Configure the route to listen on in-memory endpoints

such as direct or seda and write to mock, test, or dataset endpoints

◾ Write unit tests using JUnit or TestNG, which...– Set expectations on each producer endpoint

– Send messages to the consumer endpoint using ProducerTemplate

– Assert expectations

51

Components

● Camel's strength lies in the concept of Component

● All components are accessible via the URI syntax : e.g. <scheme name> : <hierarchical part> [ ? <query> ] [ # <fragment> ]

● Components are listed here : http://camel.apache.org/components.html

● Some components are defined as External, since they do not belong to the standard Apache Camel distribution and are available under different licenses

http://camel.apache.org/components.html

52

Asynchronous messaging

● JMS is a powerful integration technology

● Camel does not ship with a JMS provider, you need to configure it with a specific ConnectionFactory instance

Message channel

Publish Subscribe channel


53

Web Services

● Camel uses Apache CXF to access and publish Web Services (http://cxf.apache.org)

● Camel supports both WSDL first and Code First approach

● CXF can be configured via URIcxf://anAddress[?options]

● CXF can be configured via endpoint beancxf:bean:beanName

http://cxf.apache.org/

54

Databases

● Camel provides multiple ways to integrate with Databases◾ JDBC Component◾ SQL component◾ JPA Component◾ Hibernate Component (Extra component, must be downloaded

from http://code.google.com/p/camel-extra) ◾ iBatis component

http://code.google.com/p/camel-extra

55

Virtual endpoints

● In-memory messaging is a synchronous or asynchronous messaging model which happens inside or across a CamelContext

● 3 components◾ Direct

– Synchronous model

◾ Seda/VM– Asynchronous– Same parameters– No persistence– No JMS– VM can be used to send messages across CamelContexts

56

Automating tasks

● Timer and Quartz components◾ Support only consuming components

● Timer is Camel Core component◾ Very simple

● Quartz is based on the http://www.quartz-scheduler.org project◾ Allows much finer grained control

http://www.quartz-scheduler.org/

57

Aggregator

● Aggregator combines messages incoming to a single outbound message

● Aggregator identifies messages that are related

● Messages are sent to the outbound channel when completion conditions occur

● Aggregator requires the developer to define◾ Correlation identifier: an Expression◾ Completion condition: a Predicate◾ Aggregation strategy: an AggregationStrategy

58

Splitter

● Splitter allows to read a message and split it into separate messages so that they can be processed separately

● To use Splitter, the developer will need to configure an Expression, which will be evaluated when a message arrives

● Splitter will decorate the message with information that is passed to the Exchange (e.g. Number of pieces the original message has been split into, current message index ....)

59

Load Balancer

● The Camel Load Balancer component is a Processor thatimplements org.apache.camel.processor.loadbalancer.LoadBalancer interface

● Load Balancer allows to balance anything is defined in Camel routes

● Load Balancer offers multiple load balancing strategies from("direct:start").loadBalance().roundRobin().to("seda:a").to("seda:b").end();

from("seda:a").log("A received: ${body}").to("mock:a");

from("seda:b").log("B received: ${body}").to("mock:b");


60

Transactions

● Camel routes can create and enlist in transactional contexts

● Use JMS transactions for reliable retry of message delivery

● Use JDBC transactions to make processors transactional

● Use the transactional error handler provided with Camel

● Use an idempotent consumer to avoid unwanted retries

61

Concurrency

● Camel supports theCompeting ConsumersEIP pattern

● Some EIPs support concurrency◾ Aggregate, Multicast, Recipient

list, Splitter, Thread, Wire Tap

● Consumers can be for examplejms or seda components◾ Concurrent consumers normally expose

the concurrentConsumers option

62

Security

● The broad categories offered are◾ Route Security - Authentication and Authorization services to

proceed on a route or route segment– Based on Apache Shiro : handles authentication, authorization, enterprise

session management and cryptography

◾ Payload Security - Data Formats that offer encryption/decryption services at the payload level– Based on XMLDataFormat and Crypto component

◾ Endpoint Security - Security offered by components that can be utilized by endpointUri associated with the component– Some componets offer the ability to secure their endopoints (e.g. Http, jetty,

CXF, netty, mina, JMS, ....)

◾ Configuration Security - Security offered by encrypting sensitive information from configuration files– Camel allows to crypt/decrypt configuration files containing sensitive

information

63

Management & Monitoring

● The Health of Camel applications can be checked at 3 levels◾ Network◾ JVM◾ Application

● At JVM level Camel exposes its managed beans through JMX

● Applications can be monitored by use of log files● Logging is defined by EIP Log

◾ Camel supports custom logging, notification and Tracer

64

Management & Monitoring : example


Technology

Red Hat Open Day JBoss Fuse