Title
Click to edit the outline text formatSecond Outline LevelThird
Outline LevelFourth Outline LevelFifth Outline LevelSixth Outline
LevelSeventh Outline LevelEighth Outline LevelNinth Outline
Level
PGDay.IT 2015 23 Ottobre 2015 - Prato
di
Rabbitmq & Postgresql
Lucio [email protected]
Who is this guy?
Delphi developer since 1999
IT Consultant
Front end web developer
Postgresql addicted
Nonantolando.blogspot.com lucio.grenzi lucio grenzi
Agenda
Messaging system
Rabbitmq introduction
Postgresql and pg_amqp
PostgreSQL Listen exchange
Message-oriented middleware
Message-oriented middleware (MOM) is software or hardware
infrastructure supporting sending and receiving messages between
distributed systems. MOM allows application modules to be
distributed over heterogeneous platforms and reduces the complexity
of developing applications that span multiple operating systems and
network protocols
- https://en.wikipedia.org/wiki/Message-oriented_middleware
-
Why Should I use MOM?
Need to scale
Monitoring data feeds
Need things done in order
Use the cloud
Decoupling applications by separating sending and receiving
data
Messaging is asynchronous
Different protocols
Historically, message queuing has used proprietary, closed
protocols
By the time three standards have emerged which are used in open
source message queue implementations:
Advanced Message Queuing Protocol (AMQP)
Streaming Text Oriented Messaging Protocol (STOMP)
MQ Telemetry Transport (MQTT)
STOMP
Text based
Like AMQP, STOMP provides a message header with properties, and
a frame body
Does not deal in queues and topics
The broker must map onto something that it understands
internally such as a topic, queue, or exchange
Destinations are not mandated in the specification, different
brokers may support different flavours of destination
STOMP Broker
Apache ActiveMQ
HornetQ
Net::STOMP::Client (an open source client implementation in
Perl)
Open Message Queue (OpenMQ)
RabbitMQ (message broker, has support for STOMP)
Ruby server, also known as stompserver
http://stomp.github.io/implementations.html
AMQP
Designed for interoperability between different vendors
wide range of features related to messaging
Lot of fine-grained control possible with such a rich feature
set
Protocol implementations
Amqp :SwiftMQ, Apache Qpid, Apache MQ, Apache Apollo,
RabbitMQ
Stomp: Apache MQ, Apache Apollo, RabbitMQ, HornetQ, Stampy
(http://stomp.github.io/implementations.html#STOMP_Servers)
MQTT: Apache MQ, Apache Apollo, RabbitMQ, mosquitto,
WebSphereMQ, HiveMQ, IBM Message Sight
(https://github.com/mqtt/mqtt.github.io/wiki/server-support)
MQTT
Originally developed out of IBMs pervasive computing team
Provides publish-and-subscribe messaging Publish = data to
topics
Subscribe = get data from topics
Designed for resource-constrained devices and low bandwidth,
high latency networks (ideal for mobile and Internet of Things)
MQTTs strengths are simplicity (just five API methods)
MQTT Broker
ActiveMQ,
Apollo
JoramMQ
Mosquitto
RabbitMQ
http://www.scalagent.com/IMG/pdf/Benchmark_MQTT_servers-v1-1.pdf
Rabbitmq
open source message broker software (MPL)
Actively developed and with commercial support by Pivotal
written in Erlang
Client libraries to interface with the broker are available for
all major programming languages.
Gateways for HTTP, Streaming Text Oriented Messaging Protocol
(STOMP), and MQTT protocols
Amqp native clients
.NETPHPJavaPython
C /C++Node.jsGoErlang
JavascriptRubyPerlScala
DelphiGroovyCobolClojure
Amqp packet
Headers
Properties
Bytea [] data
Headers: defined by the Amqp specification
Properties: contain arbitrary, application specific,
information
Amqp message properties
Content-typeMIME content type
Content-encodingMIME content encoding
headersMessage header field table
Delivery-modeNon-persistent (1) or persistent (2)
priorityMessage priority, 0 to 9
Correlation-idApplication correlation identifier
Reply-toAddress to reply to
expirationMessage expiration specification
Message-idApplication message identifier
timestampMessage timestamp
typeMessage type name
User-idCreating user id
App-idCreating application id
reservedReserved, must be empty
Producer - Consumer
Producer: a program that sends messages
Queus: a buffer where messages are stored
Consumer: a program that waits to receive messages
Work queues
Work Queue: it is used to distribute tasks among multiple
workers.
Tasks will be shared between workers
Useful in web applications where it's impossible to handle a
complex task during a short HTTP request window
Publish/Subscribe
Deliver a message to multiple consumers (messages are going to
be broadcast to all the receivers.)
The exchange must know exactly what to do with a message it
receives (rules for that are defined by the exchange type)
Exchange types
Direct: the binding key must match the routing key exactly no
wildcard support
Fanout: it broadcasts all the messages it receives to all the
queues it knows
Topic: same as Direct, but wildcards are allowed in the binding
key
Headers: no routing key, string matching in the message headers
property
pg_amqp
Amqp client for Postgresql
Allow Postgresql to communicate with other components in the
architecture
Stable
Suggested version 9.x, but it can works too with older
versions
http://pgxn.org/dist/pg_amqp/
https://github.com/omniti-labs/pg_amqp
How to install pg_ampq (v0.3.0)
Wget
http://api.pgxn.org/dist/pg_amqp/0.3.0/pg_amqp-0.3.0.zip
Cd pg_ampq-0.3.0
Make
Make install
env PG_CONFIG=/path/to/pg_config make && make
install
shared_preload_libraries = 'pg_amqp.so' postgresql.conf
CREATE EXTENSION amqp; (Postgresql >= 9.1)
psql -d mydb -f /path/to/pgsql/share/contrib/amqp.sql
(Postgresql < 9.1)
pg_amqp (v 0.4.0)
Only supported as a PostgreSQL Extension
PG versions prior to 9.1 and means to install it without the
extension system are no longer officially supported.
Added support for upgrading extension installation from 0.3.0 to
0.4.0
Add support for amqp message properties (delivery_mode,
content_type, reply_to and correlation_id)
Setup
Insert AMQP broker information into the`amqp.broker` table
Insert into ampq.broker(host,port,vhost,username,password)
values (localhost,'5432',null,'myuser','mypassword')
How it works
A process starts and connects to PostgreSQL and runs:
SELECT amqp.publish(broker_id, 'amqp.direct', 'routing-key',
'message');
snprintf(sql, sizeof(sql), " SELECT host, port, vhost, username,
password " " FROM amqp.broker " " WHERE broker_id = %d " " ORDER BY
host DESC, port", broker_id);
Disconnect
In case of needing to disconnect from a specific broker
select amqp.disconnect(broker_id);
do nothing if it is already disconnected
PostgreSQL LISTEN Exchange
A RabbitMQ exchange type that translates PostgreSQL NOTIFY
messages to AMQP messages and publishes them to bound queues
Require Postgresql >= 9.0
Rabbitmq plugin so unobtrusive to Postgresql environment
https://github.com/aweber/pgsql-listen-exchange
Installation & configuration
Unzip file into your RabbitMQ plugins directory
rabbitmq-plugins enable pgsql_listen_exchange
To connect to PostgreSQL using the default
pgsql://postgres@localhost:5432/postgres
Edit rabbitmq.properties
[{pgsql_listen_exchange, [ {host, "localhost"}, {port, 5432},
{dbname, "postgres"}, {user, "postgres"}, {password, ""} ]}].
Send a notifcation
Psql (9.4.1)postgres=# NOTIFY channel_name, 'This is a
test';
Receive a notification
Psql (9.4.1)Postgresl=# listen channel_name
Resources
http://www.rabbitmq.com
http://blogs.vmware.com/vfabric/2013/02/choosing-your-messaging-protocol-amqp-mqtt-or-stomp.html
http://interlinked.org/tutorials/postgresql.html
Questions?
