Message QueuesA basic overview

Geshan ManandharQuality and Maintenance Lead, Namshi.com

@geshan

geshan.com.np

IndexWhat is Message Queue(MQ)MQ basics - AMQPBenefitsRabbit MQOur setup/stackWhen to use it /Practical UsecaseLogging and monitoringConclusion

What is Message Queue"message queues and mailboxes are software-engineeringcomponents used for inter-process communication (IPC), or forinter-thread communication within the same process. They use aqueue for messaging – the passing of control or of content." -WikipediaMessage queue is a software components that allow producers toproduce message to a queue managed by a message broker, thesemessage are asynchronously consumed by consumers.

Message Queue BasicsProtocols : Advanced Message Queueing Protocol(AMQP) andStreaming Text Oriented Messaging Protocol(STOMP)Types of Queue: Direct, Topic and FanoutProducer/PublisherMessage Broker - Queue ExchangeQueueConsumer

Benefits of MQDecouplingScalabilityReliabilityFanout - Same message to multiple queues (Pub/Sub)

Rabbit MQ

Rabbit MQ - featuresRobust messaging for applicationsEasy to useRuns on all major operating systemsSupports a huge number of developer platformsOpen source and commercially supported

Our Software stack/servicesrelating to with Rabbit MQ

Symfony 2 with old sound rabbit mq bundle and MonologRabbit Mq on http://www.cloudamqp.com/Service Oriented Architecture (SOA) with 3rd party APIintegrations

When to use message queuesSome practical use cases

Backend -> Frontend sync (db to NOSQL/Memcache)Communication between multiple internal systems (without API)Asynchronous 3rd party API callsNotifications like email and smsHandling large request asynchronouslyProcessing intensive tasks- like image resize

Logging and monitoring isvital

Only access and error logs from web server and database are surelynot enoughLogs are first class citizen, adopt active logging from applicationlevel.(Monolog)Use dedicated log management system like GrayLog 2Monitor application activity in real-time with services likeNewRelicLog optimally and verbose enough to give enough informationabout the context.

Conclusion/RecapCommunication between two systems can be done even withoutAPIsLanguage agnostic implementation is enable by using messagequeuesMessage queues could be that answer you were looking for, tocommunicate between different systemsFor scalability, decoupling and reliability utilize message queuesYou could potentially scale to 1 million message per second :PivotalLogging and monitoring are inevitable when doing asynchronousprocessing

Questions???

Creditshttp://keyholesoftware.com/2013/05/13/messaging-with-rabbitmq/https://www.flickr.com/photos/i8ipod/9028050858https://www.flickr.com/photos/oberazzi/318947873/

Some programming mantras to remember.