This guide covers queue, exchange and message durability, as well as othertopics related to durability, for example, durability in cluster environment.
This work is licensed under a Creative Commons Attribution 3.0 Unported License (including images and stylesheets). The source is available on Github.
This guide covers Ruby amqp gem 1.7.0 and later versions.
AMQP separates the concept of entity durability (queues, exchanges) from message persistence. Exchanges can be durable or transient. Durable exchanges survive broker restart, transient exchanges do not (they have to be redeclared when broker comes back online). Not all scenarios and use cases mandate exchanges to be durable.
Durable queues are persisted to disk and thus survive broker restarts. Queues that are not durable are called transient. Not all scenarios and use cases mandate queues to be durable.
Note that only durable queues can be bound to durable exchanges. This guarantees that it is possible to restore bindings on broker restart.
Durability of a queue does not make messages that are routed to that queue durable. If a broker is taken down and then brought back up, durable queues will be re-declared during broker startup, however, only persistent messages will be recovered.
Messages may be published as persistent and this is what makes an AMQP broker persist them to disk. If the server is restarted, the system ensures that received persistent messages are not lost. Simply publishing a message to a durable exchange or the fact that a queue to which a message is routed is durable does not make that message persistent. It all depends on the persistence mode of the message itself. Publishing messages as persistent affects performance (just like with data stores, durability comes at a certain cost to performance and vice versa). Pass :persistent => true to { yard_link AMQP::Exchange#publish } to publish your message as persistent.
TBD
Because transactions carry certain (for some applications, significant) overhead, RabbitMQ introduced an extension to AMQP 0.9.1 called publisher confirms (documentation).
The amqp gem implements support for this extension, but it is not loaded by default when you require “amqp”. To load it, use
require "amqp/extensions/rabbitmq"
and then define a callback for publisher confirms using
AMQP::Channel#confirm:
# enable publisher acknowledgements for this channel
channel.confirm_select
# define a callback that will be executed when message is acknowledged
channel.on_ack do |basic_ack|
puts "Received an acknowledgement: delivery_tag = #{basic_ack.delivery_tag}, multiple = #{basic_ack.multiple}"
end
# define a callback that will be executed when message is rejected using basic.nack (a RabbitMQ-specific extension)
channel.on_nack do |basic_nack|
puts "Received a nack: delivery_tag = #{basic_nack.delivery_tag}, multiple = #{basic_nack.multiple}"
end
Note that the same callback is used for all messages published via all exchanges on the given channel.
To achieve the degree of durability that critical applications need, it is necessary but not enough to use durable queues, exchanges and persistent messages. You need to use a cluster of brokers because otherwise, a single hardware problem may bring a broker down completely.
RabbitMQ offers a number of high availability features for both scenarios with more (LAN) and less (WAN) reliable network connections.
See the RabbitMQ clustering and high availability guides for in-depth discussion of this topic.
Whilst the use of clustering provides for greater durability of critical systems, in order to achieve the highest level of resilience for queues and messages, high availability configuration should be used. This is because although exchanges and bindings survive the loss of individual nodes by using clustering, messages do not. Without mirroring, queue contents reside on exactly one node, thus the loss of a node will cause message loss.
See the RabbitMQ high availability\ guide for more information about mirrored queues.