60

I'm trying to setup my first RabbitMQ dead letter exchange, here are the steps I'm using through the web admin interface:

  1. Create new DIRECT exchange with the name "dead.letter.test"
  2. Create new queue "dead.letter.queue"
  3. Bind "dead.letter.queue" to "dead.letter.test"
  4. Create new queue "test1" with the dead letter exchange set to "dead.letter.test"
  5. Send a message into "test1"
  6. Nack (with requeue = false) the message in "test1"

I am expecting that these steps should put a record into the "dead.letter.queue" through the "dead.letter.test" exchange. This is not happening.

I can manually put a message into the "dead.letter.test" exchange and it shows up in "dead.letter.queue" so I know that is fine.

When I look at the admin UI it shows that the DLX parameter is setup on the queue "test1".

Where am I going wrong?

jhilden
  • 12,207
  • 5
  • 53
  • 76

8 Answers8

89

Gentilissimo Signore was kind enough to answer my question on Twitter. The problem is that if your dead letter exchange is setup as DIRECT you must specify a dead letter routing key. If you just want all your NACKed message to go into a dead letter bucket for later investigation (as I do) then your dead letter exchange should be setup as a FANOUT.

Here are the updated steps that work:

  1. Create new FANOUT exchange with the name "dead.letter.test"
  2. Create new queue "dead.letter.queue"
  3. Bind "dead.letter.queue" to "dead.letter.test"
  4. Create new queue "test1" with the dead letter exchange set to "dead.letter.test"
  5. Send a message into "test1"
  6. Nack (with requeue = false) the message in "test1"
jhilden
  • 12,207
  • 5
  • 53
  • 76
  • What if i want a dead letter queue with single exchange and redirect based on the route key . Is it possible with fanout ? – Karthik Jul 16 '15 at 12:52
  • Solution to K-lyer's question: the only thing "special" about the dead letter exchange usage is that you bind it with the x-dead-letter-exchange and optional x-dead-letter-routing-key properties. If you want to redirect on a routing key just make it a direct exchange, and bind your queues to it. – Rick O'Shea Jan 21 '17 at 20:20
  • for someone who want this to work with direct exchange and single queue, while binding the queue and add queue name as routing key property – wolf_codes Sep 20 '21 at 10:23
18

Dead Letter Exchange without routing key and with direct exchange


Follow the steps these will work for sure:-
1. Create a new queue named 'dead_queue'.
2. Create an exchange named 'dead_exchange' and type of exchange should be 'direct'.
3. Bind 'dead_queue' and 'dead_exchange' without routing key.
4. Create a new queue named 'test_queue' and set its 'x-dead-letter-exchange' name as 'dead_exchange'
5. Create an exchange named 'test_exchange' and type of exchange should be 'direct'
6. Bind 'test_exchange' and 'test_queue' without routing key.

And at last we will check it. For this publish something on 'test_exchange' with argument 'expiration' set to 10000. After this when a message is publish on 'test_exchange' it will go to 'test_queue' and when a message is expired with in a queue it will look for DLX Parameter(Dead Letter Exchange name) there that message find the name 'dead_exchange' then that message will reach 'dead_exchange' deliver it to 'dead queue' .. If still you have any problem regarding this and if i miss understood your problem... write your problem i will surely look over it... Thanks..

Note: Must publish the message on 'test_exchange' because that test_queue and test_exchange binding is without routing key and it will work fine but If you publish message on 'test_queue' default exchange and routing key will be used.Then after expiration of message queue tries to deliver that dead message to dead_exchange with some default routing key and message will not go to that queue.

Sahil Gulati
  • 15,028
  • 4
  • 24
  • 42
  • Do you have any node js code for reference, I am trying to find where do I define the dead_exchange in publisher ? I already have set up a topic exchange. – user269867 Mar 28 '18 at 21:00
  • @user269867 Sure, I will look into it. Because now a days I am working on node js. We can directly talk over skype `sahil.gulati1991@outlook.com` – Sahil Gulati Apr 19 '18 at 06:22
7

If you want to use custom routing key on dead letter exchange you have to set x-dead-letter-routing-key when declaring working queue (in your case it is test1), otherwise default routing key will be used. In your case RabbitMQ broker detects cycling and simply drop rejected messages.

What you need is to have x-dead-letter-exchange=dead.letter.test and x-dead-letter-routing-key=dead.letter.queue arguments set on test1 queue.

pinepain
  • 12,453
  • 3
  • 60
  • 65
  • Zaq, thanks for the response, I tried adding the x-dead-letter-exchange and the x-dead-letter-routing-key but I'm still not getting nacked messages to go into the dead letter exchange. My goal is simple: any message in the "test1" queue that is nacked will get put into the "dead.letter.test" exchange and then any queue that is attached to that exchange will receive the message. Do I need custom routing keys on the message to accomplish this? – jhilden Feb 13 '14 at 15:28
  • Specify in question language and library you are using to deal with amqp broker. And add some code that reproduce your problem. Also specify RabbitMQ version. – pinepain Feb 13 '14 at 15:49
  • In addition, setup [Alternate Exchange](http://www.rabbitmq.com/ae.html), just in case your messages can't be routed. One of possible solution is to experiment on FANOUT exchange which definitely will rout message everywhere. – pinepain Feb 13 '14 at 23:58
  • @jhilden did you solve you problem ? I have the same issue – user269867 Mar 28 '18 at 22:11
4

If you want all your queues to have same dead letter exchange it is easier to set a general policy:

sudo rabbitmqctl -p /my/vhost/path set_policy DLX ".*" '{"dead-letter-exchange":"MyExchange.DEAD"}' --apply-to queues
cohadar
  • 4,709
  • 2
  • 20
  • 21
  • I really recommend this approach as it far more flexible and the recommended approach from the rabbitmq team. I implemented both approaches and the policy one resulted in a lot less complex code and makes it very easy to dynamically change the behavior without backing up and recreating queues. See why you should use policies instead here: https://www.rabbitmq.com/parameters.html#policies – Fredrik Corneliusson Aug 23 '17 at 14:38
  • I've tried policies on both the exchange and queue level and messages never end up in my dead letter exchange, despite it working fine if I declare through arguments when declaring queue. I agree with ^^, this is a much nicer solution, not least because I don't have to redeclare/recreate/migrate queues when I want to update the policy. – Damien Roche Jan 09 '19 at 16:17
4

Don't need to create FANOUT exchange if it is not compulsory.

You can create DIRECT exchange using the same routing key which you have used already for other exchange. And also don't need to create a new queue for the new exchange. You can use existing queues with new exchange. You just need to bind that new exchange with the queue.

Here is my receive.js file:

var amqp = require("amqplib/callback_api");
var crontab = require('node-crontab');

amqp.connect("amqp://localhost", function (err, conn) {
conn.createChannel(function (err, ch) {
    var ex = 'direct_logs';
    var ex2 = 'dead-letter-test';
    var severity = 'enterprise-1-key';

    //assert "direct" exchange
    ch.assertExchange(ex, 'direct', { durable: true });
    //assert "dead-letter-test" exchange
    ch.assertExchange(ex2, 'direct', { durable: true });

    //if acknowledgement is nack() then message will be stored in second exchange i.e. ex2="dead-letter-test"
    ch.assertQueue('enterprise-11', { exclusive: false, deadLetterExchange: ex2 }, function (err, q) {
        var n = 0;
        console.log(' [*] Waiting for logs. To exit press CTRL+C');
        console.log(q);

        //Binding queue with "direct_logs" exchange
        ch.bindQueue(q.queue, ex, severity);
        //Binding the same queue with "dead-letter-test"
        ch.bindQueue(q.queue, ex2, severity);

        ch.consume(q.queue, function (msg) {
            // consume messages via "dead-letter-exchange" exchange at every second.
            if (msg.fields.exchange === ex2) {
                crontab.scheduleJob("* * * * * *", function () {
                    console.log("Received by latest exchange %s", msg.fields.routingKey, msg.content.toString());
                });
            } else {
                console.log("Received %s", msg.fields.routingKey, msg.content.toString());
            }

            if (n < 1) {
                // this will executes first time only. Here I'm sending nack() so message will be stored in "deadLetterExchange"
                ch.nack(msg, false, false);
                n += 1;
            } else {
                ch.ack(msg)
                n = 0
            }
        }, { noAck: false });
    });
  });
});
Jay Sojitra
  • 172
  • 1
  • 9
2

Create new DIRECT exchange with the name "dead.letter.test"

Correct

Create new queue "dead.letter.queue"

Correct

Bind "dead.letter.queue" to "dead.letter.test"

Correct

Create new queue "test1" with the dead letter exchange set to "dead.letter.test"

I am assuming you are creating test1 queue and binding it to dead.letter.test exchange

Send a message into "test1"

If you want your message to be received by dead.letter.queue you will have to provide routing key while sending message and clients consuming dead.letter.queue should also use same routing key

If you are publishing without routing key then only clients who are subscribed to test1 will receive the message.

If you publish message to direct.letter.test exchange then all the queue will receive the message. It will work like a fanout exchange

So, if you want dead.letter.queue to receive message you will have to publish message in that queue or you will have to use same routing key while publishing and subscribing and publish message to exchange

Jack Daniel's
  • 2,583
  • 22
  • 28
  • Atul, the "test1" queue is not bound to any exchange, I'm just publishing directly to it for testing purposes. What I want is that when any message from "test1" is nacked that the message be put into the "dead.letter.test" exchange. – jhilden Feb 13 '14 at 15:31
  • You will have to bind it to dead.letter.test. BTW how did you created queue without mentioning which exchange? – Jack Daniel's Feb 13 '14 at 23:31
  • My guess, if you have not mentioned exchange then your queue is automatically binded to default exchange so your messages are getting published to default exchange – Jack Daniel's Feb 13 '14 at 23:32
  • Atul, you are correct, if you don't say what exchange a queue is on (which is valid) then when you send messages directly to the queue it goes through the default exchange. Either way though, if a message is nacked on a queue (regardless of how it got into that queue) shouldn't it then be sent to the dead-letter-queue associated with the original queue (test1)? – jhilden Feb 14 '14 at 14:55
  • The core idea in the messaging model in RabbitMQ is that the producer never sends any messages directly to a queue. Actually, quite often the producer doesn't even know if a message will be delivered to any queue at all. Instead, the producer sends messages to an exchange. An exchange is a very simple thing it receives messages from producers and the pushes them to queues. The exchange must know exactly what to do with a message it receives. Should it be appended to a particular queue? Should it be appended to many queues? Or should it get discarded. – Jack Daniel's Feb 17 '14 at 00:12
0

In my case the problem was because the queue had

ackMode="MANUAL"

But I never set it (because a runtime exception), use Defaul ACK instead.

Alexis Gamarra
  • 4,362
  • 1
  • 33
  • 23
0

For those who use Spring-AMQP

In my case the problem was different. I wanted a dead-letter-exchange to be of type direct. And i set both x-dead-letter-exchange and x-dead-letter-routing-key for the queue. Plus i had spring.rabbitmq.listener.simple.default-requeue-rejected=false in the application.properties.

Seems everything fine, but while debugging i noticed that my SimpleRabbitListenerContainerFactory has defaultRequeueRejected as null. So the reason was that when you declare SimpleRabbitListenerContainerFactory in your @Configuration, you create a new "non-default" bean. The default one is created for you behind the scene out of your properties. But your SimpleRabbitListenerContainerFactory in the @Config, these properties are not read, you must read it yourself and set in java code.

It happened to me, because i just copy-pasted the config from the Spring-AMQP docs when wanted to configure the concurrency. But you should do everything in one place, either in properties, like

spring.rabbitmq.listener.simple.default-requeue-rejected=false
spring.rabbitmq.listener.simple.concurrency=5
spring.rabbitmq.listener.simple.max-concurrency=10

or completely in java, like

    @Bean
    public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory(ConnectionFactory connectionFactory) {
        SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
        factory.setConnectionFactory(connectionFactory);
        factory.setConcurrentConsumers(5);
        factory.setMaxConcurrentConsumers(10);
        factory.setDefaultRequeueRejected(false);
        return factory;
    }

These 2 above are the same.

I would expect that when i use the second (java) option is still picks up properties from the application.properties and then i customize then in java, but it doesn't work like this. And yes, "copy-paste" is evil :)

Dmitrii Bocharov
  • 872
  • 6
  • 21