Dead Message Queues

By default, Guaranteed messages are removed from a durable endpoint's message spool and discarded when:

  • the number of redelivery attempts for a message exceeds the Max Redelivery value for the original destination endpoint;
  • or, a message’s TTL value has been exceeded, and the endpoint is configured to respect message TTL expiry times.

If you don't want the messages to be discarded, they can be moved to a dead message queue (DMQ) assigned to the endpoint.

How DMQs Work

Any durable queue on the same Message VPN as the endpoint that the messages are being removed from can be assigned as a DMQ.

Messages that are flagged as DMQ-eligible by the publishing client will be sent to the endpoint's DMQ rather than be discarded.

However, if an endpoint's assigned DMQ doesn't exist, then the removed Guaranteed messages will still be discarded even if they are DMQ-eligible. By default, each durable endpoint is assigned a DMQ named #DEAD_MSG_QUEUE, but this queue doesn't pre-exist and must be created by a management user.

Setting the DMQ

To assign a specific DMQ for a durable queue, enter the following CONFIG command:

solace(configure/message-spool/queue)# dead-message-queue <name>

To assign a specific DMQ for a durable topic endpoint, enter the following CONFIG command:

solace(configure/message-spool/topic-endpoint)# dead-message-queue <name>

Where:

<name> is the name of an existing queue on the same Message VPN as the given endpoint.

The no version of this command, no dead-message-queue, resets the endpoint's assigned DMQ to the default of #DEAD_MSG_QUEUE.

Moving messages to the DMQ while new messages are also being spooled could consume a large amount of event broker spool resources. Solace recommends that there always be an online consumer connected to a DMQ for expired messages to reduce overall event broker spool usage requirements.

Setting the DMQ through Endpoint Templates

You can assign a specific DMQ for client created endpoints using an endpoint template. When client created endpoints are created dynamically using an API, the endpoint name will be matched to the endpoint template. Take a look at Endpoint Templatesfor more information.

To assign a specific DMQ for a client created queue using a queue template, enter the following CONFIG command:

solace(configure/message-spool/queue-template)# dead-message-queue <name>

To assign a specific DMQ for a client created topic endpoint using a topic endpoint template, enter the following CONFIG command:

solace(configure/message-spool/topic-endpoint-template)# dead-message-queue <name>

Where:

<name> is the name of an existing queue on the same Message VPN as the given endpoint.

The no version of this command, no dead-message-queue, resets the endpoint's assigned DMQ to the default of #DEAD_MSG_QUEUE.

Setting DMQ-eligibility & TTL

Setting DMQ-eligibility

A message's DMQ-eligibility is set by the publishing client. For instructions on setting, refer to the following:

Setting TTL

A message's TTL value may also be set by a publishing client, and / or it may also have a max-ttl value, which an endpoint can set when it receives the message.

Feature Interactions

Using high-availability

Solace PubSub+ event brokers in a high-availability (HA) group should have their clocks synchronized with a Network Time Protocol (NTP) server (PTP can also be used by appliances) for Guaranteed message expiry to work as expected in redundant configurations. If the backup event broker does not have the same clock time as the primary, then the message will not expire at the expected time if there is a failover between when the message is published and when it is set to expire. NTP server parameters can be configured from the CLI through the ntp-server CONFIG command.

Using Replication

If you're using Replication, and ack propagation is enabled for the endpoint (enabled is the default value), replicated messages on the standby site that exceed their TTLs will be discarded rather than moved to a DMQ. If you disable ack propagation, then the TTL timers will run independently at both sites and lead to differences between the messages spooled at each site. Note that Solace recommends that you don't disable ack propagation.

Delayed Message Consumption

A special use of dead message queues (DMQs) is as a means to implement a function called delayed message consumption. It allows messages to be published at one time and consumed at some later time, where you specify the lag between publication and consumption.

Here's how to get things up and running.

Say you want to have a lag of delta_t between the publication of messages to an app A and their consumption by the app. In this example, messages are published on topic X.

Here are the general configuration steps:

  1. Configure app A to consume messages from queue A.
  2. Configure a queue B that's subscribed to topic X, and be sure to:
    • Enable respect-ttl
    • Set max-ttl to the desired delta_t.
  3. Set queue B's DMQ to be queue A.
  4. For each message on topic X to be sent by your publishing app you should ensure the DMQ eligible flag is set.

When your publishing app sends a message on topic X, it'll be put on queue B for delta_t. When the TTL expires, the event broker will move the message to queue A, and deliver the message to app A when app A is connected.

:  The event broker's TTL timer check runs once a second, and, in most cases, introduces a 1 second quantization into the delivery of messages from the DMQ.

Related topics