Message Replay Configuration

Message Replay allows published Guaranteed messages, and Direct messages promoted to Guaranteed, that were not rejected-to-publisher for any reason, to be stored in a replay log for an indefinite period after being received in case they need to be replayed at some later date. Messages are kept in the replay log until it's full, at which point, the oldest messages are discarded to free up space for new ones.

You can initiate replay log playback for an endpoint from either the event broker, or from clients who have at least consume privileges for the endpoint, which causes a filtered subset of the logged messages to be sent to those clients.

This topic is divided into two major areas of discussion:

Also, before using Message Replay you should consult the Deployment Notes about supported PubSub+ products and operational considerations.

Next Steps

From here there are a few different paths you can take depending on what you want to learn:

Deployment Notes

Supported PubSub+ Products

Message Replay is supported on the following Solace PubSub+ products:

  • Solace PubSub+ 3530 and 3560 appliances running release 9.1 and greater.
  • Solace PubSub+ software event brokers running release 9.1 and greater.

Advisories

You should be aware of the following Message Replay characteristics prior to deployment:

  • Message Replay consumes event broker processing resources because it needs to maintain the replay log, handle retrieval and replay of messages, and trim the replay log when it's full.
  • Slow disks negatively impact Message Replay performance.
  • Replication with Message Replay isn't supported.

Terminology

The following table presents definitions of terms that are commonly used in Message Replay discussions. The section Life Cycle of a Replayable Message shows you how a replayed message is processed and utilizes these terms.

Term Description
live message

A published message which has not been sent to all consuming clients. It exists somewhere in the datapath; that is, it was either received on ingress, sent via egress, or exists in one or more endpoints
replayable message

A live message that is received on a Message VPN that has Message Replay enabled.
logged message A replayable message which has been set aside in the replay log for later replay.
replaying endpoint An endpoint under replay.
replayed message A logged message which matches a replaying endpoint's subscriptions and has been added to the endpoint.
replay to endpoint The process of adding a logged message to a replaying endpoint, thereby creating a replayed message instance on the replaying endpoint.
replay log trimming The process of automatically deleting the oldest messages from the replay log to make room for new live data.

Types of Replay Requests

A Message Replay playback request may be initiated for an endpoint either by a client application using a Solace API when it binds to a Guaranteed endpoint, or from Broker Manager, Solace CLI or SEMP. In either case, the following Message Replay playback requests are supported:

  • Replay From-Date-And-Time: the requester specifies the date and time from which the playback is to start. Any messages in the replay log equal to, or newer than, the specified date and time that match the endpoint’s subscriptions are delivered to the consumers.
  • Replay After-Replication-Group-Message-ID: the requester specifies a replication group message ID after which the playback is to start. Any messages in the replay log received after the specified replication group message ID that match the endpoint's subscriptions are delivered to the consumers.
  • Replay From-Beginning-Of-Replay-Log: the requester specifies that the playback is to start from the oldest message in the replay log. All messages in the replay log that match the endpoint’s subscriptions are delivered to the consumers.

In all cases, once the replay has caught up to the live data stream, message delivery switches to delivery from the endpoint.

Life Cycle of a Replayable Message

Let's walk though the life cycle of a replayable message. A couple of points to keep in mind is that the life cycle is the same for both Types of Replay Requests, and in each of the following life cycle phases the underlined passages correspond to terms in the Terminology section.

  1. A live message is received on a Message VPN that has Message Replay enabled. This message is considered to be a replayable message.
    • Instructions on how to create a replay log with Solace CLI are shown in Create or Delete a replay-log.
    • Instructions on how to enable a replay log with the Solace CLI's no shutdown command are shown in shutdown.
  2. The message is successfully spooled to all its non-replaying destinations, gets added to the replay log, and is now considered to be a logged message.
  3. At some time later the event broker receives a replay request with a start time that includes the logged message. Once the replay gets to the logged message, and successfully performs replay to endpoint, it's considered a replayed message.
  4. Finally, the replayed message is delivered to a consumer and is acknowledged, which removes it from the replaying endpoint; however, it's still in the replay log until it gets trimmed.
    • For a little more discussion on automatic and manual replay log trimming, refer to Trimming the Replay Log.
    • Instructions on how to manually initiate a replay log trimming with Solace CLI can be found in trim-logged-messages.

Replaying to an Endpoint

Replaying logged messages is done on a per-endpoint basis, and can be initiated using PubSub+ Broker Manager, SEMP, Solace CLI commands, or by a client application using a Solace API when it binds to a Guaranteed endpoint. As we discussed in Types of Replay Requests, a replay request can initiate the playback of all the logged messages, all messages after a given replication group message ID, or start from a given date and time, with the condition that if a start date and time are provided, they must not be for some future date and time. In all cases, all clients bound to the replaying endpoint are disconnected, except the client requesting the replay if done through a bind request.

Topic Matching

All logged messages from the playback start time are processed, and the event broker attempts to match their topics with the replaying endpoint subscriptions. When there's a match, the event broker replays the logged message to the replaying endpoint, and it sends the original message as it was received, without modification: it uses the original topic, content, and message ID. The event broker doesn't mark the message as redelivered in any way.

Windowing

Windowing is used while replaying messages to the replaying endpoint; this means that only a certain number of messages are replayed to the endpoint before the event broker waits for a replayed message to be consumed from the replaying endpoint.

Endpoint already undergoing a replay?

Initiating a replay on an endpoint already under replay cancels the previous replay and starts a new one.

Live messages being received at an endpoint undergoing replay?

Live messages received with a replaying endpoint as a destination undergo all the usual checks against the replaying endpoint as it would do if it wasn't replaying. Once all the checks have passed, the message isn't spooled to the endpoint, but is spooled to the replay log - it will be replayed later once the replay catches up to the message.

Message gets nacked to the publisher?

A message that gets nacked to the publisher doesn't get logged to the replay log.

Replaying to Exclusive Queues

Replaying to any exclusive queue whose active consumer is using an egress selector to filter messages isn't recommended, because if an egress selector is used, and it causes replayed messages to stay indefinitely on a replaying endpoint, the replay won't complete.

Replaying to Temporary Endpoints

Initiating a replay to a temporary topic endpoint or temporary queue is not supported in versions earlier than 10.0.0. In version 10.0.0+ you can replay to a temporary topic endpoint, or temporary queue provided that you configure the event broker appropriately. For more information, see Enable Replay on Temporary Queues.

Replay States

The following table shows the states an endpoint can be in when undergoing a replay.

State Description
N/A

The initial state of all endpoints. Once a replay is started on an endpoint, this state can never be reached again by that endpoint.
Complete

No replay is in progress.
Initializing All live messages are being deleted from the endpoint before a replay starts.
Active The endpoint is currently under replay. Message replay is processing the replay log, and replayed messages are being added to the endpoint.
Pending Complete A replay has reached the end of the replay log, but there are still unacked replayed messages on the endpoint. New live messages are being delivered to the endpoint. However, it's still possible that the replay can fail and have the unacked replayed messages be deleted from the endpoint.
Failed A replay has failed and is pending acknowledgment of an unbind request it sent as a failure indication.

Trimming the Replay Log

As the replay log grows larger and fills up, older logged messages must be deleted to make room for new ones. This process is called replay log trimming.

Automatic trimming happens when the replay log grows beyond its configured max-spool-usage, and trimming always deletes from the oldest logged message.

It's also possible to manually request trimming with Solace CLI by using the trim-logged-messages command. A date and time must be provided, which will trim the log from the oldest message, excluding the one with the provided date and time.

Direct Messages & Message Replay

You should note that Message Replay is intended to be used for Guaranteed messages, or messages that have already been promoted to Guaranteed messages. Direct messages that aren't promoted to Guaranteed are not logged. So, if there's a possibility that your messages might need to be replayed, they should be published as Guaranteed.

Care must be taken when deciding to promote Direct messages as this could dramatically affect the event broker's performance, and if the event broker's Guaranteed message budget is exceeded, the Direct messages will not be logged. If you have Direct messages that you want to record in the replay log, the recommended method is to change the publisher to use Guaranteed messages.