Message Replay Configuration
Message Replay allows published Guaranteed messages, and Direct messages promoted to Guaranteed, that were not
rejected-to-publisher for any reason, to be stored in a replay log for an indefinite period after being received in case they need to be replayed at some later date. Messages are kept in the replay log until it's full, at which point, the oldest messages are discarded to free up space for new ones.
You can initiate replay log playback for an endpoint from either the event broker, or from clients who have at least consume privileges for the endpoint, which causes a filtered subset of the logged messages to be sent to those clients.
This topic is divided into two major areas of discussion:
- A presentation of the basic concepts underlying Message Replay, which includes.
- Types of Replay Requests
- Life Cycle of a Replayable Message
- Replay States
- Replaying to an Endpoint
- Trimming the Replay Log
- Direct Messages & Message Replay
- Instructions on using Solace CLI commands associated with Message Replay, which you can find in these sections.
Also, before using Message Replay you should consult the Deployment Notes about supported PubSub+ products and operational considerations.
From here there are a few different paths you can take depending on what you want to learn:
- Take a look at some simple examples showing how to configure and use Message Replay with Solace CLI in the section Message Replay Examples.
- See the details of how to use each Message Replay Solace CLI command in these sections.
- Watch the following video, Getting Started with Message Replay, that shows you how to get Message Replay up and running with PubSub+ Broker Manager.
Message Replay is supported on the following Solace PubSub+ products:
- Solace PubSub+ 3530 and 3560 appliances running release 9.1 and greater.
- Solace PubSub+ software event brokers running release 9.1 and greater.
You should be aware of the following Message Replay characteristics prior to deployment:
- Message Replay consumes event broker processing resources because it needs to maintain the replay log, handle retrieval and replay of messages, and trim the replay log when it's full.
- Slow disks negatively impact Message Replay performance.
- Replication with Message Replay isn't supported.
The following table presents definitions of terms that are commonly used in Message Replay discussions. The section Life Cycle of a Replayable Message shows you how a replayed message is processed and utilizes these terms.
A published message which has not been sent to all consuming clients. It exists somewhere in the datapath; that is, it was either received on ingress, sent via egress, or exists in one or more endpoints
A live message that is received on a Message VPN that has Message Replay enabled.
|logged message||A replayable message which has been set aside in the replay log for later replay.|
|replaying endpoint||An endpoint under replay.|
|replayed message||A logged message which matches a replaying endpoint's subscriptions and has been added to the endpoint.|
|replay to endpoint||The process of adding a logged message to a replaying endpoint, thereby creating a replayed message instance on the replaying endpoint.|
|replay log trimming||The process of automatically deleting the oldest messages from the replay log to make room for new live data.|
A Message Replay playback request may be initiated for an endpoint either by a client application using a Solace API when it binds to a Guaranteed endpoint, or from Broker Manager, Solace CLI or SEMP. In either case, the following Message Replay playback requests are supported:
- Replay From-Date-And-Time: the requester specifies the date and time from which the playback is to start. Any messages in the replay log equal to, or newer than, the specified date and time that match the endpoint’s subscriptions are delivered to the consumers.
- Replay After-Replication-Group-Message-ID: the requester specifies a replication group message ID after which the playback is to start. Any messages in the replay log received after the specified replication group message ID that match the endpoint's subscriptions are delivered to the consumers.
- Replay From-Beginning-Of-Replay-Log: the requester specifies that the playback is to start from the oldest message in the replay log. All messages in the replay log that match the endpoint’s subscriptions are delivered to the consumers.
In all cases, once the replay has caught up to the live data stream, message delivery switches to delivery from the endpoint.
Let's walk though the life cycle of a replayable message. A couple of points to keep in mind is that the life cycle is the same for both Types of Replay Requests, and in each of the following life cycle phases the underlined passages correspond to terms in the Terminology section.
- A live message is received on a Message VPN that has Message Replay enabled. This message is considered to be a replayable message.
- The message is successfully spooled to all its non-replaying destinations, gets added to the replay log, and is now considered to be a logged message.
- At some time later the event broker receives a replay request with a start time that includes the logged message. Once the replay gets to the logged message, and successfully performs replay to endpoint, it's considered a replayed message.
- Finally, the replayed message is delivered to a consumer and is acknowledged, which removes it from the replaying endpoint; however, it's still in the replay log until it gets trimmed.
Replaying logged messages is done on a per-endpoint basis, and can be initiated using PubSub+ Broker Manager, SEMP, Solace CLI commands, or by a client application using a Solace API when it binds to a Guaranteed endpoint. As we discussed in Types of Replay Requests, a replay request can initiate the playback of all the logged messages, all messages after a given replication group message ID, or start from a given date and time, with the condition that if a start date and time are provided, they must not be for some future date and time. In all cases, all clients bound to the replaying endpoint are disconnected, except the client requesting the replay if done through a bind request.
All logged messages from the playback start time are processed, and the event broker attempts to match their topics with the replaying endpoint subscriptions. When there's a match, the event broker replays the logged message to the replaying endpoint, and it sends the original message as it was received, without modification: it uses the original topic, content, and message ID. The event broker doesn't mark the message as redelivered in any way.
Windowing is used while replaying messages to the replaying endpoint; this means that only a certain number of messages are replayed to the endpoint before the event broker waits for a replayed message to be consumed from the replaying endpoint.
Endpoint already undergoing a replay?
Initiating a replay on an endpoint already under replay cancels the previous replay and starts a new one.
Live messages being received at an endpoint undergoing replay?
Live messages received with a replaying endpoint as a destination undergo all the usual checks against the replaying endpoint as it would do if it wasn't replaying. Once all the checks have passed, the message isn't spooled to the endpoint, but is spooled to the replay log - it will be replayed later once the replay catches up to the message.
Message gets nacked to the publisher?
A message that gets nacked to the publisher doesn't get logged to the replay log.
Replaying to Exclusive Queues
Replaying to any exclusive queue whose active consumer is using an egress selector to filter messages isn't recommended, because if an egress selector is used, and it causes replayed messages to stay indefinitely on a replaying endpoint, the replay won't complete.
Replaying to Temporary Endpoints
Initiating a replay to a temporary topic endpoint or temporary queue is not supported in versions earlier than 10.0.0. In version 10.0.0+ you can replay to a temporary topic endpoint, or temporary queue provided that you configure the event broker appropriately. For more information, see Enable Replay on Temporary Queues.
The following table shows the states an endpoint can be in when undergoing a replay.
The initial state of all endpoints. Once a replay is started on an endpoint, this state can never be reached again by that endpoint.
No replay is in progress.
|Initializing||All live messages are being deleted from the endpoint before a replay starts.|
|Active||The endpoint is currently under replay. Message replay is processing the replay log, and replayed messages are being added to the endpoint.|
|Pending Complete||A replay has reached the end of the replay log, but there are still unacked replayed messages on the endpoint. New live messages are being delivered to the endpoint. However, it's still possible that the replay can fail and have the unacked replayed messages be deleted from the endpoint.|
|Failed||A replay has failed and is pending acknowledgment of an unbind request it sent as a failure indication.|
As the replay log grows larger and fills up, older logged messages must be deleted to make room for new ones. This process is called replay log trimming.
Automatic trimming happens when the replay log grows beyond its configured max-spool-usage, and trimming always deletes from the oldest logged message.
It's also possible to manually request trimming with Solace CLI by using the trim-logged-messages command. A date and time must be provided, which will trim the log from the oldest message, excluding the one with the provided date and time.
You should note that Message Replay is intended to be used for Guaranteed messages, or messages that have already been promoted to Guaranteed messages. Direct messages that aren't promoted to Guaranteed are not logged. So, if there's a possibility that your messages might need to be replayed, they should be published as Guaranteed.
Care must be taken when deciding to promote Direct messages as this could dramatically affect the event broker's performance, and if the event broker's Guaranteed message budget is exceeded, the Direct messages will not be logged. If you have Direct messages that you want to record in the replay log, the recommended method is to change the publisher to use Guaranteed messages.