Message Replay Overview
Message Replay is a Solace PubSub+ feature that allows an event broker to resend messages to new or existing clients that request them, hours or even days after those messages were first received by the event broker.
With replay enabled, event brokers store persistent messages in a replay log. These messages are kept until the log is full, after which the oldest messages are removed to free-up space for new messages.
Replay can be performed on a queue or topic endpoint. When initiated, all messages in the replay log, starting at the requested replay start time, that match any subscription on the queue or topic endpoint are delivered to the endpoint. Replay can be initiated by event broker commands or client applications. Messages can be replayed from a specific date and time, or as far back as the replay log extends. The replay stream eventually joins the stream of live messages without requiring any action from the consuming application.
Replay functionality helps protect applications by giving them the ability to correct database issues they might have arising from events such as misconfiguration, application crashes, or corruption, by redelivering data to the applications.
- A replay can be specified to begin from a specific time, or from the beginning of the message replay log.
- You can initiate a replay from all Solace APIs or administratively for non-Solace protocols via Solace CLI, SEMP or Solace PubSub+ Manager.
- Message Replay can be performed for any consumer no matter what protocol or API they are using, including AMQP, MQTT, REST/WebHooks and Solace APIs.
- Wildcard subscriptions on queues and topic endpoints can be used to select which messages from the replay log should be replayed while still maintaining the original publication order across these many topics.
- Messages are replayed in the original order they would have been sent to the client – even across different topics.
- You can enable and disable Message Replay on a per Message VPN basis.
- You can configure the storage depth of the replay log on a per Message VPN basis.
- The replay log is automatically pruned as it reaches capacity.
From here there are a few paths you can take depending on what you want to learn,
- You can go to the Walkthrough section for a high-level, illustrated example of the basic operation of Message Replay.
- If you're interested in learning more about how Message Replay can be used in your network, consider reviewing the Message Replay Use-Cases.
- If you're ready to use Message Replay, you can begin learning about how to configure and use it by jumping to Message Replay Configuration.
- And there's a short video on how to get Message Replay up-and-running in under 4 minutes which you can view here:
In this section we'll take a high-level look at how Message Replay works on a Solace PubSub+ event broker. We won't go into every detail just yet, just walkthrough the basic ideas via a simplified example so you can get a sense of how Message Replay works. If you're ready for a deep dive into details and configuration, take a look at Message Replay Configuration.
The way that Message Replay works is fairly straightforward. Let's follow the journey of messages from a Publisher, through the Solace PubSub+ event broker, to a Subscriber, and see how they get logged for future playback.
Consider that you have a Publisher application that connects to the event broker and publishes messages. Those messages get added to the Queues that are subscribing, and they also get written to the Replay Log. One thing that's important to keep in mind is that the event broker isn't actually writing two copies of each message to the persistent store, only one copy is written, and the event broker keeps two references to it: there's one in the Queue that's going to deliver the message to the Subscriber, and a second that's in the Replay Log.
Ok, so then the Subscriber consumes the messages, processes them, acknowledges them, and the messages are then deleted from the Subscriber's Queue, but they remain in the Replay Log. The Replay Log continues to store messages even after they have been delivered to all Subscribers.
If some time later the Subscriber wants to get back those messages that have already been delivered, it can request a replay. The messages will be fetched from the Replay Log, and they'll be delivered to the Subscriber a second time.
You can choose to replay messages from a specific time if you know what time you want to start the replay at, or you can start replay from the oldest message available in the Replay Log.
There are 2 ways of initiating a message replay:
- If your application is using a Solace API, then you can kick off the replay request from your application.
- Your other alternative is to initiate replay from PubSub+ Manager, Solace CLI, or SEMP. Using this administration kick-off method means that a replay can go to any API: Solace API , MQTT APIs, AMQP applications, REST delivery points, or whatever protocol and API your application is using to consume events and messages can be replayed to your application.
One of the most powerful features of Message Replay is the ability to replay messages not just to a fully qualified topic, but to a wildcard topic, or to a wildcard subscription. For example, you could have a queue set up to receive messages on topics
orders/>, request a replay, and the event broker will deliver all the messages in the Replay Log that match the
orders/> wildcard subscription. This makes Message Replay quite flexible in terms of what can be delivered from the Replay Log to an application.
Messages are only replayed to the Queue for which the replay is requested
If a replay is initiated from Solace PubSub+ Manager, or Solace CLI, then one of the things you have to specify is which Queue you want to replay messages to.
If you're initiating replay from a Solace API, then the Queue that your application is connected to is where the replay is going to happen. This is important because you may have other Queues on your event broker that subscribe to that same topic, and those normally functioning applications would be very confused if they suddenly started receiving replay data.
Not surprisingly Message Replay does consume resources on a PubSub+ event broker, so as part of configuring the event broker you'll choose which of the Message VPNs will have replay enabled, and which Message VPNs don't need this capability, so you can disable Message Replay on those other Message VPNs.
The other thing you can do as part of setting up Message Replay is you can specify how many gigabytes of persistent store you want your Replay Log to consume inside of a Message VPN.
Replay Log pruning
Once the Replay Log grows to the maximum depth you've configured for it, then the event broker will automatically prune out the oldest messages from the Replay Log so that there's room for new messages as they arrive to be written to the Replay Log
In this section we’ll look at a number of use-cases where Message Replay can restore or recreate application databases.
Consider an application that's connected to one or more queues, receives messages, processes them, updates its own persistent database to reflect message content, and acknowledges receipt. This acknowledgment deletes the messages from the queue(s). Say the application then experiences some form of crash which results in the corruption of its own database. The application may be restarted with a known good database, perhaps from the beginning of the day, but any work it has done in the meantime has been lost.
In this situation, the application might request message replay from its queues from the start of day, redoing the work it had already done. During replay, new live messages will be added to the replay log. Once all logged messages have been replayed, the application automatically rejoins the live data stream.
The administrator may also initiate replay of a queue on the client’s behalf through Solace CLI, Solace PubSub+ Manager, or SEMP commands.
Continuing with the application in the previous example, where a set of administrator configured topic-to-queue mappings is used to attract messages to the application’s queues. In this case a mistake is made and the set of topics on the queue is not correct, and the error is not noticed until part way through the business day.
In this case, the administrator corrects the topic-to-queue mappings on the queue, and restarts the application with a known good state from start-of-day, or last checkpoint, or beginning-of-replay-log. The administrator or application requests message replay from its queues, redoing its work with the now corrected set of topic-to-queue mappings in play. Once all old messages, including ones that were missed due to the incorrect topic-to-queue-mapping, have been replayed to the application it automatically rejoins the live data stream.
In this use-case, an application which did not have queues or subscriptions on the event broker at the time the data was published may wish to connect to the event broker, create its queues and subscriptions, and then replay and consume the previously published data. Typical applications for this use-case include bringing on-line a new analytics or trading algorithm which requires some amount of historical data to make decisions on the subsequent live data.
In this use-case, an enterprise may wish to capture the entire history of message flow to an application so that it can be reviewed or analyzed at a later date by replaying the data that was sent to the application.
Message Replay isn't a replacement for PubSub+ Cache. There are a number of differences between the two, and they're listed in the following table. Whether you choose Message Replay or PubSub+ Cache depends on your use-cases.
|Attribute||Message Replay||PubSub+ Cache|
Records all Guaranteed messages published to the Message-VPN.
Automatically trims oldest messages from log when log’s quota is reached.
Last Value Cache
Typically keeps only the last message published on each topic.
While PubSub+ Cache can be configured to keep more than one message per topic, that is a less common use-case.
Logs Guaranteed (aka Persistent) messages.
Logs any messages that get promoted to Guaranteed messages.
Intended for caching Direct (aka Non-Persistent) messages.
Connects to the event broker using Direct messaging. If any Guaranteed messages match PubSub+ Cache subscriptions, those messages are demoted to Direct when sent to PubSub+ Cache.
|Interactions With Client API||All interactions with client APIs use Guaranteed messaging.||All interactions with client APIs use Direct messaging.|
|Topics Recorded||Logs all Guaranteed messages published in the Message-VPN.||Administrator configures the wildcard topics that are to be stored in the cache.|
|Message Delivery Order||
Replays messages in original published order.
Publish order between topics is preserved.
Cache requests are fulfilled topic-by-topic.
Order is preserved within a topic (if depth is greater than 1), but original publish order between topics is not preserved.
|Replay Request Mechanism||
|Deployment Model||Integrated feature of the PubSub+ event broker.||Software application that runs on an external Linux server.|
|Message Storage||Uses the Guaranteed messaging infrastructure to persistently store messages.||Stores cached messages in RAM.|
|Redundancy Model||Leverages the event broker’s active-standby HA model.||Deploy multiple instances for n+1 redundancy.|
|Performance and Scaling||
Built for logging messages at Guaranteed messaging rates.
Add more PubSub+ event brokers to horizontally scale Guaranteed messaging and Message Replay.
Built for very high rate Direct messaging use-cases like market data distribution.
Supports deploying multiple instances to horizontally scale.
|Availability||Integrated into the PubSub+ event broker; no extra license.||Optional add-on product that requires purchased license.|