Message Replay Overview

Message Replay is a Solace PubSub+ feature that allows a message broker to resend messages to new or existing clients that request them, hours or even days after those messages were first received by the message broker.

With replay enabled, message brokers store persistent messages in a replay log. These messages are kept until the log is full, after which the oldest messages are removed to free-up space for new messages.

Replay can be performed on a queue or topic endpoint. When initiated, all messages in the replay log, starting at the requested replay start time, that match any subscription on the queue or topic endpoint are delivered to the endpoint. Replay can be initiated by message broker commands or client applications. Messages can be replayed from a specific date and time, or as far back as the replay log extends. The replay stream eventually joins the stream of live messages without requiring any action from the consuming application.

Replay functionality helps protect applications by giving them the ability to correct database issues they might have arising from events such as misconfiguration, application crashes, or corruption, by redelivering data to the applications.

Key Concepts

  • A replay can be specified to begin from a specific time, or from the beginning of the message replay log.
  • You can initiate a replay from all Solace APIs or administratively for non-Solace protocols via Solace CLI, SEMP or Solace PubSub+ Manager.
    • Message Replay can be performed for any consumer no matter what protocol or API they are using, including AMQP, MQTT, REST/WebHooks and Solace APIs.
  • Wildcard subscriptions on queues and topic endpoints can be used to select which messages from the replay log should be replayed while still maintaining the original publication order across these many topics.
  • Messages are replayed in the original order they would have been sent to the client – even across different topics.
  • You can enable and disable Message Replay on a per Message VPN basis.
  • You can configure the storage depth of the replay log on a per Message VPN basis.
  • The replay log is automatically pruned as it reaches capacity.

Next Steps

From here there are a few paths you can take depending on what you want to learn,

  • You can go to the Walkthrough section for a high-level, illustrated example of the basic operation of Message Replay.
  • If you're interested in learning more about how Message Replay can be used in your network, consider reviewing the Message Replay Use-Cases.
  • If you're ready to use Message Replay, you can begin learning about how to configure and use it by jumping to Message Replay Concepts & Configuration.
  • And there's a short video on how to get Message Replay up-and-running in under 4 minutes which you can view here:

     

     

Walkthrough

In this section we'll take a high-level look at how Message Replay works on a Solace PubSub+ message broker. We won't go into every detail just yet, just walkthrough the basic ideas via a simplified example so you can get a sense of how Message Replay works. If you're ready for a deep dive into details and configuration, take a look at Message Replay Concepts & Configuration.

Basic Operation

 

Diagram

The way that Message Replay works is fairly straightforward. Let's follow the journey of messages from a Publisher, through the Solace PubSub+ message broker, to a Subscriber, and see how they get logged for future playback.

Message publication

Consider that you have a Publisher application that connects to the message broker and publishes messages. Those messages get added to the Queues that are subscribing, and they also get written to the Replay Log. One thing that's important to keep in mind is that the message broker isn't actually writing two copies of each message to the persistent store, only one copy is written, and the message broker keeps two references to it: there's one in the Queue that's going to deliver the message to the Subscriber, and a second that's in the Replay Log.

Message consumption

Ok, so then the Subscriber consumes the messages, processes them, acknowledges them, and the messages are then deleted from the Subscriber's Queue, but they remain in the Replay Log. The Replay Log continues to store messages even after they have been delivered to all Subscribers.

Replaying messages

If some time later the Subscriber wants to get back those messages that have already been delivered, it can request a replay. The messages will be fetched from the Replay Log, and they'll be delivered to the Subscriber a second time.

You can choose to replay messages from a specific time if you know what time you want to start the replay at, or you can start replay from the oldest message available in the Replay Log.

There are 2 ways of initiating a message replay:

  • If your application is using a Solace API, then you can kick off the replay request from your application.
  • Your other alternative is to initiate replay from PubSub+ Manager, Solace CLI, or SEMP. Using this administration kick-off method means that a replay can go to any API: Solace API , MQTT APIs, AMQP applications, REST delivery points, or whatever protocol and API your application is using to consume events and messages can be replayed to your application.

Some Other Key Characteristics

Wildcards

One of the most powerful features of Message Replay is the ability to replay messages not just to a fully qualified topic, but to a wildcard topic, or to a wildcard subscription. For example, you could have a queue set up to receive messages on topics orders/>, request a replay, and the message broker will deliver all the messages in the Replay Log that match the orders/> wildcard subscription. This makes Message Replay quite flexible in terms of what can be delivered from the Replay Log to an application.

Messages are only replayed to the Queue for which the replay is requested

If a replay is initiated from Solace PubSub+ Manager, or Solace CLI, then one of the things you have to specify is which Queue you want to replay messages to.

If you're initiating replay from a Solace API, then the Queue that your application is connected to is where the replay is going to happen. This is important because you may have other Queues on your message broker that subscribe to that same topic, and those normally functioning applications would be very confused if they suddenly started receiving replay data.

Resource configuration

Not surprisingly Message Replay does consume resources on a PubSub+ message broker, so as part of configuring the message broker you'll choose which of the Message VPNs will have replay enabled, and which Message VPNs don't need this capability, so you can disable Message Replay on those other Message VPNs.

The other thing you can do as part of setting up Message Replay is you can specify how many gigabytes of persistent store you want your Replay Log to consume inside of a Message VPN.

Replay Log pruning

Once the Replay Log grows to the maximum depth you've configured for it, then the message broker will automatically prune out the oldest messages from the Replay Log so that there's room for new messages as they arrive to be written to the Replay Log

Use-Cases

In this section we’ll look at a number of use-cases where Message Replay can restore or recreate application databases.

Application Corruption Recovery

Consider an application that's connected to one or more queues, receives messages, processes them, updates its own persistent database to reflect message content, and acknowledges receipt. This acknowledgment deletes the messages from the queue(s). Say the application then experiences some form of crash which results in the corruption of its own database. The application may be restarted with a known good database, perhaps from the beginning of the day, but any work it has done in the meantime has been lost.

In this situation, the application might request message replay from its queues from the start of day, redoing the work it had already done. During replay, new live messages will be added to the replay log. Once all logged messages have been replayed, the application automatically rejoins the live data stream.

The administrator may also initiate replay of a queue on the client’s behalf through Solace CLI, Solace PubSub+ Manager, or SEMP commands.

Application Misconfiguration Recovery

Continuing with the application in the previous example, where a set of administrator configured topic-to-queue mappings is used to attract messages to the application’s queues. In this case a mistake is made and the set of topics on the queue is not correct, and the error is not noticed until part way through the business day.

In this case, the administrator corrects the topic-to-queue mappings on the queue, and restarts the application with a known good state from start-of-day, or last checkpoint, or beginning-of-replay-log. The administrator or application requests message replay from its queues, redoing its work with the now corrected set of topic-to-queue mappings in play. Once all old messages, including ones that were missed due to the incorrect topic-to-queue-mapping, have been replayed to the application it automatically rejoins the live data stream.

Application Late-Join

In this use-case, an application which did not have queues or subscriptions on the message broker at the time the data was published may wish to connect to the message broker, create its queues and subscriptions, and then replay and consume the previously published data. Typical applications for this use-case include bringing on-line a new analytics or trading algorithm which requires some amount of historical data to make decisions on the subsequent live data.

Event Sourcing & On-Demand Analytics

In this use-case, an enterprise may wish to capture the entire history of message flow to an application so that it can be reviewed or analysed at a later date by replaying the data that was sent to the application.

Message Replay / SolCache Comparison

Message Replay isn't a replacement for SolCache. There are a number of differences between the two, and they're listed in the following table. Whether you choose Message Replay or SolCache depends on your use-cases.

Attribute Message Replay SolCache
Use Case

Message Log

Records all Guaranteed messages published to the Message-VPN.

Automatically trims oldest messages from log when log’s quota is reached.

Last Value Cache

Typically keeps only the last message published on each topic.

While SolCache can be configured to keep more than one message per topic, that is a less common use-case.

Message QoS

Logs Guaranteed (aka Persistent) messages.

Logs any messages that get promoted to Guaranteed messages.

Intended for caching Direct (aka Non-Persistent) messages.

Connects to the message broker using Direct messaging. If any Guaranteed messages match SolCache subscriptions, those messages are demoted to Direct when sent to SolCache.

Interactions With Client API All interactions with client APIs use Guaranteed messaging. All interactions with client APIs use Direct messaging.
Topics Recorded Logs all Guaranteed messages published in the Message-VPN. Administrator configures the wildcard topics that are to be stored in the cache.
Message Delivery Order

Replays messages in original published order.

Publish order between topics is preserved.

Cache requests are fulfilled topic-by-topic.

Order is preserved within a topic (if depth is greater than 1), but original publish order between topics is not preserved.

Replay Request Mechanism

Solace APIs

Management Interface

Solace APIs
Deployment Model Integrated feature of the PubSub+ message broker. Software application that runs on an external Linux server.
Message Storage Uses the Guaranteed messaging infrastructure to persistently store messages. Stores cached messages in RAM.
Redundancy Model Leverages the message broker’s active-standby HA model. Deploy multiple instances for n+1 redundancy.
Performance and Scaling

Built for logging messages at Guaranteed messaging rates.

Add more PubSub+ message brokers to horizontally scale Guaranteed messaging and Message Replay.

Built for very high rate Direct messaging use-cases like market data distribution.

Supports deploying multiple instances to horizontally scale.

Availability Integrated into the PubSub+ message broker; no extra license. Optional add-on product that requires purchased license.