Dynamic Message Routing (DMR)
As your enterprise grows, expands and becomes more global, your applications will also grow and evolve in complexity. Often those applications wind-up becoming distributed across multiple sites - which can include multiple public and private clouds as well as your legacy data centers - driving the need for an easy way to intelligently share information between multiple locations and interfaces. For example, your organization may have on-premises assets, such as IBM MQ, TIBCO or Kafka that need to connect to new application frameworks such as Spring, or to new cloud services such as Google Bigtable, BigQuery, AWS EMR or Kinesis. Or you may want to make your core data assets from DBs on mainframes or on-premise distributed systems accessible to applications and services in the cloud.
Providing this sort of large-scale, uninterrupted operation can be accomplished using a Dynamic Message Routing (DMR) network to automatically connect-up all those sites and exchange subscription information. Such a network allows the far-flung clients to be unconcerned that there could be multiple and disparate sites in the network. To the clients, the network and its details are invisible so they can go about their business of exchanging messages unencumbered by the intricacies of how to get them where they need to go.
Without DMR, you must know what publishers are producing the data you want, configure messaging connectivity to wherever that data is being published, and manually configure routing rules to move the data to where consuming applications want it. This works, but DMR makes it much simpler by automating all the labor by providing intelligent, self-learning routing capabilities.
Enterprise growth can also occur within the sites themselves. Individual message brokers have limits with respect to things like subscription counts, queue counts, message spool storage and so on. At some point even the largest, most generously provisioned message broker has its limits, but these can be overcome by networking several message brokers together with DMR and spreading the load amongst the participants. In this way, normal message brokers can be connected together to service loads far in excess of that possible by a single one.
DMR is based on established and robust Internet routing technology, and builds on the long-released Solace PubSub+ Multi-Node Routing (MNR) feature. However, where MNR only supported non-persistent messaging, DMR supports both non-persistent and persistent messaging. And DMR is easy to set up and get going as it can be configured using a Click-to-Connect wizard in Solace PubSub+ Manager. Although, if configuring via Solace CLI is what you need, you can set up DMR that way too.
- If you'd like to give DMR a try, consider jumping over to some configuration examples in DMR Examples.
- If you'd like to learn a little more about the use-cases DMR resolves, they're discussed below in the sections Multi-Site Connectivity and Horizontal Scaling.
- For more technical information on DMR, you'll find it starts over at DMR Technology.
Consider the fairly common situation where you have a number of on-premise applications all connected to a Solace PubSub+ backbone. In the above example, there are three such applications, red, blue and green, each within their own Message VPN. For reasons of cost, flexibility, security, robustness, or some other business driver, you may find that you need to migrate a number of clients of those applications to cloud providers. Solace PubSub+ software message brokers can also be deployed in those clouds to provide messaging services to both the migrated clients, and, through the use of DMR, to clients migrated to other clouds as well as those still residing on-premise.
In the example, the red and green applications have had some of their clients moved to cloud providers, but some clients have been left on-premise. The blue application has been completely migrated to the cloud. Configuration-wise, all that needs to be done to get messages flowing amongst an application's clients as if they were all still residing on-premise is to ensure DMR is enabled on each Message VPN, and configure a DMR control link between the neighboring message brokers. DMR automatically takes care of discovering the message brokers and advertising the subscriptions their clients need. Also, both direct and guaranteed messaging are supported so that the same messaging modes that were used on-premise can still be used now that the applications are running across several multiple sites.
Other sites can be added to your multi-site network as your enterprise continues to grow. For example, other public and private clouds can be added, and subscription distribution and data flow will continue to happen automatically.
You'll find instructions on how to set up and configure a Hybrid-Cloud / Multi-Site example over at Multi-Site Connectivity Configuration.
Consider a situation where you use a Solace PubSub+ software message broker as your MQTT server, with applications red, blue, and green, each within their own Message VPN.
Say, for example, you plan on increasing your deployed client count from 100,000 to 400,000 MQTT clients, all of which need to intercommunicate with QoS 1 subscriptions. A single software message broker - the largest of which supports 200,000 queues - is not sufficient at this scale. While this might be possible with an appliance, you might not be interested in maintaining hardware, and instead opt for three 200,000 software message brokers configured in a DMR network, as shown in the following diagram.
Here the applications have been spread across all three software message brokers, which are interconnected with internal DMR links, with a combined capacity of 600,000 queues. This configuration can easily handle the needed number of clients, and scale even higher by adding additional software message brokers to the network over time. No manual subscription management is needed, as the needs of each message broker is dynamically discovered.
If you're ready to set up and configure a Horizontal Scaling example, you'll find instructions over at Horizontal Scaling Configuration.
DMR works for both persistent and non-persistent messages. Also, DMR eliminates the need to predefine what data needs to flow across which Message VPN bridge with predefined topic namespaces. With DMR, all of this becomes automatic, with the same persistent QoS you enjoyed.
In addition, the Solace PubSub+ Manager includes a click-to-connect feature that can easily create a DMR mesh in a few mouse clicks.
Support of persistent messages, automatic data connectivity management, and ease of configuration make DMR the recommended choice.