High Availability in Solace Cloud

Event broker services can be deployed in high availability (HA) redundancy groups. HA redundancy uses 1:1 event broker service paring to provide fault tolerance and increase overall service availability. If one of the event broker services fails or is taken out of service, the other event broker service automatically takes over and provides service to the clients that were previously being served. A brief interruption of less than one minute occurs during the HA activity failover. In comparison, outages of 15-30 minutes occur for Enterprise-100 Standalone and standalone event broker services because they do not have an HA redundancy group.

To learn more about HA redundancy, see High Availability for Software Event Brokers.

You can test the HA redundancy of your HA event broker service, and how your client applications respond to HA failovers by performing a High Availability Switchover for Event Broker Services.

HA Concepts

Solace Cloud implements HA using an active/standby model.

With this model, one messaging node (usually the primary) is active and provides messaging services to clients, while the other (usually the backup) waits in standby mode—it only provides service should the active node fail. A third event broker acts as a monitoring node, to act as a tie-breaker and prevent split-brain scenarios that would otherwise cause both messaging nodes to become active simultaneously.

This setup requires three event broker instances:

• Primary messaging node
• Backup messaging node
• Monitoring node

These three event brokers constitute one HA event broker service, as shown in the following diagram:

The HA group is fronted by a network load balancer that routes traffic to and from the active event broker (either the primary or backup).

If the active event broker goes offline, a failure is detected within the HA redundancy group.

Subsequently, a failover occurs in the following sequence (assuming that the primary event broker was the active node):

1. The backup event broker takes over messaging activity (that is, it becomes the active node).
2. After the failed primary event broker comes back online, it resynchronizes to match the currently active backup event broker.
3. The primary event broker takes on the standby role.

HA in Public and Private Clouds

To ensure that an HA group is adequately provisioned, pods run on different worker nodes in Kubernetes. Additionally, the pods can be spread over multiple availability zones (AZ) when available. The following diagram shows a Kubernetes cluster that has worker nodes over three AZs. The Mission Control Agent schedules the messaging nodes over two AZs and the monitoring node on a third AZ. For each HA service, the primary pod is deployed in one AZ, the backup pod in a second AZ, and the monitoring pod in a third AZ. This guarantees that pods for the same HA service are not running on the same hardware.

Similarly, when deploying an HA group in virtual private clouds such as AWS, there are two network topologies available.

1. For regions with three or more AZ:

   

2. For regions with two AZ:

   

Connecting to a Cloud HA Group

Typically, event broker services are fronted by load balancers in deployments of Solace Cloud. When a load balancer is used, it abstracts the switchover between the primary and backup nodes in the event of a failure in an HA configuration. For this reason, client applications can connect to an event broker service using a single-DNS entry, whether they use Solace Messaging APIs or third-party messaging APIs, such as MQTT.

If a load balancer is not used, a host list is required. Solace Messaging APIs have built-in support for host lists. Third-party APIs do not provide support for host lists, though you can choose to implement this functionality yourself.

HA and Service Types

The following service types deploy an HA redundancy group by default:

• Professional (Standard account)
• Enterprise (Enterprise account)

Solace Cloud automates all of the configuration and setup when you create your event broker service. Once the event broker service is created, applications can use the DNS name entry provided in the connectivity tab in the console.

HA-Link Security

When a new enterprise event broker service is created, the communication between the primary and backup messaging nodes is encrypted by default, including the HA mate link and Config-sync. When you create an event broker service, you can disable encryption so that the communication is in plain text between the primary and backup event brokers. For more information, see Configuring High Availability Mate-Link Encryption. Solace recommends that you do not disable the encryption unless you trust the security restrictions of the VPC in the cloud providers or on-premises. Config-sync always remains encrypted.

The status of the mate-link encryption for an event broker service is shown in Cluster Manager on the Status tab.

Modifying the HA Mate-Link Encryption Status

If you have an existing event broker service without mate-link encryption enabled, you can enable enable it. To modify the HA mate-link encryption status for an existing event broker service perform these steps:

1. Log in to the Solace Cloud Console if you have not done so yet. The URL to access the Cloud Console differs based on your authentication scheme. For more information, see Logging In to the Solace Cloud Console.
2. On the navigation bar, select Cluster Manager .
3. Select the event broker service that you want to configure. If the event broker service is not listed, make sure you have the right environment selected. For more information, see Selecting Environments.
4. On the Service Details page, click the Manage tab.
5. On the Management Settings menu, select Authentication and Security.
6. On the Mate-Link Encryption tile, select Enable or Disable.