HA Group Configuration for macOS

This section shows you the steps to manually set up and run an HA redundancy group of PubSub+ software event broker containers in Docker Desktop (or an equivalent desktop container environment such as Rancher Desktop) on a macOS platform.

Alternatively, if you'd like to set up an HA group with Docker Compose (or equivalent), take a look at the template and instructions at GitHub: https://github.com/SolaceLabs/solace-ha-docker-compose.

Before You Begin

In the example shown, the HA configuration, which makes use of Solace PubSub+ Standard edition, is suitable for use with up to 100 client connections on the messaging nodes. However, a maximum of 1,000 client connections can be configured on your platform provided appropriate resources have been provisioned. For information on resource provisioning, refer to System Resource Requirements.

The configuration shown in this example is suitable for demonstrating and testing PubSub+ fundamentals, such as HA failover and guaranteed messaging, in non-production situations. It's but one way of setting up an HA group in a resource limited environment such as a laptop. The intent of the example is to help you become familiar with the ins-and-outs of HA configuration as a step towards using more advanced, production-oriented techniques.

Assumptions

It's assumed you have:

  1. Mac OS X Yosemite 10.10.3 or higher.
  2. Docker installed, with at least 6 GiB of virtual memory (4 GiB must be RAM) and at least 2 virtual cores dedicated to the Docker for Mac. Allocating more cores, if they can be provided, will improve the overall performance. For this example, 4 GiB of RAM, 2 GiB of swap space, and 4 virtual cores have been dedicated to the Docker for Mac. To learn about allocating memory and swap space, refer to the Docker Settings page.
  3. A host machine with 8 GB RAM and 4 CPU cores with hyperthreading enabled (8 virtual cores) is recommended.
  4. All software event broker Docker container images in the HA group must be the same: Solace PubSub+ 8.10 or higher.
  5. All messaging nodes within the HA group configured to use the same connection scaling tier.

Limitations

  1. The following features are not supported: Replication; Docker Engine Swarm mode.
  2. Multi-Node Routing (MNR) is not supported at 100 connection scale. To use MNR, you must use the 1000 connection scaling tier or higher.

Step 1: Get a Software Event Broker

First, you need to obtain an event broker Docker package, which is a compressed tar archive containing an event broker Docker repository consisting of a single event broker Docker image.

This example assumes that you are using Solace PubSub+ Standard. If you want to use another edition, you need to obtain the appropriate event broker package:

  1. Go to the Solace PubSub+ Event Broker downloads page.
  2. Scroll to the Docker section, and then select the edition of the software event broker.
  3. Fill in your details, and then click Get The Downloads.

A compressed archive file called solace-pubsub-<edition>-<version>-docker.tar.gz will be downloaded.

Once you have obtained a copy of the event broker package, you can upload it to a directory on your host and load the image using these two steps:

  1. Start Docker and open a command-line terminal.
  2. Load the image: 
    > docker load -i /Users/username/Downloads/solace-pubsub-standard-9.13.x.x-docker.tar

    In this example, the compressed tar archive of Solace PubSub+ Standard has been uploaded to Users/username/Downloads.

    When loading is finished, you can check the image with the images command.

    > docker images
REPOSITORY                TAG         IMAGE ID        CREATED        SIZE
solace-pubsub-standard    9.13.x.x    b8a61124d92f    10 days ago    1.644 GB

Step 2: Create a Docker Bridge Network

Once you have loaded the image, create a new Docker bridge network using the docker network create command. The new bridge will allow using the hostname to connect the software event brokers in the HA group.

In the example below, we have created a new Docker bridge network called solace-net.

> docker network create solace-net

Step 3: Configure the HA Group

In this step, you'll create the Docker containers to be used for the HA group's primary and backup event brokers and monitoring node using the docker create command. Make note of the following before you get started:

  • The --network solace-net option is used to connect all three containers to the solace-net Docker bridge network.
  • The HA group is set up using configuration keys. In the example below, the configuration key redundancy/enable is used in the line --env 'redundancy_enable=yes' to enable redundancy. The containers are connected to each other using their hostnames through the use of the redundancy/group/node/<name>/connectvia configuration key in the line --env 'redundancy_group_node_primary_connectvia=<hostname>'. In this example, the values of the <hostname> variable are primary, backup, and monitoring. For a complete list of configuration keys associated with HA, refer to Configuration Keys.
  • In the docker create command, the redundancy group's <pre-shared-key> must be at least 44 characters long, Base64 encoded, and the same for each node in the HA group. Refer to Pre-Shared Keys for Software Event Brokers for more details.
  • If you loaded the image from a compressed tar archive, replace solace/solace-pubsub-standard in the example with the repository and tag that corresponds with your image. For example, if you loaded version 9.13.0.16 of the Solace PubSub+ Enterprise edition, use solace-pubsub-enterprise:9.13.0.16.

To create the Docker containers and configure the HA group, perform the following steps:

  1. Run the following commands to configure the primary event broker:

    > docker create -p 212:2222 -p 213:8080 -p 214:55555 -p 215:8300 -p 216:8301 -p 217:8302 -p 218:8741 \
    --shm-size=1g \
    --ulimit nofile=2448:38048 \
    --env 'username_admin_globalaccesslevel=admin' \
    --env 'username_admin_password=admin' \
    --env 'system_scaling_maxconnectioncount=100' \
    --name=primary \
    --hostname=primary \
    --env 'routername=primary' \
    --network solace-net \
    --env 'nodetype=message_routing' \
    --env 'configsync_enable=yes' \
    --env 'redundancy_enable=yes' \
    --env 'redundancy_group_node_primary_connectvia=primary' \
    --env 'redundancy_group_node_primary_nodetype=message_routing' \
    --env 'redundancy_group_node_backup_connectvia=backup' \
    --env 'redundancy_group_node_backup_nodetype=message_routing' \
    --env 'redundancy_group_node_monitoring_connectvia=monitoring' \
    --env 'redundancy_group_node_monitoring_nodetype=monitoring' \
    --env 'redundancy_authentication_presharedkey_key=<pre-shared-key>' \
    --env 'redundancy_activestandbyrole=primary' \
    --env 'redundancy_matelink_connectvia=backup' \
    solace-pubsub-standard:9.13.x.x

    If you are using other terminals, such as Git Bash, you can copy and paste the example below.

    > docker create -p 212:2222 -p 213:8080 -p 214:55555 -p 215:8300 -p 216:8301 -p 217:8302 -p 218:8741 --shm-size=1g --ulimit nofile=2448:38048 --env username_admin_globalaccesslevel=admin --env username_admin_password=admin --env system_scaling_maxconnectioncount=100 --name=primary --hostname=primary --env routername=primary --network solace-net --env nodetype=message_routing --env configsync_enable=yes --env redundancy_enable=yes --env redundancy_group_node_primary_connectvia=primary --env redundancy_group_node_primary_nodetype=message_routing --env redundancy_group_node_backup_connectvia=backup --env redundancy_group_node_backup_nodetype=message_routing --env redundancy_group_node_monitoring_connectvia=monitoring --env redundancy_group_node_monitoring_nodetype=monitoring --env redundancy_authentication_presharedkey_key=<pre-shared-key> --env redundancy_activestandbyrole=primary --env redundancy_matelink_connectvia=backup solace-pubsub-standard:9.13.x.x

  2. Run the following commands to configure the backup event broker:

    > docker create -p 312:2222 -p 313:8080 -p 314:55555 -p 315:8300 -p 316:8301 -p 317:8302 -p 318:8741 \
    --shm-size=1g \
    --ulimit nofile=2448:38048 \
    --env 'username_admin_globalaccesslevel=admin' \
    --env 'username_admin_password=admin' \
    --env 'system_scaling_maxconnectioncount=100' \
    --name=backup \
    --hostname=backup \
    --env 'routername=backup' \
    --network solace-net \
    --env 'nodetype=message_routing' \
    --env 'configsync_enable=yes' \
    --env 'redundancy_enable=yes' \
    --env 'redundancy_group_node_primary_connectvia=primary' \
    --env 'redundancy_group_node_primary_nodetype=message_routing' \
    --env 'redundancy_group_node_backup_connectvia=backup' \
    --env 'redundancy_group_node_backup_nodetype=message_routing' \
    --env 'redundancy_group_node_monitoring_connectvia=monitoring' \
    --env 'redundancy_group_node_monitoring_nodetype=monitoring' \
    --env 'redundancy_authentication_presharedkey_key=<pre-shared-key>' \
    --env 'redundancy_activestandbyrole=backup' \
    --env 'redundancy_matelink_connectvia=primary' \
    solace-pubsub-standard:9.13.x.x

    If you are using other terminals, such as Git Bash, you can copy and paste the example below.

    > docker create -p 312:2222 -p 313:8080 -p 314:55555 -p 315:8300 -p 316:8301 -p 317:8302 -p 318:8741 --shm-size=1g --ulimit nofile=2448:38048 --env username_admin_globalaccesslevel=admin --env username_admin_password=admin --env system_scaling_maxconnectioncount=100 --name=backup --hostname=backup --env routername=backup --network solace-net --env nodetype=message_routing --env configsync_enable=yes --env redundancy_enable=yes --env redundancy_group_node_primary_connectvia=primary --env redundancy_group_node_primary_nodetype=message_routing --env redundancy_group_node_backup_connectvia=backup --env redundancy_group_node_backup_nodetype=message_routing --env redundancy_group_node_monitoring_connectvia=monitoring --env redundancy_group_node_monitoring_nodetype=monitoring --env redundancy_authentication_presharedkey_key=<pre-shared-key> --env redundancy_activestandbyrole=backup --env redundancy_matelink_connectvia=primary solace-pubsub-standard:9.13.x.x

  3. Run the following commands to configure the monitoring node:

    > docker create -p 412:2222 -p 413:8080 -p 414:55555 -p 415:8300 -p 416:8301 -p 417:8302 -p 418:8741 \
    --shm-size=1g \
    --ulimit nofile=2448:38048 \
    --env 'username_admin_globalaccesslevel=admin' \
    --env 'username_admin_password=admin' \
    --env 'system_scaling_maxconnectioncount=100' \
    --name=monitoring \
    --hostname=monitoring \
    --env 'routername=monitoring' \
    --network solace-net \
    --env 'nodetype=monitoring' \
    --env 'redundancy_enable=yes' \
    --env 'redundancy_group_node_primary_connectvia=primary' \
    --env 'redundancy_group_node_primary_nodetype=message_routing' \
    --env 'redundancy_group_node_backup_connectvia=backup' \
    --env 'redundancy_group_node_backup_nodetype=message_routing' \
    --env 'redundancy_group_node_monitoring_connectvia=monitoring' \
    --env 'redundancy_group_node_monitoring_nodetype=monitoring' \
    --env 'redundancy_authentication_presharedkey_key=<pre-shared-key>' \
    solace-pubsub-standard:9.13.x.x

    If you are using other terminals, such as Git Bash, you can copy and paste the example below.

    > docker create -p 412:2222 -p 413:8080 -p 414:55555 -p 415:8300 -p 416:8301 -p 417:8302 -p 418:8741 --shm-size=1g --ulimit nofile=2448:38048 --env username_admin_globalaccesslevel=admin --env username_admin_password=admin --env system_scaling_maxconnectioncount=100 --name=monitoring --hostname=monitoring --env routername=monitoring --network solace-net --env nodetype=monitoring --env redundancy_enable=yes --env redundancy_group_node_primary_connectvia=primary --env redundancy_group_node_primary_nodetype=message_routing --env redundancy_group_node_backup_connectvia=backup --env redundancy_group_node_backup_nodetype=message_routing --env redundancy_group_node_monitoring_connectvia=monitoring --env redundancy_group_node_monitoring_nodetype=monitoring --env redundancy_authentication_presharedkey_key=<pre-shared-key> solace-pubsub-standard:9.13.x.x

  4. After the software event broker Docker containers are configured, execute the docker start command to start the containers.

    > docker start primary backup monitoring

Step 4: Assert the Primary's Configuration

In the previous step, we enabled Config-Sync on the primary and backup event brokers using the configuration key configsync/enable in the line --env configsync_enable=yes. When enabling the Config-Sync for the first time on an HA pair, you must assert the system-level configuration of the one event broker over its mate. This “leader” event broker is typically the primary event broker in the pair.

If there is a need to manually synchronize a Message VPN configuration between two event brokers, you can assert the Message VPN configuration of one of the event brokers (it doesn’t have to be on the leader or primary event broker) over the configuration of the same Message VPN on its mate.

To assert the system-level and Message VPN-level configurations on the primary event broker, perform the following steps:

  1. Access the Solace CLI on the primary event broker:

    > docker exec -it <container-name> /usr/sw/loads/currentload/bin/cli -A

    Where,

    <container-name> is the name of the primary event broker container, which in this example is primary.

  2. Use the assert-leader router and assert-leader message-vpn commands:

    primary> enable
    primary# admin
    primary(admin)# config-sync
    primary(admin/config-sync)# assert-leader router
    WARNING: This command can delete SEMP sessions on the HA mate of this router, logging out SEMP and PubSub+ Manager users.
    Do you want to continue (y/n)? y
    Processed 1 config-sync tables.
    primary(admin/config-sync)# assert-leader message-vpn <vpn-name>
    WARNING: This command can temporarily disconnect clients on either the AD-inactive or replication-standby routers. As well in-flight messages may not be delivered to AD endpoints on the replication-standby routers if those endpoints are not currently configured the same as they are on this router. Clients connected to this router will not be affected.
    Do you want to continue (y/n)? y
    Processed 1 config-sync tables.

    Where:

    <vpn-name> is the Message VPN name, which may contain wildcards.

    The Oper Status of Config-Sync changes from Down to Up on both primary and backup event brokers. To check this, run the show config-sync User EXEC command.

  3. Verify the redundancy status of the HA group by executing the show redundancy command.

    primary(configure/redundancy)# show redundancy
    Configuration Status       : Enabled
    Redundancy Status          : Up
    Operating Mode             : Message Routing Node
    Switchover Mechanism       : Hostlist
    Auto Revert                : No
    Reundancy                  : Active/Standby
    Active-Standby Role        : Primary
    Mate Router Name           : backup
    ADB Link To Mate           : Up
    ADB Hello To Mate          : Up

                                     Primary Virtual Router   Backup Virtual Router
                                     ----------------------   ----------------------
    Activity Status                  Local Active             Shutdown
    Routing Interface                intf0: 1                 intfo:1
    Routing Interface Status         Up
    VRRP Status                      Initialize
    VRRP Priority                    250
    Message Spool Status             AD-Active
    Priority Reported By Mate        Standby

Next Steps

At this point you have an HA redundancy group running on your platform and Guaranteed Messaging is enabled. You can now do things like use the SDKPerf tool to test messaging, or test the HA group’s failover operation.

Once you have tried out some basic HA related functionality, you may be interested in some more advanced HA topics.