In-Service Migration of a Redundant Pair of Appliance Event Brokers

This procedure describes how to replace older Solace Appliance Event Broker chassis versions deployed in a high-availability (HA) redundant pair with newer CHS-3560AC-08-A chassis. The replacement is performed sequentially, replacing one appliance event broker at a time while the other remains in service, resulting in a non-service-affecting upgrade. Use this procedure when upgrading hardware for performance improvements, replacing end-of-life chassis models, or migrating to currently supported chassis versions.

To ensure continuous service, replace the backup appliance event broker first, transfer activity to it, then replace the original primary.

This procedure is for replacing functional appliance event brokers with newer hardware. For replacing a failed appliance event broker in a redundant pair, see In-Service Replacement of a Failed Appliance Event Broker in a Redundant Pair.

Supported Hardware Migrations

You can use this procedure to replace the following chassis versions with a CHS-3560AC-08-A:

  • CHS-3530AC-02-A

  • CHS-3530AC-03-A

  • CHS-3560-AC-03-A

  • CHS-3560-AC-05-A

Software Version Requirements

Before beginning this procedure, your existing appliance event brokers must be running one of the following software versions or later:

  • 10.8.1.419

  • 10.25.0.208

  • 10.25.19.5757

Compatibility Requirements

This procedure applies only to compatible hardware replacements where the following requirements are met:

Network Acceleration Board (NAB):

The following NAB replacements are supported as part of this procedure:

  • NAB-0410EM-01-A to NAB-0810EM-01-A

  • NAB-0610EM-01-A to NAB-0810EM-01-A

  • NAB-0210EM-04-A to NAB-0810EM-01-A

  • NAB-0401ET-04-A to NAB-0410EM-01-A, or NAB-0810EM-01-A

  • NAB-0801ET-04-A to NAB-0810EM-01-A

Assured Delivery Blade (ADB): Both appliance event brokers in the redundant pair must have the same ADB configuration.

The following ADB replacements are supported as part of this procedure:

  • Both the old appliance event brokers and the replacements have no ADB.

  • Both the old appliance event brokers and the replacements have the same ADB.

  • The old appliance event brokers have an ADB-04210M-01-A and the replacements have an ADB-000004-01-A.

For chassis models or NAB/ADB configurations not listed above, contact Solace Technical Support because there could be scaling differences and a custom procedure may be required.

Before You Begin

You can complete the following tasks before starting this procedure to reduce overall migration time. They can be performed for one or both replacement appliances in advance:

  • Rack the chassis: Install the replacement appliance event broker chassis in the hardware rack. See Hardware Installation.

  • Register HBA ports with the storage array: Register the replacement appliance event broker's HBA ports with the proper storage Logical Unit Number (LUN) on the external disk storage array. The WWNs required for registration are listed on the Packing List provided upon delivery; Solace can also provide them upon request. For more information, see Configuring an External Disk Array for Guaranteed Messaging.

ESD/Safety Precautions

To prevent equipment damage from Electrostatic Discharge (ESD), always ensure that the appliance event broker chassis is electrically connected to earth ground. Always follow ESD prevention procedures when removing and replacing power supplies. Use an anti-static wrist strap, or another anti-static device. If no wrist strap or mat is available, ground yourself by touching the metal part of the chassis.

Tool and Equipment Requirements

The following tools and equipment are required:

  • electrostatic discharge wrist strap
  • anti-static mat
  • replacement appliance event broker

Procedure

This procedure consists of four phases: replace the backup appliance event broker, transfer activity to the new backup, replace the primary appliance event broker, and restore normal operation. Complete each phase in sequence.

Phase 1: Replace the Backup Appliance Event Broker

Complete the following steps to replace appliance event brokers in a deployed HA redundant pair. In this example, solace-primary is the active appliance event broker and solace-backup is the appliance event broker being replaced with newer hardware.

If you require further assistance, or if you have any questions regarding this procedure, contact Solace for technical support.

When replacing an appliance event broker in a deployed HA redundant pair, do NOT physically connect the HBA links between the replacement appliance event broker and the external disk storage unit, do NOT physically connect the NAB in the replacement appliance event broker to the network, and do NOT physically connect the ADB links between the replacement appliance event broker and paired in-service appliance event broker until explicitly instructed to do so in the steps in the following procedure.

  1. Enter the following Solace Event Broker CLI show command to ensure that solace-primary is the currently active appliance event broker and it is in the correct redundancy state.
    solace-primary> enable
solace-primary# show redundancy
  2. Enter the following commands to ensure that auto-revert is not enabled. 
    solace-primary> enable
solace-primary# configure
solace-primary(configure)# redundancy
solace-primary(configure/redundancy)# no auto-revert
solace-primary(configure/redundancy)# end
solace-primary#
  3. Back up the current configuration database of the old appliance event broker that you are replacing.
    solace-backup> enable
solace-backup# copy current-config sftp://<username>@<hostname>/<path-to-remote-dir>/config_<date>.tar.gz

    For information on how to back up a current configuration to a remote destination, see Backing Up and Restoring Event Broker Configurations. Pay particular attention to the need to capture product-keys and certificate files, which are not automatically included in the backup.

  4. Power-down the old appliance event broker.
    solace-backup# power-down
This command powers off the appliance event broker and does not restart it afterwards
Do you want to continue (y/n)? y
  5. If you haven't already done so, install the replacement appliance event broker in the hardware rack. Power it up and connect a management console.

    For more information, see Hardware Installation, On / Standby Button, and Connecting a Management Console.

  6. Using the management console, run set-up and configure only the management IP address and default route for the replacement appliance event broker.

    For more information, see Setting Up a Basic Configuration and Configuring the Management Interface and Message Backbone Interface on Appliance event brokers.

  7. If you haven't already registered the replacement appliance event broker's HBA with the external disk storage array, do the following:

    1. Enter the following show command to display the HBA Port and Node names for the replacement appliance event broker:

      solace-backup> show hardware detail

    2. Using the displayed HBA Port and Node names from the output, register the HBA ports with the proper storage Logical Unit Number (LUN) on the external disk storage array.

  8. Enter the following show command to ensure that the correct version of Solace software is installed on the replacement appliance event broker. 
    solace-backup> show version

    If necessary, upgrade or downgrade the version to match that of the version used by the mate appliance event broker. For information on upgrading software on appliance event brokers, see the Solace Appliance Event Broker Release Notes. For information on downgrading appliance event brokers, see Changing Appliance Event Broker Software.

  9. To apply a saved configuration file to the replacement appliance event broker, do the following:

    1. Perform the procedure outlined in Restoring Configuration Files.

    2. Enter the following commands to ensure that auto-revert is not enabled for the replacement appliance event broker:

      solace-backup> enable
solace-backup# configure
solace-backup(configure)# redundancy
solace-backup(configure/redundancy)# no auto-revert
solace-backup(configure/redundancy)# end
solace-backup#

    3. Turn off power to the replacement appliance event broker:

      solace-backup# power-down

      The appliance event brokers do not restart automatically after this command is run.

    4. Physically connect the HBA links between the replacement appliance event broker and the external disk storage array.

    5. Connect the NAB in the replacement appliance event broker to the network.

    6. Physically connect the ADB mate links between the replacement appliance event broker and the paired in-service appliance event broker.

    7. Power up the replacement appliance event broker:

      See On / Standby Button.

    8. Assert the configuration settings of the active appliance event broker onto the replacement appliance event broker. See Asserting Event Broker System Configurations and Asserting Event Brokers VPN Configurations.

    9. Verify that the replacement appliance event broker's configuration fully matches the active appliance event broker's configuration.

      You can use the show current-config User EXEC command to obtain the sequence of CLI commands that are required to duplicate the system-level and Message VPN-level configurations used by the active appliance event broker. For more information, see Show the CLI Sequence for the Current Configuration.

  10. If necessary, restore the auto-revert redundancy setting.

    Solace recommends keeping the auto-revert redundancy configuration disabled. However, if you want to enable auto-revert, re-enable it at this point.

  11. Enter the following commands to verify that the replacement event broker is in the correct redundancy state. 
    solace-backup> enable
solace-backup# show redundancy

    If it is not in the correct redundancy state, run revert-activity.

Phase 2: Transfer Activity to the New Backup Appliance Event Broker

With the new backup appliance event broker operational, transfer activity to it so that the original primary can be safely replaced.

  1. Verify that the new backup appliance event broker is in the correct redundancy state:

    solace-backup> enable
solace-backup# show redundancy

  2. If you are using Config-Sync, verify that the new backup appliance event broker is fully synchronized:

    solace-backup# show config-sync

  3. On the primary appliance event broker, release activity to the backup:

    solace-primary# configure
solace-primary(configure)# redundancy
solace-primary(configure/redundancy)# release-activity
solace-primary(configure/redundancy)# end

  4. Verify that activity has transferred to the backup:

    solace-backup# show redundancy

    Confirm that the Message Spool Status shows AD-Active.

Phase 3: Replace the Primary Appliance Event Broker

With the new backup now serving traffic, replace the original primary appliance event broker. In this phase, solace-backup (the newly replaced appliance) is now active, and you will replace solace-primary.

Complete the following steps to replace appliance event brokers in a deployed HA redundant pair. In this example, solace-backup is the active appliance event broker and solace-primary is the appliance event broker being replaced with newer hardware.

If you require further assistance, or if you have any questions regarding this procedure, contact Solace for technical support.

When replacing an appliance event broker in a deployed HA redundant pair, do NOT physically connect the HBA links between the replacement appliance event broker and the external disk storage unit, do NOT physically connect the NAB in the replacement appliance event broker to the network, and do NOT physically connect the ADB links between the replacement appliance event broker and paired in-service appliance event broker until explicitly instructed to do so in the steps in the following procedure.

  1. Enter the following show command to ensure that solace-backup is the currently active appliance event broker and it is in the correct redundancy state.
    solace-backup> enable
solace-backup# show redundancy
  2. Enter the following commands to ensure that auto-revert is not enabled. 
    solace-backup> enable
solace-backup# configure
solace-backup(configure)# redundancy
solace-backup(configure/redundancy)# no auto-revert
solace-backup(configure/redundancy)# end
solace-backup#
  3. Back up the current configuration database of the old appliance event broker that you are replacing.
    solace-primary> enable
solace-primary# copy current-config sftp://<username>@<hostname>/<path-to-remote-dir>/config_<date>.tar.gz

    For information on how to back up a current configuration to a remote destination, see Backing Up and Restoring Event Broker Configurations. Pay particular attention to the need to capture product-keys and certificate files, which are not automatically included in the backup.

  4. Power-down the old appliance event broker.
    solace-primary# power-down
This command powers off the appliance event broker and does not restart it afterwards
Do you want to continue (y/n)? y
  5. If you haven't already done so, install the replacement appliance event broker in the hardware rack. Power it up and connect a management console.

    For more information, see Hardware Installation, On / Standby Button, and Connecting a Management Console.

  6. Using the management console, run set-up and configure only the management IP address and default route for the replacement appliance event broker.

    For more information, see Setting Up a Basic Configuration and Configuring the Management Interface and Message Backbone Interface on Appliance event brokers.

  7. If you haven't already registered the replacement appliance event broker's HBA with the external disk storage array, do the following:

    1. Enter the following show command to display the HBA Port and Node names for the replacement appliance event broker:

      solace-primary> show hardware detail

    2. Using the displayed HBA Port and Node names from the output, register the HBA ports with the proper storage Logical Unit Number (LUN) on the external disk storage array.

  8. Enter the following show command to ensure that the correct version of Solace software is installed on the replacement appliance event broker. 
    solace-primary> show version

    If necessary, upgrade or downgrade the version to match that of the version used by the mate appliance event broker. For information on upgrading software on appliance event brokers, see the Solace Appliance Event Broker Release Notes. For information on downgrading appliance event brokers, see Changing Appliance Event Broker Software.

  9. To apply a saved configuration file to the replacement appliance event broker, do the following:

    1. Perform the procedure outlined in Restoring Configuration Files.

    2. Enter the following commands to ensure that auto-revert is not enabled for the replacement appliance event broker:

      solace-primary> enable
solace-primary# configure
solace-primary(configure)# redundancy
solace-primary(configure/redundancy)# no auto-revert
solace-primary(configure/redundancy)# end
solace-primary#

    3. Turn off power to the replacement appliance event broker:

      solace-primary# power-down

      The appliance event brokers do not restart automatically after this command is run.

    4. Physically connect the HBA links between the replacement appliance event broker and the external disk storage array.

    5. Connect the NAB in the replacement appliance event broker to the network.

    6. Physically connect the ADB mate links between the replacement appliance event broker and the paired in-service appliance event broker.

    7. Power up the replacement appliance event broker:

      See On / Standby Button.

    8. Assert the configuration settings of the active appliance event broker onto the replacement appliance event broker. See Asserting Event Broker System Configurations and Asserting Event Brokers VPN Configurations.

    9. Verify that the replacement appliance event broker's configuration fully matches the active appliance event broker's configuration.

      You can use the show current-config User EXEC command to obtain the sequence of CLI commands that are required to duplicate the system-level and Message VPN-level configurations used by the active appliance event broker. For more information, see Show the CLI Sequence for the Current Configuration.

  10. If necessary, restore the auto-revert redundancy setting.

    Solace recommends keeping the auto-revert redundancy configuration disabled. However, if you want to enable auto-revert, re-enable it at this point.

  11. Enter the following commands to verify that the replacement event broker is in the correct redundancy state. 
    solace-primary> enable
solace-primary# show redundancy

    If it is not in the correct redundancy state, run revert-activity.

Phase 4: Restore Normal Operation

Verify that both new appliances are operating correctly in the redundant pair and optionally revert activity to the primary.

  1. Verify that both appliance event brokers are in the correct redundancy state:

    solace-primary# show redundancy
solace-backup# show redundancy

  2. If you are using Config-Sync, verify that both appliance event brokers are fully synchronized:

    solace-primary# show config-sync
solace-backup# show config-sync

  3. (Optional) Revert activity back to the primary appliance event broker:

    solace-backup# admin
solace-backup(admin)# redundancy revert-activity
solace-backup(admin)# end