Replacing a Failed Appliance in a Redundant Pair

This section describes in general terms how to replace a failed Solace PubSub+ appliance in a deployed high-availability (HA) redundant pair. The steps that must be performed are dependent on whether the appliance to be replaced was configured as active or backup and whether the Config‑Sync facility can be used to copy the configuration to the replacement appliance.

ESD/Safety Precautions

To prevent equipment damage from Electrostatic Discharge (ESD), always ensure that the appliance chassis is electrically connected to earth ground. Always follow ESD prevention procedures when removing and replacing power supplies. Use an anti-static wrist strap, or another anti-static device. If no wrist strap or mat is available, ground yourself by touching the metal part of the chassis.

Tool and Equipment Requirements

The following tools and equipment are required:

electrostatic discharge wrist strap
anti-static mat
replacement appliance

Procedure

Perform the following general steps provide in this example to replace a failed appliance in a deployed HA redundant pair. In this example, the solace1 is the active appliance and solace2 is the appliance that has failed and must be replaced.

If you require further assistance, or if you have any questions regarding this procedure, contact Solace for technical support.

When replacing a failed appliance in a deployed HA redundant pair, do NOT physically connect the HBA links between the replacement appliance and the external disk storage unit, do NOT physically connect the NAB in the replacement appliance to the network, and do NOT physically connect the ADB links between the replacement appliance and paired in-service appliance until explicitly instructed to do so in the steps in the following procedure.

On the currently active appliance, enter the following Solace CLI commands to ensure that auto revert is not enabled.

solace1> enable
solace1# configure
solace1(configure)# redundancy
solace1(configure/redundancy)# no auto-revert
solace1(configure/redundancy)# end
solace1#

Install the replacement appliance in the hardware rack, power it up, and connect a management console.
For more information, refer to Prerequisites & Installation, On / Standby Button, and Connecting a Management Console.
Using the management console, run set-up and configure only the management IP address and default route for the replacement appliance.
For information, refer to Setting Up a Basic Configuration and Configuring the Management Interface and Message Backbone Interface.

Enter the following show command to display the HBA Port and Node names for the replacement appliance:

solace2> show hardware detail

Example output:

...
Slot 1/3: Host Bus Adapter Blade
  Product #: HBA-0204FC-02-A
  Serial #: M54687
  Model Name: QLE2462
  Model Description: PCI-Express to 4Gb FC, Dual Channel
  Driver Version: 8.01.07-k1
 
  Fibre-Channel 1
    State: Link Up - F_Port (fabric via point-to-point)
    Speed: 2 Gbit
    Port Id: 0x031f00
    Port Name: 0x210000e08b931f25 
    Port Type: NPort (fabric via point-to-point)
    Node Name: 0x200000e08b931f25

Using the displayed HBA Port and Node names from the output, register the HBA ports with the proper storage Logical Unit Number (LUN) on the external disk storage array.
Enter the following show command to ensure that the correct version of Solace PubSub+ software is installed on the replacement appliance.
```
solace2> show version
```

If necessary, upgrade or downgrade the version to match that of the version used by the mate appliance. For information on upgrading software on appliances, refer to the Solace PubSub+ Appliance Release Notes. For information on downgrading appliances, refer to Downgrading Versions.

To apply the configuration of the active event broker to the replacement appliance, you can either apply a saved backup configuration file to the replacement mate appliance or use the Config-Sync facility to apply the configuration of the active event broker to the replacement mate appliance:

To apply a backup configuration file to the replacement appliance, refer to step 8.
To use the Config-Sync facility assert the active event broker's configuration on to the replacement appliance, refer to step 9.

To apply a saved configuration file to the replacement appliance, do the following:
1. Perform the procedure outlined in Restoring Configuration Files.
2. Enter the following commands to ensure that auto revert is not enabled for the replacement appliance:
```
solace2> enable
solace2# configure
solace2(configure)# redundancy
solace2(configure/redundancy)# no auto-revert
solace2(configure/redundancy)# end
solace2#
```
3. Turn off power to the replacement appliance:
```
solace2# power-down
```
  The appliances do not restart automatically after this command is run.
4. Physically connect the HBA links between the replacement appliance and the external disk storage array.
5. Connect the NAB in the replacement appliance to the network.
6. Physically connect the ADB mate links between the replacement appliance and the paired in-service appliance.
7. Power up the replacement appliance:
  See On / Standby Button.
8. Assert the configuration settings of the active appliance onto the replacement appliance. See Asserting Event Broker System Configurations and Asserting Event Brokers VPN Configurations.
9. Verify that the replacement appliance's configuration fully matches the active appliance's configuration.
  You can use the show current-config User EXEC command to obtain the sequence of CLI commands that are required to duplicate the system-level and Message VPN-level configurations used by the active appliance. For more information, refer to Show the CLI Sequence for the Current Configuration.
To use Config-Sync to automatically synchronize the configurations of the mate event brokers, perform the following steps for the replacement appliance:
1. Configure the Message Backbone (LAG & IP).
  See Configuring the Management Interface and Message Backbone Interface.
2. Depending on the deployed SolOS version, configure a redundancy interface or a routing interface.
  See Configuring Redundancy Parameters (version 9.5.0 and later).
  See Assigning Routing Interfaces (versions earlier than 9.5.0).
3. Configure its redundancy parameters.
  See Configuring Redundancy Parameters.
4. Configure its message spool.
  See Guaranteed Messaging Configuration.
5. Surrender activity from the given appliance to its mate:
```
solace2(configure/redundancy)# release-activity
```
6. Manually set the configuration parameters that are unique to the appliance and cannot be set through the Config-Sync facility. For a list of Config-Sync ineligible parameters, see Properties That Are Not Synchronized.
7. Enable Config-Sync:
```
solace2(configure)# config-sync
solace2(configure/config-sync)# no shutdown
```
8. Power down the appliance:
```
solace2(configure/config-sync)# end
solace2# power-down
```
  The appliances do not restart automatically after this command is run.
9. Connect the HBA links between the replacement appliance and the external disk storage array.
10. Connect the NAB in the replacement appliance to the network.
11. Connect the ADB links between the replacement appliance and the paired in-service appliance
12. Power up the appliance.
  See On / Standby Button.
13. Assert the configuration settings of the existing event broker onto the replacement event broker. See Asserting Event Broker System Configurations and Asserting Event Brokers VPN Configurations
14. Verify that the configuration is fully in sync. See Viewing Config-Sync Info.
15. Force the appliance to take activity back from its mate:
```
solace2(configure/redundancy)# no release-activity
```
16. Optionally: revert activity.
If necessary, restore the auto‑revert redundancy setting.

Solace recommends that the auto revert redundancy configuration should normally be kept disabled on an event broker. However, if you do want to use auto revert for your event broker, it should be re-enabled at this point.
Enter the following commands to verify that the replacement event broker is in the correct redundancy state.
```
solace2> enable
solace2# show redundancy
```
If it is not in the correct redundancy state, revert-activity, as required.

Provide feedback