Shutting Down Guaranteed Messaging Redundant Appliances

If you have to shut down a pair of redundant Solace PubSub+ appliances running software version 7.1 or greater that are using Guaranteed Messaging (for example, if you are moving a SAN or there is a data center power down), use the following procedure.

  1. Enter the following command for the primary event broker to ensure that it is in the correct redundancy state.
    solace-primary# show redundancy

    The command output on the primary event broker looks like this (output may vary by appliance version):

    Configuration Status : Enabled
    Auto Revert          : No
    Redundancy Mode      : Active/Active
    Mate Router Name     : solace-backup
    ADB Link To Mate     : Up
    ADB Hello To Mate    : Up
                                  Primary Virtual Router Backup Virtual Router
                                  ---------------------- ----------------------
    Activity Status               Local Active           Mate Active
    Routing Interface             1/6/lag1:1             1/6/lag1:2
    VRRP VRID                     50                     197
    Routing Interface Status      Up                     Up
    VRRP Status                   Master                 Backup
    VRRP Priority                 250                    100
    Message Spool Status          AD-Active              AD-Disabled
    Priority Reported By Mate     Standby                Active

    The Auto Revert value is a configurable option; it may be set to Yes, depending on the redundant system configuration.

  2. Enter the following command for the backup event broker to ensure that it is in the correct redundancy state.
    solace-backup# show redundancy

    The command output on the backup event broker looks like this (output may vary by appliance version):

    Configuration Status : Enabled
    Auto Revert          : No
    Redundancy Mode      : Active/Active
    Mate Router Name     : solace-primary
    ADB Link To Mate     : Up
    ADB Hello To Mate    : Up
                                  Primary Virtual Router Backup Virtual Router
                                  ---------------------- ----------------------
    Activity Status               Local Active           Mate Active
    Routing Interface             1/6/lag1:1             1/6/lag1:2
    VRRP VRID                     197                    50
    Routing Interface Status      Up                     Up
    VRRP Status                   Master                 Backup
    VRRP Priority                 250                    100
    Message Spool Status          AD-Disabled            AD-Standby
    Priority Reported By Mate     Standby                Active
  3. Enter the following commands to shut down messaging services on both event brokers:
    solace-primary# configure
    solace-primary(configure)# service msg-backbone
    solace-primary(configure/service/msg-backbone)# shutdown
    All SMF, WEB, and REST clients will be disconnected.
    Do you want to continue (y/n)? y
    solace-primary(configure/service/msg-backbone)# exit
    solace-primary(configure/service)# exit
    solace-primary(configure)#
    solace-backup# configure
    solace-backup(configure)# service msg-backbone
    solace-backup(configure/service/msg-backbone)# shutdown
    All SMF, WEB, and REST clients will be disconnected.
    Do you want to continue (y/n)? y
    solace-backup(configure/service/msg-backbone)# exit
    solace-backup(configure/service)# exit
    solace-backup(configure)#
  4. Enter the following commands for both the primary and backup event broker to ensure that auto-revert is not enabled for either event broker:
    solace-primary(configure/redundancy)# no auto-revert
    solace-primary(configure/redundancy)# end
    solace-primary#
    solace-backup(configure)# redundancy
    solace-backup(configure/redundancy)# no auto-revert
    solace-backup(configure/redundancy)# end
    solace-backup#
  5. If the event brokers are using transacted sessions, wait until all transacted sessions time-out and the count drops to 0. This could take up to 60 seconds.
  6. Enter the following command for the primary event broker to view the transacted sessions in use (the following example only shows the relevant transacted session usage lines):
    solace-primary# show message-spool detail
                                           Currently Used   Max Allowed
                                           --------------   -----------
    Queue and Topic-endpoint Spools:                   35         16000
                             Queues:                   32
                    Topic-endpoints:                    3
                Transacted Sessions:                    0         16000
    . . .
  7. Enter the following commands to shut down the message spools on both event brokers:
    solace-primary# configure
    solace-primary(configure)# hardware message-spool
    solace-primary(configure/hardware/message-spool)# shutdown
    All message spooling will be stopped.
    Do you want to continue (y/n)? y
    solace-primary(configure/hardware/message-spool)# end
    solace-primary#
    solace-backup# configure
    solace-backup(configure)# hardware message-spool
    solace-backup(configure/hardware/message-spool)# shutdown
    All message spooling will be stopped.
    Do you want to continue (y/n)? y
    solace-backup(configure/hardware/message-spool)# end
    solace-backup#
  8. Enter the following command on the primary event broker to view the total number of spooled messages, and write down the total number of spooled messages for future reference:
    solace-primary> show message-spool
  9. For the primary event broker, enter the following command to backup the ADB configuration to the event broker internal disk.
    solace-primary# admin
    solace-primary(admin)# system
    solace-primary(admin/system)# message-spool
    solace-primary(admin/system/message-spool)# backup-adb-to-disk
    On the next system startup, the backed-up ADB state on disk will overwrite
    the current ADB state.
    Do you want to continue (y/n)? y
    Moving ADB messages to disk: 100%
    Backing up ADB config to disk: 100%
    solace-primary(admin/system/message-spool)# end
    solace-primary#
  10. For the primary event broker, after the CLI output indicates that the ADB backup operation has completed, enter the following command to display the message spool contents:
    solace-primary# show message-spool detail

    Ensure that there are no messages spooled on the ADB and all messages are spooled to disk.

  11. Enter the following commands to turn off both event brokers:
    solace-primary# power-down
    This command powers off the router and does not restart it afterwards
    Do you want to continue (y/n)? y
    solace-backup# power-down
    This command powers off the router and does not restart it afterwards
    Do you want to continue (y/n)? y
  12. Perform any required maintenance tasks (for example, upgrading or replacing hardware) while the event brokers are powered down.
  13. Power up the primary event broker.
    • To power up a Solace PubSub+ 3230 or 3260, use a straightened paper clip to press the on/standby button on the front panel of the event broker.
    • To power up a Solace PubSub+ 3530 or 3560, press the on/standby button on located on the lower left-hand side of the rear panel of the event broker.

    Wait for the supercapacitor on the ADB to fully charge. This could take up to 15 minutes.

  14. Enter the following command to determine its charge level:
    solace-primary# show hardware detail
    . . .
    
    Power Module Details:
          State:                               Ok
          Charge Level:                        100%
          Estimated Time Until Fully Charged:  0 minutes
    . . .
  15. Enter the following commands on the primary event broker to start the message spool that was shut down:
    solace-primary# configure
    solace-primary(configure)# hardware message-spool
    solace-primary(configure/hardware/message-spool)# no shutdown primary
    solace-primary(configure/hardware/message-spool)# exit
    solace-primary(configure/hardware)# exit
  16. Enter the following commands on the primary event broker to start the messaging services that were shut down:
    solace-primary(configure)# service msg-backbone
    solace-primary(configure/service/msg-backbone)# no shutdown
    solace-primary(configure/service/msg-backbone)# end
    solace-primary#
  17. Power up the backup event broker.
    • To power up a Solace PubSub+ 3230 or 3260, use a straightened paper clip to press the on/standby button on the front panel of the event broker.
    • To power up a Solace PubSub+ 3530 or 3560, press the on/standby button on located on the lower left-hand side of the rear panel of the event broker.

    Wait for the supercapacitor on the ADB to fully charge. This could take up to 15 minutes.

  18. Enter the following command to determine the charge level of the supercapacitor:
    solace-backup# show hardware detail
    . . .
    
    Power Module Details:
          State:                               Ok
          Charge Level:                        100%
          Estimated Time Until Fully Charged:  0 minutes
    . . .
  19. Enter the following commands on the backup event broker to start the message spool that was shut down:
    solace-backup# configure
    solace-backup(configure)# hardware message-spool
    solace-backup(configure/hardware/message-spool)# no shutdown backup
    solace-backup(configure/hardware/message-spool)# exit
    solace-backup(configure/hardware)# exit
    solace-backup(configure)# exit
  20. Enter the following commands on the backup event broker to start the messaging services that were shut down:
    solace-backup(configure)# service msg-backbone
    solace-backup(configure/service/msg-backbone)# no shutdown
    solace-backup(configure/service/msg-backbone)# end
    solace-backup#

    Solace recommends that the auto revert redundancy configuration be kept disabled on an event broker. However, if you turned off auto-revert earlier, and you do want to use auto revert for your event brokers, you should re-enabled it at this point.

  21. Enter the following command on the primary event broker to ensure that it is in the correct redundancy state:
    solace-primary# show redundancy

    The output looks like the following (but may vary by appliance version):

    Configuration Status      : Enabled 
     Auto Revert               : No 
     Redundancy Mode           : Active/Active 
     Mate Router Name          : solace-backup 
     ADB Link To Mate          : Up 
     ADB Hello To Mate         : Up 
                                    Primary Virtual Router Backup Virtual Router 
                                    ---------------------- --------------------- 
     Activity Status                Local Active           Mate Active 
     Routing Interface              1/6/lag1:1             1/6/lag1:2 
     VRRP VRID                      50                     197 
     Routing Interface Status       Up                     Up 
     VRRP Status                    Master                 Backup 
     VRRP Priority                  250                    100 
     Message Spool Status           AD-Active              AD-Disabled 
     Priority Reported By Mate      Standby                Active
  22. Enter the following command on the backup event broker to ensure that it is in the correct redundancy state:
    solace-backup# show redundancy

    The output looks like the following (but may vary by appliance version):

    Configuration Status    : Enabled 
     Auto Revert             : No 
     Redundancy Mode         : Active/Active 
     Mate Router Name        : solace-primary 
     ADB Link To Mate        : Up 
     ADB Hello To Mate       : Up 
                                  Primary Virtual Router Backup Virtual Router 
                                  ---------------------- ---------------------- 
     Activity Status              Local Active           Mate Active 
     Routing Interface            1/6/lag1:1             1/6/lag1:2 
     VRRP VRID                    197                    50 
     Routing Interface Status     Up                     Up 
     VRRP Status                  Master                 Backup 
     VRRP Priority                250                    100 
     Message Spool Status         AD-Disabled            AD-Standby 
     Priority Reported By Mate    Standby                Active
  23. If the event brokers are not in the correct redundancy state, enter the following commands on the primary event broker to force a redundancy switch to the mate event broker, then verify that both event brokers are now in the correct redundancy state (see step 21 and 22):
    solace-primary# admin
    solace-primary(admin)# redundancy revert-activity
    solace-primary(admin)# end
    solace-primary#