Performing PubSub+ Cache Administrative Tasks

You can perform a number of administrative tasks on a per Distributed Cache basis.

Clearing PubSub+ Cache Events

You may receive a one-shot event notification if one or more PubSub+ Cache Instances enters a lost message state.

The exact event that caused the lost message state is specified in the event log immediately preceding the VPN_SOLCACHE_LOST_MSG_STATE event.

First refer to the Event Reference and follow the recommended actions for the event. Then you can clear the “Recommended Actions” for this preceding event. Then, you can clear the one-shot event to ensure that the event is raised again if further lost or discarded messages are detected (refer to Clearing One-Shot Events).

Deleting Cached Messages

To delete cached messages for a given topic for either a single PubSub+ Cache Instance or all PubSub+ Cache Instances in a Cache Cluster, enter the following commands:

solace# admin
solace(admin)# distributed-cache <name> message-vpn <vpn-name>
solace(admin/distributed-cache)# delete-messages <topic> [cache-cluster <cluster-name>] [cache-instance <instance-name>]

Where:

<name> is the name of an existing Distributed Cache.

<vpn-name> is the name of an existing Message VPN.

<topic> is the topic string for which messages are to be deleted. The topic string wildcards ‘*’ and ‘>’ can also be used. For topic syntax information, refer to Topic Support & Syntax.

<cluster-name> is the name the Cache Cluster for which you want to delete the messages cached by each of its PubSub+ Cache Instances.

<instance-name> is the name of a particular PubSub+ Cache Instance.

Starting PubSub+ Cache Instances

When the Auto Start Mode is not enabled for a PubSub+ Cache Instance (refer to Enabling/Disabling Auto Start Mode), it will remain in a “stopped” state instead of automatically restarting and resynchronizing with the Cache Manager.

To administratively start a single PubSub+ Cache Instance or all PubSub+ Cache Instances in a Cache Cluster that do not have Auto Start Mode enabled, enter the following commands:

solace# admin
solace(admin)# distributed-cache <name> message-vpn <vpn-name>
solace(admin/distributed-cache)# start [cache-cluster <cluster-name>] [cache-instance <instance-name>]

Where:

<name> is the name of an existing Distributed Cache

<vpn-name> is the name of an existing Message VPN

<cluster-name> is the name of the associated Cache Cluster

<instance-name> is the name of the associated PubSub+ Cache Instance under the Cache Cluster

Backing Up/Restoring PubSub+ Cache Instance Messages

An administrator can write the cached messages from a PubSub+ Cache Instance to a file, and then later restore the contents of a shutdown PubSub+ Cache Instance with cached messages that were saved to file.

The ability to backup and restore the messages that are cached by a PubSub+ Cache Instance can be useful for scheduled maintenance windows.

A PubSub+ Cache backup operation does not back up any remote topics that are used by Global Caching.

Backing up Cached Messages

Use the following steps to back up a PubSub+ Cache Instance’s cached messages are:

  1. Shutdown the PubSub+ Cache Instance. (Refer to Enabling/Disabling PubSub+ Cache Instances.)

    When a PubSub+ Cache Instance is shut down, it cannot receive live data or respond to cache requests, and it will not send and receive Global Caching heartbeat messages. However, the PubSub+ Cache Instance’s cached messages are maintained when it is shut down (subject to normal message lifetime aging).

  2. Back up the PubSub+ Cache Instance’s cached messages. (Refer to Backing Up Cached Messages.)
  3. Enable the PubSub+ Cache Instance. (Refer to Enabling/Disabling PubSub+ Cache Instances.)

    The Cache Manager could synchronize the PubSub+ Cache Instances with the others in the Cache Cluster.

Restoring Cached Messages

Use the following steps to restore a PubSub+ Cache Instance with the cached messages that were saved to a backup file are:

  1. Individually shutdown all of the PubSub+ Cache Instances in the Cache Cluster. Refer to Enabling/Disabling PubSub+ Cache Instances.

    It is necessary to individually shut down each PubSub+ Cache Instance in the Cache Cluster instead of shutting down their parent Cache Cluster, so that you can control the order in which the PubSub+ Cache Instances are brought back online following the restore.

  2. Run the restore operation for one PubSub+ Cache Instance so that it is restored with the cached messages that were saved to a backup file. Refer to Restoring Cached Messages.
  3. Run No Shutdown for the restored PubSub+ Cache Instance. Refer to Enabling/Disabling PubSub+ Cache Instances.

    As the first PubSub+ Cache Instance in the Cache Cluster to reconnect to the Cache Manager, this particular PubSub+ Cache Instance becomes the leader PubSub+ Cache Instance that all of the others in the Cache Cluster will synchronize with.

  4. Run No Shutdown for each of the remaining PubSub+ Cache Instances in the Cache Cluster.

    These PubSub+ Cache Instances synchronize their data from the restored PubSub+ Cache Instance. Note that it is important that you wait until the restored PubSub+ Cache Instance is online before enabling the other PubSub+ Cache Instances in the Cache Cluster. If you do not wait until the first PubSub+ Cache Instance is fully restored, another PubSub+ Cache Instance that is enabled could become the leader from which all other PubSub+ Cache Instances synchronize.

Example:

The following code snippets provide an example of the steps required to restore cached messages from a backup file.

Step 1: Shut down all PubSub+ Cache Instances in the Cache Cluster.

solace# show cache-instance *

 

Admin Oper Auto Stop Lost

Cache Instance Cache-Cluster Status Status Start On Loss Msg

-------------------- -------------------- ------ -------- ----- ------- ----

v1.d1.c1.i1 v1.d1.c1 Up Up Yes Yes No

v1.d1.c1.i2 v1.d1.c1 Up Up Yes Yes No

 

solace# configure
solace(configure)# distributed-cache v1.d1 message-vpn v1
solace(configure/distributed-cache)# cache-cluster v1.d1.c1
solacesolace(...igure/distributed-cache/cache-cluster)# cache-instance v1.d1.c1.i1
solace(...ed-cache/cache-cluster/cache-instance)## shutdown
solace(...ed-cache/cache-cluster/cache-instance)## exit
solacesolace(...igure/distributed-cache/cache-cluster)# cache-instance v1.d1.c1.i2
solace(...ed-cache/cache-cluster/cache-instance)## shutdown
solace(...ed-cache/cache-cluster/cache-instance)## end
solace# show cache-instance *

 

Admin Oper Auto Stop Lost

Cache Instance Cache-Cluster Status Status Start On Loss Msg

-------------------- -------------------- ------ -------- ----- ------- ----

v1.d1.c1.i1 v1.d1.c1 Down Down Yes Yes No

v1.d1.c1.i2 v1.d1.c1 Down Down Yes Yes No

Step 2: Start the restore operation on one PubSub+ Cache Instance.

solace# admin
solace(admin)# distributed-cache v1.d1 message-vpn v1
solace(admin/distributed-cache)# restore-cached-messages cache-instance v1.d1.c1.i1 file v1.d1.c1.i1.bkp
solace(admin/distributed-cache)# end

Step 3: Monitor the restore progress until it is complete.

The following two code snippets show the Last Restore of Cached Messages section of the cache-instance status. The first shows in-progress output, and the second shows restore complete output.

 

solace# show cache-instance v1.d1.c1.i1 remote status

 

. . .

 

Last Restore Of Cached Messages

    Last Successful Restore        : Jul 13 2012 07:43:20 PDT

    Last Status                    : In Progress, 7% complete

    Last Filename                  : /root/solCache/solCache_backups/

                                   : v1.d1.c1.i1.bkp

 

. . .

 

solace#
solace# show cache-instance v1.d1.c1.i1 remote status

 

. . .

 

Last Restore Of Cached Messages

    Last Successful Restore        : Jul 13 2012 07:44:21 PDT

    Last Status                    : Ok

    Last Filename                  : /root/solCache/solCache_backups/

: v1.d1.c1.i1.bkp

 

Step 4: After the restore has completed, run the no shutdown command on the PubSub+ Cache Instance that the restore was performed on.

solace# configure
solace(configure)# distributed-cache v1.d1 message-vpn v1
solace(configure/distributed-cache)# cache-cluster v1.d1.c1
solacesolace(...igure/distributed-cache/cache-cluster)# cache-instance v1.d1.c1.i1
solace(...ed-cache/cache-cluster/cache-instance)## no shutdown
solace(...ed-cache/cache-cluster/cache-instance)## end
solace# show cache-instance *

 

                                          Admin  Oper     Auto  Stop    Lost

Cache Instance       Cache-Cluster        Status Status   Start On Loss Msg

-------------------- -------------------- ------ -------- ----- ------- ----

v1.d1.c1.i1          v1.d1.c1             Up     Up       Yes   Yes     No

v1.d1.c1.i2          v1.d1.c1             Down   Down     Yes   Yes     No

After a restore operation, the size of the PubSub+ Cache Instance database may be smaller than the pre-backup database. This does not indicate lost messages.Rather, less memory is used for backed up messages than for live data messages.

Step 5: Perform no shutdown on all other PubSub+ Cache Instances, and wait for them to synchronize the restored messages.

solace# configure
solace(configure)# distributed-cache v1.d1 message-vpn v1
solace(configure/distributed-cache)# cache-cluster v1.d1.c1
solacesolace(...igure/distributed-cache/cache-cluster)# cache-instance v1.d1.c1.i2
solace(...ed-cache/cache-cluster/cache-instance)## no shutdown
solace(...ed-cache/cache-cluster/cache-instance)## end
solace# show cache-instance *

 

                                         Admin  Oper     Auto  Stop    Lost

Cache Instance       Cache-Cluster        Status Status   Start On Loss Msg

-------------------- -------------------- ------ -------- ----- ------- ----

v1.d1.c1.i1          v1.d1.c1             Up     Up       Yes   Yes     No

v1.d1.c1.i2          v1.d1.c1             Up     ClusSync Yes   Yes     No

 

solace# show cache-instance *

 

                                         Admin  Oper     Auto  Stop    Lost

Cache Instance       Cache-Cluster        Status Status   Start On Loss Msg

-------------------- -------------------- ------ -------- ----- ------- ----

v1.d1.c1.i1          v1.d1.c1             Up     Up       Yes   Yes     No

v1.d1.c1.i2          v1.d1.c1             Up     Up       Yes   Yes     No

Backing Up Cached Messages

To backup the cached messages for a PubSub+ Cache Instance that is shutdown, enter the following command:

solace(admin/distributed-cache)# backup-cached-messages cache-instance <instance-name> [{file <filename>} | cancel]

Where:

<instance-name> is the name of the PubSub+ Cache Instance that you want to backup.

file <filename> is the name (0-255 characters) for the backup file. This file is saved to the directory indicated by the SOLCACHE_MSG_STORE_DIRECTORY parameter in the PubSub+ Cache Instance configuration file (refer to Customizing PubSub+ Cache Instance Configuration Files). Note that cached data for global topics is not written to the file.

cancel stops a backup operation when it is in progress.

Restoring Cached Messages

When a restore operation is performed for a PubSub+ Cache Instance, the following occurs:

  • any messages that are currently cached by the PubSub+ Cache Instance are deleted
  • the messages saved to the backup file, as well as their original timestamps, are restored to the PubSub+ Cache Instance

The PubSub+ Cache Plug-in is not invoked for restored messages because the messages would have already been processed by the Plug-in when they were originally written to the cache.

When restoring messages from a backup file, messages from the backup file can be discarded if:

  • They do not match the PubSub+ Cache Instance’s configured topic subscriptions.
  • They would cause the max-msgs-per-topic to be exceeded. In this case, the oldest messages for that topic are deleted, to stay within the max‑msgs-per-topic limit.
  • They exceed the PubSub+ Cache Instance’s configured maximum message lifetime (refer to Configuring Message Lifetimes).

WARN event logs are generated to indicate how many messages were discarded for these reasons during the restore operation.

To restore the given cached messages for a PubSub+ Cache instance from a backup file, enter the following commands:

solace# admin
solace(admin)# distributed-cache <name> message-vpn <vpn-name>
solace(admin/distributed-cache)# restore-cached-messages cache-instance <instance-name> [{file <filename>} | cancel]

Where:

<name> is the name of an existing Distributed Cache

<vpn-name> is the name of an existing Message VPN

<instance-name> is the name of the PubSub+ Cache Instance that you want to restore

file <filename> is the name (0-255 characters) of the backup cached messages file to use for the restore process

cancel stops a restore operation when it is in progress