Performing Solace Cache Administrative Tasks

You can perform the following administrative tasks on a per Distributed Cache basis:

Clearing Solace Cache Events

You may receive a one-shot event notification if one or more Solace Cache Instances enter a lost message state.

The exact event that caused the lost message state is specified in the event log immediately preceding the VPN_SOLCACHE_LOST_MSG_STATE event.

First, see the Syslog Event Reference and follow the recommended actions for the event. Then, you can clear the "Recommended Actions" for this preceding event. Then, you can clear the one-shot event to ensure that the event is raised again if further lost or discarded messages are detected (see Clearing One-Shot Events).

Deleting Cached Messages

To delete cached messages for a given topic for either a single Solace Cache Instance or all Solace Cache Instances in a Cache Cluster, enter the following commands:

solace# admin
solace(admin)# distributed-cache <name> message-vpn <vpn-name>
solace(admin/distributed-cache)# delete-messages <topic> [cache-cluster <cluster-name>] [cache-instance <instance-name>]

Where:

<name> is the name of an existing Distributed Cache.

<vpn-name> is the name of an existing Message VPN.

<topic> is the topic string for which messages are to be deleted. The topic string wildcards ‘*’ and ‘>’ can also be used. For topic syntax information, see Topic Support & Syntax.

<cluster-name> is the name the Cache Cluster for which you want to delete the messages cached by each of its Solace Cache Instances.

<instance-name> is the name of a particular Solace Cache Instance.

Starting Solace Cache Instances

When Auto Start Mode is not enabled for a Solace Cache instance (see Enabling Auto Start Mode), it remains in a “stopped” state instead of automatically restarting and resynchronizing with the Cache Manager.

To administratively start a single Solace Cache instance or all Solace Cache instances in a Cache Cluster that do not have Auto Start Mode enabled, enter the following commands:

solace# admin
solace(admin)# distributed-cache <name> message-vpn <vpn-name>
solace(admin/distributed-cache)# start [cache-cluster <cluster-name>] [cache-instance <instance-name>]

Where:

<name> is the name of an existing Distributed Cache

<vpn-name> is the name of an existing Message VPN

<cluster-name> is the name of the associated Cache Cluster

<instance-name> is the name of the associated Solace Cache Instance under the Cache Cluster

Backing Up and Restoring Solace Cache Messages

An administrator can write the cached messages from a Solace Cache Instance to a file, and then later restore the contents of a shutdown Solace Cache Instance with cached messages that were saved to the file.

The ability to back up and restore the messages that are cached by a Solace Cache Instance can be useful for scheduled maintenance windows.

A Solace Cache backup operation does not back up any remote topics that are used by Global Caching.

High-Level Steps to Back up Cached Messages

Use the following steps to back up a Solace Cache Instance’s cached messages:

  1. Shut down the Solace Cache Instance. (See Enabling and Disabling Solace Cache Instances.)

    When a Solace Cache Instance is shut down, it cannot receive live data or respond to cache requests, and it does not send and receive Global Caching heartbeat messages. However, the Solace Cache Instance’s cached messages are maintained when it is shut down (subject to normal message lifetime aging).

  2. Back up the Solace Cache Instance’s cached messages. (See Backing Up Cached Messages.)
  3. Enable the Solace Cache Instance. (See Enabling and Disabling Solace Cache Instances.)

    The Cache Manager could synchronize the Solace Cache Instances with the others in the Cache Cluster.

High-Level Steps to Restore Cached Messages

Use the following steps to restore a Solace Cache Instance with the cached messages that were saved to a backup file are:

  1. Individually shutdown all of the Solace Cache Instances in the Cache Cluster. See Enabling and Disabling Solace Cache Instances.

    It is necessary to individually shut down each Solace Cache Instance in the Cache Cluster instead of shutting down their parent Cache Cluster, so that you can control the order in which the Solace Cache Instances are brought back online following the restore.

  2. Run the restore operation for one Solace Cache Instance so that it is restored with the cached messages that were saved to a backup file. See Restoring Cached Messages.
  3. Run No Shutdown for the restored Solace Cache Instance. See Enabling and Disabling Solace Cache Instances.

    As the first Solace Cache Instance in the Cache Cluster to reconnect to the Cache Manager, this particular Solace Cache Instance becomes the leader Solace Cache Instance that all of the others in the Cache Cluster synchronize with.

  4. Run No Shutdown for each of the remaining Solace Cache Instances in the Cache Cluster.

    These Solace Cache Instances synchronize their data from the restored Solace Cache Instance. You must wait until the restored Solace Cache Instance is online before enabling the other Solace Cache Instances in the Cache Cluster. If you don't wait until the first Solace Cache Instance is fully restored, another Solace Cache Instance that is enabled could become the leader the all other Solace Cache Instances synchronize from.

Example:

The following code snippets provide an example of the steps required to restore cached messages from a backup file.

Step 1: Shut down all Solace Cache Instances in the Cache Cluster.

solace# show cache-instance *
                                          Admin  Oper     Auto  Stop    Lost
Cache Instance       Cache-Cluster        Status Status   Start On Loss Msg
-------------------- -------------------- ------ -------- ----- ------- ----
v1.d1.c1.i1          v1.d1.c1             Up     Up       Yes   Yes     No
v1.d1.c1.i2          v1.d1.c1             Up     Up       Yes   Yes     No
solace# configure
solace(configure)# distributed-cache v1.d1 message-vpn v1
solace(configure/distributed-cache)# cache-cluster v1.d1.c1
solace(configure/distributed-cache/cache-cluster)# cache-instance v1.d1.c1.i1
solace(...ed-cache/cache-cluster/cache-instance)## shutdown
solace(...ed-cache/cache-cluster/cache-instance)## exit
solace(configure/distributed-cache/cache-cluster)# cache-instance v1.d1.c1.i2
solace(...ed-cache/cache-cluster/cache-instance)## shutdown 
solace(...ed-cache/cache-cluster/cache-instance)## end
solace# show cache-instance *
                                          Admin  Oper     Auto  Stop    Lost
Cache Instance       Cache-Cluster        Status Status   Start On Loss Msg
-------------------- -------------------- ------ -------- ----- ------- ----
v1.d1.c1.i1          v1.d1.c1             Down   Down     Yes   Yes     No
v1.d1.c1.i2          v1.d1.c1             Down   Down     Yes   Yes     No

Step 2: Start the restore operation on one Solace Cache Instance.

solace# admin
solace(admin)# distributed-cache v1.d1 message-vpn v1
solace(admin/distributed-cache)# restore-cached-messages cache-instance v1.d1.c1.i1 file v1.d1.c1.i1.bkp
solace(admin/distributed-cache)# end

Step 3: Monitor the restore progress until it is complete.

The following two code snippets show the Last Restore of Cached Messages section of the cache-instance status. The first shows in-progress output, and the second shows restore complete output.

solace# show cache-instance v1.d1.c1.i1 remote status
. . .
     
Last Restore Of Cached Messages
    Last Successful Restore        : Jul 13 2012 07:43:20 PDT
    Last Status                    : In Progress, 7% complete
    Last Filename                  : /root/solCache/solCache_backups/
                                   : v1.d1.c1.i1.bkp
     
    . . .
solace# 
solace# show cache-instance v1.d1.c1.i1 remote status
. . .

Last Restore Of Cached Messages
    Last Successful Restore        : Jul 13 2012 07:44:21 PDT
    Last Status                    : Ok
    Last Filename                  : /root/solCache/solCache_backups/
                                   :   v1.d1.c1.i1.bkp
     

Step 4: After the restore has completed, run the no shutdown command on the Solace Cache Instance that the restore was performed on.

solace# configure
solace(configure)# distributed-cache v1.d1 message-vpn v1 
solace(configure/distributed-cache)# cache-cluster v1.d1.c1 
solace(configure/distributed-cache/cache-cluster)# cache-instance v1.d1.c1.i1
solace(...ed-cache/cache-cluster/cache-instance)## no shutdown 
solace(...ed-cache/cache-cluster/cache-instance)## end
solace# show cache-instance *
                                          Admin  Oper     Auto  Stop    Lost
Cache Instance       Cache-Cluster        Status Status   Start On Loss Msg
-------------------- -------------------- ------ -------- ----- ------- ----
v1.d1.c1.i1          v1.d1.c1             Up     Up       Yes   Yes     No
v1.d1.c1.i2          v1.d1.c1             Down   Down     Yes   Yes     No

After a restore operation, the size of the Solace Cache Instance database may be smaller than the pre-backup database. This does not indicate lost messages. Less memory is used for backed up messages than for live data messages.

Step 5: Perform no shutdown on all other Solace Cache Instances, and wait for them to synchronize the restored messages.

solace# configure
solace(configure)# distributed-cache v1.d1 message-vpn v1
solace(configure/distributed-cache)# cache-cluster v1.d1.c1
solace(configure/distributed-cache/cache-cluster)# cache-instance v1.d1.c1.i2
solace(...ed-cache/cache-cluster/cache-instance)## no shutdown 
solace(...ed-cache/cache-cluster/cache-instance)## end
solace# show cache-instance *
                                          Admin  Oper     Auto  Stop    Lost
Cache Instance       Cache-Cluster        Status Status   Start On Loss Msg
-------------------- -------------------- ------ -------- ----- ------- ----
v1.d1.c1.i1          v1.d1.c1             Up     Up       Yes   Yes     No
v1.d1.c1.i2          v1.d1.c1             Up     ClusSync Yes   Yes     No
     
solace# show cache-instance *
                                          Admin  Oper     Auto  Stop    Lost
Cache Instance       Cache-Cluster        Status Status   Start On Loss Msg
-------------------- -------------------- ------ -------- ----- ------- ----
v1.d1.c1.i1          v1.d1.c1             Up     Up       Yes   Yes     No
v1.d1.c1.i2          v1.d1.c1             Up     Up       Yes   Yes     No

Backing Up Cached Messages

To back up the cached messages for a Solace Cache Instance that is shut down, enter the following command:

solace(admin/distributed-cache)# backup-cached-messages cache-instance <instance-name> [{file <filename>} | cancel]

Where:

<instance-name> is the name of the Solace Cache Instance that you want to backup.

file <filename> is the name (0-255 characters) for the backup file. This file is saved to the directory indicated by the SOLCACHE_MSG_STORE_DIRECTORY parameter in the Solace Cache Instance configuration file (see Customizing Solace Cache Configuration Files). Note that cached data for global topics is not written to the file.

cancel stops a backup operation when it is in progress.

Restoring Cached Messages

When a restore operation is performed for a Solace Cache Instance:

  • any messages that are currently cached by the Solace Cache Instance are deleted
  • the messages saved to the backup file, as well as their original timestamps, are restored to the Solace Cache Instance

The Solace Cache Plug-in is not invoked for restored messages because the messages would have already been processed by the Plug-in when they were originally written to the cache.

When restoring messages from a backup file, messages from the backup file can be discarded if:

  • They do not match the Solace Cache Instance’s configured topic subscriptions.
  • They would cause the max-msgs-per-topic to be exceeded. In this case, the oldest messages for that topic are deleted, to stay within the max‑msgs-per-topic limit.
  • They exceed the Solace Cache Instance’s configured maximum message lifetime (see Configuring Message Lifetimes).

WARN event logs are generated to indicate how many messages were discarded for these reasons during the restore operation.

To restore the given cached messages for a Solace Cache Instance from a backup file, enter the following commands:

solace# admin
solace(admin)# distributed-cache <name> message-vpn <vpn-name>
solace(admin/distributed-cache)# restore-cached-messages cache-instance <instance-name> [{file <filename>} | cancel]

Where:

<name> is the name of an existing Distributed Cache

<vpn-name> is the name of an existing Message VPN

<instance-name> is the name of the Solace Cache Instance that you want to restore

file <filename> is the name (0-255 characters) of the backup cached messages file to use for the restore process

cancel stops a restore operation when it is in progress