Configuring Cache Clusters

To create a Cache Cluster within an existing Distributed Cache, enter the following CONFIG command:

solace(configure/distributed-cache)# create cache-cluster <name>

To edit properties for an existing Cache Cluster, enter the following CONFIG command:

solace(configure/distributed-cache)# cache-cluster <name>

Where:

<name> is the name of the Cache Cluster

By default, a Cache Cluster is not enabled after it is created. To enable a Cache Cluster, refer to Enabling Cache Clusters.

Enabling Deliver-To-One-Overrides

A client may publish a message with a Deliver-To-One (DTO) flag, so that the message may only to be delivered to one subscriber. This may be desirable where there is a fast publisher and only some subscribers need to get the message.

However, by default, Cache Clusters use a DTO override so that every PubSub+ Cache Instance gets every message. This is the recommended configuration for Cache Clusters.

To enable the DTO override for PubSub+ Cache Instances under the Cache Cluster, enter the following CONFIG commands:

solace(configure/distributed-cache)# cache-cluster <name>
solace(...igure/distributed-cache/cache-cluster)# deliver-to-one-override

To disable the DTO override for PubSub+ Cache Instances under the Cache Cluster, enter the following CONFIG commands:

solace(configure/distributed-cache)# cache-cluster <name>
solace(...igure/distributed-cache/cache-cluster)# no deliver-to-one-override

Configuring Event Thresholds

You can configure high and low thresholds at which events are published to the message bus for a number of operating conditions experienced by PubSub+ Cache Instances in the Cache Cluster. For information on configuring these event thresholds, refer to Monitoring PubSub+ Cache.

Configuring Global Caching

The Global Caching feature allows client applications to request messages that are cached on remote Distributed Caches. For information on the Global Caching parameters that you can configure for a Cache Cluster, refer to Configuring Global Caching for a Cache Cluster.

Configuring Max Memory for PubSub+ Cache Instances

A clusterʼs PubSub+ Cache Instances should have an appropriate limit for the maximum amount of memory that they each may consume. For example, the PubSub+ Cache Instance process itself uses 128 MB of memory. Therefore, if an application is required to store, for example, 3000 MB of messages, then 3000 + 128 MB (that is, 3128 MB) must be configured. The default value is 2048 MB.

If the maximum memory limit provided for a PubSub+ Cache Instance is reached, any arriving messages that require more memory are discarded, and the PubSub+ Cache Instance triggers a single discard event to the Designated Router and enters the lost message state.

Cache instances using an Ingress Message Plug-In require additional memory as live messages received during cluster synchronization must be queued for playback through the plug-in. To accommodate the queuing the max-memory configured for instances using an Ingress Message Plug-In should be at least double the memory required to store the cached messages.

To configure the maximum memory consumption each PubSub+ Cache Instance in this Cache Cluster is allowed to consume, enter the following CONFIG command:

solace(configure/distributed-cache)# cache-cluster <name>
solace(...igure/distributed-cache/cache-cluster)# max-memory <megabytes>

Where:

<megabytes> is an integer value representing the maximum memory consumption limit in MB for each PubSub+ Cache Instance in the Cache Cluster. The valid range is 128 through 4294967294.

The no version of this command, no max-memory, resets the maximum memory consumption limit back to default.

  • You can increase or decrease the maximum memory consumption limit without stopping the Cache Cluster or its PubSub+ Cache Instances. If the limit is increased or decreased, the new configuration takes effect immediately. However, decreasing the limit below the current memory utilization does not cause the cache to delete already cached messages.
  • Monitor events are raised and logs generated when the maximum memory is reached. When it is reached:
    • A single syslog is generated and an event published, indicating “Messages Discarded, Max-Memory exceeded”.
    • A status indication is sent with all responses to cache requests, indicating the cache contents may be incomplete.
    • The PubSub+ Cache Instance, Cache Cluster, and Distributed Cache all enter the “lost message” state, and a lost message event is generated. The network operator then needs to correct the situation, and reset the event through CLI or SEMP because there is no way for the cache to know when the condition has been cleared and whether the cache contents have been brought back to a consistent state.
  • Cache instances enter the “lost message” state but do not stop when maximum memory is reached, even if configured as “stop-on-lost-message”. This is because all instances within a cluster typically reach maximum memory at the same time and stopping them all would leave no instances to respond to cache requests. Instead, cache instances at maximum memory continue to respond to cache requests but flag the responses as suspect until the “lost message” state is cleared.

Configuring Max Number of Messages Per Topic

You can configure the maximum number of messages per topic that each PubSub+ Cache Instance in this Cache Cluster may cache. When the number of messages for a topic exceeds this number, the oldest message entry is dropped to make room for the newly arrived message.

To ensure consistency, it is recommended that all Cache Clusters enabled for Global Caching (that is, both home and local Cache Clusters) be configured with the same maximum number of messages per topic.

To configure the maximum number of messages per topic, enter the following CONFIG commands:

solace(configure/distributed-cache)# cache-cluster <name>
solace(...igure/distributed-cache/cache-cluster)# max-messages-per-topic <num-messages>

Where:

<num-messages> is an integer value representing the maximum number of messages any one topic can cache. The valid range is 1 through 100000000. The default value is 1.

The no version of this command, no max-messages-per-topic, resets the maximum number of messages any one topic can cache back to the default.

Configuring Max Number of Topics

You can configure the maximum number of topics each PubSub+ Cache Instance in this Cache Cluster may cache messages for.

To configure the maximum number of topics that can be cached by PubSub+ Cache Instances in the given Cache Cluster, enter the following CONFIG commands:

solace(configure/distributed-cache)# cache-cluster <name>
solace(...igure/distributed-cache/cache-cluster)# max-topics <num-topics>

Where:

<num-topics> is the integer value representing the maximum number of topics a PubSub+ Cache Instance can contain. The default value is 2000000. For information on the valid range, see Operating Limits.

The no version of this command, no max-topics, resets the maximum number of topics back to default settings.

When the maximum number of topics is reached, any further messages arriving for new topics are discarded, and the PubSub+ Cache Instance triggers a single discard event to the Designated Router and enters the lost message state (refer to Lost Message State).

  • Monitor events are raised and logs generated when max topics is exceeded.

    When max topics is exceeded:

    • A single syslog is generated and an event published, indicating “Messages Discarded, Max-Topics exceeded”.
    • A status indication is sent with all responses to cache requests, indicating the cache contents may be suspect.
    • The PubSub+ Cache Instance, Cache Cluster, and Distributed Cache all enter the “lost message” state and a lost message event is generated. The network operator then needs to correct the situation, and reset the event through CLI or SEMP because there is no way for the cache to know when the condition has been cleared and whether the cache contents have been brought back to a consistent state.
    • If the PubSub+ Cache Instance is configured as “stop-on-lost-message” (the default), the PubSub+ Cache Instance is stopped when the lost message event occurs. The network operator then also needs to restart the PubSub+ Cache Instance after having cleared the lost message event.

    • Cache instances enter the “lost message” state but do not stop when maximum topics is exceeded even if configured as “stop-on-lost-message”. This is because all instances within a cluster typically exceed maximum topics at the same time and stopping them all would leave no instances to respond to cache requests. Instead, cache instances that have exceeded maximum topics continue to respond to cache requests but flag the responses as suspect until the “lost message” state is cleared.

Configuring Message Lifetimes

Messages held by PubSub+ Cache Instances belonging to a Cache Cluster have a set lifetime (measured in seconds) after they are cached. Once messages have reached their maximum age, they are deleted from the PubSub+ Cache Instance.

To configure a maximum lifetime for messages to be cached by PubSub+ Cache Instances in the given Cache Cluster, enter the following CONFIG commands:

solace(configure/distributed-cache)# cache-cluster <name>
solace(...igure/distributed-cache/cache-cluster)# message-lifetime <seconds>

Where:

<seconds> is an integer value representing the maximum age of any message in the cache in seconds. The valid range is 3 to 4294967294.

The no version of this command, no message-lifetime, resets the lifetime expiry age for messages held by PubSub+ Cache Instances of this Cache Cluster back to default (that is, no lifetime limit).

Enabling New Topic Advertisements

When the new-topic-advertisement option is enabled, PubSub+ Cache Instances that belong to the Cache Cluster will generate new topic advertisement events when they cache a message for a new topic. These new topic advertisements can be published over the event broker message bus to indicate to subscribing clients that there are new cached messages for that topic of interest.

To configure a Cache Cluster to generate new topic advertisements, enter the following CONFIG commands:

solace(configure/distributed-cache)# cache-cluster <name>
solace(...igure/distributed-cache/cache-cluster)# new-topic-advertisement

The no version of this command, no new-topic-advertisement, disables the generation of new topic advertisements.

Once a message is cached for a topic of interest, no new topic advertisements are generated for that topic until after the PubSub+ Cache Instance no longer contains a message with that topic and then it caches a new message with a matching topic. For information on how all cached messages for a topic can removed from the PubSub+ Cache Instance, refer to Performing PubSub+ Cache Administrative Tasks.

New topic advertisement event messages are published to the following topic on the same Message VPNs that PubSub+ Cache Instances are connected to:

#LOG/NOTICE/VPN/<rtr-name>/VPN_SOLCACHE_NEW_TOPIC/<vpn-name>/<distributed-cache-name>/<fully-qualified-topic>

For a client to receive new topic advertisement messages, the client must subscribe to that topic and event publishing over the message bus must be enabled (refer to Receiving Message Bus Events.

  • A new topic advertisement event is not returned when a local PubSub+ Cache Instance learns of a new global topic because it is the sole responsibility of the home Cache Cluster to return this event for its local topics. This prevents duplicate new topic events for the same topic.
  • When the <fully-qualified-topic> is appended to the topic prefix shown above, and it results in a topic that exceeds the maximum topic length supported by the event brokers, the <fully-qualified-topic> is truncated, so that the maximum supported topic length is not exceeded.

The payload of the new topic advertisement message published on the Solace message bus is a Solace structured data type (SDT) map containing the following fields:

Payload of New Topic Message
Keyed Data Field Data Type Description
topic String The fully qualified name of the cached topic.
distributed-cache-name String The name of the Distributed Cache the PubSub+ Cache Instance belongs to. Client applications that are not using Global Caching can directly query the particular Distributed Cache that has cached the new topic.
cache-cluster String The name of the Cache Cluster the PubSub+ Cache Instance belongs to.
cache-instance String The PubSub+ Cache Instance that learned the new topic.

A client application can get the keyed data from the received new topic advertisement. If the new advertised topic is of interest, the client application can make a cache request for messages that match that topic.

For information on how to work with SDT maps, refer to Using Structured Data for Solace enterprise messaging APIs and Solace Web messaging APIs.

Configuring Request Queue Depths

You can configure the maximum queue depth for the queue responsible for receiving cache requests for each PubSub+ Cache Instance in the Cache Cluster. If this request queue depth is surpassed, any newly arriving requests are discarded until the request queue depth returns to below maximum.

To configure the maximum queue depth, enter the following CONFIG commands:

solace(configure/distributed-cache)# cache-cluster <name>
solace(...igure/distributed-cache/cache-cluster)# request-queue-depth <num-messages>

Where:

<num-messages> is the integer value representing the maximum number of messages in the cache request queue. The valid range is 1 through 200000. The default value is 100000.

The no version of this command, no request-queue-depth, resets the maximum queue depth for the queue responsible for receiving cache requests back to default.

Although local cache requests can support larger request queue depths, when Global Caching is enabled for a Cache Cluster, a request depth of 1 is strongly recommended.

Assigning Topics to Cache Clusters

To assign a topic subscription to the given Cache Cluster, enter the following CONFIG commands. The PubSub+ Cache Instances that belong to the Cache Cluster will then cache any messages published to topics that match that subscription.

solace(configure/distributed-cache)# cache-cluster <name>
solace(...igure/distributed-cache/cache-cluster)# topic <topic-str>

Where:

<topic-str> is the assigned topic string. Refer to Topic Support & Syntax.

Enabling Cache Clusters

By default, a Cache Cluster is not enabled. When a Cache Cluster is enabled, its PubSub+ Cache Instances are not automatically enabled. They must be enabled individually. For a PubSub+ Cache Instance to be in service, it has to be enabled from the PubSub+ Cache Instance context.

To enable a given Cache Cluster, enter the following CONFIG commands:

solace(configure/distributed-cache)# cache-cluster <name>
solace(...igure/distributed-cache/cache-cluster)# no shutdown

To disable a given Cache Cluster, enter the following CONFIG commands:

solace(configure/distributed-cache)# cache-cluster <name>
solace(...igure/distributed-cache/cache-cluster)# shutdown

When a Cache Cluster is shut down, all of the PubSub+ Cache Instances that belong to the Cache Cluster are taken out of service. All topic subscriptions are withdrawn from PubSub+ Cache Instances, and they cease to cache new data and handle cache requests. However, PubSub+ Cache Instances maintain their content during this time (subject to normal message lifetime aging).