Configuring Message Caching

This section provides information on how ingress OpenMAMA messages for non‑book and order book data and dictionaries are handled when the PubSub+ Cache facility is used to provide last-value caching updates for clients. It also provides information on how to configure the SolOpenMAMA Plug-In that is required for OpenMAMA processing and handling.

When the PubSub+ Cache facility is used with the Solace OpenMAMA messaging solution, the following message exchange patterns may occur:

  • A publishing client connected to a specific Message VPN can publish messages with initial values or delta updates to well-known topics (that is, symbols), and the event broker distributes the messages to all the PubSub+ Cache Instances that belong to a Cache Cluster that is assigned a matching topic subscription. PubSub+ Cache Instances are configured to only keep the latest message for a topic rather than multiple messages.
  • A subscribing client connected to the same Message VPN that wants an initial value (with the latest updates applied to it) can send cache requests for specific topics to a Distributed Cache, Cache Cluster, or PubSub+ Cache Instance that belongs to that Message VPN. The event broker message bus load-balances the client cache requests amongst the PubSub+ Cache Instances that cache messages for the requested topic, and a single PubSub+ Cache Instance will return a cache response to the requesting client with a message with the last values cached for the requested topic.

To ensure that the OpenMAMA messages that are cached have the most current values, each PubSub+ Cache Instance must use a Solace-provided PubSub+ Cache Ingress Message Plug-In (solopenmama_plugin). The SolOpenMAMA Plug-In handles non-book, order book, and dictionary messages.

The SolOpenMAMA Plug‑In inspects each received message, and then uses OpenMAMA libraries to properly handle them. For example, a SolOpenMAMA Plug-In stores initial value messages for a topic in its parent PubSub+ Cache Instance and when it receives subsequent update messages, it applies delta values updates to the appropriate fields in the cached message for the topic. If necessary, it can also discard a received message.

The SolOpenMAMA Plug-In uses OpenMAMDA to process updates to order books according to a single dictionary file that is locally installed on the server hosting the PubSub+ Cache Instance.

Message Caching Process

When a PubSub+ Cache Instance receives an ingress OpenMAMA message for a specific topic, the SolOpenMAMA Plug-In gets the message payload, then inspects the message.

Once the message is identified as a valid OpenMAMA message (non-book, order book, data dictionary, or delete), the Plug-In then handles the message in a manner appropriate to its type and instructs the PubSub+ Cache Instance whether to cache the message and what operations to perform for the cached contents for the topic.

The Plug-In performs the following actions for the received message types:

  • initial value message—The Plug-In performs no modifications to the message, and it is written unmodified to the PubSub+ Cache Instance.
  • initial value book message—The Plug-In performs no modifications to the message, and it is written unmodified to the PubSub+ Cache Instance.
  • initial value dictionary message—The Plug-In performs no modifications to the message, and it is written unmodified to the PubSub+ Cache Instance.
  • update message—The Plug-In retrieves the cached message for the symbol from the PubSub+ Cache Instance, then applies the updates from the new message to the appropriate fields. The updated message is then written back to the PubSub+ Cache Instance.
  • update book message—The Plug-In retrieves the cached message for the order book from the PubSub+ Cache Instance, then applies the updates from the new message to the appropriate fields. The updated message is then written back to the PubSub+ Cache Instance.
  • delete message—The Plug-In instructs the PubSub+ Cache Instance to delete the cached message for the given topic, and then discard the received delete message. A delete message can be received for non-book, book, or dictionary data.
  • unexpected message—The Plug-In discards the received message and generates a warn log message.

For more detailed information on how the SolOpenMAMA Plug-In handles specific, enumerated Open MAMA messages, refer to OpenMAMA Message Type Handling.

The SolOpenMAMA Plug-In differs from customer‑defined PubSub+ Cache Ingress Plug-Ins in that the actions to perform for received OpenMAMA messages are predefined by Solace. This ensures that OpenMAMA messages are handled in a standard, predictable manner. (For more detailed information on the operation and configuration of customer‑defined PubSub+ Cache Ingress Plug-Ins, refer to Using Ingress Message Plug-Ins.)

Data Loss Detection

A PubSub+ Cache Instance stores reference copies of OpenMAMA messages for a particular set of topics, but there is no centralized network entity that is able to replay missing data from OpenMAMA messages to the PubSub+ Cache Instance. Therefore, the PubSub+ Cache Plug‑In cannot check for data loss or perform any specific actions in the event of data loss for a symbol.

For information on how PubSub+ Cache handles lost messages, refer to Configuring Stop On Lost Message Behavior.

Configuring the SolOpenMAMA Plug-In

The configuration of a SolOpenMAMA Plug-In is provided through a special mama.properties file used for the SolOpenMAMA Plug-In. This mama.properties file, which differs from the configuration file of the same name used by the Solace Middleware Bridge, is required for each PubSub+ Cache Instance used to handle OpenMAMA messages. As of the SolOpenMAMA 7.5.0 release, the PubSub+ Cache instance handles messages as follows:

  • Non-book messages—The PubSub+ Cache instance can handle both payload version 1 and 2 messages. Update messages are only cached if their payload version matches that of the initial message received for the topic. You can replace an initial message with a new initial of a different payload version, but an update message with a different payload version than the initial will not be cached.
  • Book messages—The PubSub+ Cache instance requires book messages to be published using payload version 2. Book messages using payload version 1 will be dropped by the PubSub+ Cache instance.

Clients using SolOpenMAMA versions prior to 7.5.0 can continue operating, but payload version 1 book messages will be discarded by PubSub+ Cache.

Typically, the configuration file is located in the same directory that the SolOpenMAMA Plug‑In was installed in on the Linux server hosting the PubSub+ Cache Instance. The location of the mama.properties file used for the SolOpenMAMA Plug-In must be specified in the WOMBAT_PATH environment variable.

To modify the configuration of a PubSub+ Cache Instance’s SolOpenMAMA Plug-In, you can use a text editor to edit its mama.properties file. Any changes made to this file only take effect after the SolOpenMAMA Plug-In successfully initializes.

You can perform the following configuration changes for the SolOpenMAMA Plug‑In:

  • Plug-In dictionary file—To configure the dictionary file a SolOpenMAMA Plug‑In uses to handle Book Order messages, edit the mama.solace.cacheplugin.dictionary.filename property of the SolOpenMAMA Plug-In’s mama.properties file. The data dictionary that is specified for a SolOpenMAMA Plug-In must follow the OpenMAMA dictionary file format. A dictionary file may be used for several OpenMAMA publisher sources (specified through the <source> element in the property hierarchy).

    This dictionary file must be saved on the server hosting the PubSub+ Cache Instance. By default, a file named dictionary is used.

    The Plug-In does not monitor changes to the dictionary files. Therefore, if a dictionary is updated while it is in use, the PubSub+ Cache Instance must be restarted for the updates to take effect.

  • handling of updates for non-cached entries—When a SolOpenMAMA Plug-In receives a non-book update message that has no existing cache entry, it can either instruct the PubSub+ Cache Instance to discard the message (the default behavior) or cache the message.

    To configure the handling behavior for update messages for which there are no existing cache entries, edit the mama.solace.cacheplugin.update.
    update_before_initial     property of the mama.properties file. Specify discard to indicate that the PubSub+ Cache Instance should discard the updates, or specify cache to indicate that the PubSub+ Cache Instance should cache them.

  • handling of submessages—When a SolOpenMAMA Plug-In receives a non-book update message that has SubMsg fields (mama field type: MAMAM_FIELD_TYPE_MSG) data, it can either:
    • replace the existing values—The value of a submsg field in the cached message is replaced with the value of the corresponding submsg field in the received message. This is the default behavior.
    • merge the values—The value of a submsg field in the received message is merged with value of the submsg fields in the cached message. When a merge is used, if a submsg itself contains a submsg field, the merge is recursively applied to submsg field that are at the same hierarchy level between the cached message and the update message.

      To configure the handling of non-book update messages that have SubMsg fields, edit the mama.solace.cacheplugin.submsg_field_alg property of the mama.properties file. Specify replace (the default value) or merge, to indicate the handling action you want the SolOpenMAMA Plug-In to perform.

Tuning Message Caching Performance

PubSub+ Cache and the SolOpenMAMA Plug-In use fixed size buffers to store incoming messages. To optimize performance and memory utilization, the buffer sizes must be tuned in each PubSub+ Cache Instances’ configuration file to match your application’s expected message sizes. For example, the following lines added to a PubSub+ Cache Instance configuration file set the buffer sizes to 4096, 8192, 16384, 32768, and 65536 bytes.

# Data buffer sizes
GLOBAL_DBQUANTA_SIZE_0 4096
GLOBAL_DBQUANTA_SIZE_1 8192
GLOBAL_DBQUANTA_SIZE_2 16384
GLOBAL_DBQUANTA_SIZE_3 32768
GLOBAL_DBQUANTA_SIZE_4 65536

For best performance in terms of CPU utilization, GLOBAL_DBQUANTA_SIZE_4 must be larger than most of the expected messages. For example, if 95% of the messages are smaller than 65536 bytes, GLOBAL_DBQUANTA_SIZE_4 should be set to 65536.

Message processing for messages larger than GLOBAL_DBQUANTA_SIZE_4 is more CPU-intensive, but when these large messages are freed the associated memory is guaranteed to be freed back to the operating system for use by other processes.

A PubSub+ Cache Instance process must be restarted for changes to its buffer sizes to take effect.

PubSub+ Cache Logging

Each PubSub+ Cache Instance generates Message VPN-level events that are published over the event broker message bus. The Cache Manager on the Designated Router listens to these cache events and consolidates them with the event broker’s own events, and then these are redirected to the OpenMAMA logging facility. For more information on how SolOpenMAMA logging, refer to Logging.

Any MAMA logs generated by a SolOpenMAMA Plug-In are included in the PubSub+ Cache Instance’s PubSub+ Cache logs. To differentiate them from other PubSub+ Cache logs, they are prefixed as Mama Logs.

For a complete list of PubSub+ Cache events, refer to Syslog Event Reference, which lists and describes Solace PubSub+ syslog messages related to system-wide, Message VPN, and local client Solace PubSub+ events.

For more information on PubSub+ Cache eventing, refer to Monitoring PubSub+ Cache.