Configuring VPN-Level Replication Settings

By default, the use of the replication facility is not enabled for a Message VPN. To use the replication facility, replication must be enabled for the Message VPNs at both sites.

When you are using redundant event brokers at a site, configure the replication settings for one Message VPN, and then they will be automatically propagated to its mate. (Replication bridge configuration settings are an exception—they are not automatically propagated between replication sites.)

Configuring ACK Propagation Intervals

After a client consumes a message from a durable endpoint, an acknowledgment (that is, an ‘ACK’) is generated so that the message is removed from the message spool. (An ACK is also generated if a message is deleted from the message spool.)

By default, for a Message VPN with an active replication state, after clients consume a set number of messages from its endpoints, an ACK message is propagated to its mate Message VPN with a standby replication state. When this ACK message is received by the corresponding durable endpoints on the mate replication Message VPN, those endpoints can then also consume those same messages that were replicated to it.

If the messages interval is not reached, a one-second periodic timer ensures that acks are propagated in a timely manner. This ACK propagation interval only affects the bandwidth utilization on the replication bridge and likely will not need to be adjusted.

To set an interval for propagating ACK messages, enter the following CONFIG commands:

solace(configure)# message-vpn <vpn-name>
solace(configure/message-vpn)# replication
solace(configure/message-vpn/replication)# ack-propagation
solace(...ssage-vpn/replication/ack-propagation)# interval messages [messages]

Where:

  • messages is the number of consumed messages that must be acknowledged before an ACK message is propagated to the same Message VPN with a replication standby state on the mate site. The default value is 20.

The no version of this command, no messages, resets the ack-propagation interval back to the default value.

By default, endpoints in an active Replicated Message VPN propagate consumer acks to the standby Replicated Message VPN; however, ACK propagation can be enabled/disabled on a per-endpoint basis (for queues, see Enabling / Disabling Propagating Consumer Acks to Replicated VPNs; for topic endpoints, see Enabling/Disabling Propagating Consumer Acks to Replicated VPNs).

Configuring Replication Bridges

A replication bridge is used to connect a Message VPN with an active replication state and a Message VPN of the same name with a standby replication state at the mate site. When replication is enabled for a Message VPN, a replication bridge named #MSGVPN_REPLICATION_BRIDGE is automatically created.

Config-Sync will not automatically synchronize this object or property. Therefore, if the event broker is being used in a high-availability (HA) redundant configuration or in a replicated site, you must manually configure this object/property on each mate event broker or replicated Message VPN.

To determine whether an object/property is synchronized by Config-Sync, look up the command used to configure the object/property in the CLI Command Reference or type the command in the Solace CLI, ending the command with "?". The Help lists whether the object/property is synchronized.

Any uni-directional client profiles that are used for the replication bridges are excepted—these are synchronized between the replication mates.

You can perform the following configurations for the replication bridge for the given Message VPN:

Configuring Authentication Schemes

Authentication can be set up on a replication bridge to provide a secure connection between replication sites. Basic and client certificate authentication modes are available. Where Basic uses basic username and password authentication for replication bridge connections, and Client Certificate uses client certificate authentication for replication bridge connections.

In an authentication scenario, the replication-standby Message VPN acts as the client and the replication-active Message VPN acts as the authenticating server.

Configuring a Client Username for Basic Authentication

To set a client username to use for basic replication bridge authentication, enter the following CONFIG commands:

solace(configure/message-vpn/replication)# bridge
solace(configure/message-vpn/replication/bridge)# authentication basic
solace(...plication/bridge/authentication/basic)# client-username <name> [password <password>]

Where:

  • <name> is the client username for the bridge to use to log in to the replication standby Message VPN. The username may contain up to 189 characters.
  • <password> is an optional password that will be also required for authentication if set. The password may contain up to 128 characters.

The no version of this command, no client username, removes the client username and any password associated with it.

  • A client username can only be configured for the replication bridge when the replication bridge is shutdown.
  • A client username used to establish a replication bridge connection must be given permission to create bridge connections (see Allowing Bridge Connections).

Configuring a Certificate File for Client Certificate Authentication

To set the certificate file to use for client certificate replication bridge authentication, enter the following CONFIG commands:

solace(configure/message-vpn/replication)# bridge
solace(configure/message-vpn/replication/bridge)# authentication client-certificate
solace(...dge/authentication/client-certificate)# certificate-file <filename>

Where:

  • <filename> is the file name of the certificate to use for authentication. The certificate must be located in the /usr/sw/jail/certs directory of the event broker and have an extension of .pem. Once a certificate is configured, a copy of it is saved internally. The file in the certs directory is no longer required.

The no version of this command, no certificate file, disassociates the certificate file from the current replication bridge.

For authentication to succeed, the common name provided in the certificate file must exist as a client username on the remote Message VPN. The client username must be enabled, and the client profile that is used by that client username must have allow-bridge-connections enabled. The provided client certificate must also be enabled on the remote Message VPN and adhere to the specified client certificate authentication settings on the remote Message VPN.

The certificate file can only be configured while the replication bridge is shutdown.

If no client certificate file is configured for the replication bridge, the bridge will present the event broker’s server certificate for authentication.

Enabling the Use of Compressed Data

Compression can be enabled for a replication bridge’s data connections to make better use of available WAN bandwidth. The default setting is no compression.

To enable compression, enter the following CONFIG command:

solace(configure/message-vpn/replication/bridge)# compressed-data

The no version of this command, no compressed-data, disables the use of compression.

Before configuring compression for a replication bridge while a Message VPN is in the standby state, replication on the Message VPN must be disabled (see Enabling Replication).

Configuring Message Spool Window Sizes

The default value for a replication bridge's message spool window size is expected to work well in most typical deployments. Configuring an excessively large message spool window size can negatively impact network performance. Before changing this parameter, contact Solace for assistance in choosing the appropriate value for your network conditions.

The message spool window size dictates how many outstanding configuration messages may be sent over a replication bridge before the Message VPN with an active replication state must receive an acknowledgment.

To configure the message spool window size, enter the following CONFIG commands:

solace(configure/message-vpn/replication/bridge)# message-spool
solace(...-vpn/replication/bridge/message-spool)# window-size <number>

Where:

  • <number> is an integer from 1 to 65536 that sets how many messages can be in the window. The default is 255.

The no version of this command, no message-spool window-size, resets the message spool window size value back to the default value.

Before configuring the message spool window size, the replication bridge connection must be disabled by shutting down the replication feature for the Message VPN (see Enabling Replication).

Configuring Retry Delays

To configure the number of seconds that must pass before retrying a connection to the replication mate over the replication bridge, enter the following CONFIG commands:

solace(configure/message-vpn/replication/bridge)# retry-delay <seconds>
Where:
  • <seconds> is an integer from 1 to 255 that specifies the number of seconds to wait before retrying a connection to the replication mate. The default is 3.

The no version of this command, no retry-delay, resets the number of seconds to wait back to the default value.

Before configuring a retry-delay value, the replication bridge connection must be disabled by shutting down the replication feature for the Message VPN (see Enabling Replication).

Enabling TLS/SSL Encryption for Bridge Connections

TLS/SSL encryption can be enabled on replication bridges.

To enable, enter the following CONFIG command:

solace(configure/message-vpn/replication/bridge)# ssl

To disable, enter the following CONFIG command:

solace(configure/message-vpn/replication/bridge)# no ssl

Setting Client Profiles for Uni-Directional Bridges

By default, when the bridge connection from the Message VPN with an active replication state is established to the Message VPN of the same name with a standby replication state at the mate site, the TCP parameters from a non‑editable, default client profile, #client-profile, are used (this default client profile has a TCP max window size of 2 MB). However, if the TCP parameters used by the #client‑profile client profile are not suitable for your network, you can use those from another client profile on the Message VPN with an active replication state.

To specify a client profile that will provide the TCP parameters to use for the uni‑directional bridge link, enter the following CONFIG commands:

solace(configure/message-vpn/replication/bridge)# unidirectional
solace(...vpn/replication/bridge/unidirectional)# client‑profile <name>

Where:

  • <name> is the name of a client profile that exists on the Message VPN with an active replication state. A client profile name can contain up to 32 alphanumeric characters (case‑sensitive).

The no version of this command, no client-profile, removes the current client profile assignment, and reassigns the default client profile, #client-profile.

Configuring Replication Queues

When replication is enabled for a Message VPN, a replication queue (named #MSGVPN_REPLICATION_DATA_QUEUE) is automatically created. Messages are spooled here before they are replicated on the replication mate.

You can set the following replication queue configuration parameters:

Max Spool Usage

To configure the maximum message spool disk space that can be used for the replication queue for the Message VPN, enter the following CONFIG commands:

solace(configure/message-vpn/replication/queue)#
solace(configure/message-vpn/replication/queue)# max-spool-usage <size>

Where:

  • <size> is an integer value specifying the maximum amount of message spool disk space permitted for the replication queue in MB.

The no version of this command, no max-spool-usage, resets the permitted message spool usage quota available for use by the queue back to the default value.

Reject Message To Sender on Discard

The reject-msg-to-sender-on-discard option sets whether the replication queue will provide a client with a negative acknowledgment (that is, a “nack”) when published Guaranteed messages are not enqueued and are discarded. When using replicated transactions, the client will also get a negative acknowledgment if the transaction cannot be put on the replication queue. By default, the reject-msg-to-sender-on-discard option is enabled.

The reject-msg-to-sender-on-discard option can be configured on a Message VPN for a replication queue. Normally, this value should not be disabled, because disabling it will allow Guaranteed messages or transactions that cannot be replicated to be accepted. However, if the replication queue is full because of a long term failure of the standby site, this option can be disabled so that non-replicated service can be provided.

To configure how the replication queue should behave when Guaranteed messages and transactions are not enqueued and are discarded, enter the following CONFIG commands:

solace(configure/message-vpn/replication/queue)#
solace(configure/message-vpn/replication/queue)# reject-msg-to-sender-on-discard

Where:

  • reject-msg-to-sender-on-discard enables the reject-msg-to-sender-on-discard option for the queue so that session events (that is, nacks) are sent to the publishing clients whenever messages or transactions are not enqueued and are discarded. These messages are not allowed to be enqueued on any other endpoints.

The no version of this command, no reject-msg-to-sender-on-discard, disables the reject-msg-to-sender-on-discard option for the queue so that no session events are returned to the publishing client whenever messages or transactions are not enqueued on the replication queue and are discarded.

Configuring to Reject Messages When Sync Ineligible

If a Message VPN with an active replication state is using synchronous replication when propagating replicated messages and transactions to its mate Message VPN with a standby replication state, and the replication bridge is very slow or goes down, the Message VPN can either switch to an asynchronous replication mode (this is the default action), or the synchronous replication mode can be maintained and error indications (that is, nacks) are sent back to clients publishing to replicated messages and transactions when they cannot be propagated to the standby Message VPN.

To configure the given Message VPN so that its synchronous replication mode should be maintained and nacks should be sent back to the publishing clients, enter the following CONFIG command on one event broker at the active site:

solace(configure/message-vpn/replication)# reject-msg-when-sync-ineligible

The no version of this command, no reject-msg-when-sync-ineligible, configures the given Message VPN to switch to asynchronous replication when there are problems with the replication bridge. This is the default behavior.

  • When configured to reject messages when sync ineligible and the replication service is degraded (that is, the Message VPN is sync ineligible), then new synchronous XA transactions will fail to start.
  • These commands may be executed when clients are connected and publishing messages and whether replication is enabled or not.

Configuring Replicated Topic Subscriptions

To configure a topic pattern to use for replicating messages to the corresponding Message VPN on the mate site, enter the following CONFIG command on one event broker at the active site:

solace(configure/message-vpn/replication)# create replicated-topic <topic-pattern>

Where:

  • topic pattern is the topic pattern to use for replicating messages to the corresponding Message VPN on the replication mate. Typically, a topic pattern is a topic subscription. In this case, only messages whose topics match the configured replicated topic subscription are copied to the corresponding Message VPN on the mate replication site. To enable the replication of messages delivered to queues, the topic pattern may also represent a queue. For example, a topic pattern for a queue could be “#P2P/QUE/<queueName>”.

You can also specify a topic subscription exception by adding a leading "!" character to the topic subscription. In this case, messages whose topics match the configured topic subscription exception are not copied to the corresponding Message VPN on the replication site. For example, if you add a topic subscription "animals/f*" and a topic subscription exception "!animals/fox", messages published to "animals/frog" are copied to the replication site; however, messages published to "animals/fox" are not. For more information about subscription exceptions, see System-Level Subscription Exception Configuration.

The no version of this command, no replicated-topic, removes the specified replicated topic pattern.

  • The topic pattern can include the * and > wildcard characters.
  • Since Direct messages that match a replicated topic will be promoted to Guaranteed messages, it's recommended to avoid the use of wildcards that can match Direct messages to prevent unnecessary message promotion.
  • Messages that match the replicated-topic are replicated either synchronously or asynchronously, according to the configured replication mode (see Replication Mode below). The replication mode is specified on a per-replicated topic-basis.
  • When using local and XA transactions, published messages in the transaction must match the topic pattern to be replicated. If a transaction contains some messages that match a replicated topic and some that do not, only those matching messages are propagated to the replication mate as part of the transaction.

Replication Mode

To set the replication mode for a replicated topic subscription, enter the following CONFIG commands:

solace(configure/message-vpn/replication)# create replicated-topic <topic-pattern>
solace(...sage-vpn/replication/replicated-topic)# replication-mode {sync | async}

Where:

  • sync is a synchronous replication mode. When the synchronous replication mode is used, a message with a topic that matches a replicated topic subscription is not acknowledged by the event broker until the message is replicated to the mate replication site. After the message is spooled locally, message replication and publisher acknowledgment are serialized.
  • async is an asynchronous replication mode. When the asynchronous replication mode is used, a message with a topic that matches a replicated topic subscription is not acknowledged by the event broker until the message is spooled locally. After the message is spooled, message replication and publisher acknowledgment occur asynchronously. Async is the default replication mode.

The no version of this command, no replication-mode, sets the replication mode to the default value.

  • When using transactions, the replication mode of the replicated topic subscription is ignored. The replication mode of transactions is determined by the Message VPN setting. See Replication Mode for more information.
  • Topic subscription exceptions apply only to subscriptions using the same replication mode. For example, if you configure an asynchronous topic subscription "animals/f*" and a synchronous topic subscription exception "!animals/fox" messages published to "animals/fox" are received on the mate replication site.

Displaying Replicated Topic Subscriptions

To show the configured replicated topic subscriptions for the entire system, or on a per‑Message VPN-basis, enter the following User EXEC command:

solace> show replicated-topic <topic> [message-vpn <vpn-name>] [replication-mode {sync | async}] [count <num-elements>]

Where:

  • <topic> is a string representation of the replicated topic pattern (that is, either a topic subscription or a queue name). The topic pattern can include CLI wildcard constructs such as * and >.
  • message-vpn <vpn-name> is a Message VPN name. The Message VPN name can contain from 1 to 32 characters and may contain wildcards * or ?.
  • replication-mode filters the replicated topic subscriptions based on replication mode.
  • sync asks to list only topic subscriptions that are configured to use a synchronous replication mode.
  • async asks to list only topic subscriptions that are configured to use an asynchronous replication mode.
  • count specifies the number of display elements to show. <num-elements> is the integer specifying the number of elements. Valid values are 1 through 4294967295.
solace> show replicated-topic * message-vpn vpn-1 replication-mode async 

Message VPN: vpn-1
Replication
Topic                                                                     Mode
------------------------------------------------------------------------------
t-1-1-1                                                                  async
t-1-1-2                                                                  async
t-1-1-3                                                                  async

Configuring Transaction Replication Mode

The transaction replication mode controls whether transactions are replicated synchronously or asynchronously. Synchronous transactions wait for the transaction to complete on the standby site, while asynchronous transactions only wait of the transaction to be stored on the replication queue of the active site. Asynchronous transactions provide better performance, but they allow for message duplication in some failover scenarios. It is also possible to lose messages or transactions in replication queue in the event of non-recoverable failure of the active site.

To configure the transaction replication mode of the Message VPN, enter the following CONFIG command in the Solace CLI:

solace(configure/message-vpn/replication)# transaction-replication-mode {async| sync}

Where:

  • async sets the transaction replication mode of the Message VPN to asynchronous (the default state).
  • sync sets the transaction replication mode of the Message VPN to synchronous.

The no version of this command sets the mode to async.

  • Changes to this setting do not affect in-progress transaction. Those transaction will complete using their previous replication mode. Newly started transaction will use the new setting.
  • The replication mode of the replicated topic used to match the messages is ignored when using transactions. For example, messages published to an asynchronous topic inside a synchronous transaction use synchronous handling.

Configuring VPN Replication States

To configure the replication state of the Message VPN, enter the following CONFIG command:

solace(configure/message-vpn/replication)# state {standby|active}

Where:

  • standby sets the replication state of the Message VPN to standby (the default state).
  • active sets the replication state of the Message VPN to active.
  • When this command is executed while replication is shutdown, it sets the replication state that is to be used when replication is enabled. When this command is executed while replication is enabled, it changes the replication state on the fly.
  • When the replication state transitions from active to standby, all clients except #MSGVPN_REPLICATION_BRIDGE and those with allow-clients-when-standby set in their client profile are disconnected from the event broker. After the clients are disconnected, those clients are not allowed to reconnect while the VPN remains in standby state.
  • When the replication state transitions from active to standby, all Active and Idle XA transactions are canceled since they have not reached a state where they need to be persisted. An application server transaction manager will properly recover these transactions.

Enabling Replication

By default, replication is disabled on a Message VPN.

Config-Sync will not automatically synchronize this object or property. Therefore, if the event broker is being used in a high-availability (HA) redundant configuration or in a replicated site, you must manually configure this object/property on each mate event broker or replicated Message VPN.

To determine whether an object/property is synchronized by Config-Sync, look up the command used to configure the object/property in the CLI Command Reference or type the command in the Solace CLI, ending the command with "?". The Help lists whether the object/property is synchronized.

  • To enable the replication feature for the Message VPN, enter the following CONFIG command: 
    solace(configure/message-vpn/replication)# no shutdown [force-use-existing-queue | force-recreate-queue]

    Where:

    • force‑use‑existing‑queue preserves any messages on a pre-existing replication queue, and the existing replication queue is reused. Before using this option ensure that the messages are not old or otherwise unsuitable to be propagated; this option can only be specified when there is a pre-existing replication queue.
    • force-recreate-queue causes any messages on a pre-existing replication queue to be discarded and then the replication queue is recreated. Note that this option can only be specified when there is a preexisting replication queue.

    Typically the no shutdown command can be used without any further arguments—it enables the replication facility for the Message VPN and creates a new bridge and replication queue objects. However, if you encounter a situation where, even though replication was disabled, a previous replication queue still exists, you can use the force options with the command to handle the preexisting replication queue and any messages spooled to it.

  • To disable replication for a Message VPN enter the following CONFIG command for the given Message VPN: 
    solace(configure/message-vpn/replication)# shutdown

Disabling replication for a Message VPN deletes its associated replication bridge and replication queue objects, which can also delete any enqueued messages. Therefore, it is recommended that you ensure that there are no messages enqueued on a replication Queue before shutting down replication.