Using System Scaling Parameters

You can increase certain system scaling limits using system scaling parameters. You can specify system scaling parameters using configuration keys when you create a software broker instance (see Setting Scaling Parameters Using Configuration Keys). You can also change system scaling parameters while the software broker is running using the Solace CLI (see Scaling Up an Existing Single Software Event Broker and Scaling Up an Existing HA Group).

For details about the minimum system resources required, see System Resource Requirements.

To see the current values of the system scaling parameters, run the show system command in the Solace CLI, and look at the Scaling section of the output:

solace> show system

System Uptime: 1d 0h 45m 21s
Last Restart Reason: User request

Scaling:
   Max Bridges: 25
   Max Connections: 100
   Max Queue Messages: 100M
   Max Kafka Bridges: 0
   Max Kafka Broker Connections: 0
   Max Subscriptions: 500000

Topic Routing:
   Subscription Exceptions: Enabled
   Subscription Exceptions Defer: Enabled

You can configure the following system scaling parameters:

• Maximum Number of Client Connections
• Maximum Number of Queue Messages
• Maximum Number of VPN Bridges
• Maximum Number of Subscriptions
• Maximum Number of Kafka Bridges
• Maximum Number of Kafka Broker Connections

Increasing a scaling parameters causes an immediate event broker restart.

If you increase the value of these system scaling parameters, the event broker will require additional resources. For more information, see Additional System Resources.

Maximum Number of Client Connections

You can increase the number of concurrent client connections supported by the event broker using the max-connections system scaling parameter. By default, the event broker is limited to 100 concurrent client connections, but it can be provisioned for up to 200,000 connections assuming that sufficient system resources are provided.

To scale parameters independent of max-connections, you must upgrade your PubSub+ event broker to version 10.7.1 or later. See Event Broker Releases for release timelines of individual scaling parameters.

PubSub+ Broker	Default Concurrent Client Connections	Maximum Possible Concurrent Client Connections
PubSub+ Standard	100	1,000
PubSub+ Enterprise	100	200,000

For default maximum client connections values, see maxconnectioncount in Configuration Keys.

For PubSub+ Standard Edition, if you set the max-connections system scaling parameter higher than the maximum supported, the broker will fail to start.

The number of current client connections cannot be changed on a broker configured as a monitoring node in an HA group.

A number of related system limits (for example, the maximum number of client usernames) are automatically scaled to values appropriate for the permitted number of client connections; see the System Limits and Alerts spreadsheet (available from the Solace Products site for Appliance and PubSub+ Enterprise customers) for more information regarding those system limits.

The following types of clients use connections that count against the total number of max-connections:

• Messaging clients (SMF, MQTT, AMQP, or REST): 1 connection each

• Internal clients (VPN, VPN Bridge, Kafka Bridge Sender or Receiver, or DMR link per VPN): 1 connection each

Setting the Number of Concurrent Open Files for Container Images

To ensure reliable operation of the event broker, you must also configure the maximum number of concurrent open files per process when the maximum number of client connections is increased. The maximum number of concurrent open files for the container processes is governed by the nofiles resource limit. Most container runtimes support setting rlimit values using the ulimit parameter when creating a container. For example, in Docker Engine and Podman add the --ulimit nofile=4096:1048576 command line parameter when you create the container. The System Resource Calculator recommends values for these limits.

The event broker will fail to start if the hard limit is less than 1048576. For ease of use, we recommend using 4096:1048576 or greater for all event broker instances.

Maximum Number of Queue Messages

You can increase the number of queue messages (references to messages queued for delivery to consumers) supported by the event broker using the max-queue-messages system scaling parameter.

For PubSub+ event broker versions 10.8.1 and later, for new broker instances, the default value for the maximum number of queue messages is governed by the value of the max-connections system scaling parameter:

• When max-connections is 100, the default value for the maximum number of queue messages is 100 million.

• When max-connections is 1K or higher, the default value for the maximum number of queue messages is 240 million.

PubSub+ Broker	Maximum Possible Queue Messages
PubSub+ Standard	240,000,000
PubSub+ Enterprise	3,000,000,000

For default maximum queue messages values, see maxqueuemessagecount in Configuration Keys.

Running an event broker with 100M queue messages is not recommended in a production environment as it may result in performance issues.

For PubSub+ Standard Edition, if you set the max-queue-messages system scaling parameter higher than the maximum supported, the broker will fail to start.

The maximum number of queue messages cannot be changed on a broker configured as a monitoring node in an HA group.

Maximum Number of VPN Bridges

You can increase the number of VPN bridges for a single event broker using the max-bridges system scaling parameter. A VPN bridge consumes a client connection. Therefore, don't configure max-bridges values higher than max-connections.

This independent scaling parameter is available for PubSub+ event broker versions 10.7.1 and later.

The value of the maximum number of VPN bridge subscriptions is: 

• 100,000 when the max-bridges value is 5000.

• 20,000 when the max-bridges value is 25 or 500.

PubSub+ Broker	Maximum Possible VPN Bridges
PubSub+ Standard	500
PubSub+ Enterprise	5,000

For default maximum VPN bridges values, see maxbridgecount in Configuration Keys.

Because max-bridges is dependent on max-connections, if you for example increase max-bridges from 25 to 5000 and max-connections remains at 100, not all 5000 VPN bridges may be allocated. The Solace CLI shows a warning in such a scenario.

connect-via is limited to 500 outbound VPN bridge connections. This means that in an event mesh, the configuration of 5000 VPN bridges must be done as an inbound connection to the core event brokers.

Maximum Number of Subscriptions

You can increase the number of subscriptions for a single event broker using the max-subscriptions system scaling parameter. This helps to achieve flexibility to deploy different sizes of the event broker within your event mesh without worrying about exceeding the subscription limits of the smaller event brokers in your event mesh . Note that you must allocate additional RAM to the smaller event brokers to accommodate the larger subscription tables. Each client requires at least one subscription. Therefore, don't configure max-connections values higher than max-subscriptions.

This independent scaling parameter is available for PubSub+ event broker versions 10.8.1 and later.

PubSub+ Broker	Maximum Possible Subscriptions
PubSub+ Standard	500,000
PubSub+ Enterprise	5,000,000

For default maximum subscriptions values, see maxsubscriptioncount in Configuration Keys.

Because max-connections is dependent on max-subscriptions, if your max-connections is for example at 100 and you increase your max-subscriptions from 50,000 to 500,000, not all 500,000 subscriptions may be allocated. The Solace CLI shows a warning in such a scenario.

For PubSub+ event broker versions earlier than 10.8.1, all event brokers in the same network must have the same value for max-subscriptions; otherwise, subscription resource limits might be exceeded on the event brokers, and message forwarding will not be reliable.

Maximum Number of Kafka Bridges

You can increase the number of Kafka bridges (Kafka senders + Kafka receivers) supported by the event broker using the max-kafka-bridges system scaling parameter. By default, the event broker is limited to 0 bridges, but it can be provisioned for up to 200 bridges assuming that sufficient system resources are provided. In addition, keep in mind that you should not set the max-kafka-bridges value to greater than the max-connections value set for the system.

PubSub+ Broker	Maximum Possible Kafka Bridges
PubSub+ Standard or Enterprise	200

For default maximum subscriptions values, see maxkafkabridgecount in Configuration Keys.

The maximum number of Kafka bridges cannot be changed on a broker configured as a monitoring node in an HA group.

Maximum Number of Kafka Broker Connections

You can increase the number of concurrent Kafka broker connections supported by the event broker using the max-kafka-broker-connections system scaling parameter. By default, the event broker is limited to 0 concurrent Kafka broker connections, but it can be provisioned for up to 10000 connections assuming that sufficient system resources are provided.

You can expect each Kafka sender and receiver to require approximately one connection from this budget for every Kafka broker in the Kafka cluster that they are configured to send messages to or receive messages from. If different senders and receivers connect to the same Kafka cluster these connections are all independent of one another and count separately against this limit. You should, in such cases, use a common sender or receiver with more queue or topic bindings to aid in scaling. The number of bindings does not affect the number of required connections.

When deciding on the maximum number of concurrent Kafka broker connections your deployment will support, you must ensure that enough process IDs (PIDs) are available to your platform. Kafka bridging requires that 1000 + the value you configure for max-kafka-broker-connections PIDs are available. For more information about setting per Node and per Pod PIDs limits in Kubernetes, see Process ID Limits And Reservations.

PubSub+ Broker	Maximum Possible Concurrent Kafka Broker Connections
PubSub+ Standard or Enterprise	10,000

For default maximum subscriptions values, see maxkafkabrokerconnectioncount in Configuration Keys.

The maximum number of Kafka broker connections cannot be changed on a broker configured as a monitoring node in an HA group.

If the event broker exceeds the value configured for the maximum number of Kafka broker connections, Kafka senders and receivers on the event broker may become operationally down with failure reasons such as Too many other broker connections in use or Too many broker connections required. To prevent this, you can monitor the current number of Kafka broker connections using the show kafka and show message-vpn <vpn-name> kafka CLI commands (see Monitoring Kafka Bridges). In addition, the event broker generates SYSTEM_KAFKA_BROKER_CONNECTIONS_HIGH and VPN_KAFKA_BROKER_CONNECTIONS_HIGH events when it approaches the maximum value. For more information, see Solace PubSub+ Syslog Events.

Multiple Broker Considerations

If you have multiple event brokers in an event mesh, you must be aware of the following considerations when increasing the scaling parameters: 

• Except for HA group setups, the values of the system scaling parameters can be arbitrarily different on each software event broker in the event mesh.

• For max-subscriptions specifically, because the allocation of SMRP can be unpredictable, if you increase to a higher subscription limit on one event broker, you must update the other event brokers in the event mesh to the same value as soon as possible.

Additional System Resources

The resource requirements for the minimum (default) values of the system scaling parameters are listed in System Resource Requirements.

If you increase the value of any system scaling parameters, the event broker will require additional resources. To see how the system resource requirements change with higher values of system scaling parameters, use the System Resource Calculator.