SolOS-TR Message Bus Management Syslog Events

System-wide SolOS syslog Events

Guaranteed Messaging syslog Messages

SYSTEM_AD_DELIVERED_UNACKED_MSGS_EXCEED

Description

This event is sent when the number of delivered but not yet acknowledged guaranteed messages exceed the maximum available for the message broker.

The maximum value depends on the model and cannot be modified. Refer to the 'SolOS System Limits and Alerts' section for the maximum value.

To view the current value that the threshold monitors, see the 'Delivered Unacked Msgs Utilization' field of 'show message-spool detail' CLI command output. The current value is expressed as a percentage of the maximum value.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

Identify guaranteed messaging flows with large number of unacknowledged messages with the 'show queue * stats' and 'show topic-endpoint * stats' CLI commands, and disconnect the associated clients.

Message Format

SYSTEM_AD_DELIVERED_UNACKED_MSGS_EXCEED: - - Delivered unacked limit <limit> exceeded

SYSTEM_AD_DELIVERED_UNACKED_MSGS_HIGH

Description

This event is sent when the number of delivered but not yet acknowledged Guaranteed messages rises above the set threshold.

The threshold value can be changed under 'configure hardware message-spool event delivered-unacked'. It defaults to 80%.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

See the 'Recommended Actions' of the SYSTEM_AD_DELIVERED_UNACKED_MSGS_EXCEED event.

Message Format

SYSTEM_AD_DELIVERED_UNACKED_MSGS_HIGH: - - Delivered unacked threshold <thresholdPercentage>% reached: <currentUnackedMsgsPercentage>%

SYSTEM_AD_DELIVERED_UNACKED_MSGS_HIGH_CLEAR

Description

This event is sent when the number of delivered but not yet acknowledged Guaranteed messages falls below the clear threshold.

The threshold value can be changed under configure hardware message-spool event delivered-unacked. It defaults to 60%.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

Description

This event is sent during start-up, when the flash is deemed unable to be relied upon to perform a successful save.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

ERROR

Recommended Action

Copy the message text exactly as it appears on the console or in the event log, contact your Solace technical support representative, and provide the representative with the gathered information.

Message Format

SYSTEM_AD_FLASH_FAILED: - - ADB flash failed

SYSTEM_AD_INGRESS_FLOWS_HIGH

Description

This event is sent when the number of ingress flows on the message broker reaches or exceeds the set threshold value.

To set the threshold value associated with this event, use the 'hardware message-spool event ingress-flows thresholds 'set-value/set-percentage' Config CLI command. The default is 80% of the maximum value (see the SYSTEM_AD_MAX_INGRESS_FLOWS_EXCEEDED event).

To view the threshold value, see the 'Ingress Flows - Set Value' field of the 'show message-spool detail' CLI command output.

To view the current value that the threshold monitors, see the 'Total Ingress Flows - Currently Used' field of the 'show message-spool detail' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

The thresholds can be increased if the large number of ingress flows is expected; otherwise, see the 'Recommended Actions' of the SYSTEM_AD_MAX_INGRESS_FLOWS_EXCEEDED event.

Message Format

SYSTEM_AD_INGRESS_FLOWS_HIGH: - - Ingress flows threshold <threshold>(<thresholdPercentage>%) reached: <currentIngressFlows>

SYSTEM_AD_INGRESS_FLOWS_HIGH_CLEAR

Description

This event is sent when the number of ingress flows on the message broker falls below the clear threshold value.

To set the threshold value associated with this event, use the 'hardware message-spool event ingress-flows thresholds 'clear-value/clear-percentage' Config CLI command. The default is 60% of the maximum value (see the SYSTEM_AD_MAX_INGRESS_FLOWS_EXCEEDED event).

To view the threshold value, see the 'Ingress Flows - Clear Value' field of the 'show message-spool detail' CLI command output.

To view the current value that the threshold monitors, see the 'Total Ingress Flows - Currently Used' field of the 'show message-spool detail' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

SYSTEM_AD_INGRESS_FLOWS_HIGH_CLEAR: - - Ingress flows threshold <threshold>(<thresholdPercentage>%) cleared: <currentIngressFlows>

SYSTEM_AD_MAX_EGRESS_FLOWS_EXCEEDED

Description

This event is sent when the number of egress flows exceeds the maximum available for the message broker.

The maximum value is fixed at 16000 and cannot be modified.

To view the current value that the threshold monitors, see the 'Total Egress Flows - Currently Used' field of the 'show message-spool detail' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

Remove any unused egress flows. Otherwise the maintainers of the client application should reduce egress flows use by the client application(s) to ease demand.

Message Format

SYSTEM_AD_MAX_EGRESS_FLOWS_EXCEEDED: - - Egress flows maximum exceeded: <maxEgressFlows>

SYSTEM_AD_MAX_ENDPOINTS_EXCEEDED

Description

This event is sent when the number of endpoints exceeds the maximum available for the message broker.

The maximum value is fixed at 16000 and cannot be modified.

To view the current value that the threshold monitors, see the 'Queue and Topic-endpoint Spools - Currently Used' field of the 'show message-spool detail' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

Remove any unused endpoints. Otherwise reduce endpoint use to ease demand.

Message Format

SYSTEM_AD_MAX_ENDPOINTS_EXCEEDED: - - Endpoints maximum exceeded: <numEndpoints>

SYSTEM_AD_MAX_INGRESS_FLOWS_EXCEEDED

Description

This event is sent when the number of ingress flows exceeds the maximum available for the message broker.

The maximum value is fixed at 16000 and cannot be modified.

To view the current value that the threshold monitors, see the 'Total Ingress Flows - Currently Used' field of the 'show message-spool detail' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

Remove any unused ingress flows. Otherwise the maintainers of the client application should reduce ingress flows use by the client application(s) to ease demand.

Message Format

SYSTEM_AD_MAX_INGRESS_FLOWS_EXCEEDED: - - Ingress flows maximum exceeded: <maxIngressFlows>

SYSTEM_AD_MAX_MSG_CACHE_USAGE_EXCEEDED

Description

This event is sent when the guaranteed message cache usage exceeds the maximum usage limit configured for the message broker.

To set the maximum usage limit associated with this event use the 'hardware message-spool max-cache-usage' Config EXEC CLI command. The default is 10% of available NAB resources.

To view the maximum usage limit see the 'Max Cache Usage' field of the 'show message-spool detail' User EXEC CLI command output.

To view the current value that the maximum usage monitors, see the 'Current Cache Usage' field of the 'show message-spool detail' User EXEC CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

No immediate action is required. However, if this message recurs repeatedly, indicating a continued condition, contact your Solace technical support representative.

Message Format

SYSTEM_AD_MAX_MSG_CACHE_USAGE_EXCEEDED: - - Guaranteed Message Cache Usage Limit (<maxCacheUsage>) exceeded, cache misses occurring

SYSTEM_AD_MESSAGE_BUFFER_LEVEL_LOW

Description

This event is sent when the broker detects a sustained low level of available Guaranteed message buffers which may impact the broker's ability to process incoming Guaranteed messages.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

ERROR

Recommended Action

Copy the message text exactly as it appears on the console or in the event log, contact your Solace technical support representative, and provide the representative with the gathered information.

Description

This event is sent when the message spool changes state. The most common states are 'AD-Active' and 'AD-Standby'.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

NOTICE

Recommended Action

This event is informational. No action is required.

Message Format

SYSTEM_AD_MSG_SPOOL_CHG: - - Message spool on <virtualRouter> Virtual Router operational state change from <oldState> to <newState> <reason>

SYSTEM_AD_MSG_SPOOL_DEFRAG_END

Description

This event is sent when message spool defragmentation has ended.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

NOTICE

Recommended Action

This event is informational. No action is required, although some exit conditions can indicate a state preventing defragmentation of the entire spool which administrators may choose to act on. The exit condition text will provide additional information. An example of this could be an uncommitted XA transaction exit condition which is blocking progress beyond a certain point in the message spool. In a case like this, administrators can choose to commit or roll back the transaction to allow a subsequent defragmentation operation to progress further.

Description

This event is sent when the system starts up and the ADB is unable to restore its image from non-volatile memory. When this happens, the Flash Card State will be 'Restore Failed' and the message-spool will not be operational.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

ERROR

Recommended Action

Replacement of the failed ADB/flash should be scheduled as soon as possible. If message-spool operation is required in the interim at the risk of losing guaranteed messages, then the Admin CLI command 'override-flash-failure' may be used. Contact your Solace technical support representative for assistance.

Message Format

Description

This event is sent when the number of transacted sessions exceeds the maximum available for the message broker.

The maximum value is fixed at 16000 and cannot be modified.

To view the current value that the threshold monitors, see the 'Transacted Sessions - Currently Used' field of the 'show message-spool detail' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

The maintainers of the client application(s) establishing the transacted sessions should be consulted to ensure transacted sessions are being committed in a timely fashion. Otherwise reduce transacted session to ease demand.

Message Format

SYSTEM_AD_TRANSACTED_SESSIONS_EXCEED: - - Transacted sessions limit <limit> exceeded

SYSTEM_AD_TRANSACTED_SESSIONS_HIGH

Description

This event is sent when the number of transacted sessions active on the message broker reaches or exceeds the set threshold value.

To set the threshold value associated with this event, use the 'hardware message-spool event transacted-sessions thresholds set-value/set-percentage' Config CLI command. The default is 80% of the maximum value (see the SYSTEM_AD_TRANSACTED_SESSIONS_EXCEED event).

To view the threshold value, see the 'Transacted Sessions - Set Value' field of the 'show message-spool detail' CLI command output.

To view the current value that the threshold monitors, see the 'Transacted Sessions - Currently Used' field of the 'show message-spool detail' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

The thresholds can be increased if the large numbers of transacted session is expected; otherwise, see the 'Recommended Actions' of the SYSTEM_AD_TRANSACTED_SESSIONS_EXCEED event.

Message Format

SYSTEM_AD_TRANSACTED_SESSIONS_HIGH: - - Transacted sessions threshold <threshold>(<thresholdPercentage>%) reached: <currentSessions>

SYSTEM_AD_TRANSACTED_SESSIONS_HIGH_CLEAR

Description

This event is sent when the number of transacted sessions active on the message broker falls below the clear threshold value.

To set the threshold value associated with this event, use the 'hardware message-spool event transacted-sessions thresholds clear-value/clear-percentage' Config CLI command. The default is 60% of the maximum value (see the SYSTEM_AD_TRANSACTED_SESSIONS_EXCEED event).

To view the threshold value, see the 'Transacted Sessions - Clear Value' field of the 'show message-spool detail' CLI command output.

To view the current value that the threshold monitors, see the 'Transacted Sessions - Currently Used' field of the 'show message-spool detail' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

SYSTEM_AD_TRANSACTED_SESSIONS_HIGH_CLEAR: - - Transacted sessions threshold <threshold>(<thresholdPercentage>%) cleared: <currentSessions>

SYSTEM_AD_TRANSACTED_SESSION_RESOURCE_UTILIZATION_EXCEEDED

Description

This event is sent when the amount of system-wide resources required to support the current number of messages pending in transacted sessions exceeds the total resources available.

The maximum value is fixed at 100% and cannot be modified.

To view the current value that the threshold monitors, see the 'Transacted Session Resource Utilization' field of the 'show message-spool' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

Reduce the number of pending messages by having fewer transacted sessions or having fewer messages per transaction.

Message Format

SYSTEM_AD_TRANSACTED_SESSION_RESOURCE_UTILIZATION_EXCEEDED: - - Transacted Session Resource Utilization limit (<limit> %) exceeded

SYSTEM_AD_TRANSACTED_SESSION_RESOURCE_UTILIZATION_HIGH

Description

This event is sent when the amount of system-wide resources required to support the current number of messages pending in transacted sessions reaches the configured high threshold of the total resources available.

The threshold value can be configured in 'configure hardware message-spool event transacted-session-resources'. The threshold defaults to 80%.

To view the current value that the threshold monitors, see the 'Transacted Session Resource Utilization' field of the 'show message-spool' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

See the 'Recommended Actions' for the TRANSACTED_SESSION_RESOURCE_UTILIZATION_EXCEEDED event.

Message Format

SYSTEM_AD_TRANSACTED_SESSION_RESOURCE_UTILIZATION_HIGH: - - Transacted Session Resource Utilization Threshold (<threshold> %) reached: <percentage> %

SYSTEM_AD_TRANSACTED_SESSION_RESOURCE_UTILIZATION_HIGH_CLEAR

Description

This event is sent when the amount of system-wide resources required to support the current number of transacted sessions falls below the configured clear threshold of the total resources available.

The threshold value can be configured in 'configure hardware message-spool event transacted-session-resources'. The threshold defaults to 60%.

To view the current value that the threshold monitors, see the 'Transacted Session Resource Utilization' field of the 'show message-spool' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

SYSTEM_AD_TRANSACTED_SESSION_RESOURCE_UTILIZATION_HIGH_CLEAR: - - Transacted Session Resource Utilization Threshold (<threshold> % ) reached cleared: <percentage> %

This event message is sent when the appliance detects a non-service affecting hardware condition on the ADB (for example, a degraded capacitor).

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

Message Format

SYSTEM_ADB_NONCRITICAL_HARDWARE_NOTIFICATION: - - Slot <fabric>/<slot> Non-critical hardware notification detected on <cardTypeAndDevNum> info <infoStr>

Authentication syslog Messages

SYSTEM_AUTHENTICATION_ADMIN_BIND_FAIL

Description

This event is sent when an attempt by the administrative connection to bind to a LDAP server fails. Further attempts to authenticate using this server will fail if this issue continues to recur.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

Consult the reason field in the event text which explains why the bind failed. A common reason is because the far-end LDAP server rejected the distinguished-name or password passed in the bind request. In such a case verify that the 'Admin DN' and 'Admin Password' specified in the associated ldap-profile are correct. See the 'show ldap-profile <>' CLI command.

Message Format

SYSTEM_AUTHENTICATION_ADMIN_BIND_FAIL: - - Profile: <profileName> admin connection bind to LDAP server <hostname> failed. Reason: <error>

SYSTEM_AUTHENTICATION_ADMIN_CONN_DOWN

Description

This event is sent when the administration connection to a LDAP server has gone down.

Note: The connection status is not polled; it is determined when an attempt is made to use the connection.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

Consult the reason field in the event text which explains why the connection went down. A common reason is because the far-end LDAP server closed the connection usually due to a session timeout. Logs and status information on the LDAP server-side should be consulted to determine the root cause.

Another common reason for administrative connections being torn down is due to the subsequent bind attempt being unsuccessful. In this case there will be a preceding SYSTEM_AUTHENTICATION_ADMIN_BIND_FAIL event in the log.

Note: The message broker will attempt to re-establish the administration connection the next time an authentication is required. As a result, this log by itself might not indicate an issue so long as the re-establishment is successful.

Message Format

SYSTEM_AUTHENTICATION_ADMIN_CONN_DOWN: - - Profile: <profileName> admin connection to LDAP server <hostname> down. Reason: <error>

SYSTEM_AUTHENTICATION_ADMIN_CONN_UP

Description

This event is sent when the administration connection to a LDAP server has gone up.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

No action is required, this event is for informational purposes only.

Message Format

SYSTEM_AUTHENTICATION_ADMIN_CONN_UP: - - Profile: <profileName> admin connection to LDAP server <hostname> up.

SYSTEM_AUTHENTICATION_BIND_CONN_DOWN

Description

This event is sent when the bind connection to a LDAP server failed to be established or was disconnected after being established. Attempts to authenticate using this server will fail if this issue continues to recur.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

Consult the reason field in the event text, which explains why the connection went down. Logs and status information on the LDAP server-side should be consulted to determine the root cause.

Message Format

SYSTEM_AUTHENTICATION_BIND_CONN_DOWN: - - Profile: <profileName> bind connection to LDAP server <hostname> down. Reason: <error>

SYSTEM_AUTHENTICATION_BIND_CONN_UP

Description

This event is sent when the bind connection to a LDAP server has gone up.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

No action is required, this event is for informational purposes only.

Message Format

SYSTEM_AUTHENTICATION_BIND_CONN_UP: - - Profile: <profileName> bind connection to LDAP server <hostname> up.

SYSTEM_AUTHENTICATION_CRL_DOWNLOAD_FAILED

Description

This event is sent when an attempt to download the CRL file for a Certificate Authority failed.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

Consult the reason field in the event text, which explains why the failure occurs.

Message Format

SYSTEM_AUTHENTICATION_CRL_DOWNLOAD_FAILED: - - CA: <caName> failed to download the CRL file. Reason: <reason>

SYSTEM_AUTHENTICATION_CRL_DOWNLOAD_SUCCESS

Description

This event is sent when an attempt to download the CRL file for a Certificate Authority.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

SYSTEM_AUTHENTICATION_CRL_DOWNLOAD_SUCCESS: - - CA: <caName> completed CRL download

SYSTEM_AUTHENTICATION_DISCOVERY_FAILED

Description

An attempt to retrieve discovery endpoints for an OAuth Profile failed.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

This event is informational. No action is required.

Message Format

SYSTEM_AUTHENTICATION_DISCOVERY_FAILED: - - Unable to refresh discovery endpoints for Profile <profileName> from URI <uri>. Reason: <reason>

SYSTEM_AUTHENTICATION_DISCOVERY_SUCCESS

Description

An attempt to retrieve discovery endpoints for an OAuth Profile succeeded.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

SYSTEM_AUTHENTICATION_DISCOVERY_SUCCESS: - - Refresh discovery endpoints for Profile <profileName> completed from URI <uri>.

SYSTEM_AUTHENTICATION_DUPLICATE_ISSUER

Description

Two OAuth Profiles have the same issuer. As the issuer is used to uniquely identify the Profile on login, only the first Profile will be used.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

Disable one of the OAuth Profiles or change its issuer.

Message Format

SYSTEM_AUTHENTICATION_DUPLICATE_ISSUER: - - Profile <profileName> has same issuer as Profile <otherProfileName> (<issuer>). The first profile will be used.

SYSTEM_AUTHENTICATION_JWKS_DOWNLOAD_FAILED

Description

An attempt to download an OAuth Profile JWKS failed.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

This event is informational. No action is required.

Message Format

SYSTEM_AUTHENTICATION_JWKS_DOWNLOAD_FAILED: - - Unable to refresh JWKS for Profile <profileName> from URI <uri>. Reason: <reason>

SYSTEM_AUTHENTICATION_JWKS_DOWNLOAD_SUCCESS

Description

An attempt to download an OAuth Profile JWKS succeeded.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

SYSTEM_AUTHENTICATION_JWKS_DOWNLOAD_SUCCESS: - - Refresh JWKS for Profile <profileName> completed from URI <uri>.

SYSTEM_AUTHENTICATION_SEMP_USERNAME_LOCKOUT

Description

This event is sent after a sequence of consecutive failed SEMP authentication attempts for an internal username causes authentication for the username to be temporarily locked out.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

Ensure the username and password are correct. This event may indicate an attempt to gain unauthorized access to the server.

Message Format

SYSTEM_AUTHENTICATION_SEMP_USERNAME_LOCKOUT: - - Username <username> temporarily locked out from SEMP authentication due to <maxFail> consecutive failed authentication attempts in <failInterval> seconds. Lock will be released in <lockDuration> seconds.

SYSTEM_AUTHENTICATION_SESSION_CLOSED

Description

This event is sent when a user ends a CLI, shell, scp, or sftp session. With LDAP or RADIUS authentication, username is the remote user, user is the local user. With su, username is the current user, user is the target user.

Event Level Severity

NOTICE

Recommended Action

This event is informational. No action is required.

Message Format

SYSTEM_AUTHENTICATION_SESSION_CLOSED: - - <sessionType> session <sessionUniqueId> closed for user <username> (<user>)

SYSTEM_AUTHENTICATION_SESSION_DENIED

Description

This event is sent when a user unsuccessfully attempts to authenticate for a CLI, SEMP, shell, scp, or sftp session.

Event Level Severity

NOTICE

Recommended Action

Ensure authentication tokens (like username and password) are correct, and external services are configured correctly and available. CLI/SEMP, shell and scp/sftp access each have separate usernames, so ensure the correct username is used.

Message Format

SYSTEM_AUTHENTICATION_SESSION_DENIED: - - Denied <sessionType> session <sessionUniqueId> <authenticationType> authentication for <username> (<user>). Reason: <reason>

Message Format

SYSTEM_CHASSIS_BLADE_POST_CRITICAL_FAILURE: - - Slot <fabric>/<slot> Critical hardware failure: <bladeStr> Power-on Self Test (POST) failed

SYSTEM_CHASSIS_BLADE_POST_NONCRITICAL_NOTIFICATION

Description

This event message is sent when the appliance detects a recoverable hardware failure during Power-On Self Test (POST)

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

No immediate action is required. However, if this message recurs repeatedly, indicating a persistent condition, contact your Solace technical support representative.

Message Format

SYSTEM_CHASSIS_BLADE_POST_NONCRITICAL_NOTIFICATION: - - Slot <fabric>/<slot> Non-critical hardware notification: <bladeStr> Power-on Self Test (POST) failed

SYSTEM_CHASSIS_BLADE_UP

Description

This event is sent when an appliance blade is brought online.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

SYSTEM_CHASSIS_BLADE_UP: - - Blade <fabricAndSlot> (<type>) up

SYSTEM_CHASSIS_BOOT_DISK_FAIL

Description

This event is sent when there is a boot disk failure on the appliance.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

CRIT

Recommended Action

Copy the message text exactly as it appears on the console or in the event log, contact your Solace technical support representative, and provide the representative with the gathered information. Also provide the output of the CLI commands 'show hardware detail' and 'show disk detail' CLI commands along with any other relevant information.

Message Format

SYSTEM_CHASSIS_BOOT_DISK_FAIL: - - Flash Bootdisk Failed

SYSTEM_CHASSIS_DISK_DOWN

Description

This event is sent when the disks are not fully redundant, due to disk shutdown, disk failure, or incomplete RAID synchronization.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

Verify the administrative state with the CLI command 'show disk'. If the disk is administratively shutdown, enable it with the 'hardware disk no shutdown' Config CLI command. If the disk is administratively enabled, copy the message text exactly as it appears on the console or in the event log, contact your Solace technical support representative, and provide the representative with the gathered information. Also provide the output of the CLI commands 'show hardware detail' and 'show disk detail' CLI commands along with any other relevant information.

Description

This event is sent when the disk utilization of a monitored disk partition reached the maximum value (i.e. 100%). The monitored partitions include the root internal disk partition (md2 mounted as /), the primary internal disk partition (md6 mounted as /usr/sw), as well as the two external spool partitions (externalSpool/p1 mounted as /usr/sw/externalSpool/p1) and (externalSpool/p2 mounted as /usr/sw/externalSpool/p1) if the message-spool is enabled.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

SolOS loads no longer required should be deleted from the disk in order to free up space. The CLI 'delete-load' command can be used to remove loads from the SolOS loads/ directory. In addition if there are files in /usr/sw/support these should be removed unless they are patch directories.

Message Format

SYSTEM_CHASSIS_DISK_UTILIZATION_MAX: - - Disk utilization on <diskPartition> reached maximum

SYSTEM_CHASSIS_FABRIC_POST_CRITICAL_FAILURE

Description

This event message is sent when the appliance detects a critical hardware failure during Power-On Self Test (POST)

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

CRIT

Recommended Action

Run gather-diagnostics and contact your Solace technical support representative.

Message Format

Description

This event is sent when the appliance detects a soft error in a hardware component.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

ERROR

Recommended Action

There is a low probability that performance has been affected. However, for any errors reporting an ADB or TRB FPGA CRC error, we recommend an immediate HA fail-over in those instances. All errors should be followed by a restart of the affected broker in the next available maintenance window. If this message recurs after restart, possibly indicating a continued condition, contact your Solace technical support representative.

Message Format

SYSTEM_CHASSIS_HARDWARE_SOFT_ERROR: - - Slot <fabric>/<slot> Hardware soft error detected on <cardTypeAndDevNum> info <infoStr>

SYSTEM_CHASSIS_INODE_UTILIZATION_HIGH

Description

This event is sent when the iNode utilization of a monitored disk partition, expressed as a percentage of the total iNodes in the partition, rises above the set threshold value. The monitored partitions include the root internal disk partition (md2 mounted as /), the primary internal disk partition (md6 mounted as /usr/sw), as well as the two external spool partitions (externalSpool/p1 mounted as /usr/sw/externalSpool/p1) and (externalSpool/p2 mounted as /usr/sw/externalSpool/p1) if the message-spool is enabled.

To view the current value that the threshold monitors, login to the message broker as 'root' user, then see the 'IUse%' field for the given mount point in the 'df -i' linux command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

Contact your Solace technical support representative for assistance.

Message Format

SYSTEM_CHASSIS_INODE_UTILIZATION_HIGH: - - iNode utilization on <diskPartition> (<threshold> %) reached: <value> %

SYSTEM_CHASSIS_INODE_UTILIZATION_HIGH_CLEAR

Description

This event is sent when the iNode utilization of a monitored disk partition, expressed as a percentage of the total iNodes in the partition, falls below the clear threshold value.

To view the current value that the threshold monitors, login to the message broker as 'root' user, then see the 'IUse%' field for the given mount point in the 'df -i' linux command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

SYSTEM_CHASSIS_INODE_UTILIZATION_HIGH_CLEAR: - - iNode utilization on <diskPartition> (<threshold> %) cleared: <value> %

SYSTEM_CHASSIS_INODE_UTILIZATION_MAX

Description

This event is sent when the iNode utilization of a monitored disk partition reached the maximum value (i.e. 100%). The monitored partitions include the root internal disk partition (md2 mounted as /), the primary internal disk partition (md6 mounted as /usr/sw), as well as the two external spool partitions (externalSpool/p1 mounted as /usr/sw/externalSpool/p1) and (externalSpool/p2 mounted as /usr/sw/externalSpool/p1) if the message-spool is enabled.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

Contact your Solace technical support representative for assistance.

Message Format

SYSTEM_CHASSIS_INODE_UTILIZATION_MAX: - - iNode utilization on <diskPartition> reached maximum (100 %)

SYSTEM_CHASSIS_PHYSICAL_CPU_FAIL

Description

This event is sent during start-up if the number of physical CPUs present is lower than the expected number. After start-up, this event cannot be raised because a physical CPU failure after start-up will terminate operation.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

CRIT

Recommended Action

Copy the message text exactly as it appears on the console or in the event log, contact your Solace technical support representative, and provide the representative with the gathered information. Also provide the system configuration information along with any other relevant information.

Message Format

SYSTEM_CHASSIS_PHYSICAL_CPU_FAIL: - - Physical CPU(s) failed: expected <expected> cpu(s), present <present> cpu(s)

SYSTEM_CHASSIS_PHYSICAL_MEMORY_FAIL

Description

This event is sent duringstart-up if the amount of physical memory present is less than expected. After start-up this event cannot be raised because physical memory failures will terminate operation.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

CRIT

Recommended Action

Copy the message text exactly as it appears on the console or in the event log, contact your Solace technical support representative, and provide the representative with the gathered information. Also provide the system configuration information along with any other relevant information.

Message Format

SYSTEM_CHASSIS_PHYSICAL_MEMORY_FAIL: - - Physical memory failed: expected <expected> kB, present <present> kB

SYSTEM_CHASSIS_POST_CRITICAL_FAILURE

Description

This event message is sent when the message broker detects a critical failure during Power-On Self Test (POST)

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

CRIT

Recommended Action

Run gather-diagnostics and contact your Solace technical support representative.

Message Format

SYSTEM_CHASSIS_POST_CRITICAL_FAILURE: - - Critical hardware failure: Chassis Power-on Self Test (POST) failed

SYSTEM_CHASSIS_POST_NONCRITICAL_NOTIFICATION

Description

This event message is sent when the message broker detects a recoverable failure during Power-On Self Test (POST)

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

No immediate action is required. However, if this message recurs repeatedly, indicating a persistent condition, contact your Solace technical support representative.

Message Format

SYSTEM_CHASSIS_POST_NONCRITICAL_NOTIFICATION: - - Non-critical hardware notification: Chassis Power-on Self Test (POST) failed

SYSTEM_CHASSIS_POWER_MODULE_DOWN

Description

This event is sent if one of the redundant power supplies fails, or has its AC power source removed, and as a result the appliance is in, or transitioning into, a non-redundant power state.

That is, this event is sent when the number of active power supplies detected on the appliance is less than that required for the power redundancy setting configured through the 'hardware power-redundancy' Global CONFIG command.

Note: This event is raised under the following conditions:

1) In a 2+2 or 2+1 redundant power configuration (available only on Solace PubSub+ 3260) after a power supply failure leaves just two power supplies active.

2) In a 1+1 redundant power configuration (available only on Solace PubSub+ 3230, Solace PubSub+ 3530, and SolacePubSub+ 3560) after a power supply failure leaves just one power supply active.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

Verify that AC power is being delivered to the module, power cords are firmly inserted, and a green LED is lit at the back of the power module. Reseat the power supply in the chassis. Replace the power supply if these steps fail to clear the condition.

Message Format

SYSTEM_CHASSIS_POWER_MODULE_DOWN: - - Power Module <module> down

SYSTEM_CHASSIS_POWER_MODULE_DOWN_INFO

Description

This event is sent if one of the redundant power supplies fail or has its AC power source removed, and the appliance maintains a redundant power state.

That is, this event is sent when the number of active power supplies detected on the appliance after one power supply goes down still meets that required for the power redundancy setting configured through the hardware power-redundancy Global CONFIG command.

Note: This event is only raised in a 2+2 redundant power configuration (available only on Solace PubSub+ 3260) after a single power supply failure leaves the system with still three active power supplies.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

Verify that AC power is being delivered to the module, power cords are firmly inserted, and a green LED is lit at the back of the power module. Replace the power supply if these steps fail to clear the condition.

Message Format

SYSTEM_CHASSIS_POWER_MODULE_DOWN_INFO: - - Power Module <module> down

SYSTEM_CHASSIS_POWER_MODULE_UP

Description

This event is sent when one of the power modules return to normal operation.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

SYSTEM_CHASSIS_POWER_MODULE_UP: - - Power Module <module> up

SYSTEM_CHASSIS_STORAGE_ELEMENT_USAGE_HIGH

Description

This event is sent when the disk usage for a monitored storage element exceeds the threshold value.

Usage thresholds are calculated based on system scaling parameters. To view the threshold value, see the 'show storage-element <storage-element-name> detail' CLIcommand.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

Message Format

SYSTEM_CHASSIS_STORAGE_ELEMENT_USAGE_HIGH: - - Storage element <name> (<path>) disk usage threshold (<threshold> MB) reached: <usage> MB

Description

The system blades installed in the chassis are installed in a slot configuration which is unsupported by Solace. The system could appear stable however the future quality and/or performance of the system cannot be predicted by Solace.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

Re-install the system blades in a supported configuration. See Solace documentation for supported configurations.

Message Format

SYSTEM_CHASSIS_UNSUPPORTED_BLADE_CONFIGURATION: - - Blades are installed in an unsupported configuration.

SYSTEM_CHASSIS_UNSUPPORTED_CONFIGURATION

Description

The hardware present in the chassis is unable to provide complete support for the configuration. Details of the unsupported configuration, and possible recommended actions follow the initial message.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

See the details for recommended actions.

Message Format

SYSTEM_CHASSIS_UNSUPPORTED_CONFIGURATION: - - The system configuration is not supported by the hardware: <details>

SYSTEM_CHASSIS_UNSUPPORTED_DISK_CONFIGURATION

Description

The disk drives are installed in a slot configuration which is unsupported by Solace. The system could appear stable however the future quality and/or performance of the system cannot be predicted by Solace.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

Reinstall the disk so the system raid group is in slot 1 and slot 2. See solace documentation for supported configurations.

Message Format

SYSTEM_CHASSIS_UNSUPPORTED_DISK_CONFIGURATION: - - <type> disk is unsupported as disk (<disk>)

SYSTEM_CLIENT_ACL_CONNECT_DENIAL

Description

This event is sent when one or more client connect attempts were denied in the last 60s due at an Access Control List (ACL) client-connect rule.

Event Level Severity

INFO

Recommended Action

Consult the ACL log through the 'show log acl' CLI command for details about the most recent connection attempts that were denied. If the denial was not intended modify the ACL rules for the associated acl-profile using the 'acl-profile <> message-vpn <> client-connect' Config CLI command

Message Format

SYSTEM_CLIENT_ACL_CONNECT_DENIAL: - - <connectDenials> connect denials were recorded in the last minute due to client connect ACL configuration

SYSTEM_CLIENT_ACL_PUBLISH_DENIAL

Description

This event is sent when one or more messages, published within the last 60s, was rejected on ingress because the topic matched a publish-topic ACL rule.

Event Level Severity

INFO

Recommended Action

Consult the ACL log through the 'show log acl' CLI command for details about the most recent publish attempts that were denied. If the denial was not intended modify the ACL rules for the associated acl-profile using the 'acl-profile <> message-vpn <> publish-topic' Config CLI command.

Message Format

SYSTEM_CLIENT_ACL_PUBLISH_DENIAL: - - <publishDenials> publish denials were recorded in the last minute due to ACL profile configuration

SYSTEM_CLIENT_ACL_SUBSCRIBE_DENIAL

Description

This event is sent when one or more subscriptions, received from clients within the last 60s, was rejected because the topic matched a subscribe-topic ACL rule.

Event Level Severity

INFO

Recommended Action

Consult the ACL log through the 'show log acl' CLI command for details about the most recent subscriptions that were not accepted. If the rejection was not intended modify the ACL rules for the associated acl-profile using the 'acl-profile <> message-vpn <> subscribe-topic' Config CLI command.

Message Format

SYSTEM_CLIENT_ACL_SUBSCRIBE_DENIAL: - - <subscribeDenials> subscribe denials were recorded in the last minute due to ACL profile configuration

SYSTEM_CLIENT_CONNECTIONS_EXCEEDED

Description

This event is sent when the number of clients attempting to connect to the message broker exceeds the maximum connections available.

The maximum connections available is 6000 for Solace PubSub+ 3230 and Solace PubSub+ 3530 and 9000 for Solace PubSub+ 3260 and Solace PubSub+ 3560.

To view the current value that the threshold monitors, see the 'Total Clients Connected' field of the 'show stats client' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

See the 'Recommended Actions' for the SYSTEM_CLIENT_CONNECTIONS_HIGH event.

Message Format

SYSTEM_CLIENT_CONNECTIONS_EXCEEDED: - - Connections limit (<threshold>) exceeded Service (<service>)

SYSTEM_CLIENT_CONNECTIONS_HIGH

Description

This event is sent when the number of clients connected to the message broker rises above the set threshold value.

To set the threshold value associated with this event, use the 'service event connections thresholds set-value|set-percentage' or 'service smf event connections thresholds set-value|set-percentage' Config CLI command. The default is 80% of the maximum value (see the SYSTEM_CLIENT_CONNECTIONS_EXCEEDED event)

To view the threshold value, see the 'Event Threshold' table of the 'show service' CLI command output.

To view the current value that the threshold monitors, see the 'Total Clients Connected' field of the 'show stats client' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

The thresholds can be increased if a large number of clients connected to the message broker is expected. Otherwise, if possible, contact the maintainers of the client applications connected to the message broker to verify if the number of connections being used by those applications is reasonable. If neither of those options are possible a CLI operator can forcefully disconnect individual clients through the 'client <> message-vpn <> disconnect' Admin CLI command. However unless one or more of the configuration objects required for client login (for example, client-profile, message-vpn, authentication profiles) have been shutdown beforehand the client(s) may subsequently re-connect. An alternative to shutting down configuration objects is to use client-connect ACL rules that can be configured to limit which clients can connect to the message broker.

Warning: shutting down configuration objects necessary for client login will affect *all* clients using those objects.

Message Format

SYSTEM_CLIENT_CONNECTIONS_HIGH: - - Connections threshold (<threshold>) reached: <value> Service (<service>)

SYSTEM_CLIENT_CONNECTIONS_HIGH_CLEAR

Description

This event is sent when the number of clients connected to the message broker falls below the set threshold value.

To set the threshold value associated with this event, use the 'service event connections thresholds clear-value|clear-percentage' or 'service smf event connections thresholds clear-value|clear-percentage' Config CLI command. The default is 60% of the maximum value (see the SYSTEM_CLIENT_CONNECTIONS_EXCEEDED event)

To view the threshold value, see the 'Event Threshold' table of the 'show service' CLI command output.

To view the current value that the threshold monitors, see the 'Total Clients Connected' field of the 'show stats client' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

SYSTEM_CLIENT_CONNECTIONS_HIGH_CLEAR: - - Connections threshold (<threshold>) cleared: <value> Service (<service>)

SYSTEM_CLIENT_CONNECT_AUTH_FAIL

Description

This event is sent when user authentication fails for a client trying to connect to the message broker.

Event Level Severity

WARN

Recommended Action

Authorization failures are typically the result of an incorrect password or an authentication configuration error. Verify the authentication configuration is correct for the given Message VPN through the 'show message-vpn <>'. See the 'Auth Type' and 'Auth Profile' and 'Radius Domain' fields (if applicable) of the resultant show display.

Message Format

SYSTEM_CLIENT_CONNECT_AUTH_FAIL: - - Message VPN (<messageVpn>) <type> username <clientUsername> authentication failed from <IP>:<port> using scheme <authScheme> - <reason>: <respDesc>

SYSTEM_CLIENT_CONNECT_FAIL

Description

This event is sent when a client tries to connect to the message broker but the message broker is unable or unwilling to accept the connection.

Event Level Severity

INFO

Recommended Action

Consult the reason field in the event text which explains why the connection attempt was denied. The most typical reasons fall into two categories: configured objects necessary for connect attempts to succeed have been shutdown (for example, message-vpn, client-profile, authentication profiles) or some system limits have been exceeded (for example, system-wide limit of number of concurrent client connections or subscriptions). Another reason is that the specified Message VPN may be non-existent.

Message Format

SYSTEM_CLIENT_CONNECT_FAIL: - - Message VPN (<messageVpn>) <type> username <clientUsername> clientname <clientname> connect failed from <IP>:<port> - <reason>: <respDesc>

Value	Description
A required claim did not match the configured value	A required claim did not match the configured value
A required claim was not found	A required claim was not found
A token contains an illegal expiry value	A token contains an illegal expiry value
A token has an issued at (iat) timestamp in the future	A token has an issued at (iat) timestamp in the future
A token is not yet valid	A token is not yet valid
AD is not enabled	AD is not enabled
ADCTRL Parse Error	Received an ADCTRL request that fails to parse correctly.
Already Bound	The request to bind a queue endpoint to a client failed because it is already bound to that client.
Already Exists	The resource already exist. The request to create it has failed.
An IdToken or AccessToken was unparseable	An IdToken or AccessToken was unparseable
An administrator has annulled this login by changing the configuration	An administrator has annulled this login by changing the configuration
An error was encounter when retrieving the Authorization Groups from the Authorizing Server	An error was encounter when retrieving the Authorization Groups from the Authorizing Server
Assured Delivery Not Ready	Assured Delivery is not ready. One possible reason is that AD is not enabled. Another possible reason is that AdDp is not synchronized.
Bad ClientCtrl MsgType	Bad ClientCtrl MsgType
Basic Authentication Is Shutdown	Basic Authentication Is Shutdown
Binding to an LVQ within an XASession is not allowed	Binding to an LVQ within an XASession is not allowed
Bridge To Self	Bridge To Self
Bridging Not Allowed	Bridging Not Allowed
Browser Not Allowed	Request to bind to a Topic Endpoint failed because browsers are not allowed for a topic endpoint.
Browsing Not Supported on Partitioned Queue	Browsing Not Supported on Partitioned Queue
Busy - Retry Later	Busy - Retry Later
Cache Instance Not Running On Same Router	Cache Instance Not Running On Same Router
Cache Instance Would Cause Memory Limits To Be Exceeded	Cache Instance Would Cause Memory Limits To Be Exceeded
Cannot Spool to Disk	Cannot Spool to Disk
Certificate Chain Too Long	Certificate Chain Too Long
Certificate Error	Certificate Error
Certificate Expired	Certificate Expired
Certificate Not Yet Valid	Certificate Not Yet Valid
Client Certificate Authentication Is Shutdown	Client Certificate Authentication Is Shutdown
Client Initiated Replay Not Allowed on Non-Exclusive Queue	The request attempted to start replay on a non-exclusive queue, which is not permitted. Replay on non-exclusive queues must be started administratively via the router’s management interface.
Client Initiated Replay Not Allowed on Non-Exclusive Topic Endpoint	The request attempted to start replay on a non-exclusive topic-endpoint, which is not permitted. Replay on non-exclusive queues must be started administratively via the router’s management interface.
Client Initiated Replay Not Allowed on telemetry queue	The request attempted to start replay on telemetry queue, which is not permitted.
Client Initiated Replay from Browser Flow Not Allowed	The request attempted to start replay as a browser flow, which is not permitted. Only active consumers can start request replays.
Client Initiated Replay from Inactive Flow Not Allowed	The request attempted to start replay as an inactive flow, which is not permitted. Only active consumers can start request replays.
Client Name Already In Use	Client Name Already In Use
Client Name Parse Error	Client Name Parse Error
Client Username Is Shutdown	Client Username Is Shutdown
Client exceeded max transaction	Client exceeded max transaction
Client-Connect ACL Denied	Client-Connect ACL Denied
ClientCtrl Parse Error	ClientCtrl Parse Error
Compression Is Shutdown	Compression Is Shutdown
Consumer Ack Propagation Enabled when Replication is Standby	Consumer Ack Propagation Enabled when Replication is Standby
Control message discard	Control message discard
Copy-from queue missing	Copy-from queue missing
Copy-from queue template missing	Copy-from queue template missing
Copy-from topic-endpoint missing	Copy-from topic-endpoint missing
Copy-from topic-endpoint template missing	Copy-from topic-endpoint template missing
Dead Message Queue Not Allowed	Request to create Queue Endpoint failed because the requested Dead Message Queue is set to itself
Delete In Progress	Request to create Queue Endpoint failed because a prior request to delete the Queue Endpoint is still in progress.
Destination group error	Destination group error
Document Is Too Large	The incoming message is too large for the appliance to handle.
Duplicate message	Duplicate message
Durable Topic Endpoint Shutdown	Attempt to bind to a Durable Topic Endpoint (DTE) failed because the DTE has shutdown.
Dynamic Clients Not Allowed	Dynamic Clients Not Allowed
Endpoint Modified	Endpoint Modified
Endpoint Property Mismatch	Endpoint Property Mismatch
Endpoint Shutdown	Endpoint Shutdown
Errored message	Errored message
Exceeded Spool File Limit	Exceeded Spool File Limit
Flow Name In Use	Request to open AD flow failed because the flow name is already in use.
GSS Invalid Authentication	GSS Invalid Authentication
Header Parse Error	Fail to parse the header information of an incoming message.
In Progress	In Progress
Inactive Connection	A connection has been found to be inactive. The appliance is about to clean it up or close the connection down.
Incompatible bridge transport configuration	The local and remote bridges have incompatible transport configurations.
Incompatible config-sync transport configuration	The local and remote config-sync have incompatible transport configurations.
Incomplete Authentication Configuration	Incomplete Authentication Configuration
Internal Error	An internal error has occurred.
Introspection failed because of an HTTPS error	Introspection failed because of an HTTPS error
Introspection is required but no introspection endpoint is configured	Introspection is required but no introspection endpoint is configured
Introspection response is not valid JSON	Introspection response is not valid JSON
Introspection timed out waiting for a response from a DNS or Authorization Server	Introspection timed out waiting for a response from a DNS or Authorization Server
Invalid Connection Owner	More than one client is attempting to perform transactions on the same session, which is not allowed.
Invalid FlowId	Request to close or unbind a flow failed, because no flow id or an invalid flow id is provided.
Invalid Local DeliverToOne Priority	Invalid Local DeliverToOne Priority
Invalid Network DeliverToOne Priority	Invalid Network DeliverToOne Priority
Invalid Queue Name	An operation on a Queue Endpoint failed because the Queue Endpoint Name is invalid
Invalid Queue or Topic Endpoint Durability	Bind to Queue Endpoint failed, because the durability in the request is not compatible with the Queue Endpoint.
Invalid Selector	Request to create Selector for Topic Endpoint failed. This is a generic response.
Invalid Sol-Session Header	Invalid Sol-Session Header
Invalid Topic Endpoint Name	An operation on a Topic Endpoint failed because the Topic Endpoint Name is invalid.
Invalid Topic Name	When configuring type topic for a queue endpoint, the topic provide in the configuration is invalid.
Invalid Transacted Session Id	Invalid Transacted Session Id
Invalid Transacted Session Name	Invalid Transacted Session Name
Invalid Transaction Id	Invalid Transaction Id
Invalid Username	Attempt to login with an invalid Username.
Invalid Virtual Router Address	Invalid Virtual Router Address
Invalid Virtual Router Address	Received Bind Request from a client which is not configured to access AD service.
Invalid parameter combination	Invalid parameter combination
JNDI Parse Error	JNDI Parse Error
Kerberos Authentication Is Shutdown	Kerberos Authentication Is Shutdown
Kerberos Authentication Token Too Long	Kerberos Authentication Token Too Long
Kerberos GSS Token Missing	Kerberos GSS Token Missing
Kerberos Principal Too Long For Client Name	Kerberos Principal Too Long For Client Name
Kerberos Service Principal Not Found	Kerberos Service Principal Not Found
LDAP local error	LDAP local error
Local Message Quota Exceeded	Messages are being discarded in an egress queue because the queue depth has been exceeded (too many messages and cannot send the messages out fast enough).
Login Respond Error	Fail to send a login response. One possible reason is that the connection is closed.
Logout In Progress	Request to login Client failed because a prior request to logout the Client is still in progress.
Low Priority Msg Congestion	Low Priority Msg Congestion
MQTT CleanSession=0 not allowed with redundancy when message-spool and config-sync are disabled	MQTT CleanSession=0 not allowed with redundancy when message-spool and config-sync are disabled
MQTT Session Is Shutdown	MQTT Session Is Shutdown
MQTT Session Owner Mismatch	MQTT Session Owner Mismatch
Max Client Queue and Topic Endpoint Flow Exceeded	Queue Endpoint bind failed because the Maximum Queue and Topic Endpoint Flow for this client has been exceeded.
Max Num Shared Subscriptions Exceeded	Max Num Shared Subscriptions Exceeded
Max Num Subscriptions Exceeded	Max Num Subscriptions Exceeded
Max Router Publishing Flow Exceeded	Open Flow failed because the Maximum Publishing Flow for this router has been exceeded.
Max Router Selectors Exceeded	Request to create Selector for Topic Endpoint failed because the maximum number of Selectors has been exceeded for this router.
Max Transacted Sessions Exceeded	Max Transacted Sessions Exceeded
Max Transactions Exceeded	Max Transactions Exceeded
Max clients exceeded for queue	When attempting to bind a client to a queue, the bind failed because there are too many clients bound to that queue.
Max clients exceeded for topic endpoint	When attempting to bind a client to a topic endpoint, the bind failed because there are too many clients bound to that topic endpoint.
Max message usage exceeded	Max message usage exceeded
Message Consume Failure	Message Consume Failure
Message Has Zero Length	A message of size zero was received. For XMLDOC protocol, the appliance has received a zero length message.
Message Publish Failure	Message Publish Failure
Message Publish Failure: Invalid flow-id from the publisher	Message Publish Failure: Invalid flow-id from the publisher
Message Publish Failure: No open flow from the publisher	Message Publish Failure: No open flow from the publisher
Message Publish Failure: Received fewer messages than expected from the publisher	Message Publish Failure: Received fewer messages than expected from the publisher
Message Publish Failure: Received more messages than expected from the publisher	Message Publish Failure: Received more messages than expected from the publisher
Message Publish Failure: Received one or more wrong messages from the publisher	Message Publish Failure: Received one or more wrong messages from the publisher
Message Size Out Of Range	Request to create Queue Endpoint failed because the requested message size is too big.
Message VPN Not Allowed	Message VPN Not Allowed
Message VPN Unavailable	Message VPN Unavailable
Mismatched Endpoint Error ID	Endpoint error ID in the bind request does not match the endpoint's error ID.
Mismatched User Type Specified	Mismatched User Type Specified
Missing or Invalid Token(s) in OAuth Login	Missing or Invalid Token(s) in OAuth Login
Multiple tokens were provided but OAuth role is resource server	Multiple tokens were provided but OAuth role is resource server
Neighbor Handshake Denied - Duplicate Router Name or Wrong Neighbor Initiating Connection	Neighbor Handshake Denied - Duplicate Router Name or Wrong Neighbor Initiating Connection
Neighbor Handshake Denied - Neighbor Sending From Wrong IP Address	Neighbor Handshake Denied - Neighbor Sending From Wrong IP Address
Neighbor Handshake Denied - Operating Mode Mismatch	Neighbor Handshake Denied - Operating Mode Mismatch
Neighbor Handshake Denied - Unrecognized Neighbor IP Address	Neighbor Handshake Denied - Unrecognized Neighbor IP Address
Neighbor Handshake Denied - Unrecognized Neighbor Router Name	Neighbor Handshake Denied - Unrecognized Neighbor Router Name
Neighbor Handshake Denied - Wrong Destination Router Name	Neighbor Handshake Denied - Wrong Destination Router Name
No Authorization Groups in enabled state were configured on the appliance	No Authorization Groups in enabled state were configured on the appliance
No Authorization Groups were retrieved from the Authorizing Server	No Authorization Groups were retrieved from the Authorizing Server
No JSON Web Key exists to decode the token	No JSON Web Key exists to decode the token
No More Non-Durable Queue or Topic Endpoint	Create Queue Endpoint failed because needed resources cannot be allocated.
No More Selector Resources	Request to create Selector for Topic Endpoint failed because there are no more Selector Resources.
No RADIUS profile available	No RADIUS profile available
No Subscription Match	No Subscription Match
No Transaction Started	No Transaction Started
No Transport Buffer	The appliance fail to allocate memory to construct a request or respond message. The message is dropped.
No access level configured	No access level configured
No compatible bidirectional bridge	There is no local bridge compatible with the remote bridge to form a bidirectional connection.
No issuer ('iss' claim) was found in the token, there is no active default profile, and neither the Issuer Identifier nor Profile Name SMF parameter was provided	No issuer ('iss' claim) was found in the token, there is no active default profile, and neither the Issuer Identifier nor Profile Name SMF parameter was provided
No matching configured Authorization Group was found (or all such matching groups are disabled)	No matching configured Authorization Group was found (or all such matching groups are disabled)
NoLocal Discard	NoLocal Discard
Non-XASession Cannot Bind to XA Endpoint	Non-XASession Cannot Bind to XA Endpoint
Not Compatible With Forwarding Mode	Not Compatible With Forwarding Mode
Not Enough Space	Not Enough Space
Not Found	Not Found
Not Ready	The appliance is not ready. A possible reason is that the Routing component is shutdown.
OAuth Authentication is Shutdown	OAuth Authentication is Shutdown
OAuth Profile discovery has failed or has not yet finished	OAuth Profile discovery has failed or has not yet finished
OAuth Token Too Long	OAuth Token Too Long
Ok	Ok
Only local transacted sessions can bind to Delayed Redelivery endpoint	A bind to an endpoint configured with delayed redelivery was requested, but the session is not locally transacted
Out of Replay Resources	A replay was requested, but the router does not have sufficient resources to fulfill the request, due to too many active replays.
Out of buffers	Out of buffers
Out of internal event handling resources	The client could not connect because the internal resources necessary to process their events were temporaily unavailable
Out of order message	Out of order message
Out of transaction resources	Out of transaction resources
Partition is not ready	Partition is not ready
Partitioned Queue Support Required	Partitioned Queue Support Required
Peer Reset	A connection failure has occurred due to some transport error. This is a generic failure response.
Permission Not Allowed	The AD request failed. The operation is not allowed. This is a generic response to many types of request where the operation failed because of an incorrect permission level. e.g. a client is not allowed to delete a Queue Endpoint that is created by CLI.
Protocol Violation	This event is seen when an error occurs during a protocol exchange. When the appliance attempt to send a control message as part of the protocol, but the message failed to send, resulting in a protocol violation.
Publish ACL Denied	Publish ACL Denied
Publisher Name Already In Use	Publisher Name Already In Use
Publisher msg count mismatch	Publisher msg count mismatch
Queue Not Found	A message is being spooled through AD, but the queue that is supposed to receive the message is not found.
Queue Shutdown	Attempt to bind to a Queue Endpoint failed because the Queue has shutdown.
Quota Out Of Range	Request to create Queue Endpoint failed because the requested disk quota is too big.
RADIUS profile contains no functional Servers	RADIUS profile contains no functional Servers
Redelivery delay support required	The client does not support redelivery delay and has tried to bind to the endpoint configured with delayed redelivery
Remote router spooling not supported	Remote router spooling not supported
Replay Cancelled	The flow was replaying messages as requested by an earlier bind request, but the replay was administratively cancelled.
Replay Disabled	Replay is not enabled in the queue or DTE’s message VPN.
Replay Failed	An unexpected error has happened during message replay on the queue or DTE.
Replay Log Modified	A replay was ongoing on the queue or DTE and the replay log was modified.
Replay Message Unavailable	The flow was replaying messages as requested by an earlier bind request, but the replay failed because messages to be replayed were trimmed from the replay log.
Replay Not Supported on Anonymous Queue Without Subscriptions	The request attempted to start replay on an anonymous queue without subscriptions, which is not supported.
Replay Start Message Unavailable	A replay was requested, but the requested start message is not available in the replay log.
Replay Start Time Not Available	A replay was requested, but the requested start time is not available in the replay log.
Replay Started	A replay was started on the queue or DTE, either by another client or by the router.
Replayed Message Rejected by Queue	The router attempted to replay a message, but the queue rejected the message to sender.
Replayed Message Rejected by Topic Endpoint	The router attempted to replay a message, but the topic endpoint rejected the message to sender.
Replication Is Shutdown	Replication Is Shutdown
Replication Is Standby	Replication Is Standby
Request On Non-static Interface	Request On Non-static Interface
Request Time-out	The client has taken too long to provide a request and a timeout occurs.
Revocation Check Failed	Revocation Check Failed
Router Name Mismatch	Router Name Mismatch
SMF Parse Error	SMF Parse Error
SMF Plain Text Is Shutdown	SMF Plain Text Is Shutdown
SMF SSL Is Shutdown	SMF SSL Is Shutdown
SMF TTL Exceeded	SMF TTL Exceeded
SMP Parse Error	SMP Parse Error
SSL Downgrade Not Allowed	SSL Downgrade Not Allowed
SSL certificate not trusted common name	SSL certificate not trusted common name
SSL connecting to non-SSL not allowed	SSL connecting to non-SSL not allowed
Selector Does Not Match	Non-Exclusive DTE bind failed becaused of the different selector presented
Selectors Not Supported on Partitioned Queue	Selectors Not Supported on Partitioned Queue
Service Unavailable	AD Service is unavailable. This is a generic response when an egress flow is deleted.
Service Unavailable	Service Unavailable
Session already exists	Session already exists
Shared Subscription Permission Denied	Shared Subscription Permission Denied
Shared Subscriptions Not Supported On Queues	Shared Subscriptions Not Supported On Queues
Shared Subscriptions Not Supported On Topic Endpoints	Shared Subscriptions Not Supported On Topic Endpoints
Spool Over Quota. Message VPN limit exceeded	A message is being spooled through AD, but the spool quota for the vpn has been exceeded.
Spool Over Quota. Queue or Topic endpoint limit exceeded	A message is being spooled through AD, but the spool quota for the endpoint has been exceeded.
Spool Over Quota. Router limit exceeded	A message is being spooled through AD, but the spool quota for the router has been exceeded.
Subscriber Name Already In Use	Subscriber Name Already In Use
Subscriber msg reassigned	Subscriber msg reassigned
Subscription ACL Denied	Subscription ACL Denied
Subscription Already Exists	Subscription Already Exists
Subscription Attributes Conflict With Existing Subscription	Subscription Attributes Conflict With Existing Subscription
Subscription Does Not Match	None-first Client failed to bind with Non-exclusive DTE because subscription topic doesn't match existing one.
Subscription Manager Denied	Subscription Manager Denied
Subscription Not Found	Subscription Not Found
Subscription Parse Error	Subscription Parse Error
Sync Replication Ineligible	Sync Replication Ineligible
Temporary lockout	The account is temporarily locked out due to too many failed authentication attempts
The Connection Does Not Support Bind to Non-exclusive Queue	The Connection Does Not Support Bind to Non-exclusive Queue
The Cut-Through Forwarding Mode is Not Allowed	The Cut-Through Forwarding Mode is Not Allowed
The IdToken and AccessToken have conflicting data, they were not generated compatibly	The IdToken and AccessToken have conflicting data, they were not generated compatibly
The Issuer Identifier of the profile specified by the provided Profile Name SMF Parameter does not match the specified Issuer Identifier SMF Parameter	The Issuer Identifier of the profile specified by the provided Profile Name SMF Parameter does not match the specified Issuer Identifier SMF Parameter
The JWT's signature is invalid	The JWT's signature is invalid
The LDAP Server is administratively shutdown	The LDAP Server is administratively shutdown
The LDAP Server is down	The LDAP Server is down
The LDAP Server rejects the filter	The LDAP Server rejects the filter
The RADIUS profile is shutdown	The RADIUS profile is shutdown
The Store-And-Forward Forwarding Mode is Not Allowed	The Store-And-Forward Forwarding Mode is Not Allowed
The \active\ member was missing from the introspection response	The \active\ member was missing from the introspection response
The \alg\ header parameter was not \RS256\, \RS384\ or \RS512\	The \alg\ header parameter was not \RS256\, \RS384\ or \RS512\
The \alg\ header parameter was not \RS256\, \RS384\, \RS512\, \ES256\, \ES384\ or \ES512\	The \alg\ header parameter was not \RS256\, \RS384\, \RS512\, \ES256\, \ES384\ or \ES512\
The \alg\ header parameter was not found	The \alg\ header parameter was not found
The \aud\ claim did not match the configured client ID	The \aud\ claim did not match the configured client ID
The \aud\ claim did not match the configured client ID, and no required audience is configured	The \aud\ claim did not match the configured client ID, and no required audience is configured
The \aud\ claim did not match the configured required audience	The \aud\ claim did not match the configured required audience
The \exp\ claim was invalid	The \exp\ claim was invalid
The \exp\ claim was not found	The \exp\ claim was not found
The \iss\ claim did not match the OAuth Profile's issuer	The \iss\ claim did not match the OAuth Profile's issuer
The \iss\ claim was not found	The \iss\ claim was not found
The \scope\ claim did not contain all configured scopes or was otherwise invalid	The \scope\ claim did not contain all configured scopes or was otherwise invalid
The \scope\ claim was not found	The \scope\ claim was not found
The \sub\ claim from the userinfo response did not match the \sub\ claim in the IdToken	The \sub\ claim from the userinfo response did not match the \sub\ claim in the IdToken
The \sub\ claim was not found	The \sub\ claim was not found
The \sub\ claim was too long or otherwise invalid	The \sub\ claim was too long or otherwise invalid
The \typ\ header parameter did not match the configured value	The \typ\ header parameter did not match the configured value
The \typ\ header parameter was not found	The \typ\ header parameter was not found
The introspection attempt failed due to a transport problem	The introspection attempt failed due to a transport problem
The introspection response indicated an error	The introspection response indicated an error
The introspection response indicated that the token is not active	The introspection response indicated that the token is not active
The introspection response was invalid	The introspection response was invalid
The login cannot be processed due to resource limitations	The login cannot be processed due to resource limitations
The login cannot proceed against a shutdown Provider	The login cannot proceed against a shutdown Provider
The login cannot proceed against an inactive Profile	The login cannot proceed against an inactive Profile
The login gave a Provider Name that does not correspond to a configured Profile or Provider	The login gave a Provider Name that does not correspond to a configured Profile or Provider
The login gave no Provider Name and no active default Profile or Provider could be found	The login gave no Provider Name and no active default Profile or Provider could be found
The login provided a Profile Name SMF parameter that does not correspond to an active Profile	The login provided a Profile Name SMF parameter that does not correspond to an active Profile
The login provided an Issuer Identifier SMF parameter that does not correspond to an active Profile	The login provided an Issuer Identifier SMF parameter that does not correspond to an active Profile
The mandatory AccessToken contained no data	The mandatory AccessToken contained no data
The mandatory IdToken contained no data	The mandatory IdToken contained no data
The required Access Level or Authorization Group Claim was found but the value is invalid	The required Access Level or Authorization Group Claim was found but the value is invalid
The required Access Level or Authorization Group Claim was not found	The required Access Level or Authorization Group Claim was not found
The required Audience Claim was found but did not match the configured value	The required Audience Claim was found but did not match the configured value
The required Audience Claim was found but the value is invalid	The required Audience Claim was found but the value is invalid
The required Audience Claim was not found	The required Audience Claim was not found
The required Username Claim was found but did not match API username (and validation is configured)	The required Username Claim was found but did not match API username (and validation is configured)
The required Username Claim was found but the value is invalid	The required Username Claim was found but the value is invalid
The required Username Claim was not found	The required Username Claim was not found
The token issuer ('iss' claim) did not match the issuer of any active profile	The token issuer ('iss' claim) did not match the issuer of any active profile
The transaction has timed out	The transaction has timed out
The userinfo request attempt failed due to a transport problem	The userinfo request attempt failed due to a transport problem
The userinfo request failed because of a an HTTPS error	The userinfo request failed because of a an HTTPS error
The userinfo request timed out waiting for a response from a DNS or Authorization Server	The userinfo request timed out waiting for a response from a DNS or Authorization Server
The userinfo response indicated an error	The userinfo response indicated an error
Token Has Expired	Token Has Expired
Too Many Clients	Client login failed because there are too many clients.
Too Many Connections For Client-Username	Too Many Connections For Client-Username
Too Many Connections For VPN	Too Many Connections For VPN
Too Many Publishers	The request has failed because there are too many publishers. For a Transacted Session, there should only be one publisher. This event is seen if more than one publisher is found for a Transacted Session.
Too Many Subscribers	Too many subscribers. For a Transacted Session, the Maximum Egress Flows has been reached.
Too many Client-Usernames	Too many Client-Usernames
Topic Parse Error	Topic Parse Error
Topic or Selector Modified on Durable Topic Endpoint	A replay was in progress on a Durable Topic Endpoint (DTE) when its topic or selector was modified, causing the replay to fail.
Transacted Session Name In Use	Transacted Session Name In Use
Transacted Session Recreated	Transacted Session Recreated
Transacted session is associated	Transacted session is associated
Transacted session is local	Transacted session is local
Transaction Failure: The number of messages in the transaction exceeds the limit	Transaction Failure: The number of messages in the transaction exceeds the limit
Transaction already exists	Transaction already exists
Transaction is active	Transaction is active
Transaction is heuristically committed	Transaction is heuristically committed
Transaction is heuristically rolled back	Transaction is heuristically rolled back
Transaction is not associated to session	Transaction is not associated to session
Transaction is not idle	Transaction is not idle
Transaction is processing request	Transaction is processing request
Transaction is rollback only	Transaction is rollback only
Transaction not associated to session	Transaction not associated to session
Transaction not heuristically completed	Transaction not heuristically completed
Transaction not prepared	Transaction not prepared
Transaction replication fail	Transaction replication fail
Transaction replication timeout	Transaction replication timeout
Transaction session not found	Transaction session not found
Transform Result Too Big	Transform Result Too Big
Transform error	Transform error
Unauthenticated Authentication Not Allowed	Unauthenticated Authentication Not Allowed
Unhandled Protocol Specified	The protocol specified in the incoming message is unsupported.
Unknown Client Name	Unknown Client Name
Unknown Durable Topic Endpoint	In an operation that requires a Durable Topic Endpoint (DTE), the DTE in question cannot be found.
Unknown Flow Name	Request to open AD flow failed because the flow name is not found.
Unknown Parameter	Received a message with an unknown parameter in the header. This type is also used for configuration errors where an unknown parameter has been entered.
Unknown Protocol	Received a message with a protocol that is not known.
Unknown Queue	Request to operate on a queue failed because an unknown queue id is provided.
Unknown Start Location Type	The request attempted to start a replay but provided an unknown start location type.
Unknown Transacted Session	Unknown Transacted Session
Unknown Transacted Session Name	Unknown Transacted Session Name
Unsubscribe not allowed, client(s) bound to DTE	When client attempt to unsubscribe from a topic, the unsubscribed failed because the client is bound to a DTE.
Unsupported Bridge Version	Unsupported Bridge Version
Unsupported Login Protocol	Received a login attempt that uses an unsupported protocol type.
Unsupported Protocol Version	Received a message with an unsupported Protocol Version.
Unsupported SSL downgrade value	Unsupported SSL downgrade value
Untrusted Certificate	Untrusted Certificate
User profile deny guaranteed	User profile deny guaranteed
Userinfo lookup is required but no userinfo endpoint is configured	Userinfo lookup is required but no userinfo endpoint is configured
Userinfo response is not valid JSON	Userinfo response is not valid JSON
VPN exceeded max transaction	VPN exceeded max transaction
Web-transport Plain Text Is Shutdown	Web-transport Plain Text Is Shutdown
Web-transport SSL Is Shutdown	Web-transport SSL Is Shutdown
XA Transacted Sessions Cannot Bind to Partitioned Queues	A bind to a partitioned queue was requested, but the session is an XA, transacted session
XA join not supported	XA join not supported
XA transactions are not supported	XA transactions are not supported
XASession Cannot Bind to Non-XA Endpoint	XASession Cannot Bind to Non-XA Endpoint
XMLLINK Parse Error	Received an XmlLink message that contains bad data or missing parameters.

Value	Description
Assured Delivery Not Ready	The appliance is unable to process the request because Assured Delivery is not ready. Possibly AD is disabled.
Bad Request	The incoming request could not be understood by the appliance doe to malformed syntax.
Conflict	The request could not be completed due to a conflict with the current state of the resource. For a bi-directional bridge, this reply is used to indicate a compatible bridge is found. The existing client connection should be reused, and the newly created (conflicting) client connection should be taken down.
Created	Successful processing, resulting in the creation of a new resource.
Forbidden	The appliance understood the request, but is refusing to fulfill it.
Gateway Time-out	The request has taken too long for the appliance to process. The appliance has timed out and reply with this message.
Ignore	HTTP protocol: Processing. When the appliance receives this message, it is used as an HTTP keep alive event, and the message is ignored.
Internal Server Error	The appliance encountered an unexpected condition which prevented it from fulfilling the request.
Message dropped on connection close	A message to be sent out on a connection is dropped due to connection close.
Message dropped on connection error	A message to be sent out on a connection is dropped due to connection error.
No Content	This is a response to successful login.
No SEMP Session Available	The broker cannot allocate a session for a SEMP client because there are no free sessions available.
Not Found	The appliance has not found anything matching the request.
Not Implemented	The appliance does not support the functionality required to fulfill the request.
OK	Successful processing.
Out Of Server Resources	The appliance is unable to process the request due to lack of resources.
Service Unavailable	The appliance is currently unable to handle the request due to a temporary overloading or maintenance of the server.
Unauthorized	For an authentication request, this reply indicates that authentication has failed. For other request types, this reply indicates that the user does not have sufficient authorization to perform the request.
Unknown	An unknown error has occurred.

SYSTEM_CLIENT_EG_MSG_RATE_HIGH

Description

This event is sent when the aggregate egress message rate rises above the set threshold value.

To set the threshold value associated with this event, use the 'snmp-server trap egress-msg-rate thresholds set-value' Config CLI command. The default is 4000000 msgs/sec.

To view the threshold value, see the 'egress-msg-rate - Set Value' field of the 'show snmp trap' CLI command output.

To view the current value that the threshold monitors, see the 'Current Rate (1 sec sample) - Egress (msg/sec)' field of the 'show stats client' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

The thresholds can be increased if the message rate is expected and messages lost due to congestion, if any, are tolerable. Otherwise use the 'show client * [message-vpn <>] sorted-stats current-egress-message-rate-per-second' CLI command to determine which clients are most responsible for the large egress message-rate. Follow up with the maintainers of these client applications as necessary.

Message Format

SYSTEM_CLIENT_EG_MSG_RATE_HIGH: - - Egress Message Rate threshold (<threshold> msg/sec) reached: <rate>

SYSTEM_CLIENT_EG_MSG_RATE_HIGH_CLEAR

Description

This event is sent when the aggregate egress message rate falls below the clear threshold value.

To set the threshold value associated with this event, use the 'snmp-server trap egress-msg-rate thresholds clear-value' Config CLI command. The default is 3900000 msgs/sec.

To view the threshold value, see the 'egress-msg-rate - Clear Value' field of the 'show snmp trap' CLI command output.

To view the current value that the threshold monitors, see the 'Current Rate (1 sec sample) - Egress (msg/sec)' field of the 'show stats client' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

SYSTEM_CLIENT_EG_MSG_RATE_HIGH_CLEAR: - - Egress Message Rate threshold (<threshold> msg/sec) cleared: <rate>

SYSTEM_CLIENT_ING_MSG_RATE_HIGH

Description

This event is sent when the aggregate ingress message rate rises above the set threshold value.

To set the threshold value associated with this event, use the 'snmp-server trap ingress-msg-rate thresholds set-value' Config CLI command. The default is 4000000 msgs/sec.

To view the threshold value, see the 'ingress-msg-rate - Set Value' field of the 'show snmp trap' CLI command output.

To view the current value that the threshold monitors, see the 'Current Rate (1 sec sample) - Ingress (msg/sec)' field of the 'show stats client' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

The thresholds can be increased if the message rate is expected and messages lost due to congestion, if any, are tolerable. Otherwise use the 'show client * [message-vpn <>] sorted-stats current-ingress-message-rate-per-second' CLI command to determine which clients are most responsible for the large ingress message-rate. Follow up with the maintainers of these client applications as necessary.

Message Format

SYSTEM_CLIENT_ING_MSG_RATE_HIGH: - - Ingress Message Rate threshold (<threshold> msg/sec) reached: <rate>

SYSTEM_CLIENT_ING_MSG_RATE_HIGH_CLEAR

Description

This event is sent when the aggregate ingress message falls below the clear threshold value.

To set the threshold value associated with this event, use the 'snmp-server trap ingress-msg-rate thresholds clear-value' Config CLI command. The default is 3900000 msgs/sec.

To view the threshold value, see the 'ingress-msg-rate - Clear Value' field of the 'show snmp trap' CLI command output.

To view the current value that the threshold monitors, see the 'Current Rate (1 sec sample) - Ingress (msg/sec)' field of the 'show stats client' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

SYSTEM_DNS_NAME_SERVER_DOWN

Description

This event is generated when a name server that was previously reachable has become unreachable.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

This event is informational. No action is required.

Message Format

SYSTEM_DNS_NAME_SERVER_DOWN: - - Name server <serverName> is operationally down.

SYSTEM_DNS_NAME_SERVER_UP

Description

Description

This event is sent when the number of connections to Kafka Brokers, across all Kafka Senders and Receivers, exceeds the maximum available for the Solace Broker. The number of connections required depends on:

1) the number of operational Kafka Senders and Receivers

2) the number of Kafka Brokers in the Kafka Clusters

Each Kafka Sender and Receiver requires approximately 1 connection for every Kafka Broker in its Kafka Cluster.

This event will likely be accompanied with Kafka Senders and Receivers becoming operationally down with failure reasons such as 'Too many other broker connections in use' or 'Too many broker connections required'.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

Identify Kafka Receivers which connect to the same Kafka Cluster, and collapse them together by using multiple Topic Bindings with a single Kafka Receiver instead.

Identify Kafka Senders which connect to the same Kafka Cluster, and collapse them together by using multiple Queue Bindings with a single Kafka Sender instead.

Consider increasing the number of Kafka Broker Connections which are allowed by changing the appropriate system scaling parameter. Increasing this limit may require significantly more system resources.

Once sufficient Kafka Broker Connections are available, individual Kafka Senders and Receivers may need to be disabled and re-enabled in order to become operational.

Message Format

SYSTEM_KAFKA_BROKER_CONNECTIONS_EXCEEDED: - - Kafka broker connections limit (<limit>) exceeded

SYSTEM_KAFKA_BROKER_CONNECTIONS_HIGH

Description

This event is sent when the number of connections to Kafka Brokers, across all Kafka Senders and Receivers, exceeds the set threshold limit.

The default is 80% of the maximum value (see the SYSTEM_KAFKA_BROKER_CONNECTIONS_EXCEEDED event) and cannot be modified.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

See the 'Recommended Actions' for the SYSTEM_KAFKA_BROKER_CONNECTIONS_EXCEEDED event.

Message Format

SYSTEM_KAFKA_BROKER_CONNECTIONS_HIGH: - - Kafka broker connections limit threshold (<thresholdPercentage>%) reached: <actualPercentage>%

SYSTEM_KAFKA_BROKER_CONNECTIONS_HIGH_CLEAR

Description

This event is sent when the number of connections to Kafka Brokers, across all Kafka Senders and Receivers, falls below the clear threshold limit.

SYSTEM_LOGGING_LOST_EVENTS

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

CRIT

Recommended Action

Copy the message text exactly as it appears on the console or in the event log, contact your Solace technical support representative, and provide the representative with the gathered information. Also provide the system configuration information, along with any other relevant information.

Message Format

SYSTEM_NAB_CRITICAL_HARDWARE_FAILURE: - - Slot <fabric>/<slot> Critical hardware failure: component <compStr> type <typeStr> info <infoStr>

SYSTEM_NAB_CRITICAL_HARDWARE_SOFT_FAILURE

Description

This event message is sent when the appliance detects an error in the NAB firmware.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

CRIT

Recommended Action

Copy the message text exactly as it appears on the console or in the event log, contact your Solace technical support representative, and provide the representative with the gathered information. Also provide the system configuration information, along with any other relevant information.

Message Format

SYSTEM_NAB_CRITICAL_HARDWARE_SOFT_FAILURE: - - Slot <fabric>/<slot> Critical hardware soft failure: component <compStr> type <typeStr> info <infoStr>

SYSTEM_NAB_NONCRITICAL_HARDWARE_NOTIFICATION

Description

This event message is sent when the appliance detects a recoverable failure on the NAB (for example, an Error Correcting Code [ECC] error) which has been automatically repaired.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

No immediate action is required. However, if this message recurs repeatedly, indicating a continued condition, contact your Solace technical support representative.

Message Format

SYSTEM_NAB_NONCRITICAL_HARDWARE_NOTIFICATION: - - Slot <fabric>/<slot> Non-critical hardware notification: component <compStr> type <typeStr> info <infoStr>

SYSTEM_NAB_NONCRITICAL_HARDWARE_SOFT_FAILURE

Description

This event message is sent when the NAB hardware is placed into an invalid state by software. This is a self-recovering condition.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

NOTICE

Recommended Action

No immediate action is required. However, if this message recurs repeatedly, indicating a continued condition, contact your Solace technical support representative.

Message Format

SYSTEM_NAB_NONCRITICAL_HARDWARE_SOFT_FAILURE: - - Slot <fabric>/<slot> Non-critical hardware soft notification: component <compStr> type <typeStr> info <infoStr>

SYSTEM_NAB_ZIP_ENGINE_RESTART_COMPLETE

Description

The NAB has successfully restarted the ZIP engine to recover from a fault.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

NOTICE

Recommended Action

No immediate action is required. However, if this message recurs repeatedly, indicating a continued condition, contact your Solace technical support representative.

Message Format

SYSTEM_NAB_ZIP_ENGINE_RESTART_COMPLETE: - - Slot <fabric>/<slot> ZIP engine restart complete: ZIP info: ingress decompression <forcedIngressErrorCount>, egress compression <forcedEgressErrorCount>; ZIP command buffers remaining: <zipCmdBufsFree>.

SYSTEM_NAB_ZIP_ENGINE_RESTART_INITIATED

Description

SYSTEM_PTP_SYNC_DOWN

Description

This event is generated when PTP is no longer in sync.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

Check management network and PTP master for issues.

Message Format

SYSTEM_PTP_SYNC_DOWN: - - PTP sync is operationally down.

SYSTEM_PTP_SYNC_UP

Description

Description

This event is generated when a message is discarded because it cannot be sent to a neighbor.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

No immediate action is required. However, you should clear and reset this one-shot event through the 'routing cspf neighbor <> clear-event egress-message-discard' Admin CLI command to allow it to be generated again.

Run the 'show cspf neighbor <> detail' CLI command on the message broker to display if the event is raised and if so when it is raised.

Message Format

SYSTEM_ROUTING_CSPF_NBR_EGRESS_MSG_DISCARD: - - CSPF Neighbor <nbrName> Connection (<srcIpPort> - <destIpPort>)

SYSTEM_ROUTING_CSPF_NBR_UP

Description

This event is sent when a neighbor link is established. This means at least one TCP connection has been established with the neighbor.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

SYSTEM_ROUTING_CSPF_NBR_UP: - - CSPF Neighbour <destIpPort> (<nbrRouterName>) changed from <oldState> to <newState> due to: <reason>

SYSTEM_ROUTING_SMRP_DB_CLEANUP_COMPLETE

Description

The message broker has completed cleaning up stale subscriptions from its database.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

SYSTEM_ROUTING_SMRP_DB_CLEANUP_COMPLETE: - - SMRP DB cleanup complete for router-name <routerName>; reason: <reason>. <cleaned>/<total> subscriptions cleaned up.

SYSTEM_ROUTING_SMRP_DB_CLEANUP_HOLDDOWN_INTERVAL_COMPLETE

Description

The hold down period for cleaning up stale subscriptions for a router-name has expired. Subscriptions which have not been re-added by their clients will start being removed from the database.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

SYSTEM_ROUTING_SMRP_DB_CLEANUP_HOLDDOWN_INTERVAL_COMPLETE: - - SMRP DB cleanup hold down interval for router-name <routerName> is complete; reason: <reason>.

SYSTEM_ROUTING_SMRP_DB_CLEANUP_HOLDDOWN_INTERVAL_STARTED

Description

The message broker has taken ownership (activity) for a router-name and is waiting for clients to connect. Once this hold down period expires, existing subscriptions in the database which have not been re-added are removed.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

SYSTEM_ROUTING_SMRP_DB_CLEANUP_HOLDDOWN_INTERVAL_STARTED: - - SMRP DB cleanup hold down interval for router-name <routerName> was started. Waiting for clients to re-add subscriptions before DB cleanup.

SYSTEM_ROUTING_SMRP_ROUTER_NAME_DELETED

Description

A message broker that had been reachable in the network has been deleted. This may be due to an administrative action or an age-out situation after being unreachable for 30 minutes.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

If the event is unexpected check CSPF connectivity on the message broker that has been deleted to ensure it is active on the same CSPF network as this message broker.

Message Format

SYSTEM_ROUTING_SMRP_ROUTER_NAME_DELETED: - - SMRP router-name <routerName> deleted: reason <reason>.

SYSTEM_ROUTING_SUBSCRIPTIONS_LOAD_FACTOR_HIGH

Description

This event message is sent when the percentage of subscription resources in use by the message broker rises above the set threshold value. There are multiple resources, and this event is driven by the highest utilization of them.

To set the threshold value associated with this event, use the 'memory-event subscription-load-factor thresholds set-percentage' Config CLI command. The default is 80%

To view the threshold value, see the 'Subscriptions Load Factor (%) - Set Value' field of the show memory' CLI command output.

To display the current value that the threshold monitors, and the name of the resource with highest utilizaiton, see the 'Subscription Load Factor' field of the 'show memory' CLI command.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

The number of subscriptions does not map directly to subscription load factor although there is a rough correlation. Resources are required for local client and queue subscriptions, and also remote subscriptions learned from other brokers in the MNR or DMR network.

The 'show smrp subscriptions summary' command can be used to see how many local and remote subscriptions the broker has.

Resource usage is also influenced by many other factors, such as subscription length, topic-level patterns, and amount of topic duplication. For more detailed analysis of resource usage contact a Solace technical support representative.

Message Format

SYSTEM_ROUTING_SUBSCRIPTIONS_LOAD_FACTOR_HIGH: - - Subscriptions load factor high threshold (<thresholdPercentage>%) exceeded: <actualPercentage>%

SYSTEM_ROUTING_SUBSCRIPTIONS_LOAD_FACTOR_HIGH_CLEAR

Description

This event message is sent when the percentage of subscription resources in use by the message broker falls below the clear threshold value. There are multiple resources, and this event is driven by the highest utilization of them.

To set the threshold value associated with this event, use the 'memory-event subscription-load-factor thresholds clear-percentage' Config CLI command. The default is 60%

To view the threshold value, see the 'Subscriptions Load Factor (%) - Clear Value' field of the show memory' CLI command output.

SYSTEM_SOLCACHE_MAX_MEMORY_EXCEED

Description

This is an event sent when the amount of memory used by a cache process exceeds the maximum allowed. As a result, messages have been discarded by the cache process.

To set the maximum value associated with this event use the 'mqtt retain-cache max-memory-global' Config CLI command. The default is 4294967294 MB.

To view the maximum value, see the 'Max Memory (global) (MB)' field of the 'show mqtt' CLI command output.

This event will force the cache instance that raised the event to enter the lost message state if it is not already in it. See the VPN_SOLCACHE_LOST_MSG_STATE event for further information.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

The maximum amount of memory allowed for a cache process can be increased if the large memory usage is expected and more is available on the host system on which the cache process is running. Otherwise you can do one of the following to reduce the memory used by the cache process:

1) Reduce the maximum number of individual topics cached. See the 'distributed-cache <> cache-cluster <> max-topics' Config CLI command.

2) Reduce the number of messages cached per individual topic. See the 'distributed-cache <> cache-cluster <> max-messages-per-topic' Config EXEC CLI command.

3) Reduce the number of topics cached and/or redistribute the cached topic space amongst other cache clusters to ease the request rate experienced by this cache process.

Message Format

SYSTEM_SOLCACHE_MAX_MEMORY_EXCEED: - - <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	WARN
evt-type	String	SYSTEM
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	SYSTEM_SOLCACHE_MAX_MEMORY_EXCEED
evt-version	INT32	Version Number
user-id	String	ID that uniquely identifies the cache process sending the event

SYSTEM_SOLCACHE_MAX_REQUEST_QUEUE_DEPTH_EXCEED

Description

This is an event sent when the cache request queue depth used by a cache process exceeds the maximum allowed. As a result, messages have been discarded by the cache process.

To set the maximum value associate with this event use the CACHE_MAX_REQUEST_QUEUE_DEPTH_GLOBAL configuration file parameter. The default is 2147483647.

There is currently no supported way to view the maximum value.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

The maximum number of queued cache requests allowed for a cache process can be increased if the large number of cache requests is expected.

Message Format

SYSTEM_SOLCACHE_MAX_REQUEST_QUEUE_DEPTH_EXCEED: - - <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	WARN
evt-type	String	SYSTEM
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	SYSTEM_SOLCACHE_MAX_REQUEST_QUEUE_DEPTH_EXCEED
evt-version	INT32	Version Number
user-id	String	ID that uniquely identifies the cache process sending the event

SYSTEM_SOLCACHE_MEMORY_HIGH

Description

This event is sent when the amount of memory used by a cache process rises above the set threshold percentage of maximum (see SYSTEM_SOLCACHE_MAX_MEMORY_EXCEED).

The default is 80% of the maximum value (see the SYSTEM_SOLCACHE_MAX_MEMORY_EXCEED event) and can not be modified.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

If the large message rate is expected and not resulting in message loss, then either the maximum memory (SYSTEM_SOLCACHE_MAX_MEMORY_EXCEED) value should be increased or the threshold percentage can be increased. Otherwise reduce the memory used by the cache process as described in the Recommended Actions of the SYSTEM_SOLCACHE_MAX_MEMORY_EXCEED event.

Message Format

SYSTEM_SOLCACHE_MEMORY_HIGH: - - <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	WARN
evt-type	String	SYSTEM
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	SYSTEM_SOLCACHE_MEMORY_HIGH
evt-version	INT32	Version Number
user-id	String	ID that uniquely identifies the cache process sending the event

SYSTEM_SOLCACHE_MEMORY_HIGH_CLEAR

Description

This event is sent when the amount of memory used by a cache process falls below the clear threshold percentage of maximum memory.

The default is 60% of the maximum value (see the SYSTEM_SOLCACHE_MAX_MEMORY_EXCEED event) and can not be modified.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

SYSTEM_SOLCACHE_MEMORY_HIGH_CLEAR: - - <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	INFO
evt-type	String	SYSTEM
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	SYSTEM_SOLCACHE_MEMORY_HIGH_CLEAR
evt-version	INT32	Version Number
user-id	String	ID that uniquely identifies the cache process sending the event

SYSTEM_SOLCACHE_REQUEST_QUEUE_DEPTH_HIGH

Description

This event is sent when the number of queued cache requests exceeds the set threshold percentage of maximum (see SYSTEM_SOLCACHE_MAX_REQUEST_QUEUE_DEPTH_EXCEED).

To set the threshold percentage value associate with this event use the CACHE_MAX_REQUEST_QUEUE_DEPTH_GLOBAL_SET_THRESHOLD configuration file parameter. The default is 80%.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

If a large number of queued cache requests is expected and not resulting in request loss, then either the maximum request queue depth (SYSTEM_SOLCACHE_MAX_REQUEST_QUEUE_DEPTH_EXCEED) value should be increased or the threshold percentage should be increased. Otherwise reduce the cache request burst of the system as to not overwhelm the cache process.

Message Format

SYSTEM_SOLCACHE_REQUEST_QUEUE_DEPTH_HIGH: - - <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	WARN
evt-type	String	SYSTEM
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	SYSTEM_SOLCACHE_REQUEST_QUEUE_DEPTH_HIGH
evt-version	INT32	Version Number
user-id	String	ID that uniquely identifies the cache process sending the event

SYSTEM_SOLCACHE_REQUEST_QUEUE_DEPTH_HIGH_CLEAR

Description

This event is sent when the number of queued cache requests falls below the clear threshold percentage of maximum (see SYSTEM_SOLCACHE_MAX_REQUEST_QUEUE_DEPTH_EXCEED).

To set the threshold percentage value associate with this event use the CACHE_MAX_REQUEST_QUEUE_DEPTH_GLOBAL_CLEAR_THRESHOLD configuration file parameter. The default is 60%.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

SYSTEM_SOLCACHE_REQUEST_QUEUE_DEPTH_HIGH_CLEAR: - - <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	INFO
evt-type	String	SYSTEM
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	SYSTEM_SOLCACHE_REQUEST_QUEUE_DEPTH_HIGH_CLEAR
evt-version	INT32	Version Number
user-id	String	ID that uniquely identifies the cache process sending the event

SYSTEM_SSL_CONNECTIONS_HIGH_CLEAR: - - SSL Connections threshold (<threshold>) cleared: <value>

Description

To view the current value associated with the Message VPN-based event that the threshold monitors, see the 'Current Spool Usage (MB)' field of the 'show message-spool message-vpn <>' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

The thresholds can be increased if the large amount of spool usage is expected; otherwise, see the 'Recommended Actions' for the SYSTEM_AD_MSG_SPOOL_QUOTA_EXCEED event.

Message Format

VPN_AD_MSG_SPOOL_HIGH: <vpnName> - Message VPN (<ID>) <qtype> <qname> message spool threshold <threshold> kB (<thresholdPercentage>%) reached: <value> kB

VPN_AD_MSG_SPOOL_HIGH_CLEAR

Description

This event is sent when the aggregate size of all messages enqueued to a specific endpoint falls below the clear threshold value, or the aggregate size of all messages enqueued within a Message VPN falls below the set threshold for that specific Message VPN.

To set the threshold value associated with the endpoint-based event, use the 'message-spool message-vpn <> queue|topic-endpoint event spool-usage thresholds clear-value/clear-percentage' Config CLI command. The default is 60% of the maximum value (see the VPN_AD_MSG_SPOOL_QUOTA_EXCEED event)

To set the threshold value associated with the Message VPN-based event, use the 'message-spool message-vpn <> event spool-usage thresholds clear-value/clear-percentage' Config CLI command. The default is 60% of the maximum value (see the VPN_AD_MSG_SPOOL_QUOTA_EXCEED event)

To view the threshold value associated with the endpoint-based event, see 'Spool usage (MB) - Clear Value' field of the 'show queue|topic-endpoint <> message-vpn <>' CLI command output.

To view the threshold value associated with the Message VPN-based event, see 'Spool usage (MB) - Clear Value' field of the 'show message-spool message-vpn <>' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

VPN_AD_MSG_SPOOL_HIGH_CLEAR: <vpnName> - Message VPN (<ID>) <qtype> <qname> message spool threshold <threshold> kB (<thresholdPercentage>%) cleared: <value> kB

VPN_AD_MSG_SPOOL_QUOTA_EXCEED

Description

This event is sent when the aggregate size of all messages enqueued to a specific endpoint exceeds the configured maximum for that endpoint or the aggregate size of all messages enqueued within a Message VPN exceeds the configured maximum for that specific Message VPN.

When a message is discarded from an individual endpoint because the quota has been exceeded, an event such as the following is raised:

2010-04-23T21:42:02+0000 <local3.warning> hostname event: VPN: VPN_AD_MSG_SPOOL_QUOTA_EXCEED: vpnName - Message VPN (2) Topic endpoint myTE message spool quota (102400 kB) exceeded, messages discarded

Whereas when a message is discarded from a Message VPN because the quota has been exceeded, an event such as the following is raised:

2010-04-23T21:54:42+0000 <local3.warning> hostname event: VPN: VPN_AD_MSG_SPOOL_QUOTA_EXCEED: vpnName - Message VPN (2) vpnName message spool quota (153600 kB) exceeded; messages discarded.

To set the maximum value associated with the endpoint-based event, use the 'message-spool message-vpn <> queue|topic-endpoint max-spool-usage' Config CLI command. The default is 4000 MB.

To set the maximum value associated with the Message VPN-based event, use the 'message-spool message-vpn <> max-spool-usage' Config CLI command. The default is 30000 MB.

To view the maximum value associated with the endpoint-based event, see 'Quota (MB)' field of the 'show queue|topic-endpoint <> message-vpn <>' CLI command output.

To view the maximum value associated with the Message VPN-based event, see 'Max Allowed Spool Usage (MB)' field of the 'show message-spool message-vpn <>' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

See the 'Recommended Actions' for the SYSTEM_AD_MSG_SPOOL_QUOTA_EXCEED event.

Message Format

VPN_AD_MSG_SPOOL_QUOTA_EXCEED: <vpnName> - Message VPN (<ID>) <qtype> <qname> message spool quota (<quota> kB) exceeded, messages discarded

VPN_AD_MSG_SPOOL_REJECT_LOW_PRIORITY_MSG_LIMIT_EXCEED

Description

This event is sent when the messages enqueued to a specific endpoint exceeds the configured reject-low-priority-msg-limit. No new low priority messages will be admitted to the endpoint.

To set the reject-low-priority-msg-limit value associated with the endpoint, use the 'reject-low-priority-msg-limit' Config CLI command of 'message-spool message-vpn <> queue|topic-endpoint. The default is 0 (that is, disabled).

To view the reject-low-priority-msg-limit value associated with the endpoint, see 'Reject Low-Priority-Msg Limit' field of the 'show queue|topic-endpoint <> message-vpn <>' CLI command output.

To view the current state of Low-Priority-Msg Congestion associated with the endpoint, see 'Low-Priority-Msg Congestion State' field of the'show queue|topic-endpoint <> message-vpn <>' CLI command output.

Event Level Severity

WARN

Recommended Action

Application specific behavior, no action is required.

Message Format

VPN_AD_MSG_SPOOL_REJECT_LOW_PRIORITY_MSG_LIMIT_EXCEED: <vpnName> - Message VPN (<ID>) <qtype> <qname> reject-low-priority-msg-limit (<limit>) exceeded

VPN_AD_MSG_SPOOL_REJECT_LOW_PRIORITY_MSG_LIMIT_HIGH

Description

This event is sent when the messages enqueued to a specific endpoint exceeds the set threshold value of configured reject-low-priority-msg-limit.

To set the threshold value associated with the endpoint-based event, use the 'event reject-low-priority-msg-limit thresholds set-value/set-percentage' Config CLI command of 'message-spool message-vpn <> queue|topic-endpoint'. The default is 80% of the maximum value

To view the threshold value associated with the endpoint-based event, see 'Reject Low-Priority-Msg Limit - Set Value' field of the 'show queue|topic-endpoint <> message-vpn <>' CLI command output.

Event Level Severity

WARN

Recommended Action

Application specific behavior, no action is required.

Message Format

VPN_AD_MSG_SPOOL_REJECT_LOW_PRIORITY_MSG_LIMIT_HIGH: <vpnName> - Message VPN (<ID>) <qtype> <qname> reject-low-priority-msg-limit threshold <threshold> (<thresholdPercentage>%) reached: <value>

VPN_AD_MSG_SPOOL_REJECT_LOW_PRIORITY_MSG_LIMIT_HIGH_CLEAR

Description

This event is sent when the messages enqueued to a specific endpoint falls below the clear threshold value of configured reject-low-priority-msg-limit.

To set the threshold value associated with the endpoint-based event, use the 'event reject-low-priority-msg-limit thresholds clear-value/clear-percentage' Config CLI command of 'message-spool message-vpn <> queue|topic-endpoint'. The default is 60% of the maximum value

To view the threshold value associated with the endpoint-based event, see 'Reject Low-Priority-Msg Limit - Clear Value' field of the 'show queue|topic-endpoint <> message-vpn <>' CLI command output.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

VPN_AD_MSG_SPOOL_REJECT_LOW_PRIORITY_MSG_LIMIT_HIGH_CLEAR: <vpnName> - Message VPN (<ID>) <qtype> <qname> reject-low-priority-msg-limit threshold <threshold> (<thresholdPercentage>%) cleared: <value>

VPN_AD_PARTITIONED_QUEUE_REBALANCE_COMPLETED

Description

This event is raised when a partitioned queue completes rebalancing its partitions across connected clients.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

VPN_AD_PARTITIONED_QUEUE_REBALANCE_COMPLETED: <vpnName> - VPN (<ID>) queue <qname> partition rebalance completed

VPN_AD_PARTITIONED_QUEUE_REBALANCE_STARTED

Description

This event is raised when a partitioned queue starts rebalancing its partitions across connected clients.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

VPN_AD_PARTITIONED_QUEUE_REBALANCE_STARTED: <vpnName> - VPN (<ID>) queue <qname> partition rebalance started

VPN_AD_PARTITIONED_QUEUE_SCALING_COMPLETED

Description

This event is raised when a partitioned queue completes scaling its partition count.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

VPN_AD_PARTITIONED_QUEUE_SCALING_COMPLETED: <vpnName> - VPN (<ID>) queue <qname> partition scaling completed

VPN_AD_PARTITIONED_QUEUE_SCALING_STARTED

Description

This event is raised when a partitioned queue starts scaling its partition count.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

VPN_AD_PARTITIONED_QUEUE_SCALING_STARTED: <vpnName> - VPN (<ID>) queue <qname> partition scaling started

VPN_AD_QENDPT_CREATE

Description

This event is sent when a message endpoint is created.

Event Level Severity

NOTICE

Recommended Action

This event is informational. No action is required.

Message Format

VPN_AD_QENDPT_CREATE: <vpnName> - Message VPN (<ID>) <qtype> <qname> created

VPN_AD_QENDPT_DELETE

Description

This event is sent when a CLI-created endpoint is deleted from the CLI or when the bind count of a temporary endpoint drops to zero.

Event Level Severity

NOTICE

Recommended Action

This event is informational. No action is required.

Message Format

VPN_AD_QENDPT_DELETE: <vpnName> - Message VPN (<ID>) <qtype> <qname> deleted, final statistics - spool(<lastMsgIdSpooled>, <lowMsgIdAcked>, <highMsgIdAcked>, <msgRedelivered>, <msgTransportRetransmits>, <spoolUsageExceeded>, <bytesSpooled>, <msgsSpooled>) bind(<bindRequests>, <bindRespOk>, <bindRespAlreadyBound>, <bindRespExceededClients>, <bindRespOther>)

VPN_AD_REPLAY_STATE_TRANSITION

Description

This event is sent when a message replay transitions to a new state.

Event Level Severity

NOTICE

Recommended Action

This event is informational. No action is required.

Message Format

VPN_AD_REPLAY_STATE_TRANSITION: <vpnName> - Message VPN <vpnName> (<ID>) <endpointType> <endpointName> message replay state transitioned from (<originalState>) to (<newState>), Seconds Since Replay Started: (<secondsSinceReplayStarted>).

VPN_AD_REPLAY_STATE_TRANSITION_TO_FAILED

Description

This event is sent when a message replay transitions to the failed state.

Event Level Severity

WARN

Recommended Action

This event is informational. No action is required.

Message Format

VPN_AD_REPLAY_STATE_TRANSITION_TO_FAILED: <vpnName> - Message VPN <vpnName> (<ID>) <endpointType> <endpointName> message replay state transitioned from (<originalState>) to (Failed), Seconds Since Replay Started: (<secondsSinceReplayStarted>), Reason: (<reason>).

VPN_AD_SEQUENCED_TOPIC_NUMBER_CHANGE

Description

This event is sent when the sequence number for a sequenced topic is administratively changed through the 'message-spool <> message-vpn <> sequenced-topic' Config CLI command.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Description

Two OAuth Profiles have the same issuer. As the issuer is used to uniquely identify the Profile on login, only the first Profile will be used.

Event Level Severity

WARN

Recommended Action

Disable one of the OAuth Profiles or change its issuer.

Message Format

VPN_AUTHENTICATION_DUPLICATE_ISSUER: <vpnName> - Message VPN (<ID>) Profile <profileName> has same issuer as Profile <otherProfileName> (<issuer>). The first profile will be used.

VPN_BRIDGING_BRIDGE_STALLED: <vpnName> - Message VPN (<ID>) <name> Bridge <bridgeName> from <remoteRouterName> VPN <remoteMessageVpn> stalled.

VPN_BRIDGING_LINK_CONFIG_MISMATCH

Description

This event message is sent when there is an attempt to establish a bi-directional bridge but both sides are using incompatible transport configurations. For example, the local bridge is configured to use SSL or compression but the remote bridge is not.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

Ensure that both sides of a bi-directional bridge are using identical transport configurations.

Message Format

VPN_BRIDGING_LINK_CONFIG_MISMATCH: <vpnName> - Message VPN (<ID>) <name> Bridge <bridgeName> from <remoteRouterName> VPN <remoteMessageVpn> config mismatch: <reason>

VPN_BRIDGING_LINK_DOWN

Description

This event message is sent when a Message VPN bridging link has gone down.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

Determine whether there have been any configuration changes on either side of the bridging link. This event is expected if a Message VPN or bridge is administratively disabled or if any other configuration change is performed which affects messaging connectivity. If no such change has been made, validate IP connectivity between both ends of the bridge. This is done by using the 'ping msg-backbone:<address>' CLI command on appliances, or 'ping <address>' on software brokers. The <address> of the bridge can be obtained using the 'show bridge <> detail' CLI command, and consulting the list of remote message-VPNs that are configured for the bridge.

Message Format

VPN_BRIDGING_LINK_DOWN: <vpnName> - Message VPN (<ID>) <name> Bridge <bridgeName> from <remoteRouterName> VPN <remoteMessageVpn> down: <reason>

VPN_BRIDGING_LINK_REJECTED

Description

This event message is sent when a Message VPN bridging link request is rejected by the remote message broker.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

NOTICE

Recommended Action

Consult the reason field in the event text which explains why the link request was rejected.

Message Format

VPN_BRIDGING_LINK_REJECTED: <vpnName> - Message VPN (<ID>) <name> Bridge <bridgeName> from <remoteRouterName> VPN <remoteMessageVpn> rejected: <reason>

VPN_BRIDGING_LINK_TTL_EXCEEDED

Description

This is a one-shot event message sent when a message has been discarded due to it exceeding the allowed number of bridging hops.

Event Level Severity

WARN

Recommended Action

This event typically indicates a loop in the network VPN bridging topology. Validate the VPN bridges configured in this network using the 'show bridge *' and 'show bridge <> detail' CLI commands. Ensure the Maximum TTL parameters are set correctly and there is no bridging loop configured.

This one-shot event message needs to be cleared and reset through the 'bridge <> message-vpn <> clear-event ttl-exceeded' Admin CLU command to be generated again.

Message Format

VPN_BRIDGING_LINK_TTL_EXCEEDED: <vpnName> - Message VPN (<ID>) <name> Bridge <bridgeName> from <remoteRouterName> VPN <remoteMessageVpn> TTL Exceeded

See the 'Recommended Actions' for the SYSTEM_CLIENT_CONNECTIONS_HIGH event.

Message Format

VPN_CLIENT_USERNAME_CONNECTIONS_EXCEEDED: <vpnName> - Message VPN (<ID>) <name> client-username <clientUsername> Maximum connections exceeded : <threshold> Service (<service>)

VPN_CLIENT_USERNAME_CONNECTIONS_HIGH

Description

This event is sent when the number of client connections associated with specified client-username rises above the set threshold value.

To set the threshold value associated with this event, use the 'client-profile <> event connections-per-client-username event thresholds set-value/set-percentage' Config CLI command. The default is 80% of the maximum value.

To view the threshold value, see the 'Connections per client-username - Set Value' field of the 'show client-profile <> detail' CLI command output.

To view the current value that the threshold monitors, see the '# Clients' field of the 'show client-username <> message-vpn <> detail' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

The threshold can be increased if the number of clients using the client username is expected. Otherwise reduce client username use to ease demand.

Message Format

VPN_CLIENT_USERNAME_CONNECTIONS_HIGH: <vpnName> - Message VPN (<ID>) <name> client-username <clientUsername> connections threshold <threshold>(<thresholdPercentage>%) reached: <currentConns> Service (<service>)

VPN_CLIENT_USERNAME_CONNECTIONS_HIGH_CLEAR

Description

This event is sent when the number of client connections associated with specified client-username falls below the clear threshold value.

To set the threshold value associated with this event, use the 'client-profile <> event connections-per-client-username event thresholds clear-value/clear-percentage' Config CLI command. The default is 60% of the maximum value.

To view the threshold value, see the 'Connections per client-username - Clear Value' field of the 'show client-profile <> detail' CLI command output.

To view the current value that the threshold monitors, see the '# Clients' field of the 'show client-username <> message-vpn <> detail' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

VPN_CLIENT_USERNAME_CONNECTIONS_HIGH_CLEAR: <vpnName> - Message VPN (<ID>) <name> client-username <clientUsername> connections threshold <threshold>(<thresholdPercentage>%) cleared: <currentConns> Service (<service>)

Kafka syslog Messages

VPN_KAFKA_BROKER_CONNECTIONS_EXCEEDED

Description

This event is sent when the number of connections to Kafka Brokers, across all Kafka Senders and Receivers of the VPN, exceeds the maximum available for the VPN. The number of connections required depends on:

1) the number of operational Kafka Senders and Receivers

2) the number of Kafka Brokers in the Kafka Clusters

Each Kafka Sender and Receiver requires approximately 1 connection for every Kafka Broker in its Kafka Cluster.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

Identify Kafka Receivers of this VPN which connect to the same Kafka Cluster, and collapse them together by using multiple Topic Bindings with a single Kafka Receiver instead.

Identify Kafka Senders of this VPN which connect to the same Kafka Cluster, and collapse them together by using multiple Queue Bindings with a single Kafka Sender instead.

Consider increasing the number of Kafka Broker Connections which are allowed for this VPN by changing the max-broker-connections parameter.

Once sufficient Kafka Broker Connections are available, individual Kafka Senders and Receivers may need to be disabled and re-enabled in order to become operational.

Message Format

VPN_KAFKA_BROKER_CONNECTIONS_EXCEEDED: <vpnName> - Message VPN (<ID>) <name> Kafka broker connections limit (<limit>) exceeded

VPN_KAFKA_BROKER_CONNECTIONS_HIGH

Description

This event is sent when the number of connections to Kafka Brokers, across all Kafka Senders and Receivers of the VPN, exceeds the set threshold limit for the VPN.

The default is 80% of the maximum value (see the VPN_KAFKA_BROKER_CONNECTIONS_EXCEEDED event) and cannot be modified.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

See the 'Recommended Actions' for the VPN_KAFKA_BROKER_CONNECTIONS_EXCEEDED event.

Message Format

VPN_KAFKA_BROKER_CONNECTIONS_HIGH: <vpnName> - Message VPN (<ID>) <name> Kafka broker connections limit threshold (<thresholdPercentage>%) reached: <actualPercentage>%

VPN_KAFKA_BROKER_CONNECTIONS_HIGH_CLEAR

Description

This event is sent when the number of connections to Kafka Brokers, across all Kafka Senders and Receivers of the VPN, falls below the clear threshold limit for the VPN.

The default is 60% of the maximum value (see the VPN_KAFKA_BROKER_CONNECTIONS_EXCEEDED event) and cannot be modified.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

VPN_KAFKA_BROKER_CONNECTIONS_HIGH_CLEAR: <vpnName> - Message VPN (<ID>) <name> Kafka broker connections limit threshold (<thresholdPercentage>%) cleared: <actualPercentage>%

VPN_KAFKA_RECEIVER_DOWN

Description

This event message is sent when a Kafka Receiver is down and no longer communicating with its Kafka Cluster.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

Determine whether there have been any configuration changes on either the Kafka Receiver or on the Kafka Cluster it is connecting to. This event is expected if a Message VPN or Kafka Receiver is administratively disabled or if any other configuration change is performed which affects messaging connectivity. If no such change has been made, validate IP connectivity to the bootstrap addresses by using the 'ping <address>' CLI command.

Message Format

VPN_KAFKA_RECEIVER_DOWN: <vpnName> - Message VPN (<ID>) <name> Kafka-Receiver <kafkaName> down (<reason>)

VPN_KAFKA_RECEIVER_UP

Description

This event message is sent when a Kafka Receiver is up and communicating with its Kafka Cluster.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

This event message is informational. No action is required.

Message Format

VPN_KAFKA_RECEIVER_UP: <vpnName> - Message VPN (<ID>) <name> Kafka-Receiver <kafkaName> up

VPN_KAFKA_SENDER_DOWN

Description

This event message is sent when a Kafka Sender is down and no longer communicating with its Kafka Cluster.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

Determine whether there have been any configuration changes on either the Kafka Sender or on the Kafka Cluster it is connecting to. This event is expected if a Message VPN or Kafka Sender is administratively disabled or if any other configuration change is performed which affects messaging connectivity. If no such change has been made, validate IP connectivity to the bootstrap addresses by using the 'ping <address>' CLI command.

Message Format

VPN_KAFKA_SENDER_DOWN: <vpnName> - Message VPN (<ID>) <name> Kafka-Sender <kafkaName> down (<reason>)

VPN_KAFKA_SENDER_UP

Description

VPN_REPLICATION_SERVICE_DEGRADED

Description

This event is sent to indicate that asynchronous replication-mode will be used for synchronous replication-mode messages.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

This event indicates that replicated messages cannot be delivered to the standby site as fast as they are being received by the active site. Synchronous replication reverts to asynchronous replication to avoid excessive acknowledgment delays to publishers producing synchronous replicated messages. Verify stable link connectivity by consulting the 'Queue State' field of the 'show message-vpn <> replication' CLI command and by checking the event logs for CLIENT_CLIENT_UNBIND events for the #MSGVPN_REPLICATION_DATA_QUEUE. Next analyze the connectivity between the replication sites to determine if there is sufficient bandwidth for the replicated messages, and if the replication bridge message-spool window-size is sufficient for the typical round trip time on the inter-site connection.

Message Format

VPN_REPLICATION_SERVICE_DEGRADED: <vpnName> - Message VPN (<ID>) <name> Replication service degraded due to slow message replication after <syncTimeSec> seconds of normal operation. Synchronous replication mode has been reverted to asynchronous replication mode.

VPN_REPLICATION_SERVICE_DEGRADED_CLEAR

Description

Description

The cache instance has successfully completed writing all cached messages to disk.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

VPN_SOLCACHE_BACKUP_COMPLETED: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	INFO
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_BACKUP_COMPLETED
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_BACKUP_FAILED

Description

The cache instance has failed to start a backup operation.

Event Level Severity

WARN

Recommended Action

Examine the reason for the backup failure and if the necessary corrective action is understood, then proceed with that action and try the backup again. An example of such a problem would be an out of disk space error.

If the problem is not understood or persists, contact your Solace technical support representative for assistance.

Message Format

VPN_SOLCACHE_BACKUP_FAILED: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	WARN
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_BACKUP_FAILED
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_BACKUP_STARTED

Description

The cache instance has successfully opened the backup file and has started to write cached messages to disk.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

VPN_SOLCACHE_BACKUP_STARTED: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	INFO
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_BACKUP_STARTED
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_CLUSTER_SYNC_FAIL

Description

This event is sent when a cache cluster synchronization fails due to the given reason. Cache cluster synchronization allows cache instances within a cluster to fetch cached messages from each other to ensure each instance has the same view of the messages cached. Failures to synchronize prevent new cache instances from coming online while synchronization retry mechanisms attempt to ensure synchronization will eventually succeed.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

Consult the reason in the event text, which explains why the synchronization failed. As an example, a 'Message Lost Indications Rxd' indicates that the message broker discarded messages destined for the cache instance likely due to the cache instance being unable to handle the large stream of incoming cache topics from its peer cache instance(s). If it is not apparent based on the reason why the synchronization failed and subsequent synchronization retries continue to fail, copy the message text exactly as it appears on the console or in the event log, contact your Solace technical support representative, and provide the representative with the system configuration information along with any other relevant information.

Message Format

VPN_SOLCACHE_CLUSTER_SYNC_FAIL: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	WARN
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_CLUSTER_SYNC_FAIL
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_CONFIG_SYNC_FAIL

Description

This event is sent if the management configuration cannot be applied to the cache instance. When started, the cache instance downloads its management configuration from the message broker. If this download operation is unsuccessful this event is raised, and the cache instance might not operate as expected.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

Consult the 'due to' reason in the event text, which explains why the synchronization failed. This event can be caused by internal processing errors encountered while processing the download messages. In such a case the download will be attempted again after a brief timeout period. If it continues to fail, contact your Solace technical support representative, and provide the system configuration information along with any other relevant information.

Message Format

VPN_SOLCACHE_CONFIG_SYNC_FAIL: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	WARN
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_CONFIG_SYNC_FAIL
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_CONNECT_MEMORY_QUOTA_FAIL

Description

This event only occurs for cache instances that run on their managing message broker. It is sent when a registration attempt fails (that is, it will occur in tandem with the VPN_SOLCACHE_REGISTRATION_FAIL event) because accepting the registration would result in the aggregate sum of configured max-memory values, for all cache instances residing on the message broker, exceeding 8192 MB.

See the VPN_SOLCACHE_MAX_MEMORY_EXCEED event for a description of how to set/view the max-memory value for an individual cache-instance.

Event Level Severity

WARN

Recommended Action

There are two options to correct the issue:

1) Reduce the max-memory configuration for cache instances already running on the managing message broker. This requires a restart of all affected cache instances. See the 'distributed-cache <> cache-cluster <> max-memory' Config CLI command.

2) Run one or more cache instances on a host other than their managing message broker.

Message Format

VPN_SOLCACHE_CONNECT_MEMORY_QUOTA_FAIL: <vpnName> - Cache Instance (<ID>) <name> Connect failed, system memory quota (<quota> MB) exceeded

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	WARN
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_CONNECT_MEMORY_QUOTA_FAIL
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_DATA_BYTE_RATE_HIGH

Description

This event is sent when the incoming byte rate per second for cached data, experienced by the cache instance, exceeds the set threshold value.

To set the threshold value associated with this event, use the 'distributed-cache <> cache-cluster <> event data-byte-rate set-value' Config CLI command. The default is 250000000 bytes/sec.

To view the threshold value, see the 'Data Byte Rate (bytes/sec) - Set Value' field of the 'show cache-cluster <> detail' CLI command output.

To view the current value that the threshold monitors, see the 'Incoming Data Byte (1 sec sample) (bytes/s)' field of the 'show cache-instance <> remote status' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

The thresholds can be increased if the large byte rate is expected and not resulting in message loss. Otherwise reduce the number of topics cached or publishing rates to ease the rate experienced by the cache instance.

Message Format

VPN_SOLCACHE_DATA_BYTE_RATE_HIGH: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	WARN
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_DATA_BYTE_RATE_HIGH
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_DATA_BYTE_RATE_HIGH_CLEAR

Description

This event is sent when the incoming byte rate per second for cached data, experienced by the cache instance, falls below the clear threshold value.

To set the threshold value associated with this event, use the 'distributed-cache <> cache-cluster <> event data-byte-rate clear-value' Config CLI command. The default is 187500000 bytes/sec.

To view the threshold value, see the 'Data Byte Rate (bytes/sec) - Clear Value' field of the 'show cache-cluster <> detail' CLI command output.

To view the current value that the threshold monitors, see the 'Incoming Data Byte (1 sec sample) (bytes/s)' field of the 'show cache-instance <> remote status' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

VPN_SOLCACHE_DATA_BYTE_RATE_HIGH_CLEAR: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	INFO
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_DATA_BYTE_RATE_HIGH_CLEAR
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_DATA_MSG_RATE_HIGH

Description

This event is sent when the incoming message rate for data destined to be cached by the cache instance exceeds the set threshold value.

To set the threshold value associated with this event, use the 'distributed-cache <> cache-cluster <> event data-message-rate set-value' Config CLI command. The default is 48000 msgs/sec.

To view the threshold value, see the 'Data Message Rate (msgs/sec) - Set Value' field of the 'show cache-cluster <> detail' CLI command output.

To view the current value that the threshold monitors, see the 'Incoming Data Msg (1 sec sample) (msgs/s)' field of the 'show cache-instance <> remote status' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

The thresholds can be increased if the large message rate is expected and not resulting in message loss. Otherwise reduce the number of topics cached or publishing rates to ease the rate experienced by the cache instance.

Message Format

VPN_SOLCACHE_DATA_MSG_RATE_HIGH: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	WARN
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_DATA_MSG_RATE_HIGH
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_DATA_MSG_RATE_HIGH_CLEAR

Description

This event is sent when the incoming message rate for data destined to be cached by the cache instance falls below the clear threshold value (default is 36000).

To set the threshold value associated with this event, use the 'distributed-cache <> cache-cluster <> event data-message-rate clear-value' Config CLI command. The default is 36000 msgs/sec.

To view the threshold value, see the 'Data Message Rate (msgs/sec) - Clear Value' field of the 'show cache-cluster <> detail' CLI command output.

To view the current value that the threshold monitors, see the 'Incoming Data Msg (1 sec sample) (msgs/s)' field of the 'show cache-instance <> remote status' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

VPN_SOLCACHE_DATA_MSG_RATE_HIGH_CLEAR: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	INFO
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_DATA_MSG_RATE_HIGH_CLEAR
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_DELETE_MESSAGE

Description

This event is sent when a SolCache Instance performs either a manual message delete operation (through the 'distributed-cache <> delete-messages' Admin CLI command), or a scheduled delete message operation (through the 'distributed-cache <> scheduled-delete-message' Config CLI command) delete message operation. Two events are sent per operation: one indicating the operation was started, and one indicating the operation completed. They are differentiated by the event text which indicate one of the following: 'Starting manual', 'Completed manual', 'Starting scheduled', or 'Completed scheduled'

Event Level Severity

NOTICE

Recommended Action

This event is informational. No action is required.

Message Format

VPN_SOLCACHE_DELETE_MESSAGE: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	NOTICE
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_DELETE_MESSAGE
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_GLOBAL_PREFIX_STATUS

Description

When a cache instance establishes or loses connectivity to a home cache cluster, this event is sent to indicate the connectivity status change. As well, the binary metadata of the event sent over the message bus will contain the list of global topic prefixes affected. The status reported will reported will be either 'DOWN' or 'CONNECTED'.

Event Level Severity

NOTICE

Recommended Action

This event is informational. No action is required.

Message Format

VPN_SOLCACHE_GLOBAL_PREFIX_STATUS: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	NOTICE
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_GLOBAL_PREFIX_STATUS
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name
evt-suffix	String	Distributed Cache Name
status	string	lost or restored
home-cache-cluster	string	Home Cache Cluster Name

VPN_SOLCACHE_HEARTBEAT_TIMEOUT

Description

When a SolCache Instance fails to see heartbeat responses from the cache manager, it will log this occurrence and cease sending events until connectivity with the cache manager is restored. This does not affect cache operation as it will continue to cache messages and respond to cache requests.

Event Level Severity

WARN

Recommended Action

This event is informational. If there is a single occurrence no action is required. If the event occurs more than once for a given cache instance it should be investigated as it may indicate a problem with communication between the cache instance and its cache manager.

Message Format

VPN_SOLCACHE_HEARTBEAT_TIMEOUT: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	WARN
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_HEARTBEAT_TIMEOUT
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_LOSS_OF_NOTIFICATION

Description

This local event message is raised when one of the following occurs: a burst of events is generated, overloading the local Cache Instance event queue; there is a problem with the Cache Manager; or there is a problem with the communication between the Cache Manager and the Cache Instance. This event is raised in tandem, locally, with the event messages that could not be sent back to the cache manager.

Event Level Severity

WARN

Recommended Action

If this event occurs frequently, it is suggested that the Cache Instance's event queue depth should be increased. The event queue depth is configured with the CACHE_MAX_EVENT_QUEUE_DEPTH cache property.

Message Format

VPN_SOLCACHE_LOSS_OF_NOTIFICATION: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

VPN_SOLCACHE_LOST_MSG_STATE

Description

This is a one-shot event sent when the cache instance enters the lost message state. This occurs when a preceding event has resulted in a lost or discarded message. The preceding event will fall into one of the following categories:

1) Configured resource limits for a cache instance have been exceeded thereby triggering a discard (for example, VPN_SOLCACHE_MAX_MEMORY_EXCEED, VPN_SOLCACHE_MAX_TOPICS_EXCEED).

2) Reception of a message discard indication from the message bus (for example, VPN_SOLCACHE_ROUTER_DISCARD)

3) Events which may result in a discard although unlike 1) and 2) it is unknown with certainty that a discard has actually occurred (for example, VPN_SOLCACHE_MSG_BUS_DISCONNECTED).

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

The exact event which caused the lost message state to be entered will be specified in the event log immediately preceding the VPN_SOLCACHE_LOST_MSG_STATE event. First follow the 'Recommended Actions' for this preceding event. Then, as per one-shot event semantics, the VPN_SOLCACHE_LOST_MSG_STATE can be cleared to ensure the event is raised again if further lost or discarded messages are detected. See the 'distributed-cache <> clear-event lost-message' Admin CLI command.

Note: If the 'stop-on-lost-message' flag has been configured on the cache instance (see the 'distributed-cache <> cache cluster <> cache-instance <> stop-on-lost-message' Config CLI command) the lost message state MUST be cleared before the cache instance will start accepting further cache requests.

Message Format

VPN_SOLCACHE_LOST_MSG_STATE: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	WARN
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_LOST_MSG_STATE
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_LOST_MSG_STATE_CLEAR

Description

This event is sent when a cache instance leaves the lost message state. This can happen either due to an explicit command from the CLI (see the 'distributed-cache <> clear-event lost-message' Admin CLI command) or after a successful cluster synchronization with its peer cache instances.

Note: As a side-effect of clearing the VPN_SOLCACHE_LOST_MSG_STATE event the VPN_SOLCACHE_PARSE_ERROR and VPN_SOLCACHE_ROUTER_DISCARD one-shot events are also cleared, if previously set.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

VPN_SOLCACHE_LOST_MSG_STATE_CLEAR: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	INFO
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_LOST_MSG_STATE_CLEAR
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_MAX_MEMORY_EXCEED

Description

This is an event sent when the amount of memory used by a cache instance exceeds the maximum allowed. As a result, messages have been discarded by the cache instance.

To set the maximum value associated with this event use the 'distributed-cache <> cache-cluster <> max-memory' Config CLI command. The default is 2048 MB.

To view the maximum value, see the 'Max Memory (MB)' field of the 'show cache-cluster <> detail' CLI command output.

To view the current value that the threshold monitors, see the 'Total Mem Utilization' field of the 'show cache-instance <> remote status' CLI command output.

This event will force the cache instance to enter the lost message state if it is not already in it. See the VPN_SOLCACHE_LOST_MSG_STATE event for further information.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

The maximum amount of memory allowed for a cache instance can be increased if the large memory usage is expected and more is available on the host system on which the cache instance is running. Otherwise you can do one of the following to reduce the memory used by the cache instance:

1) Reduce the maximum number of individual topics cached. See the 'distributed-cache <> cache-cluster <> max-topics' Config CLI command.

2) Reduce the number of messages cached per individual topic. See the 'distributed-cache <> cache-cluster <> max-messages-per-topic' Config EXEC CLI command.

3) Reduce the number of topics cached and/or redistribute the cached topic space amongst other cache clusters to ease the request rate experienced by this cache instance.

Message Format

VPN_SOLCACHE_MAX_MEMORY_EXCEED: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	WARN
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_MAX_MEMORY_EXCEED
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_MAX_REQUEST_QUEUE_DEPTH_EXCEED

Description

This event is sent when the number of cache requests enqueued to a cache instance exceeds the maximum allowed. As a result, requests have been discarded by the cache instance.

To set the maximum value associated with this event use the 'distributed-cache <> cache-cluster <> request-queue-depth' Config CLI command. The default is 100000.

To view the maximum value, see the 'Request Queue Depth' field of the 'show cache-cluster <> detail' CLI command output.

To view the current value that the threshold monitors, see the 'Cache Request Queue Depth - Current Value' field of the 'show cache-instance <> remote status' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

The maximum queue depth can be increased if large bursts of cache requests are expected and latencies in responding to the requests can be tolerated by the requesting applications. Otherwise reduce the request rate to the cache instance from the client applications. If this is not possible, verify that the CPU utilization of the cache instance process is near 100% (see the 'CPU Utilization (%)' field of the show cache-instance <> remote status' User EXEC CLI command output). If so, a host with a faster processor may improve the queue processing rate for the cache instance. If not, find other processes that are running on the same CPU as the cache instance process, and as a result the CPU is running near 100% utilization. If so, move those processes to another CPU. Contact your Solace technical support representative for assistance if necessary.

Message Format

VPN_SOLCACHE_MAX_REQUEST_QUEUE_DEPTH_EXCEED: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	WARN
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_MAX_REQUEST_QUEUE_DEPTH_EXCEED
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_MAX_TOPICS_EXCEED

Description

This event is sent when the number of unique topics cached by a cache instance exceeds the maximum allowed. As a result, messages have been discarded by the cache instance

To set the maximum value associated with this event, use the 'distributed-cache <> cache-cluster <> max-topics' Config CLI command. The default is 2000000.

To view the maximum value, see the 'Max Topics' field of the 'show cache-cluster <> detail' CLI command output.

To view the current value that the threshold monitors, see the 'Cached Topics - Current Value' field of the 'show cache-instance <> remote status' CLI command output.

To view the current individual topics, use the 'show cache-instance <> remote topics' CLI command output.

This event forces the cache instance to enter the lost message state if it is not already in it. See the VPN_SOLCACHE_LOST_MSG_STATE event for further information.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

The maximum number of cached topics allowed can be increased if the large number of topics is expected; otherwise, you can do one of the following to reduce the number of topics cached by a given cache instance:

1) Reduce the number of topic subscriptions configured in the cache cluster. See the 'distributed-cache <> cache-cluster <> [no] topic' Config EXEC CLI command.

2) Subdivide the wildcarded topic subscriptions and distribute them between multiple clusters.

Message Format

VPN_SOLCACHE_MAX_TOPICS_EXCEED: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	WARN
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_MAX_TOPICS_EXCEED
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_MEMORY_HIGH

Description

This event is sent when the amount of memory used by a cache instance rises above the set threshold percentage of maximum (see VPN_SOLCACHE_MAX_MEMORY_EXCEED.

To set the threshold value associated with this event, use the 'distributed-cache <> cache-cluster <> event max-memory thresholds set-value' Config CLI command. The default is 80% of the maximum value (see the VPN_SOLCACHE_MAX_MEMORY_EXCEED event)

To view the threshold value, see the 'Max Memory (%) - Set Value' field of the show cache-cluster <> detail' CLI command output.

To view the current value that the threshold monitors, see the 'Total Mem Utilization' field of the 'show cache-instance <> remote status' CLI command output.

This event forces the cache instance to enter the lost message state. See the VPN_SOLCACHE_LOST_MSG_STATE event for further information.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

If the large message rate is expected and not resulting in message loss, then either the maximum memory (VPN_SOLCACHE_MAX_MEMORY_EXCEED) value should be increased or the threshold percentage can be increased. Otherwise reduce the memory used by the cache instance as described in the Recommended Actions of the VPN_SOLCACHE_MAX_MEMORY_EXCEED event.

Message Format

VPN_SOLCACHE_MEMORY_HIGH: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	WARN
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_MEMORY_HIGH
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_MEMORY_HIGH_CLEAR

Description

This event is sent when the amount of memory used by a cache instance falls below the clear threshold percentage of maximum memory.

To set the threshold value associated with this event, use the 'distributed-cache <> cache-cluster <> event max-memory thresholds clear-value' Config CLI command. The default is 60% of the maximum value (see the VPN_SOLCACHE_MAX_MEMORY_EXCEED event)

To view the threshold value, see the 'Max Memory (%) - Clear Value' field of the 'show cache-cluster <> detail' CLI command output.

To view the current value that the threshold monitors, see the 'Total Mem Utilization' field of the 'show cache-instance <> remote status' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

VPN_SOLCACHE_MEMORY_HIGH_CLEAR: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	INFO
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_MEMORY_HIGH_CLEAR
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_MSG_BUS_CONNECTED

Description

When a cache instance establishes a connection to the message broker message bus this log is emitted. This log is generated before the cache instance establishes connectivity with the solCacheMgr.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

NOTICE

Recommended Action

This event is informational. No action is required.

Message Format

VPN_SOLCACHE_MSG_BUS_CONNECTED: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	NOTICE
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_MSG_BUS_CONNECTED
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_MSG_BUS_CONNECT_FAILED

Description

This log is emitted when a cache instance fails to establish a connection to the message broker message bus.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

Verify the communication path between the cache instance and the message broker. If the problem persists, contact your Solace technical support representative for assistance.

Message Format

VPN_SOLCACHE_MSG_BUS_CONNECT_FAILED: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	WARN
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_MSG_BUS_CONNECT_FAILED
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_MSG_BUS_DISCONNECTED

Description

The transport connection to the message broker has been reset. The cache instance will immediately attempt to reconnect to the message bus, then follow the actions indicated by the 'stop-on-lost-message' setting for the cache instance.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

The cache instance will recover based on the 'stop-on-lost-message' setting. This will require operator intervention to clear the 'lost-message' event. See the 'distributed-cache <> clear-event lost-message' Admin CLI command. If 'stop-on-lost-message' is true, the operator must also manually start the cache instance. See the 'distributed-cache <> start cache-instance' Admin CLI command. If the problem persists, contact your Solace technical support representative for assistance.

Message Format

VPN_SOLCACHE_MSG_BUS_DISCONNECTED: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	WARN
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_MSG_BUS_DISCONNECTED
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_NEW_TOPIC

Description

When new topic advertisements are enabled from the CLI (see the 'distributed-cache <> cache cluster <> new-topic-advertisement' Config CLI command), this event is sent whenever a SolCache Instance learns of a new local topic. This event is not sent when a SolCache Instance learns of a new global topic because it is the sole responsibility of the home Cache Cluster to send this event for its local topics to prevent duplicate VPN_SOLCACHE_NEW_TOPIC events for the same topic.

Event Level Severity

NOTICE

Recommended Action

This event is informational. No action is required.

Message Format

VPN_SOLCACHE_NEW_TOPIC: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	NOTICE
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_NEW_TOPIC
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name
evt-suffix	String	distributed-cache-name/new-topic-string
topic	String	new-topic-string

VPN_SOLCACHE_PARSE_ERROR

Description

This is a one-shot event sent when a cache instance is unable to parse an incoming message. As a result, messages have been discarded by the cache instance.

This event will force the cache instance to enter the lost message state, if it is not already in it. Clearing of this one-shot event occurs as a side-effect of clearing the lost message state. See the VPN_SOLCACHE_LOST_MSG_STATE_CLEAR event for further information.

Event Level Severity

WARN

Recommended Action

This message indicates an internal error in the Solace APIs or SolCache. Contact your Solace technical support representative for assistance, if necessary.

The cache instance lost message state should be cleared to ensure this event is raised again if it occurs in the future. See the VPN_SOLCACHE_LOST_MSG_STATE_CLEAR event for further information.

Message Format

VPN_SOLCACHE_PARSE_ERROR: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	WARN
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_PARSE_ERROR
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_PLUGIN_FAIL

Description

This event is sent when a custom-designed cache plugin returns SOLCACHE_INGRESS_OP_FAIL. SolCache enters lost-message state

Note: If the 'stop-on-lost-message' flag has been configured on the cache instance (see the 'distributed-cache <> cache cluster <> cache-instance <> stop-on-lost-message' Config EXEC CLI command) the lost message state MUST be cleared before the cache instance will start accepting further cache requests.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

Consult with the designers/developers of your custom solCache plugin.

Message Format

VPN_SOLCACHE_PLUGIN_FAIL: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	WARN
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_PLUGIN_FAIL
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_REGISTERED

Description

This event is sent when the cache instance successfully registers with the managing message broker.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

VPN_SOLCACHE_REGISTERED: <vpnName> - Cache Instance (<ID>) <name> registered

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	INFO
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_REGISTERED
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_REGISTRATION_FAIL

Description

This event is sent when the message broker rejects a registration request from the cache instance. This is often due to missing/errored configuration or a lack of resources on the managing message broker. See the Description for VPN_SOLCACHE_CONNECT_MEMORY_QUOTA_FAIL for an example.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

Consult the reason in the event text which will provide a more precise explanation for the failure.

Message Format

VPN_SOLCACHE_REGISTRATION_FAIL: <vpnName> - Message VPN (<ID>) <name> Cache Instance <cache> registration failed, <reason>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	WARN
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_REGISTRATION_FAIL
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_REQUEST_MSG_RATE_HIGH

Description

This event is sent when the incoming message rate for cache requests experienced by the cache instance exceeds the set threshold value.

To set the threshold value associated with this event, use the 'distributed-cache <> cache-cluster <> event request-rate set-value' Config EXEC CLI command. The default is 25000 msgs/sec.

To view the threshold value, see the 'Request Rate (msgs/sec) - Set Value' field of the 'show cache-cluster <> detail' CLI command output.

To view the current value that the threshold monitors, see the 'Cache Request (1 sec sample) (msgs/s)' field of the 'show cache-instance <> remote status' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

The thresholds can be increased if the large request rate is expected and not resulting in lost requests. Otherwise increase the number of cache instances in the cluster or reduce the number of topics cached and/or redistribute the cached topic space amongst other cache clusters to ease the request rate experienced by this cache instance.

Message Format

VPN_SOLCACHE_REQUEST_MSG_RATE_HIGH: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	WARN
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_REQUEST_MSG_RATE_HIGH
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_REQUEST_MSG_RATE_HIGH_CLEAR

Description

This event is sent when the incoming message rate for cache requests experienced by the cache instance falls below the clear threshold value.

To set the threshold value associated with this event, use the 'distributed-cache <> cache-cluster <> event request-rate clear-value' Config EXEC CLI command. The default is 1000 msgs/sec.

To view the threshold value, see the 'Request Rate (msgs/sec) - Clear Value' field of the 'show cache-cluster <> detail' CLI command output.

To view the current value that the threshold monitors, see the 'Cache Request (1 sec sample) (msgs/s)' field of the 'show cache-instance <> remote status' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

VPN_SOLCACHE_REQUEST_MSG_RATE_HIGH_CLEAR: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	INFO
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_REQUEST_MSG_RATE_HIGH_CLEAR
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_REQUEST_QUEUE_DEPTH_HIGH

Description

This event is sent when the number of cache requests enqueued to a cache instance rises above the set percentage of Request Queue Depth.

To set the threshold value associated with this event, use the 'distributed-cache <> cache-cluster <> event request-queue-depth thresholds set-value' Config CLI command. The default is 80% of the maximum value (see the VPN_SOLCACHE_MAX_REQUEST_QUEUE_DEPTH_EXCEED event).

To view the threshold value, see the 'Request Queue Depth (%) - Set Value' field of the 'show cache-cluster <> detail' CLI command output.

To display the current value that the threshold monitors, see the 'Cache Request Queue Depth - Current Value' field of the 'show cache-instance <> remote status' CLI command.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

The thresholds can be increased if less warning before exceeding Max Request Queue Depth is tolerable. Otherwise consult the Recommended Actions of the VPN_SOLCACHE_MAX_REQUEST_QUEUE_DEPTH_EXCEED event for further information.

Message Format

VPN_SOLCACHE_REQUEST_QUEUE_DEPTH_HIGH: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	WARN
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_REQUEST_QUEUE_DEPTH_HIGH
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_REQUEST_QUEUE_DEPTH_HIGH_CLEAR

Description

This event is sent when the number of cache requests enqueued to a cache falls below the clear threshold percentage of Request Queue Depth

To set the threshold value associated with this event, use the 'distributed-cache <> cache-cluster <> event request-queue-depth thresholds clear-value' Config CLI command. The default is 60% of the maximum value (see the VPN_SOLCACHE_MAX_REQUEST_QUEUE_DEPTH_EXCEED event).

To view the threshold value, see the 'Request Queue Depth (%) - Clear Value' field of the show cache-cluster <> detail' CLI command output.

To display the current value that the threshold monitors, see the 'Cache Request Queue Depth - Current Value' field of the 'show cache-instance <> remote status' CLI command.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

VPN_SOLCACHE_REQUEST_QUEUE_DEPTH_HIGH_CLEAR: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	INFO
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_REQUEST_QUEUE_DEPTH_HIGH_CLEAR
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_RESPONSE_MSG_RATE_HIGH

Description

This event is sent when the outgoing message rate for data messages that are sent by the cache instance in response to cache requests exceeds the set threshold value.

To set the threshold value associated with this event, use the 'distributed-cache <> cache-cluster <> event response-rate set-value' Config CLI command. The default is 80000 msgs/sec.

To view the threshold value, see the 'Response Rate (msgs/sec) - Set Value' field of the 'show cache-cluster <> detail' CLI command output.

To view the current value that the threshold monitors, see the 'Outgoing Data Msg (1 sec sample) (msgs/s) - Current Value' field of the 'show cache-instance <> remote status' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

The thresholds can be increased if the large response rate is expected and not resulting in discards towards the requesting application. Otherwise throttle the response rate by either:

1) Modify applications to request more specific topics, containing less wildcards.

2) Reduce the CACHE_REPLY_SIZE_LIMIT SolCache configuration parameter. The outgoing data message rate is based on the rate of individual data messages returned data. However, it is possible for multiple data messages to be returned in a single cache instance reply. This can result in a reported outgoing data message rate that far exceeds the rate individual cache instance replies are being returned. By reducing the number of individual data messages that can be returned in a single cache instance reply, the outgoing data Message rate can be reduced. See the CACHE_REPLY_SIZE_LIMIT help in solCacheConfig.txt for more information.

Message Format

VPN_SOLCACHE_RESPONSE_MSG_RATE_HIGH: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	WARN
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_RESPONSE_MSG_RATE_HIGH
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_RESPONSE_MSG_RATE_HIGH_CLEAR

Description

This event is sent when the frequency of cache response rate in messages per second for each Cache Instance under the Cache Cluster falls below the clear threshold value (default is 1000).

To set the threshold value associated with this event, use the 'distributed-cache <> cache-cluster <> event response-rate clear-value' Config CLI command. The default is 1000 msgs/sec.

To view the threshold value, see the 'Response Rate (msgs/sec) - Clear Value' field of the 'show cache-cluster <> detail' CLI command output.

To view the current value that the threshold monitors, see the 'Outgoing Data Msg (1 sec sample) (msgs/s) - Current Value' field of the 'show cache-instance <> remote status' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

VPN_SOLCACHE_RESPONSE_MSG_RATE_HIGH_CLEAR: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	INFO
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_RESPONSE_MSG_RATE_HIGH_CLEAR
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_RESTORE_COMPLETED

Description

The cache instance has successfully completed reading all cached messages from disk.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

VPN_SOLCACHE_RESTORE_COMPLETED: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	INFO
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_RESTORE_COMPLETED
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_RESTORE_FAILED

Description

The cache instance has failed to start a restore operation.

Event Level Severity

WARN

Recommended Action

Examine the reason for the restore failure and if the necessary corrective action is understood, then proceed with that action and try the restore again.

If the problem is not understood or persists, contact your Solace technical support representative for assistance.

Message Format

VPN_SOLCACHE_RESTORE_FAILED: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	WARN
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_RESTORE_FAILED
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_RESTORE_MAX_MSGS_PER_TOPIC_EXCEEDED

Description

During a restore operation, the cache instance has found messages that exceed the maximum number permitted for a given topic so those messages have been discarded.

Event Level Severity

INFO

Recommended Action

This event is informational. If expected, no action is required.

If not expected, contact your Solace technical support representative for assistance.

Message Format

VPN_SOLCACHE_RESTORE_MAX_MSGS_PER_TOPIC_EXCEEDED: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	INFO
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_RESTORE_MAX_MSGS_PER_TOPIC_EXCEEDED
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_RESTORE_STARTED

Description

The cache instance has successfully opened the backup file and has started to read cached messages from disk.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

VPN_SOLCACHE_RESTORE_STARTED: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	INFO
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_RESTORE_STARTED
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_RESTORE_TOPIC_NOT_SUBSCRIBED

Description

During a restore operation, the cache instance has found messages with a topic that is not currently part of its subscription set so those messages have been discarded.

Event Level Severity

INFO

Recommended Action

This event is informational. If expected, no action is required.

If not expected, contact your Solace technical support representative for assistance.

Message Format

VPN_SOLCACHE_RESTORE_TOPIC_NOT_SUBSCRIBED: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	INFO
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_RESTORE_TOPIC_NOT_SUBSCRIBED
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_ROUTER_DISCARD

Description

This is a one-shot sent when a cache instance receives message discard indications from the message bus.

Event Level Severity

WARN

Recommended Action

The connection should re-establish itself automatically. Contact your Solace Systems technical support representative should the condition persist.

Message Format

VPN_SOLCACHE_ROUTER_DISCARD: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	WARN
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_ROUTER_DISCARD
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_START_CMD

Description

This event is sent when the 'distributed-cache <> start' Admin CLI command is received by a cache instance.

Event Level Severity

NOTICE

Recommended Action

This event is informational. No action is required.

Message Format

VPN_SOLCACHE_START_CMD: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	NOTICE
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_START_CMD
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_STATE_CHANGE

Description

This event is sent when the state of a cache instance changes. The state can be one of 'DOWN', 'STOPPED', 'STOPPED_LOST_MSGS', 'REGISTER', 'CONFIG_SYNC', 'CLUSTER_SYNC', or 'UP'.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

NOTICE

Recommended Action

This event is informational. No action is required.

Message Format

VPN_SOLCACHE_STATE_CHANGE: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	NOTICE
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_STATE_CHANGE
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_SUBSCRIBE_FAIL

Description

This event is sent when the cache instance is unable to add or remove its configured topics from the message broker.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

Consult the message broker statistics for the cache instance client along with the message broker event logs to see why the add/remove subscription request failed. In particular, the 'Total Client Messages - Client Control Messages - Add/Remove Subscription Messages' fields of the 'show client <> stats detail' CLI command typically highlights the issue.

Message Format

VPN_SOLCACHE_SUBSCRIBE_FAIL: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	WARN
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_SUBSCRIBE_FAIL
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_TOPICS_HIGH

Description

This event is sent when the number of unique topics cached by a cache instances rises above the set percentage of Max Topics

To set the threshold value associated with this event, use the 'distributed-cache <> cache-cluster <> event max-topics thresholds set-value' Config CLI command. The default is 80% of the maximum value (see the VPN_SOLCACHE_MAX_TOPICS_EXCEED event)

To view the threshold value, see the 'Max Topics (%) - Set Value' field of the 'show cache-cluster <> detail' CLI command output.

To display the current value that the threshold monitors, see the 'Cached Topics - Current Value' field of the 'show cache-instance <> remote status' CLI command.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

The thresholds can be increased if less warning before exceeding Max Topics is tolerable. Otherwise consult the Recommended Actions of the VPN_SOLCACHE_MAX_TOPICS_EXCEED event for further information.

Message Format

VPN_SOLCACHE_TOPICS_HIGH: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	WARN
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_TOPICS_HIGH
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_TOPICS_HIGH_CLEAR

Description

This event is sent when the number of cached topics per cache instance falls below the clear threshold percentage of Max Topics.

To set the threshold value associated with this event, use the 'distributed-cache <> cache-cluster <> event max-topics thresholds clear-value' Config CLI command. The default is 60% of the maximum value (see the VPN_SOLCACHE_MAX_TOPICS_EXCEED event)

To view the threshold value, see the 'Max Topics (%) - Clear Value' field of the 'show cache-cluster <> detail' CLI command output.

To view the current value that the threshold monitors, see the 'Cached Topics - Current Value' field of the 'show cache-instance <> remote status' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

VPN_SOLCACHE_TOPICS_HIGH_CLEAR: <vpnName> - Cache Instance (<ID>) <name> <remoteText>

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	INFO
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_TOPICS_HIGH_CLEAR
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_SOLCACHE_UNREGISTERED

Description

This event is sent when the cache instance successfully unregisters from the managing message broker.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

VPN_SOLCACHE_UNREGISTERED: <vpnName> - Cache Instance (<ID>) <name> unregistered

When this event is received over the message bus, the following data can be retrieved from the User Property Map in that message. See the Solace Message Bus Management Guide for more details.

Field Name	Type	Description
evt-level	String	INFO
evt-type	String	VPN
evt-rtr-name	String	Name of the message broker providing message bus service to the SolCache Instance
evt-name	String	VPN_SOLCACHE_UNREGISTERED
evt-vpn-name	String	The name of the associated Message VPN.
evt-version	INT32	Version Number
distributed-cache-name	String	Distributed Cache Name
cache-cluster	String	Cache Cluster Name
cache-instance	String	Cache Instance Name

VPN_VPN_MAX_SUBSCRIPTIONS_EXCEEDED: <vpnName> - Message VPN (<ID>) <name> Maximum subscriptions exceeded : <value>

VPN_VPN_STATE_CHANGE

Description

This event message is sent when a Message VPN changes operational state from down to up, or vice versa.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

This event message is informational. No action is required.

Message Format

VPN_VPN_STATE_CHANGE: <vpnName> - Message VPN (<ID>) <name> State Changed to: <operationalState>

VPN_VPN_SUBSCRIPTIONS_HIGH

Description

This event is sent when the number of subscriptions reaches or exceeds the set threshold value for the associated Message VPN.

To set the threshold associated with this event use the 'message-vpn <> event subscriptions thresholds set-value/set-percentage' Config CLI command. The default is 80% of the maximum value (see the VPN_VPN_MAX_SUBSCRIPTIONS_EXCEEDED event).

To view the threshold value, see the 'Subscriptions (#subs) - Set Value' field of the 'show message-vpn <>' CLI command output.

To view the current value that the threshold monitors, see the 'Total Local Unique Subscriptions' field of the 'show message-vpn <>' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

The thresholds can be increased if the large number of subscriptions is expected; otherwise, see the 'Recommended Actions' of the SYSTEM_CLIENT_SUBSCRIPTIONS_HIGH event.

Message Format

VPN_VPN_SUBSCRIPTIONS_HIGH: <vpnName> - Message VPN (<ID>) <name> Subscriptions threshold <threshold>(<thresholdPercentage>%) reached: <currentSubs>

VPN_VPN_SUBSCRIPTIONS_HIGH_CLEAR

Description

This event is sent when the number of subscriptions falls below the clear threshold value for the associated Message VPN.

To set the threshold associated with this event use the 'message-vpn <> event subscriptions thresholds clear-value/clear-percentage' Config CLI command. The default is 60% of the maximum value (see the VPN_VPN_MAX_SUBSCRIPTIONS_EXCEEDED event).

To view the threshold value, see the 'Subscriptions (#subs) - Clear Value' field of the 'show message-vpn <>' CLI command output.

To view the current value that the threshold monitors, see the 'Total Local Unique Subscriptions' field of the 'show message-vpn <>' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

This event message is informational. No action is required.

Message Format

VPN_VPN_SUBSCRIPTIONS_HIGH_CLEAR: <vpnName> - Message VPN (<ID>) <name> Subscriptions threshold <threshold>(<thresholdPercentage>%) cleared: <currentSubs>

Client SolOS syslog Events

Ad syslog Messages

CLIENT_AD_EGRESS_FLOWS_HIGH

Description

This event is sent when the number of egress flows for a client reaches or exceeds the set threshold specified by the associated client profile for that client.

To set the threshold value associated with this event, use the 'client-profile <> event egress-flows thresholds 'set-value/set-percentage' Config CLI command. The default is 80% of the maximum value (see the CLIENT_AD_MAX_EGRESS_FLOWS_EXCEEDED event).

To view the threshold value, see the 'Egress Flows - Set Value' field of the 'show client-profile <> detail' CLI command output.

To view the current value that the threshold monitors, see the 'Total Egress Flows' field of the 'show client <> message-vpn <> message-spool' CLI command output.

Event Level Severity

WARN

Recommended Action

The thresholds can be increased if large number of egress flows is expected; otherwise, see the 'Recommended Actions' of the CLIENT_AD_MAX_EGRESS_FLOWS_EXCEEDED event.

Message Format

CLIENT_AD_EGRESS_FLOWS_HIGH: <vpnName> <clientName> Client (<ID>) <name/clientUsername> egress flows threshold <threshold>(<thresholdPercentage>%) reached: <currentEgressFlows>.

CLIENT_AD_EGRESS_FLOWS_HIGH_CLEAR

Description

This event is sent when the number of egress flows, for a client, falls below the clear threshold specified by the associated client profile for that client.

To set the threshold value associated with this event, use the 'client-profile <> event egress-flows thresholds 'clear-value/clear-percentage' Config CLI command. The default is 60% of the maximum value (see the CLIENT_AD_MAX_EGRESS_FLOWS_EXCEEDED event).

To view the threshold value, see the 'Egress Flows - Clear Value' field of the 'show client-profile <> detail' CLI command output.

To view the current value that the threshold monitors, see the 'Total Egress Flows' field of the 'show client <> message-vpn <> message-spool' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

CLIENT_AD_EGRESS_FLOWS_HIGH_CLEAR: <vpnName> <clientName> Client (<ID>) <name/clientUsername> egress flows threshold <threshold>(<thresholdPercentage>%) cleared: <currentEgressFlows>.

CLIENT_AD_INGRESS_FLOWS_HIGH

Description

This event is sent when the number of ingress flows, for a client, reaches or exceeds the set threshold specified by the associated client profile for that client.

To set the threshold value associated with this event, use the 'client-profile <> event ingress-flows thresholds 'set-value/set-percentage' Config CLI command. The default is 80% of the maximum value (see the CLIENT_AD_MAX_INGRESS_FLOWS_EXCEEDED event).

To view the threshold value, see the 'Ingress Flows - Set Value' field of the 'show client-profile <> detail' CLI command output.

To view the current value that the threshold monitors, see the 'Total Ingress Flows' field of the 'show client <> message-vpn <> message-spool' CLI command output.

Event Level Severity

WARN

Recommended Action

The thresholds can be increased if large number of egress flows is expected; otherwise, see the 'Recommended Actions' of the CLIENT_AD_MAX_INGRESS_FLOWS_EXCEEDED event.

Message Format

CLIENT_AD_INGRESS_FLOWS_HIGH: <vpnName> <clientName> Client (<ID>) <name/clientUsername> ingress flows threshold <threshold>(<thresholdPercentage>%) reached: <currentIngressFlows>.

Description

This event is raised when a partition is assigned to a client.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

CLIENT_AD_PARTITIONED_QUEUE_ASSIGNED: <vpnName> <clientName> Queue <qname> partition <pnum> assigned to <type> (<ID>) <name/clientUsername>

CLIENT_AD_TRANSACTED_SESSIONS_EXCEED

Description

This event is sent when the number of transacted sessions, for the specified client, exceeds the total available by the associated client profile for that client.

To set the maximum value associated with this event use the 'client-profile <> message-spool max-transacted-sessions' Config CLI command. The default is 10.

To view the maximum value, see the 'Message Spool - Max Transacted Sessions' field of the 'show client-profile <> detail' CLI command output.

The number of transacted sessions shown by the 'show client <> message-vpn <> transaction-stats' CLI command determines the current value that the threshold monitors.

Event Level Severity

WARN

Recommended Action

The maximum can be increased if the client application is behaving correctly and the large number of transacted sessions is expected. Otherwise see the 'Recommended Actions' of the SYSTEM_AD_TRANSACTED_SESSIONS_EXCEED event.

Note: Configuration changes to the client-profile affect *all* clients associated with that profile. Otherwise the maintainers of the client application should reduce transacted session use by the client to ease demand.

Message Format

CLIENT_AD_TRANSACTED_SESSIONS_EXCEED: <vpnName> <clientName> Client (<ID>) <name/clientUsername> transacted sessions limit <limit> exceeded

CLIENT_AD_TRANSACTED_SESSIONS_HIGH

Description

This event is sent when the number of transacted sessions, for the specified client, reaches or exceeds the set threshold value.

To set the threshold value associated with this event, use the 'client-profile <> event transacted-sessions thresholds set-value|set-percentage' Config CLI command. The default is 80% of the maximum value (see the CLIENT_AD_TRANSACTED_SESSIONS_EXCEED event)

To view the threshold value, see the 'Transacted sessions (#) - Set Value' field of the 'show client-profile <> detail' CLI command output.

The number of transacted sessions shown by the 'show client <> message-vpn <> transaction-stats' CLI command determines the current value that the threshold monitors.

Event Level Severity

WARN

Recommended Action

The threshold can be increased if the client application is behaving correctly and the large number of transacted sessions is expected. Otherwise see the 'Recommended Actions' for the CLIENT_AD_TRANSACTED_SESSIONS_EXCEED event.

Message Format

CLIENT_AD_TRANSACTED_SESSIONS_HIGH: <vpnName> <clientName> Client (<ID>) <name/clientUsername> transacted sessions threshold <threshold>(<thresholdPercentage>%) reached: <currentSessions>

CLIENT_AD_TRANSACTED_SESSIONS_HIGH_CLEAR

Description

This event is sent when the number of transacted sessions, for the specified client, falls below the clear threshold value.

To set the threshold value associated with this event, use the 'client-profile <> event transacted-sessions thresholds clear-value|clear-percentage' Config CLI command. The default is 60% of the maximum value (see the CLIENT_AD_TRANSACTED_SESSIONS_EXCEED event)

To view the threshold value, see the 'Transacted sessions (#) - Clear Value' field of the 'show client-profile <> detail' CLI command output.

The number of transacted sessions shown by the 'show client <> message-vpn <> transaction-stats' CLI command determines the current value that the threshold monitors.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Description

This event is sent when the number of transactions, for the specified client, falls below the clear threshold value.

To set the threshold value associated with this event, use the 'client-profile <> event transactions thresholds clear-value|clear-percentage' Config CLI command. The default is 60% of the maximum value (see the CLIENT_AD_TRANSACTIONS_EXCEED event)

To view the threshold value, see the 'Transacted sessions (#) - Clear Value' field of the 'show client-profile <> detail' CLI command output.

To view the current value that the threshold monitors, see the sessions shown by the 'show client <> message-vpn <> transaction-stats detail' CLI command output.

Description

This event is sent when a client connects to the message broker.

The field SSLDowngradedToPlainText is deprecated; use SSLNegotiatedTo instead.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

CLIENT_CLIENT_CONNECT: <vpnName> <clientName> <type> (<ID>) <name/clientUsername> OriginalClientUsername(<originalClientUsername>) WebSessionId (<webSessionId>) connected to <applianceAddr> from <clientAddr> version(<sdkVersion>) platform(<platform>) SslVersion(<tlsVersion>) SslCipher(<tlsCipherDescription>) APIuser(<userIdString>) authScheme(<authScheme>) authorizationGroup(<authorizationGroup>) clientProfile(<clientProfile>) ACLProfile(<aclProfile>) SSLDowngradedToPlainText(<tlsDowngradedToPlainText>) SSLNegotiatedTo(<tlsNegotiatedTo>) SslRevocation(<tlsRevocation>) Capabilities(<capabilities>) SslValidityNotAfter(<tlsValidityNotAfter>)

CLIENT_CLIENT_CONNECT_MQTT

Description

This event is sent when a client connects to the message broker using MQTT protocol.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

CLIENT_CLIENT_CONNECT_MQTT: <vpnName> <clientName> <type> (<ID>) <name/clientUsername> OriginalClientUsername(<originalClientUsername>) ClientId (<sessionId>) connected to <applianceAddr> from <clientAddr> SslVersion(<tlsVersion>) SslCipher(<tlsCipherDescription>) authScheme(<authScheme>) Clean(<clean>) Will(<will>) SslRevocation(<tlsRevocation>) MqttVersion(<mqttVersion>) SessionExpiry(<sessionExpiry>)

CLIENT_CLIENT_CREATE_ENDPOINT

Description

This event is sent when a message endpoint is created.

Event Level Severity

NOTICE

Recommended Action

This event is informational. No action is required.

Message Format

CLIENT_CLIENT_CREATE_ENDPOINT: <vpnName> <clientName> <type> (<ID>) <name/clientUsername> Create <qendptDurability> <qendptType> <qendptName>: From (<templateEndpoint>), AccessType (<accessType>), Quota (<spoolQuota>MB), MaxMessageSize (<maxMessageSize>B), AllOthersPermission(<allOthersPermission>), RespectTTL(<respectTTL>), RejectMsgToSenderOnDiscard(<rejectMsgToSenderOnDiscard>), MaxRedelivery(<maxRedelivery>)

CLIENT_CLIENT_CREATE_ENDPOINT_FAILED

Description

This event is sent when a message endpoint fails to be created.

Event Level Severity

NOTICE

Recommended Action

Consult the reason field in the event text, which explains why the endpoint creation request failed. This is most common with client-created endpoints where client signalled parameters such as max-message-size or max-spool-usage can exceed the system limits.

Message Format

CLIENT_CLIENT_CREATE_ENDPOINT_FAILED: <vpnName> <clientName> <type> (<ID>) <name/clientUsername> Create <qendptDurability> <qendptType> <qendptName>: AccessType (<accessType>), Quota (<spoolQuota>MB), MaxMessageSize (<maxMessageSize>B), AllOthersPermission(<allOthersPermission>), RespectTTL(<respectTTL>), RejectMsgToSenderOnDiscard(<rejectMsgToSenderOnDiscard>), MaxRedelivery(<maxRedelivery>), failed, reason: <reason>

CLIENT_CLIENT_DELETE_ENDPOINT

Description

This event is sent when a message endpoint is deleted.

Event Level Severity

NOTICE

Recommended Action

This event is informational. No action is required.

Message Format

CLIENT_CLIENT_DELETE_ENDPOINT: <vpnName> <clientName> <type> (<ID>) <name/clientUsername> Delete <qendptDurability> <qendptType> <qendptName>

CLIENT_CLIENT_DISCONNECT

Description

This event is sent when a client disconnects from the message broker.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

CLIENT_CLIENT_DISCONNECT: <vpnName> <clientName> <type> (<ID>) <name/clientUsername> WebSessionId (<webSessionId>) reason(<disconnectReason>) final statistics - dp(<controlMsgsReceived>, <controlMsgsDelivered>, <topicMsgsReceived>, <topicMsgsDelivered>, <totalMsgsReceived>, <totalMsgsDelivered>, <controlBytesReceived>, <controlBytesDelivered>, <topicBytesReceived>, <topicBytesDelivered>, <totalBytesReceived>, <totalBytesDelivered>, <curMsgRateIngress>, <curMsgRateEgress>, <avgMsgRateIngress>, <avgMsgRateEgress>, <deniedDuplicateClients>, <discardsNoSubscriptionMatch>, <discardsTopicParseError>, <discardsParseError>, <discardsMsgTooBig>, <discardsTransmitCongestion>) conn(<recvQBytes>, <sendQBytes>, <clientAddr>, <state>, <outOfOrder>, <fastRetransmit>, <timedRetransmit>) zip(<compressedBytesReceived>, <compressedBytesDelivered>, <uncompressedBytesReceived>, <uncompressedBytesDelivered>, <compressionRatioIngress>, <compressionRatioEgress>, <curCompressedByteRateIngress>, <curCompressedByteRateEgress>, <curUncompressedByteRateIngress>, <curUncompressedByteRateEgress>, <avgCompressedByteRateIngress>, <avgCompressedByteRateEgress>, <avgUncompressedByteRateIngress>, <avgUncompressedByteRateEgress>) web(<webMsgsReceived>, <webMsgsDelivered>, <webBytesReceived>, <webBytesDelivered>, <webOutOfOrder>, <webFastRetransmit>, <webTimedRetransmit>), SslVersion(<tlsVersion>), SslCipher(<tlsCipherDescription>)

CLIENT_CLIENT_DISCONNECT_MQTT

Description

This event is sent when an MQTT client disconnects from the message broker.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

CLIENT_CLIENT_DISCONNECT_MQTT: <vpnName> <clientName> <type> (<ID>) <name/clientUsername> ClientId (<sessionId>) reason(<disconnectReason>) final statistics - dp(<controlMsgsReceived>, <controlMsgsDelivered>, <topicMsgsReceived>, <topicMsgsDelivered>, <totalMsgsReceived>, <totalMsgsDelivered>, <controlBytesReceived>, <controlBytesDelivered>, <topicBytesReceived>, <topicBytesDelivered>, <totalBytesReceived>, <totalBytesDelivered>, <curMsgRateIngress>, <curMsgRateEgress>, <avgMsgRateIngress>, <avgMsgRateEgress>, <deniedDuplicateClients>, <discardsNoSubscriptionMatch>, <discardsTopicParseError>, <discardsParseError>, <discardsMsgTooBig>, <discardsTransmitCongestion>) conn(<recvQBytes>, <sendQBytes>, <clientAddr>, <state>, <outOfOrder>, <fastRetransmit>, <timedRetransmit>) mqtt(<mqttConnectReceived>, <mqttConnAckSent>, <mqttConnAckErrSent>, <mqttPublishReceived>, <mqttPublishSent>, <mqttPubAckReceived>, <mqttPubAckSent>, <mqttPubRecSent>, <mqttPubRelReceived>, <mqttPubCompSent>, <mqttSubscribeReceived>, <mqttSubAckSent>, <mqttSubAckErrSent>, <mqttUnsubscribeReceived>, <mqttUnsubscribeSent>, <mqttPingReqReceived>, <mqttPingRespSent>, <mqttDisconnectReceived>), SslVersion(<tlsVersion>), SslCipher(<tlsCipherDescription>), WillSent(<willSent>), SessionExpiry(<sessionExpiry>)

CLIENT_CLIENT_EGRESS_MSG_DISCARD

Description

This event is generated when a message is discarded because it cannot be sent to a client. However, it is only sent out approximately once every 60 seconds for the specified client.

Event Level Severity

INFO

Recommended Action

Run the 'show client <> stats detail' CLI command on the message broker and consult the 'Egress Discards' section of the resultant display to determine the reason(s) for the discards. These reason(s) will dictate the recommended actions if any. For example, if the discards are due to 'Transmit Congestion' this is an indication the client is unable to handle the rate of traffic offered to it by the message broker.

Message Format

Description

This event is sent when the number of subscriptions exceeds the maximum available for the specified client.

To set the maximum value associated with this event use the 'client-profile <> max-subscriptions' Config CLI command. The default is 500000.

To view the maximum value, see the 'Max # Subscriptions' field of the 'show client-profile <> detail' CLI command output.

To view the current value that the threshold monitors, see the 'Subscriptions' field of the 'show client <> message-vpn <>' CLI command output.

This event is duplicated into the system.log file in addition to the event.log file.

Event Level Severity

WARN

Recommended Action

The maximum can be increased if a large number of subscriptions is expected; otherwise, see the 'Recommended Actions' of the SYSTEM_CLIENT_SUBSCRIPTIONS_HIGH event.

Message Format

CLIENT_CLIENT_MAX_SUBSCRIPTIONS_EXCEEDED: <vpnName> <clientName> Client (<ID>) <name/clientUsername> Maximum subscriptions exceeded : <value>

CLIENT_CLIENT_MESSAGE_TOO_BIG

Description

This is a one-shot event sent when a direct or guaranteed message size exceeds what is allowed by the message broker. This event is only raised for clients using the SMF protocol.

Event Level Severity

WARN

Recommended Action

No immediate action is required. However, you should clear and reset this one-shot event through the 'client <> message-vpn <> clear-event message-too-big' Admin CLI command to allow it to be generated again.

Message Format

CLIENT_CLIENT_MESSAGE_TOO_BIG: <vpnName> <clientName> Client (<ID>) <name/clientUsername> Message Too Big threshold (<threshold> bytes) exceeded: <size>, topic <topic>

CLIENT_CLIENT_NAME_CHANGE

Description

Through Solace APIs, client applications can change their name after connecting to and authenticating with the message broker. This event is sent when a client performs such a name change.

Event Level Severity

NOTICE

Recommended Action

This event is informational. No action is required.

Message Format

CLIENT_CLIENT_NAME_CHANGE: <vpnName> <clientName> Client (<ID>) <name/clientUsername> changed name from <oldname>

CLIENT_CLIENT_NAME_CHANGE_FAILED

Description

Through Solace APIs, client applications can change their name after connecting to and authenticating with the message broker. This event is sent when a client attempts to perform such a name change but is rejected by the message broker.

Event Level Severity

INFO

Recommended Action

Consult the reason field in the event text which explains why the name change request failed. A common reason is because another client with the specified name is already connected to the message broker. In this case the maintainers of the client application originating the conflicting name change request can be consulted.

Message Format

CLIENT_CLIENT_NAME_CHANGE_FAILED: <vpnName> <clientName> Client (<ID>) <name/clientUsername> New name(<newName>), reason(<reason>)

CLIENT_CLIENT_OPEN_FLOW

Description

This event is sent when a Solace API client opens a publisher session flow to the message broker to publish Guaranteed messages.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Description

This event is sent when all messages are removed from a topic endpoint due to the topic or selector associated with the endpoint changing.

Event Level Severity

NOTICE

Recommended Action

This event is informational. No action is required.

Message Format

CLIENT_CLIENT_TOPIC_ENDPOINT_FLUSHED: <vpnName> <clientName> <type> (<ID>) <name/clientUsername> <topic> <selector> changed, all messages removed from Topic Endpoint <qendptName>

CLIENT_CLIENT_TRANSACTED_SESSION_CLOSE

Description

This event is sent when a Solace API client closes a transacted session to the message broker.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

CLIENT_CLIENT_TRANSACTED_SESSION_CLOSE: <vpnName> <clientName> Client (<ID>) <name/clientUsername> Transacted Session (<transactedSessionId>) <sessionName> closed, final statistics requests (commits <committedRequests>, rollback <rollbackRequests>, failed <failedRequests>), messages (published <publishedMessages>, consumed <consumedMessages>)

CLIENT_CLIENT_TRANSACTED_SESSION_OPEN

Description

This event is sent when a Solace API client opens a transacted session to the message broker.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

CLIENT_CLIENT_TRANSACTED_SESSION_OPEN: <vpnName> <clientName> Client (<ID>) <name/clientUsername> Transacted Session (<transactedSessionId>) <sessionName> open

CLIENT_CLIENT_UNBIND

Description

This event is sent when a client unbinds from a message endpoint. The possible unbind reason given in the event are listed below:

Reason	Description
Client issued unbind	The client issued an unbind request to the router
Client disconnected	The client disconnected without an unbind request
Message-spool shutdown	The message-spool was shutdown or unavailable
Endpoint egress shutdown	The endpoint egress flow was shutdown or removed

Event Level Severity

NOTICE

Recommended Action

This event is informational. No action is required.

Message Format

CLIENT_CLIENT_UNBIND: <vpnName> <clientName> <type> (<ID>) <name/clientUsername> Unbind to <validInvalid> (<flowId>), ForwardingMode(<forwardingMode>), final statistics - flow(<windowSize>, <usedWindow>, <unackedMessages>, <lowMsgIdAckPending>, <highMsgIdAckPending>, <windowClosed>, <msgRedelivered>, <msgTransportRetransmits>, <msgDeliveredStoreAndForward>, <msgDeliveredCutThrough>), isActive(<isActive>), Reason(<reason>)

CLIENT_CLIENT_UNSUBSCRIBE

Description

This event is sent when all messages are removed from a topic endpoint due to an subscription unsubscribe request received from a client.

Event Level Severity

NOTICE

Recommended Action

This event is informational. No action is required.

Message Format

CLIENT_CLIENT_UNSUBSCRIBE: <vpnName> <clientName> <type> (<ID>) <name/clientUsername> Unsubscribe Topic Endpoint <qendptName>, all messages removed

CLIENT_CLIENT_XA_TRANSACTED_SESSION_CLOSE

Description

This event is sent when a Solace API client closes a XA transacted session to the message broker.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

CLIENT_CLIENT_XA_TRANSACTED_SESSION_CLOSE: <vpnName> <clientName> Client (<ID>) <name/clientUsername> XA Transacted Session (<transactedSessionId>) <sessionName> closed, final statistics requests (starts <startsRequests>, suspends <suspendsRequests>, resumes <resumesRequests>, ends <endsRequests>, end-fails <endFailsRequests>, end-rollbacks <endRollbacksRequests>, prepares <preparesRequests>, one-phase-commits <onePhaseCommitRequests>, two-phase-commits <twoPhaseCommitRequests>, rollbacks <rollbackRequests>, forgets <forgetsRequests>), failures (starts <startsFailed>, suspends <suspendsFailed>, resumes <resumesFailed>, ends <endsFailed>, end-fails <endFailsFailed>, end-rollbacks <endRollbacksFailed>, prepares <preparesFailed>, one-phase-commits <onePhaseCommitFailed>, two-phase-commits <twoPhaseCommitFailed>, rollbacks <rollbackFailed>, forgets <forgetsFailed>), messages (published <publishedMessages>, consumed <consumedMessages>)

CLIENT_CLIENT_XA_TRANSACTED_SESSION_OPEN

Description

This event is sent when a Solace API client opens a XA transacted session to the message broker.

Event Level Severity

INFO

Recommended Action

This event is informational. No action is required.

Message Format

CLIENT_CLIENT_XA_TRANSACTED_SESSION_OPEN: <vpnName> <clientName> Client (<ID>) <name/clientUsername> XA Transacted Session (<transactedSessionId>) <sessionName> open

Table of Contents

System-wide SolOS syslog Events

Guaranteed Messaging syslog Messages

SYSTEM_AD_DELIVERED_UNACKED_MSGS_EXCEED

SYSTEM_AD_DELIVERED_UNACKED_MSGS_HIGH

SYSTEM_AD_DELIVERED_UNACKED_MSGS_HIGH_CLEAR

SYSTEM_AD_DISK_USAGE_EXCEEDED

SYSTEM_AD_DISK_USAGE_HIGH

SYSTEM_AD_DISK_USAGE_HIGH_CLEAR

SYSTEM_AD_EGRESS_FLOWS_HIGH

SYSTEM_AD_EGRESS_FLOWS_HIGH_CLEAR

SYSTEM_AD_ENDPOINTS_HIGH

SYSTEM_AD_ENDPOINTS_HIGH_CLEAR

SYSTEM_AD_FLASH_FAILED

SYSTEM_AD_INGRESS_FLOWS_HIGH

SYSTEM_AD_INGRESS_FLOWS_HIGH_CLEAR

SYSTEM_AD_MAX_EGRESS_FLOWS_EXCEEDED

SYSTEM_AD_MAX_ENDPOINTS_EXCEEDED

SYSTEM_AD_MAX_INGRESS_FLOWS_EXCEEDED

SYSTEM_AD_MAX_MSG_CACHE_USAGE_EXCEEDED

SYSTEM_AD_MESSAGE_BUFFER_LEVEL_LOW

SYSTEM_AD_MESSAGE_BUFFER_LEVEL_LOW_CLEAR

SYSTEM_AD_MSG_CACHE_USAGE_HIGH

SYSTEM_AD_MSG_CACHE_USAGE_HIGH_CLEAR

SYSTEM_AD_MSG_COUNT_UTILIZATION_EXCEED

SYSTEM_AD_MSG_COUNT_UTILIZATION_HIGH

SYSTEM_AD_MSG_COUNT_UTILIZATION_HIGH_CLEAR

SYSTEM_AD_MSG_SPOOL_CHG

SYSTEM_AD_MSG_SPOOL_DEFRAG_END

SYSTEM_AD_MSG_SPOOL_DEFRAG_START

SYSTEM_AD_MSG_SPOOL_HIGH

SYSTEM_AD_MSG_SPOOL_HIGH_CLEAR

SYSTEM_AD_MSG_SPOOL_QUOTA_EXCEED

SYSTEM_AD_RESTORE_FAILED

SYSTEM_AD_SPOOL_FILES_EXCEEDED

SYSTEM_AD_SPOOL_FILES_HIGH

SYSTEM_AD_SPOOL_FILES_HIGH_CLEAR

SYSTEM_AD_TRANSACTED_SESSIONS_EXCEED

SYSTEM_AD_TRANSACTED_SESSIONS_HIGH

SYSTEM_AD_TRANSACTED_SESSIONS_HIGH_CLEAR

SYSTEM_AD_TRANSACTED_SESSION_RESOURCE_UTILIZATION_EXCEEDED

SYSTEM_AD_TRANSACTED_SESSION_RESOURCE_UTILIZATION_HIGH

SYSTEM_AD_TRANSACTED_SESSION_RESOURCE_UTILIZATION_HIGH_CLEAR

SYSTEM_AD_TRANSACTIONS_EXCEED

SYSTEM_AD_TRANSACTIONS_HIGH

SYSTEM_AD_TRANSACTIONS_HIGH_CLEAR

Adb syslog Messages

SYSTEM_ADB_CRITICAL_HARDWARE_FAILURE

SYSTEM_ADB_CRITICAL_HARDWARE_SOFT_FAILURE

SYSTEM_ADB_NONCRITICAL_HARDWARE_NOTIFICATION

Authentication syslog Messages

SYSTEM_AUTHENTICATION_ADMIN_BIND_FAIL

SYSTEM_AUTHENTICATION_ADMIN_CONN_DOWN

SYSTEM_AUTHENTICATION_ADMIN_CONN_UP

SYSTEM_AUTHENTICATION_BIND_CONN_DOWN

SYSTEM_AUTHENTICATION_BIND_CONN_UP

SYSTEM_AUTHENTICATION_CRL_DOWNLOAD_FAILED

SYSTEM_AUTHENTICATION_CRL_DOWNLOAD_SUCCESS

SYSTEM_AUTHENTICATION_DISCOVERY_FAILED

SYSTEM_AUTHENTICATION_DISCOVERY_SUCCESS

SYSTEM_AUTHENTICATION_DUPLICATE_ISSUER

SYSTEM_AUTHENTICATION_JWKS_DOWNLOAD_FAILED

SYSTEM_AUTHENTICATION_JWKS_DOWNLOAD_SUCCESS

SYSTEM_AUTHENTICATION_SEMP_USERNAME_LOCKOUT

SYSTEM_AUTHENTICATION_SESSION_CLOSED

SYSTEM_AUTHENTICATION_SESSION_DENIED

SYSTEM_AUTHENTICATION_SESSION_OPENED

SYSTEM_AUTHENTICATION_SHELL_ACCESS_DENIED

SYSTEM_AUTHENTICATION_SHELL_ACCESS_GRANTED

SYSTEM_AUTHENTICATION_TLS_START_FAIL

Cfgsync syslog Messages

SYSTEM_CFGSYNC_DOWN

SYSTEM_CFGSYNC_UP

Chassis syslog Messages

SYSTEM_CHASSIS_AVAILABLE_STORAGE_LOW

SYSTEM_CHASSIS_AVAILABLE_STORAGE_LOW_CLEAR

SYSTEM_CHASSIS_BLADE_DOWN

SYSTEM_CHASSIS_BLADE_POST_CRITICAL_FAILURE

SYSTEM_CHASSIS_BLADE_POST_NONCRITICAL_NOTIFICATION

SYSTEM_CHASSIS_BLADE_UP