Using Audit Logs and System Logs

Solace Cloud provides ways for you to receive logs and be notified of system events that occur. Depending on your deployment and the security policies, you may want to consider how to integrate these into your existing security infrastructure. For more information about logs that you can use for security audits, see Audit Logs.

In addition to overall audit logs in the Cloud Console, there are system logs available for each event broker service. These are useful for you to monitor the activities, performance, capacity, or operations that occur on your event broker services. You can access these logs in two ways:

  • Using the SysLog Forwarding feature, where the event broker services can be configured to forward logs to target servers you own. For more information, see Event Broker Service Logs.
  • Using Solace Insights Advanced Monitoring to access the logs collected from the central monitoring service. For more information, see Solace Insights.

Audit Logs

The audit logs provide records of user activity for security and compliance for your Solace Cloud enterprise account. You can view, monitor, and track the sequence of the following activities in the Solace Cloud Console:

  • IAM operations, such as user login
  • user management activities, such as user activation or role changes
  • event broker service lifecycle events such as the creation, modification, or deletion

With correctly authenticated and authorized users (including Administrators), you have a system that provides access to only those individuals who have the correct, pre-defined privileges. To ensure that the system authenticates and authorizes the right individuals, the system logs events such as brute force password attacks in addition to expected access to data and configuration changes.

You can view, filter, and download audit logs from Solace Cloud to monitor what occurs on your system. For more information, see Using Audit Logs.

Event Broker Service Logs

System logs are useful to understand the operations that occur on event broker services. These logs pertain to system, Message VPN, and client logs. You can forward these logs to your own monitoring system to manage the health of your system. For more information, see Forwarding Logs to an External System.

The same logs are used by Solace Insights, which permits you to monitor the health of your event broker services in your account (in addition to other useful monitoring features). For more information, see Solace Insights.

A subset of the event broker service logs are collected and sent to our central monitoring system, which Solace uses to monitor the health of your event broker services.

There are more logs available from the event broker services. You can use the Syslog Forwarding feature to enable the distribution of these logs in the Solace Cloud Console.

Limited logs are collected which are required for Solace Cloud to function correctly, which are as follows:

  • command.log—An audit log of all administrative commands run on the event broker service. The command action is logged and includes the user account that issued the command as well as the IP address of the connection from where the command was issued.
  • system.log—A notification log for significant system-level health events (e.g., redundancy state changes). For a summary of the logs that are collected, see System Logs Collected.
  • gather-diagnostics—An encrypted, diagnostics dump of the system state and logs that Solace can use to troubleshoot issues.
  • System metrics—Metrics for capacity monitoring and planning. The central monitoring service collects various logs from the event broker services. These logs are used for notifications and for advanced monitoring through Solace Insights. For detailed information on the information collected, see Metrics Collected.
  • Heartbeats—Health checks for various components of the event broker services are logged.
  • Response codes and status—Solace Home Cloud actions (upgrades, service creation, and deletion, etc.), confirmation as to whether the action completed as intended are collected.

Summary of Log Information Collected

The following is a summary of the log information collected by the Solace Cloud for monitoring of event broker service health. There are two categories of information that are collected:

System Logs Collected

A number of system logs are collected from the event broker services. These system logs are required to monitor the health and performance of the event broker services and utilized by Solace Insights for monitoring (via Datadog monitors). For information about the list of Datadog monitors and metrics available, see Solace Insights Monitors for Datadog Reference and Solace Insights Metrics and Checks.

The following is the list of system logs collected. For detailed information about each of the logs collected, see Solace PubSub+ Syslog Events.

  • SYSTEM_AD_MAX_EGRESS_FLOWS_EXCEEDED
  • SYSTEM_AD_MAX_INGRESS_FLOWS_EXCEEDED
  • SYSTEM_AD_MSG_SPOOL_QUOTA_EXCEED
  • SYSTEM_AD_DELIVERED_UNACKED_MSGS_EXCEED
  • SYSTEM_AD_TRANSACTED_SESSION_RESOURCE_UTILIZATION_EXCEEDED
  • SYSTEM_AD_DISK_USAGE_EXCEEDED
  • SYSTEM_AD_MSG_COUNT_UTILIZATION_EXCEED
  • SYSTEM_AD_TRANSACTED_SESSIONS_EXCEED
  • SYSTEM_AD_SPOOL_FILES_EXCEEDED
  • SYSTEM_AD_TRANSACTIONS_EXCEED
  • SYSTEM_AD_MAX_ENDPOINTS_EXCEEDED
  • SYSTEM_HA_REDUN_STATE_DOWN
  • SYSTEM_CFGSYNC_DOWN
  • SYSTEM_AD_TRANSACTIONS_HIGH
  • SYSTEM_AD_SPOOL_FILES_HIGH
  • SYSTEM_AD_DELIVERED_UNACKED_MSGS_HIGH
  • SYSTEM_AD_DISK_USAGE_HIGH
  • SYSTEM_AD_EGRESS_FLOWS_HIGH
  • SYSTEM_AD_ENDPOINTS_HIGH
  • SYSTEM_AD_INGRESS_FLOWS_HIGH
  • SYSTEM_AD_MSG_COUNT_UTILIZATION_HIGH
  • SYSTEM_AD_MSG_SPOOL_CHG
  • SYSTEM_AD_MSG_SPOOL_HIGH
  • SYSTEM_AD_SPOOL_FILES_HIGH
  • SYSTEM_AD_TRANSACTED_SESSIONS_HIGH
  • SYSTEM_AD_TRANSACTED_SESSION_RESOURCE_UTILIZATION_HIGH
  • SYSTEM_AD_TRANSACTIONS_HIGH

Metrics Collected

Logs are collected by a third-party, central monitoring service called Datadog. Insights Agents on the event broker services collect the statistics and send them over a secure, encrypted connection to the central monitoring service. For more information about the central monitoring service and Insights Agents, see Central Monitoring Service and Insights Agents.

The state information, metrics, and statistics collected by the Insights Agents are listed in Solace Insights Metrics and Checks section. These metrics are available for Advanced Monitoring in Solace Insights.