In addition to overall audit logs in the Cloud Console, there are system logs available for each event broker service. These are useful for you to monitor the activities, performance, capacity, or operations that occur on your event broker services. You can access these logs in two ways:
- Using the SysLog Forwarding feature, where the event broker services can be configured to forward logs to target servers you own. For more information, see Event Broker Service Logs.
- Using PubSub+ Insights Advanced Monitoring to access the logs collected from the central monitoring service. For more information, see PubSub+ Insights.
Audit Logs
The audit logs provide records of user activity for security and compliance for your PubSub+ Cloud enterprise account. You can view, monitor, and track the sequence of the following activities in the PubSub+ Cloud Console:
- IAM operations, such as user login
- user management activities, such as user activation or role changes
- event broker service lifecycle events such as the creation, modification, or deletion
With correctly authenticated and authorized users (including administrators), you have a system that provides access to only those individuals who have the correct, pre-defined privileges. To ensure that the system authenticates and authorizes the right individuals, the system logs events such as brute force password attacks in addition to expected access to data and configuration changes.
You can view, filter, and download audit logs from PubSub+ Cloud to monitor what occurs on your system. For more information, see Using Audit Logs.
Event Broker Service Logs
System logs are useful to understand the operations that occur on event broker services. These logs pertain to system, Message VPN, and client logs. You can forward these logs to your own monitoring system to manage the health of your system. For more information, see Forwarding Logs to an External System.
The same logs are used by PubSub+ Insights, which permits you to monitor the health of your event broker services in your account (in addition to other useful monitoring features). For more information, see PubSub+ Insights.
A subset of the event broker service logs are collected and sent to our central monitoring system, which Solace uses to monitor the health of your event broker services.
There are more logs available from the event broker services. You can use the Syslog Forwarding feature to enable the distribution of these logs in the PubSub+ Cloud Console.
Limited logs are collected which are required for PubSub+ Cloud to function correctly, which are as follows:
command.log— An audit log of all administrative commands run on the event broker service. The command action is logged and includes the user account that issued the command as well as the IP address of the connection from where the command was issued.system.log— A notification log for significant system-level health events (e.g., redundancy state changes). For a summary of the logs that are collected, see System Logs Collected.gather-diagnostics— A diagnostics dump of the system state and the logs that can be used to assist in troubleshooting issues.- System metrics — Metrics for capacity monitoring and planning. The central monitoring service collects various logs from the event broker services. These logs are used for notifications and for advanced monitoring through PubSub+ Insights. For detailed information on the information collected, see Metrics Collected.
- Heartbeats — Health checks for various components of the event broker services are logged.
- Response codes and status — PubSub+ Home Cloud actions (upgrades, service creation, and deletion, etc.), confirmation as to whether the action completed as intended are collected.
Summary of Log Information Collected
The following is a summary of the log information collected by the PubSub+ Cloud for monitoring of event broker service health. There are two categories of information that are collected:
System Logs Collected
A number of system logs are collected from the event broker services. These system logs are required to monitor the health and performance of the event broker services and utilized by PubSub+ Insights for monitoring (via Datadog monitors). For information about the list of Datadog monitors and metrics available, see PubSub+ Insights Monitors for Datadog Reference and PubSub+ Insights Metrics and Checks.
The following is the list of system logs collected. For detailed information about each of the logs collected, see Solace PubSub+ Syslog Events.
- SYSTEM_AD_MAX_EGRESS_FLOWS_EXCEEDED
- SYSTEM_AD_MAX_INGRESS_FLOWS_EXCEEDED
- SYSTEM_AD_MSG_SPOOL_QUOTA_EXCEED
- SYSTEM_AD_DELIVERED_UNACKED_MSGS_EXCEED
- SYSTEM_AD_TRANSACTED_SESSION_RESOURCE_UTILIZATION_EXCEEDED
- SYSTEM_AD_DISK_USAGE_EXCEEDED
- SYSTEM_AD_MSG_COUNT_UTILIZATION_EXCEED
- SYSTEM_AD_TRANSACTED_SESSIONS_EXCEED
- SYSTEM_AD_SPOOL_FILES_EXCEEDED
- SYSTEM_AD_TRANSACTIONS_EXCEED
- SYSTEM_AD_MAX_ENDPOINTS_EXCEEDED
- SYSTEM_HA_REDUN_STATE_DOWN
- SYSTEM_CFGSYNC_DOWN
- SYSTEM_AD_TRANSACTIONS_HIGH
- SYSTEM_AD_SPOOL_FILES_HIGH
- SYSTEM_AD_DELIVERED_UNACKED_MSGS_HIGH
- SYSTEM_AD_DISK_USAGE_HIGH
- SYSTEM_AD_EGRESS_FLOWS_HIGH
- SYSTEM_AD_ENDPOINTS_HIGH
- SYSTEM_AD_INGRESS_FLOWS_HIGH
- SYSTEM_AD_MSG_COUNT_UTILIZATION_HIGH
- SYSTEM_AD_MSG_SPOOL_CHG
- SYSTEM_AD_MSG_SPOOL_HIGH
- SYSTEM_AD_SPOOL_FILES_HIGH
- SYSTEM_AD_TRANSACTED_SESSIONS_HIGH
- SYSTEM_AD_TRANSACTED_SESSION_RESOURCE_UTILIZATION_HIGH
- SYSTEM_AD_TRANSACTIONS_HIGH
Metrics Collected
Logs are collected by a third-party, central monitoring service called Datadog. Datadog agents on the event broker services collect the statistics and send them over a secure, encrypted connection to the central monitoring service. For more information about the central monitoring service and Datadog agents, see Central Monitoring Service and Datadog Agents.
The state information, metrics, and statistics collected by the Datadog agents are listed in PubSub+ Insights Metrics and Checks section. These metrics are available for Advanced Monitoring in PubSub+ Insights.