PubSub+ Insights Overview

PubSub+ Insights is a monitoring service that you subscribe to and if your account is enabled with Insights, you can access it from the PubSub+ Cloud Console. PubSub+ Insights provides you with the ability see historical and real-time metrics in the PubSub+ Cloud Console for your event broker services and access logs. This can help you monitor, operate, capacity plan your event broker services and event meshes all from the PubSub+ Cloud Console.

As part of your PubSub+ Insights subscription, you can also access Advanced Monitoring capabilities. The Advanced Monitoring options gives you additional capabilties to detect issues, recover them, and monitor performance. Advanced Monitoring comes with default dashboards that leverage our best practices to monitor event broker services and alerting capabilities. These capabilities include access to detailed information for all event broker services shown in in the dashboards that permit to you deep-dive and have better insight to your event streaming capabilities.  To access Advanced Monitoring, you must enable it For more information, see Using PubSub+ Insights Advanced Monitoring (Beta).

If you require a subscription, contact Solace. Insights provides a single entry point to see historical and real-time metrics for monitoring your event broker services to allow you to better manage them. Insights collects metrics to allow you to:

  • monitor the health and stability of your Message VPNs, endpoints, clients, and VPN Bridges
  • analyze capacity and bottlenecks so that you better manage your services and perform capacity management
  • proactively monitor your services to minimize downtime
  • build an understanding of application behavior
  • have in-depth insights from our advanced monitoring capabilities that includes access to dashboards, metrics, monitors, and logs

Insights takes care of collecting metrics to build visualizations and useful information for you so that you can monitor your system. To better understanding the Insights, see Using PubSub+ Insights. Alternatively, you can also use Syslog  Forwarding to send command/event logs if you have your own monitoring system, but if you have Insights enabled, you may find the visualizations and notification emails complementary to your existing monitoring system. For more information about forwarding logs from your event broker service, see Forwarding Logs to an External System.

For more information using Insights, understanding notifications, and managing notifications, see:

Using PubSub+ Insights

When have you Insights enabled, the Monitoring tab is populated with dashboard and historical information. With Insights, you are able to see summarized information and use a notification service that sends notification emails to the email accounts based on established metric thresholds. As part of the summary information, there are many useful visualizations. One useful visualization shows the overall health and status of your service as follows:

  • OK — There has not been any service interruptions for the service.
  • Interrupted — There has been anywhere from one to five minutes of interruption time for the service
  • Failed — There has been more than five minutes of interruption time for the service.

For more information about visualizations, notifications, and metrics, see the following topics:

 

The Monitoring tab provides you dashboards and graphical summary information. On the Monitoring tab, there are two drop-down menus:

  • From the left-most drop-down, you can choose from summaries of dashboards and graphs that includes: 
    • Summary — dashboard and historical information for your service that includes a health bar, messages rates, byte rates, Guaranteed Messages (Queue usage), Message spool usage, and discard rates.
    • Connections by Protocol — dashboard and historical information of the number of clients connected to the service by API protocol, REST, or via the Web
    • Messages Sent & Received — dashboard and historical information of incoming and outgoing messages sent to clients
    • Message Bytes & Rates — historical information of data sent and average incoming/outgoing data rates over secure (TLS) and non-secure connections, average incoming/outgoing message rates, client data received and sent
    • Guaranteed Messaging— dashboard and historical information of spool usage, incoming flows, outgoing flows ,transacted sessions, and transactions
    • Subscriptions — dashboard and historical information of subscriptions, unique subscriptions, remote subscriptions, unique local subscriptions, and export subscriptions complete percentage
  • From the right-most drop-down menu, you can choose the time interval of information as follows:
    • Last Hour — the last 60 minutes populated with data points aggregated at 20 second intervals
    • Last Day — the last 24 hours populated with data points aggregated at 5 minute intervals
    • Last Week — the last 7 days populated with data points aggregated at one hour intervals
    • Last Month — the last 31 days populated with data points aggregated at 4 hour intervals

    The visualized data is refreshed in the specified Update Interval period located at the bottom of the page (in this case, it is one minute). You can see the details about when it was last refreshed at the bottom of the page. For example, if the Monitoring tab is first viewed at 10:05, the Last hour time frame is from 9:05 to 10:05 and refreshed only after one minute to display data from 9:06 to 10:06.

Understanding Metric Collection

With PubSub+ Insights, broker metrics are collected to allow dashboards and visuals to be built. Metrics are collected at a high frequency interval providing a large amount of data points. To manage the data volume used in visualizations, metric data is aggregated at fixed intervals. The interval size is auto-selected to best fit the selected time frame.

Insights monitors use third-party monitoring software provider called Datadog. PubSub+ Cloud uses Datadog to: 

  • collect metrics through the use of Datadog monitors
  • send notifications for events that occur on event broker service,
  • configure thresholds and levels

At this time, direct access to Datadog isn't available.

Understanding Visualizations

You can select the time ranges for your data to see the metrics that have been collected to visualize your data. PubSub+ Cloud sends data every ten seconds to the monitors. The data that's used in the graphs and charts are cached for the refresh interval specified at the bottom of the monitoring page for each time range. If you refresh the page or switch to another time interval range that has been previously loaded by another user (including yourself), the same data from the cache is shown for that same time interval period. For this reason, you might not see changes reflected immediately for changes to the metrics or the visualizations for your service. It's also important to note that the data on the visualizations (or graphs) on the Insights page will automatically refresh based on the Update Interval (in minutes) specified at the bottom of the Monitoring tab.

Insights constructs visualizations from data points. The following are terms you should know to understand how the these data points are determined for the visualizations:

now
The point in time the data is first loaded.
ticks
On the x-axis, there are gradations that are automatically calculated by the charting engine depending on the number of data points (periods).
time aggregation
The aggregation type applied per period in a time-series result. This aggregated data becomes a data point.
periodicity
The size of the periods that data for the selected time frame is aggregated (the interval which data is aggregated). Each period is defined an aggregation of the values from the beginning of the period to the beginning of the next period. That aggregation is your data point.

Data points for the visualizations are determined based on the aggregation of data. The data that we've collected is visualized as line-graphs (time-series data) and charts (category-based data).

The granularity of data becomes more coarse as the time frame increases. Data within the period is aggregated using time aggregation.For information about aggregation, see Time Aggregation. The periodicity that used depends on the selected time range you select as shown in the following table: 

Time Range Relative Range Periodicity
Last hour (now - 1h) to now 20 second
Last day (now - 1d) to now 5 minutes
Last week (now - 7d) to now 1 hour
Last month (now - 31d) to now 4 hours

The appropriate periodicity is selected based on the time frame selected and the amount of data available.

Time Aggregation

Time aggregation is done by averaging data that's collected. This data is aggregated into larger intervals and the values are averaged over the selected periodicity. For example, to display 8: 00 PM on the line chart for a four hour periodicity, all the values from the period from 8:00 PM to 12:00 AM are averaged together to from a data point. Let's say in this case, there are three values (100, 335, 500) in that four-hour period, then the value 312 is the time aggregated value for 8 PM. Then if for the next four hours there are four values (498, 500, 502, 500), then the average is 500.

Understanding Notifications

A PubSub+ Insights subscription provides you with the ability to enable notifications for managing your event broker services. Insights uses Datadog (a third-party software provider) to:

  • collect metrics through the use of monitors
  • send notifications for events that occur on event broker service,
  • configure thresholds

By default, notification emails are not enabled. To enable notifications for your account, see Enabling Notifications for Your Account. Once enabled, notification emails indicate useful information such as:

  • the name of the monitor
  • the issue
  • a description of the problem
  • recommended action
  • the severity
  • time and date of the problem
  • details of the affected event broker service (service name, organization, current threshold)

The links in the notification email are not accessible at this time as access to Datadog isn't available.

Here's an example of the notification email with the severity of Alert when the Queue for Guaranteed Messages rises above 95% of the capacity:

Notification emails have three levels of severity as follows:

  • Alerts — A serious error where services are not available or operational. When an alert notification maps to a threshold level, this notification indicates that 95% of the available capacity has been reached.
  • Warning — An error where the event broker service is experiencing degradation or about to experience a loss of service if the issue is not resolved. When a warning notification maps to a threshold level, this notification indicates that 80% of the available capacity has been reached. These notifications often indicate a more serious problem may occur.
  • Recovery — Recovery notifications always follow the related Alert or Warning notification. For example, if you had received an Alert that indicated that the metric level had reached 95%, you'll get a corresponding Recovery notification when the metric level goes back below 95%.

The standard notifications are grouped into the following categories and utilize these Insights monitors:

For more information about the Insights monitors, see PubSub+ Insights Monitors Reference .

Managing Monitoring and Notifications

Notification emails are not enabled by default when you first subscribe to PubSub+ Insights. You must enable your account to receive notifications. In addition to enabling your account to receive email, you can disable it or change the email addresses that notifications are sent to. You can also see the length of time that logs are retained for. For more information, see the following:

Enabling Notifications for Your Account

To start receiving notifications, you must enable it for your account. Notifications are sent to up to five email addresses. This could be an email account that you used for your PubSub+ Cloud account or a different email address. Follow these steps to start receiving notification emails to one or more email addresses:

  1. Log in to your PubSub+ Cloud account.
  2. From the navigation bar, click User & Account Account Details.
  3. On the Account Details page, select the Insights Settings tab.
  4. In the Message Service Alerts & Notification:Inactive tile, type up to five emails (with each email delimited by a comma) in the List of emails for notifications field, and then click Activate Notifications.

If you need to disable notifications or change the email notifications are sent to, see Disabling Notifications for Your Account and Changing the Emails for Your Account.

Changing the Emails for Your Account

Follow these steps to change the email addresses to send email notifications to:

  1. Log in to your PubSub+ Cloud account.
  2. On the navigation bar, click your User & Account Account Details.
  3. On the Account Details page, select the Insights Settings tab.
  4. In the Message Service Alerts & Notification:Active tile, enter or remove emails in the List of emails for notifications field, and then click Update Notifications.

New notifications emails are now sent to the emails that you have specified.

Disabling Notifications for Your Account

To disable notification emails, follow these steps:

  1. Log in to your PubSub+ Cloud account.
  2. On the navigation bar, click your User & Account Account Details.
  3. On the Account Details page, select the Insights Settings tab.
  4. In the Message Service Alerts & Notification:Active tile, click Deactivate Notifications, and in the Deactivate Notifications dialog, click Deactivate.

Notification emails are no longer sent. To enable notification emails again, you must enable it for your account. For more information , see Enabling Notifications for Your Account.

Viewing the Log Retention Period

By default, monitoring data and logs from event broker services are retained for 30 days to help in analyzing, debugging, or troubleshooting your applications. You may require the logs to be retained for a longer time for additional requirements, such as security. If so, you can open a support ticket from the Cloud Console to request an increase of up to 90 days. After you open your ticket, we'll contact you.

To see the current log retention time for your services, follow these steps:

  1. Log in to your PubSub+ Cloud account.
  2. On the navigation bar, click your User & Account Account Details.
  3. On the Account Details page, select the Insights Setting tab and you'll see the log retention time in the Logs Retention tile.