Chatting with the Activity Monitor (Experimental)

This feature is in the Experimental stage and under active development. Configuration schemas and behavior are subject to change. We recommend that you do not use this feature in production environments.

The Activity Monitor is a built-in agent you talk to like any other agent in your deployment. Instead of performing a task, it answers questions about the tasks other agents and workflows have already run: how many ran, how long they took, whether they succeeded, how much large language model (LLM) token usage they drove, and what feedback users gave on the responses. This page covers what the Activity Monitor can tell you and how to ask it. For an overview of the other activities in the application, see Using Agent Mesh.

The Activity Monitor is a different tool than the Activities page described in Monitoring Activity. The Activities page is a live, per-task view: you select one task and watch its execution graph as it happens. The Activity Monitor is a chat-driven, aggregate view: you ask it a question in natural language, and it summarizes many tasks over a time window into a report.

What the Activity Monitor Does

The Activity Monitor is read-only. It does not change agent configuration, cancel tasks, or fix anything it finds. If you ask it to resolve a problem, it explains what it found and recommends an action instead of attempting one.

It can do four things:

  • Produce an activity report for a time window, covering total tasks, failures, duration, and token usage broken down by agent and by workflow.

  • List the tasks that failed in a time window.

  • Trace a single task, including the peer agents or workflow nodes it touched, so you can see where a failure originated.

  • Summarize and look up user feedback (thumbs-up and thumbs-down ratings and comments), and explain why a user gave a particular rating.

Enabling the Activity Monitor

The Activity Monitor is on by default. An administrator can disable it by setting the SAM_FEATURE_ACTIVITY_MONITOR environment variable to false on the deployment. For more information about setting feature flags, see Environment Variables.

The Activity Monitor is seeded automatically and appears alongside the Orchestrator and any other built-in agents in the Deployed tab of the Agents page (Builder > Agents). For more information about that page, see Viewing Agents.

Starting a Conversation

Open the agent selector in the chat toolbar and select the Activity Monitor, the same way you select any other agent. For more information about the agent selector, see Chatting with Agents. You don't need special access to talk to it beyond the access your account already has to invoke agents. What the Activity Monitor tells you about, however, depends on your access. See Access and Visibility.

Requesting an Activity Report

Describe the time window you want in your message. The Activity Monitor understands relative phrasing ("the last 24 hours", "today", "the past 7 days") and absolute date ranges ("from June 1 to June 15"). If you don't specify a window, it defaults to the last 24 hours. It reports on a maximum of 90 days in a single request; ask for a narrower window if the Activity Monitor rejects your request for exceeding that limit.

You can also narrow a report to one agent or workflow by naming it in your request, for example, "How did the billing-agent do this week?" A narrowed report omits the by-agent and by-workflow breakdowns, because every row would restate the one agent you asked about.

Example requests:

Give me an activity report for the last 24 hours.
How many tasks failed last week?
Show me token usage for the release-notes-agent over the past 7 days.

The Activity Monitor returns two things: a short chat message stating the headline result, and an HTML report that opens in the chat preview pane the same way any other artifact does. For more information about artifacts and the preview pane, see Working with Artifacts.

Reading the Report

The report opens with a verdict banner summarizing the window as one of No activity, All succeeded, Mostly succeeded, Mixed outcomes, or Mostly failed, based on the proportion of tasks that failed. Below the banner, a row of tiles shows total tasks, success rate, failure count, 95th-percentile duration, and the number of tasks still running in the background.

The rest of the report breaks the window down into:

  • A status chart showing the mix of completed, failed, canceled, and in-progress tasks.

  • A duration histogram.

  • Token usage and task-count trends over the window.

  • A By Agent table and a By Workflow table, each with per-handler invocation counts, failure counts, and token totals. The By Workflow table also expands to show which agents ran inside each workflow.

  • A Users table, collapsed by default, lists every user active in the window and notes if the count hit a safety limit set for very large deployments. It only appears when you hold the activity:*:read scope and the report isn't already narrowed to one user. See Access and Visibility.

  • A Recent Failures table, capped at 10 rows. If there are more, the report tells you how many, and you can ask the Activity Monitor to list them all. See Looking Up Failed Tasks.

Looking Up Failed Tasks

Ask for the full list of failures instead of the capped preview in a report, for example, "List every task that failed yesterday". The Activity Monitor returns a paginated table, ordered newest first, with 20 rows per page by default. To get more rows on one page, ask for a specific number of failures, up to 100, for example, "List the 50 most recent failures". Ask for the next page by number if there are more results than fit on one page.

Tracing a Single Task

Give the Activity Monitor a task ID and ask it to explain or trace that task, for example, "Why did task 019fdc7d-4b90-7d63-8faa-b63ee755f7dd fail?" You can copy a task ID from the Task ID column of a failure list. The Activity Monitor replies in the chat with a trace that contains:

  • The task's execution tree: the tasks above it (if a peer agent or workflow delegated to it) and the tasks below it (if it delegated further). A marker identifies the task you asked about.

  • The task's event timeline.

  • A root cause section, taken from the task's error message or its last recorded event, if the task ended without completing.

  • Any other tasks in the tree that ended without completing, with each one's relation to your task.

If the task ended without completing, the Activity Monitor follows the trace with recommendations.

On the Activities page, Ask AI in the header of a selected task opens a chat with the Activity Monitor with the task ID already filled in. For more information, see Asking the Activity Monitor About a Task in Monitoring Activity.

The trace includes only the event timeline of the task you asked about. For the other tasks in the tree, the trace shows their status and duration. To see another task's events, ask about that task directly. If the tree is too large to show in full, the trace shows part of it and says so. To see more, ask about the lowest task shown.

Asking About Feedback

The Activity Monitor can answer questions about the thumbs-up and thumbs-down feedback users leave on agent and workflow responses. It chooses between three ways of answering, based on your question.

Feedback Overviews

Ask how an agent or workflow is doing, or which are rated worst, for example, "How is the hr-assistant doing this week?" or "Summarize all feedback from the last 7 days". The Activity Monitor returns tables of thumbs-up and thumbs-down counts and the negative ratio for each agent and workflow, with the worst ratio first. A summary that isn't narrowed to an agent, a workflow, or a user also lists up to 10 of the most recent negative comments. If your request could match more than one agent, the Activity Monitor asks which one you mean. If a name that you provide doesn't match any agent or workflow that you can access, the Activity Monitor tells you which name didn't match and answers for the names that match.

Individual Feedback Entries

Ask for the feedback from a named user, or the comments on a named agent or workflow, for example, "Show me the negative comments on the hr-assistant from the last 7 days" or "What feedback has user1@example.com given recently?" The Activity Monitor returns the newest entries with their rating and comment. A request must name an agent, a workflow, a user, or a feedback ID. If you don't hold a read_feedback scope, you can ask about the feedback you wrote yourself without naming anything. A single answer shows at most 50 entries. If there are more, the Activity Monitor tells you the list was cut off. You can ask follow-up questions about the entries it returned, such as which comments mention timeouts, without it fetching the data again.

Explaining One Piece of Feedback

Ask why a user gave a specific rating, for example, "Why did user2 give the sales-copilot a thumbs-down yesterday?" The Activity Monitor finds the entry, then replays the chat session up to the rated response. You can also give it the entry's feedback ID or the task ID of the rated response. Ask Activity Monitor on the Feedback History tab sends the feedback ID. For more information, see Reaching the Conversation Behind a Rating. The replay shows each turn's request, execution tree, and timeline, followed by a short list of what likely went wrong. It replays at most the last 20 turns of a session.

The Activity Monitor explains one piece of feedback at a time. If you ask it to open the sessions for many entries, it explains the limit and offers a list of the entries instead, so you can choose one.

If a session is old, parts of it might no longer be retained. When you own the session or hold the activity:*:read scope, the Activity Monitor shows the turns that remain. If the rated response itself is no longer retained, the Activity Monitor reports that the response is no longer retained. The Activity Monitor leaves earlier turns that are no longer retained out of the replay without mentioning them, because it has no record that those turns existed. When the Activity Monitor leaves out turns beyond the most recent 20, it reports that those turns were left out. If you rely on trace access, the Activity Monitor refuses the session when the rated response is no longer retained. See Access to Feedback.

Access and Visibility

By default, the Activity Monitor shows you only your own task activity: reports, failure lists, and traces are scoped to tasks your account started or that were run on your behalf. This default scope requires no special permission beyond the standard access your account already has to invoke agents.

An administrator can grant a role the activity:*:read scope to allow cross-user visibility: reports and failure lists for every user in the deployment, and the ability to ask about a specific user's activity by user ID. The activity:*:read scope follows the same scope-based access model used throughout Solace Agent Mesh. For more information, see RBAC Reference.

Access to Feedback

Feedback is limited by the read_feedback scope, the same scope that controls the Feedback dashboard:

  • If your role grants read_feedback for certain agents or workflows, you see feedback on those agents and workflows only. Asking about any other agent or workflow gives the same answer as asking about one that doesn't exist.

  • If you hold any read_feedback scope, feedback summaries and lists include only the agents and workflows that your role grants. They don't add feedback that you wrote yourself on other agents or workflows.

  • If you hold neither a read_feedback scope nor the activity:*:read scope, you see only the feedback you wrote yourself, as a list of individual entries. The Activity Monitor doesn't produce feedback overviews (counts and ratios) for you.

  • The read_trace scope on its own does not let you read feedback.

  • The activity:*:read scope lets you see all feedback.

Replaying the session behind a piece of feedback requires more than reading the feedback. The Activity Monitor checks the following cases in order and applies the first case that matches your access:

  1. If you hold the activity:*:read scope, the session opens.

  2. If you own the session (you started every turn of it), the session opens when every task in it is yours or one that Agent Mesh ran on its own. If another user's task is part of the session, the Activity Monitor refuses the session, even if your read_trace scope covers that task.

  3. Otherwise, your read_trace scope must cover every agent that took part in every turn of the session. A turn that started from a workflow is never opened this way.

If the case that applies to you isn't met, the Activity Monitor refuses the whole session. If you can read the feedback entry, the Activity Monitor shows you the entry and tells you that you don't have access to the session. If you can't, it reports that the feedback was not found, which is the same response it gives for feedback that doesn't exist.

The activity:*:read scope does not have a separate content tier. If you can see a task, whether because it's your own or because you hold this scope, you can see its content, including the text of the request that started it.

Troubleshooting

The following issues are the most common problems you might encounter using the Activity Monitor.

Cross-User Request Rejected

You asked the Activity Monitor about another user's activity, and it reported the request was rejected. Your account does not hold the activity:*:read scope required for cross-user reporting.

To resolve this issue:

  1. Ask your administrator to grant a role with the activity:*:read scope.

For more information about the scope model, see Access and Visibility.

No Access to a Feedback Session

You asked the Activity Monitor to explain a piece of feedback, and it reported that you can see the feedback but don't have access to its session. Reading a feedback entry does not by itself give you access to its session. For more information, see Access to Feedback.

To resolve this issue:

  1. Ask your administrator whether your role has trace access to every agent involved, or the activity:*:read scope. Trace access never opens a session that started from a workflow, so for those sessions you need to own the session or hold the activity:*:read scope.

  2. If you rely on trace access and the feedback is old, the response it rated might no longer be retained. In that case, trace access can't open the session.

Feedback Reported as Not Found

You asked the Activity Monitor to explain a piece of feedback, and it reported the feedback was not found. The Activity Monitor returns this same response when the feedback does not exist and when you can't read the feedback entry, so a not-found result does not prove that the feedback doesn't exist.

To resolve this issue:

  1. Confirm that the feedback ID came from a list the Activity Monitor gave you or from Ask Activity Monitor on the Feedback History tab, or that the task ID you gave is the rated response's task ID.

  2. Ask your administrator whether your role has the read_feedback scope for the agent or workflow that received the feedback.

Task Reported as Not Found

You asked the Activity Monitor to trace a task you believe exists, and it reported the task was not found. The Activity Monitor returns this same response both when a task ID does not exist and when it exists but you do not have access to it, so a not-found result does not confirm the task never ran.

To resolve this issue:

  1. Confirm you have the correct task ID from the Activities page or from a report's failure list.

  2. If you believe the task belongs to another user, ask an administrator with the activity:*:read scope to look it up.

Related Topics