Smart Context Management
Smart Context Management is the set of techniques Solace Agent Mesh uses to keep the text that a large language model (LLM) reads small. It works mainly by storing large content as an artifact and giving the model a short reference to it, together with tools to read or process the content only when the task requires it. For more information, see Artifacts.
Agent Mesh uses several techniques to manage context, and not all of them can be measured. The Smart Context Savings card, marked Estimated, on the Cost and Token Usage page counts only four mechanisms. For more information, see Counted in the Savings Estimate. Prompt caching and history compaction, which the figure does not count, are described under Also Part of Smart Context Management.
Context Size and Cost
An agent completes a task in a loop. Agent Mesh sends the conversation to the model, the model calls a tool, and Agent Mesh sends the conversation, now including the tool result, to the model again. The loop repeats until the task is done. Every call in that loop re-sends the whole conversation, so a tool result that stays in the conversation is billed again on every later call, not once.
Content that Agent Mesh keeps out of the conversation avoids that cost on every call that follows. At about 4 bytes per token, a 40,000-byte tool result adds about 10,000 input tokens to each later call. If the task makes eight more calls, keeping the result out of the conversation avoids about 80,000 input tokens.
Counted in the Savings Estimate
Agent Mesh counts each of the following mechanisms because it records exactly how many bytes it kept out of the conversation. Except where noted, the settings that this section names are agent settings. For more information about where to set them, see Settings That Control Context Size.
Tool-Result Artifact Routing
Every tool returns its output to the model as a tool result. Agent Mesh checks the type and size of that output before it reaches the model. Unless the tool asks for different handling, Agent Mesh routes the output in one of three ways:
-
Binary output, such as an image, a PDF, or an archive, is always saved as an artifact, whatever its size.
-
Text output that is within the inline cap stays in the conversation, and Agent Mesh also saves a copy as an artifact. The inline cap is 8 KiB (8,192 bytes) by default, set with
tool_result_auto_artifact_threshold_bytes. -
Text output above the inline cap is saved as an artifact only. The model receives a reference and a preview, not a truncated copy. If the agent has no tool that can read the saved artifact, Agent Mesh also tells the model to repeat the call with a narrower request.
A tool can also ask Agent Mesh to handle its output differently:
-
A tool can ask Agent Mesh to always save its output. For example, the
web_requesttool always saves what it fetches. -
A tool can name its own size limit for a result. That limit replaces the inline cap for that result, and the
tool_result_inline_truncation_bytessetting caps it. -
A tool that asks for inline output without naming a limit still gets the inline cap, and Agent Mesh saves a larger result instead of truncating it.
The reference that the model receives lists the artifact's filename, version, size in bytes, MIME type, and description, plus a preview of the start of a text artifact. The preview is 500 bytes by default, set with tool_result_preview_length_chars. For data files, the reference also includes the structure that Agent Mesh inferred from the data, such as column names. The following example is illustrative, and the preview is shortened:
{
"artifacts_created": [
{
"filename": "release_notes.md",
"version": 0,
"size_bytes": 31240,
"mime_type": "text/markdown",
"description": "Release notes fetched from the product site",
"preview": "# Release notes for version 4.2"
}
]
}
The model reads more of the artifact only when the task requires it, with the artifact tools. The load_artifact tool loads a whole artifact or a slice of it, and the artifact_grep tool searches inside it. For more information about the artifact tools, see Built-In Tools.
The policy applies to the tools that return files or other content attached to their results, and it differs by tool type:
-
Built-in tools follow the preceding rules.
-
A tool that runs in the Secure Tool Runtime marks each content object that it returns with a disposition, a field that the tool's code sets on the object to tell Agent Mesh how to handle it. The values are
auto,artifact,artifact_with_preview, andinline. Agent Mesh handles the tool's output as follows:The Secure Tool Runtime saves each file that the tool writes to its output directory as an artifact, whatever the file size.
Content with the
auto,artifact, orartifact_with_previewdisposition is saved the same way, and the agent records each artifact without a preview.Content with the
inlinedisposition stays in the conversation within the inline cap, and Agent Mesh saves it as an artifact when it is larger.The Secure Tool Runtime also saves any inline result payload larger than 100 KiB (102,400 bytes). That limit is the
tool_output_llm_return_max_bytessetting in the configuration of the Secure Tool Runtime, separate from the agent's setting of the same name.
-
Model Context Protocol (MCP) tools use their own thresholds and handling:
Agent Mesh saves an MCP result when it is larger than 2,048 bytes or contains image, audio, or resource content.
For image and audio content, the model receives only a description of its type and size.
Agent Mesh replaces a result that is larger than 4,096 bytes with an artifact reference and, for JSON or YAML content, a short structural summary.
A text result between those two sizes reaches the model and is also saved.
When a text result is a JSON object or array, the model receives it as JSON, which uses fewer tokens than the same text as a quoted string. A workflow node that calls the tool, and the saved artifact, receive the text as the MCP server sent it.
An MCP server can also return structured content, a JSON copy of the result that it sends alongside the text. When the text repeats that copy, the model receives the content only once.
The savings estimate does not count MCP results.
Uploaded-File References
When a user attaches a file in the Agent Mesh UI, or through an entrypoint such as Slack, Microsoft Teams, or WhatsApp, Agent Mesh saves the file as an artifact before the model sees the message. What the model receives depends on the file:
-
A text file of 2 KiB (2,048 bytes) or less is included in the message, and the artifact is also available to the model's tools.
-
A larger text file, or a file that is not text, is replaced by a short metadata summary: the description, source, type, and size of the file, plus its inferred structure, such as the column names of a table. The model reads the content with a tool when the task requires it:
For a text file, the model calls
load_artifactto load all or part of the file, orartifact_grepto search it.For an image, the summary tells the model to call
load_artifact.For other files, such as a PDF or a Word document,
load_artifactdoes not extract the text. The agent requires a file conversion tool, such asconvert_pdf_to_markdown,pdf_extract_text, orconvert_file_to_markdown. For more information, see Built-In Tools.
Set max_inline_artifact_bytes to 0 to stop including file content in the message. Two other settings send uploaded images and PDFs to the model directly instead. For more information, see Settings That Control Context Size.
Data Tools
The data analysis tools compute inside the tool, so the model can answer a question about a large data set without reading the data:
-
The
query_data_with_sqltool runs a read-only SQL query against data artifacts. -
The
transform_data_with_jmespathtool applies a JMESPath expression to a JSON, YAML, or CSV artifact.
Each tool saves the full result as a new artifact. The model receives a preview of up to 50 rows and 2,048 bytes, whichever limit is reached first, a flag that shows whether the preview was truncated, and the name of the artifact that holds the full result. The preview always includes the first row. You can change the preview limits for a tool with max_result_preview_rows and max_result_preview_bytes in the tool's tool_config, not in the agent's app_config. For more information about each tool, see Built-In Tools.
Tool Results Above the Tool-Result Limit
A tool result that carries only structured data, such as the result of list_artifacts or artifact_grep, skips the inline cap and reaches the model as it is, up to the tool-result limit of 64 KiB (65,536 bytes) that tool_result_inline_truncation_bytes sets. Agent Mesh saves a larger result as an artifact. The model receives the result's status, the first 2 KiB of its message, its error details, and its small fields, with a reference to the artifact and, when it fits, a preview. The estimate counts the size of the result minus the size of what the model receives.
Estimate Calculation
The estimate follows four steps:
-
Agent Mesh records the number of bytes that each of the four mechanisms kept out of the conversation, minus the size of the reference, preview, or summary that the model received instead. Only text counts. Binary content, such as an image or a PDF, counts as zero, because Agent Mesh would never have sent its bytes to the model as text.
-
Agent Mesh converts the bytes to tokens at about 4 bytes per token, the same approximation it uses elsewhere to size conversation history.
-
Agent Mesh counts those tokens on every later call that the agent makes to the model for the same request, because each call re-sends the conversation. In a later turn of the session, the agent receives an earlier tool result as a one-line placeholder instead of the reference, so Agent Mesh stops counting the reference. When the model's maximum input size is known, Agent Mesh counts no more avoided tokens on a call than that size minus the size of the messages that the call sent, because messages beyond that size would have failed instead of costing more. A reference stops counting when history compaction removes it from the conversation. The extra call that compaction makes, the call that names a session, and any model call that a tool makes add nothing to the estimate.
-
Agent Mesh prices the tokens on each call at the average price that call paid per input token, across uncached input tokens, cache reads, and cache writes. When a call reports no input tokens, Agent Mesh uses the model's input price instead.
For the 40,000-byte tool result described earlier, the estimate counts the same 80,000 input tokens. At an illustrative average of $3 per million input tokens, the estimate is $0.24.
The figure is labeled Estimated for these reasons:
-
The token count is an approximation, because Agent Mesh does not run the model's tokenizer on the content it keeps out.
-
The estimate assumes that the agent would have made the same later calls if the content had stayed in the conversation. A model that receives the full content can take a different path through the task than a model that receives a reference.
-
The estimate prices the avoided tokens at each call's average input price. If the content had stayed in the conversation, the provider might have cached a different share of it, so the real avoided cost can be higher or lower than the estimate.
-
A text file that a Secure Tool Runtime tool saves counts at its full size, because the agent records it without a preview, even if the tool would never have returned the file's content to the model.
The token figure covers every measured call. A measured call is a call that an agent makes to the model while it works on a task and Agent Mesh is measuring savings. Step 3 of the calculation lists the calls that do not count. The dollar figure covers only measured calls to a model that has an input price. When a time window mixes priced and unpriced models, the dollar figure leaves out the tokens avoided on the unpriced calls. When no measured call has a price, or every price is $0, the card shows tokens only. When no call was measured, the card shows a dash and a message that savings were not measured in that period. This happens, for example, for a period before you upgraded to a release that measures savings. A conversation that started before that upgrade counts only the content that Agent Mesh kept out after it, so its savings read low until history compaction removes the earlier references from the conversation. When calls were measured but nothing was kept out, it shows $0.00.
Techniques Not Counted in the Estimate
The estimate leaves out the following techniques. They reduce context, but Agent Mesh cannot measure what the model would have received without them, so the dashboard shows no savings for them:
-
Skills load on demand. The
load_skilltool lists only the name and description of each skill. The full instructions load when the model calls the tool, and they stay in the context for the rest of the session. For more information, see Skills. -
Tool-list filtering removes tools that the model cannot use yet, such as tools owned by a skill that has not loaded. When role-based access control is configured, it also removes tools and peer agents that the user is not authorized to call.
-
Peer delegation keeps work in a separate context. The delegating agent receives the peer agent's reply, not the peer's own conversation or tool results. For more information about peer agents, see What Is an Agent?.
-
MCP results are not counted. Agent Mesh saves large MCP results as artifacts, as Tool-Result Artifact Routing describes, but the estimate does not record the bytes that they keep out.
-
Some results over the tool-result limit are shortened instead of saved. Above the limit,
load_artifactreturns the lines that fit and, when the content has more than one line, names the line to continue from. The estimate counts nothing for that result, because the model already holds a reference to the artifact. If Agent Mesh cannot save a result above the limit, the model receives the beginning of the result instead, a successful result's status changes topartial, and the estimate counts nothing for it. -
Prompt caching and history compaction are also excluded. For more information, see Also Part of Smart Context Management.
Settings That Control Context Size
The settings in the following table are agent settings. Where you set them depends on how you define the agent:
-
In a declarative configuration file for the
agentkind, put the settings underspec.additionalConfigurationswith the camelCase names in the following table. For more information, see Configure Settings Not Yet Modeled as Fields. -
In a component configuration file, put the settings under
app_configwith the snake_case names in the following table.
The following table lists each setting with its default.
app_config Setting |
additionalConfigurations Key |
Default | Effect |
|---|---|---|---|
tool_result_auto_artifact_threshold_bytes (alias tool_output_save_threshold_bytes) |
toolResultAutoArtifactThresholdBytes
|
8192
|
The largest text result that stays in the conversation. |
tool_result_inline_truncation_bytes (alias tool_output_llm_return_max_bytes) |
toolResultInlineTruncationBytes
|
65536
|
The tool-result limit: the largest result that the model receives from a tool that returns only structured data. It also caps the inline size that a single tool can request. |
tool_result_preview_length_chars
|
toolResultPreviewLengthChars
|
500
|
The length of the preview of a text artifact, in bytes. Tools that supply their own preview, such as web_request, ignore this setting. |
tool_result_processing_enabled
|
toolResultProcessingEnabled
|
true
|
When false, a result with files or attached content reaches the model as its status, message, and structured data only, and the tool-result limit no longer applies. |
max_inline_artifact_bytes
|
maxInlineArtifactBytes
|
2048
|
The largest text upload that is included in the message. A value of 0 never includes file content. |
enable_inline_vision
|
enableInlineVision
|
false
|
Sends uploaded images to the model directly. |
max_inline_vision_images
|
maxInlineVisionImages
|
5
|
The most images that Agent Mesh sends directly in one message. |
max_inline_vision_bytes
|
maxInlineVisionBytes
|
20971520
|
The most image bytes (20 MiB) that Agent Mesh sends directly in one message. |
enable_inline_pdf
|
enableInlinePdf
|
false
|
Sends a PDF to the model directly when it contains an embedded image that is at least 256 pixels on its shorter side, or a page with under 50 bytes of text. |
max_inline_pdf_count
|
maxInlinePdfCount
|
5
|
The most PDFs that Agent Mesh examines in one message. |
max_inline_pdf_pages
|
maxInlinePdfPages
|
100
|
The most pages that a PDF can have and still be sent directly. |
max_inline_pdf_bytes
|
maxInlinePdfBytes
|
20971520
|
The most PDF bytes (20 MiB) that Agent Mesh sends directly in one message. |
mcp_tool_response_save_threshold_bytes
|
mcpToolResponseSaveThresholdBytes
|
2048
|
MCP results larger than this value are saved as artifacts. Image, audio, and resource content is always saved. |
mcp_tool_llm_return_max_bytes
|
mcpToolLlmReturnMaxBytes
|
4096
|
MCP results larger than this value are replaced by an artifact reference. |
auto_summarization.enabled
|
autoSummarization.enabled
|
true
|
Turns automatic history compaction on or off. |
auto_summarization.compaction_percentage
|
autoSummarization.compactionPercentage
|
0.25
|
The fraction of the oldest history that compaction summarizes. |
auto_summarization.summary_max_tokens
|
autoSummarization.summaryMaxTokens
|
5000
|
The maximum length of the summary, in tokens. |
Two settings add content to the conversation instead of keeping it out. Both are off by default, and both require a model that accepts images or PDFs as input:
-
With
enable_inline_vision, Agent Mesh sends uploaded images to the model directly, up to five images and 20 MiB per message by default. -
With
enable_inline_pdf, Agent Mesh sends a PDF to the model directly when the PDF contains an embedded image that is at least 256 pixels on its shorter side, or a page with under 50 bytes of text. A PDF that is all text gets the metadata summary that Uploaded-File References describes. Agent Mesh examines up to five PDFs per message by default. Each PDF can have up to 100 pages, and the PDFs that Agent Mesh sends together can total 20 MiB. A PDF that exceeds a limit falls back to the metadata summary.
Setting tool_result_processing_enabled to false changes what Agent Mesh sends to the model for a result with files or attached content. For such a result, anything that a tool returns as a file, or as content attached to the result rather than in its structured data, never reaches the model. Examples include:
-
Fetched pages
-
Query rows that a Secure Tool Runtime tool returns inline
-
The references to files that Secure Tool Runtime tools save
A built-in tool that returns a file or fetched page saves no artifact. The setting also removes the tool-result limit, so a result that a tool reports only as structured data, such as the result of load_artifact, list_artifacts, or artifact_grep, reaches the model whole at any size. The setting does not change the following behavior:
-
The Secure Tool Runtime still saves its own files.
-
Images that
load_artifactreturns still reach the model. -
MCP results are unaffected at every size.
The following example shows how to lower the inline cap and raise the preview length and the upload limit for one agent in a declarative configuration file. Add the block to the agent's existing spec:
# agent declarative config
spec:
additionalConfigurations:
toolResultAutoArtifactThresholdBytes: 4096
toolResultPreviewLengthChars: 1000
maxInlineArtifactBytes: 4096
A higher cap keeps more content in the conversation, which raises the cost of every later call. A lower cap saves more, but the model then makes more artifact tool calls to read the content that the task requires. Prompt caching is a model setting rather than an agent setting, and the next section describes it.
Also Part of Smart Context Management
Prompt caching and history compaction are part of Smart Context Management, but the savings estimate does not count them and shows no savings for either.
Prompt Caching
Agent Mesh marks the stable parts of each LLM request, so that providers that support prompt caching can reuse them at a reduced price. The marks cover the system prompt, the tool definitions, and the most recent messages in the conversation. The marks on the recent messages move forward on every call, so the growing conversation stays cached.
You choose the strategy on the model configuration. In the Agent Mesh UI, it is the Prompt Caching Strategy under Advanced Settings of the model form. In YAML, it is the cache_strategy parameter, which accepts the values in the following table.
| Value | Agent Mesh UI Option | Behavior |
|---|---|---|
5m
|
5 minutes (default) | Cached content stays available for 5 minutes between calls. |
1h
|
1 hour | Cached content stays available for 1 hour between calls. |
none
|
Disabled | Agent Mesh adds no cache marks. |
Whether a provider honors the marks, and whether it reports cached-token counts, depends on the provider and on how Agent Mesh reaches it. For more information about the model form, see Configuring Models.
History Compaction
When a model reports that its context limit was exceeded, Agent Mesh summarizes the oldest part of the conversation with one extra LLM call, replaces that part with the summary, and retries the call. By default, compaction summarizes about the oldest 25% of the conversation, limits the summary to 5,000 tokens, and runs up to three times for one task. Agent Mesh turns compaction on by default. Users can also compact a conversation on demand from the context-usage indicator in the Agent Mesh UI, which appears when a conversation uses about half of the model's maximum input size. On-demand compaction summarizes the same share of the conversation as automatic compaction.
The effect of compaction depends on how the conversation would have continued without it, so the savings estimate does not count it. For more information, see History Compaction.
Next Steps
For more information about configuring an agent, see Creating Agents.