Configuring Models

A model configuration tells Solace Agent Mesh how to reach a large language model (LLM): which provider hosts it, which model to call, and the credentials to authenticate with. Each configuration has a short alias (for example, general) that agents reference in place of the underlying provider details. You create and manage these configurations from the Agent Mesh UI, with no YAML to write and no restart.

This page covers creating and managing model configurations from the Models page in the UI. For installation instructions, see Install and Deploy. To manage the same configurations as version-controllable YAML instead, see Managing Configuration as Code.

How Agent Mesh Uses Models

Every agent reasons with a model, but an agent does not name a provider or hold an API key itself. Instead, each agent references a model configuration by its alias through the Model picker in the agent editor. This indirection lets you point many agents at one configuration and change the underlying provider or model in a single place.

A model configuration is composed of a few parts:

  • Display name: the alias agents and teammates use to reference the model, such as general.

  • Model provider: the service that hosts the model, such as Anthropic, OpenAI, or a Custom OpenAI-compatible endpoint.

  • Model name: the specific model to call, such as claude-sonnet-4-6.

  • Authentication: the credentials Agent Mesh uses to reach the provider, such as an API key.

  • Parameters: optional settings that shape responses, such as temperature, maximum tokens, and prompt caching.

Every deployment starts with two built-in model aliases:

  • general: the model new agents and built-in AI features use.

  • planning: the model the Orchestrator agent uses to plan and delegate work.

You connect these aliases to an LLM provider yourself, either during first-run setup or later on this page. Until a default alias is connected to a provider, it is flagged as not configured and agents cannot use it. Both aliases always carry a Default badge and cannot be deleted, though you can edit them to point at a different provider or model.

Every model an agent uses is a chat model. A deployment can also register a decision model (Early Access), a structured model from TypeSafe, OpenAI, or Google that answers typed questions. Only workflow decision nodes and decision-model evaluators can call a decision model. Agents, LLM-as-a-Judge evaluators, and other features that use a model cannot. For more information, see Adding a Decision Model (Early Access) and Decision-Model Evaluators (Early Access).

Opening the Models Page

On the navigation bar, expand System Settings and select Models. The page lists every model configuration with the following columns:

  • Name: the configuration's alias.

  • Model: the model name it calls.

  • Model Provider: the provider that hosts it.

  • Input and Output: the model's prices per 1,000,000 tokens in US dollars (USD). A dash marks an unpriced model. To see a priced model's cache rates, hover over or focus its Input price. For more information, see Pricing Models.

  • Last Updated: when the configuration was last changed.

Select the Name, Model, or Model Provider column header to sort by that column. The built-in general and planning configurations carry a Default badge, and a default configuration that has not yet been connected to a provider is flagged so you can finish setting it up. Each row has an Actions menu, and Add Model creates a new configuration.

Adding a Model

The following steps add a model configuration for an Anthropic model and make it available to your agents.

  1. On the Models page, click Add Model.

  2. If the form shows Model Type, leave Chat (LLM) model selected. If it does not, go to the next step. The form shows Model Type only while workflow decision nodes or decision-model evaluators are enabled.

  3. Enter a Display Name and a Description. The display name is the alias agents reference, so keep it short and descriptive, such as analysis. The description tells teammates what the model is suited for.

  4. Select a Model Provider. The list includes Amazon Bedrock, Anthropic, Microsoft Foundry (formerly Azure OpenAI), Google AI Studio, Google Vertex AI, Ollama, OpenAI, and Custom, which covers any provider that implements the OpenAI-compatible API protocol. Select Anthropic.

  5. Enter the connection details the provider requires:

  6. Set Model Name. When the provider and key are valid, the list populates with the models available to your account. You can also type a model name directly.

  7. (Optional) Set Max Input Tokens to the model's context-window limit.

  8. (Optional) Under Pricing, enter what the model costs per 1,000,000 tokens in USD, in Input Price / 1M Tokens and Output Price / 1M Tokens. To set separate rates for cached tokens, select Cached input tokens have different prices, then enter Cache Read Price / 1M Tokens and Cache Write Price / 1M Tokens. For more information, see Pricing Models.

  9. (Optional) Expand Advanced Settings to set Temperature, Max Tokens, a Prompt Caching Strategy, or provider-specific Custom Parameters.

  10. Click Test Connection to confirm Agent Mesh can reach the model. A failed test shows the provider's error message inline.

  11. Click Add to save the configuration. It appears on the Models page, ready for agents to reference.

Some providers, such as Custom and Ollama, add an Authentication Type field with API Key, OAuth2, and None options. For a local or self-hosted, OpenAI-compatible endpoint, select Custom or Ollama, set Authentication Type to None, and provide only the API Base URL; no API key is required.

For a Microsoft Foundry model on a deployment that enables it, the Authentication Type list also includes Azure Workload Identity, which lets the model sign in to Azure without a stored key. For more information, see Authenticating Models with Azure Workload Identity.

If Test Connection fails but the details are correct, you can still save the configuration: select Add, then Save Anyway in the Connection Test Failed dialog. This is useful when the provider is not reachable from where you are working.

Entering an API Base URL for a Custom Model

For a chat model whose provider is Custom, API Base URL accepts either of two forms:

  • A base URL. For example, https://llm.example.com. Agent Mesh sends chat requests to the base URL followed by /v1/chat/completions. If the base URL already ends in /v1, Agent Mesh drops that /v1 first, so https://llm.example.com/v1 reaches the same endpoint. When you select a Model Name, Agent Mesh lists the available models from the base URL followed by /v1/models.

  • The provider's complete chat completions URL. Use this form for a provider that does not serve its API under /v1. Enter the full URL, ending in /chat/completions. Agent Mesh calls that URL exactly as you wrote it, and lists models from the /models URL beside it. A trailing / is allowed, and a query string is kept. For example, Volcengine Ark serves chat completions at https://ark.cn-beijing.volces.com/api/v3/chat/completions, so Agent Mesh lists its models at https://ark.cn-beijing.volces.com/api/v3/models.

A Requests go to line under API Base URL shows the chat completions URL Agent Mesh calls for the value you entered, so you can confirm which form you used. The line assumes that the model uses the OpenAI-compatible chat completions API, so it does not apply to a Model Name that selects another provider, as described in the limits below.

The complete chat completions URL applies only to chat models whose provider is Custom, or OpenAI when you configure the model with the CLI. Other providers and decision models treat API Base URL as a base URL, as before. The following limits also apply to a complete chat completions URL:

  • A Model Name that selects another provider's API treats API Base URL as a base URL. This includes a name that starts with another provider's prefix, such as groq/, and a name that contains claude or anthropic without an openai/ prefix. To send a Claude model through an OpenAI-compatible gateway at a complete chat completions URL, start the Model Name with openai/, for example openai/claude-sonnet-4-6.

  • When you configure the model with the CLI, a Model Name that contains gemini without an openai/ prefix also treats API Base URL as a base URL.

  • Models that accept a request only through the OpenAI Responses API do not work with a complete chat completions URL. These include the OpenAI -pro models, and some GPT-5 models when the request includes tools. With a provider that serves its API under /v1, use a base URL for these models instead.

Earlier Agent Mesh releases do not recognize a complete chat completions URL. They append /v1/chat/completions to it. Upgrade every Agent Mesh component before you use this form.

Adding a Decision Model (Early Access)

This feature is in the Early Access stage and under active development. Configuration schemas and behavior are subject to change. We recommend that you do not use this feature in production environments.

A decision model is the model that workflow decision nodes and decision-model evaluators call to answer typed questions. A decision node classifies its input and routes on the answer, and a decision-model evaluator scores an agent's response. For more information about decision nodes, see Routing with Decision Nodes (Early Access). For more information about decision-model evaluators, see Decision-Model Evaluators (Early Access).

Before you begin, complete these prerequisites:

  • Get an API key from your decision model's provider: TypeSafe, OpenAI, Google (a Google AI Studio API key for the Gemini API), or a pass-through service that serves the TypeSafe API, such as OpenRouter.

  • Confirm that workflow decision nodes or decision-model evaluators are enabled. The Add Model form offers decision models while either feature is enabled.

    • Workflow decision nodes are enabled by default. An administrator can disable them by setting the SAM_FEATURE_WORKFLOW_DECISION_NODE environment variable to false.

    • Decision-model evaluators are enabled by default. An administrator can disable them by setting the SAM_FEATURE_EVAL_DECISION_EVALUATOR environment variable to false.

A workflow decision node or a decision-model evaluator sends data to a third party. That third party is the decision model's provider (TypeSafe, OpenAI, or Google), or a pass-through service such as OpenRouter that forwards the call. For a workflow that you save in the Agent Mesh UI or apply with sam config apply, registering a decision model is what allows that data to leave your deployment. The same is true for an evaluator that names the decision model. Review the provider's data-processing terms before you register one.

To add a decision model, perform these steps:

  1. On the Models page, click Add Model.

  2. Under Model Type, select Decision model, which carries an Early Access badge. The form hides Max Input Tokens and Advanced Settings, which do not apply to a decision model.

  3. Enter a Display Name and a Description. The display name is the alias that a workflow's decision.model_alias field or an evaluator's model field names, such as ticket-router.

  4. Select a Model Provider: TypeSafe, OpenAI, or Google AI Studio (for the Google Gemini Decisions API). All three carry the Early Access badge of decision models.

  5. Enter the connection details:

    • API Base URL — For TypeSafe, leave the field blank to use the TypeSafe API at https://api.typesafe.ai, or enter a pass-through that serves the same API, such as https://openrouter.ai/api. For OpenAI, leave the field blank to use the OpenAI API at https://api.openai.com/v1, or enter another base URL that serves the OpenAI Decisions API, such as a regional OpenAI API endpoint. For Google AI Studio, leave the field blank to use the Gemini API at https://generativelanguage.googleapis.com/v1beta, or enter another base URL that serves the Gemini Decisions API.

    • API Key — Enter the API key from your provider or pass-through service. A decision model always authenticates with an API key.

  6. Enter or select the Model Name:

    • For TypeSafe, enter a version such as jev-1.13.0, or typesafe/jev-1.13 through OpenRouter. Use a pinned version rather than an alias such as jev-latest, which the provider can move to a newer version and change how deployed workflows route.

    • For a TypeSafe model that reads images, enter a Jev version, such as jev-1.14.0, with the TypeSafe API base URL. With the OpenRouter API base URL, enter typesafe/jev-1.14 or openai/gpt-6-luna-decisions. For more information, see Sending Images to the Decision Model.

    • For OpenAI, select gpt-6-luna from the list.

    • For Google AI Studio, select gemini-decision-1 from the list.

  7. (Optional) Under Pricing, enter what the model costs per 1,000,000 tokens in USD, in Input Price / 1M Tokens and Output Price / 1M Tokens. The form offers no cache rates for a decision model. For more information, see Pricing Models.

  8. Click Test Connection to confirm that Agent Mesh can reach the model. For a decision model, the test asks the model one yes-or-no question.

  9. Click Add to save the configuration. The model appears on the Models page with a Decision badge, and Open Details shows its Model Type as Decision model.

Be aware of the following considerations about decision models:

  • A model's type cannot change after it is created.

  • A decision model's provider cannot change after it is created. The Edit form shows the provider but does not let you change it. To move to another provider, add a decision model on that provider and point your workflows at it.

  • The Model picker in the agent editor does not list decision models.

  • You cannot rename a decision model while a workflow's decision.model_alias names it, in either the workflow's saved version or its deployed version.

  • Deleting a decision model undeploys the deployed workflows that name it. For more information, see Managing Models.

Managing Models

Select a configuration's Actions menu to reach its lifecycle actions:

  • Open Details: view the configuration's provider, model, and audit information.

  • Edit: reopen the editor to change the provider, model, credentials, or parameters. Stored credentials are masked; enter a new value only to replace one.

  • Delete: remove the configuration.

The Delete Model dialog lists the agents and workflows that use the configuration. Deleting the configuration undeploys the ones that are deployed, and the dialog marks each of them Will be undeployed. Agents and workflows that are saved but not deployed still reference the deleted configuration's alias, so update them to use another configuration before you deploy them. To confirm the deletion, type the configuration's alias. The built-in general and planning configurations are required, so you cannot delete them; selecting Delete on either one opens an Unable to Delete dialog instead.

Pricing Models

Agent Mesh records the cost of every LLM call from the prices on the model configuration the call used, and a budget sets a monthly spending limit on that cost. For more information, see Managing Model Spend with Budgets.

A model's price has up to four rates, each in US dollars per 1,000,000 tokens with at most four decimal places:

  • Input: prompt tokens not served from the provider's cache.

  • Output: completion tokens, including reasoning tokens.

  • Cache Read: prompt tokens served from the provider's cache.

  • Cache Write: tokens written into the provider's cache.

Enter the input and output rates together. The cache rates are optional. Enter them together, and only after you price input and output. If you leave both cache rates blank, Agent Mesh prices cached tokens at the input rate.

A model with no price is unpriced, not free. Agent Mesh records the tokens of a call to an unpriced model but no cost, so the call does not count toward any budget. To record a model as costing nothing, enter 0 for both the input and output rates. To leave a model unpriced, leave its rates blank.

A price change applies to calls that Agent Mesh records after the change; it never changes the cost of calls already recorded.

Agent Mesh can price some tokens differently from how your provider bills them:

  • Agent Mesh prices cache writes at the cache-write rate only when the provider reports them separately, as providers such as Anthropic and Amazon Bedrock do. Otherwise, it prices them at the input rate. Anthropic charges more for a 1-hour cache write than for a 5-minute one, but Agent Mesh prices both at the model's cache-write rate.

  • Agent Mesh prices image and audio input at the model's input rate, but some providers charge a different rate for it.

Prices are visible only to users who hold model_config:*:read.

The Agent Mesh UI offers only input and output rates for a decision model. Agent Mesh records the calls of a decision model and refuses them while a budget is spent, like the calls of any other model.

Raising the Stream Idle Timeout

By default, Agent Mesh abandons a streamed model response when the provider sends no data for 300 seconds, and the request fails with a stream idle timeout error. Some providers send a tool call only after the model finishes generating it, so a large tool-call argument, such as a complete HTML page, can leave the stream silent for several minutes. If an agent's requests fail with this error, raise the timeout on the model that the agent references.

To raise the stream idle timeout, perform these steps:

  1. On the Models page, open the model's Actions menu and select Edit.

  2. Expand Advanced Settings.

  3. Under Custom Parameters, click New Pair, enter stream_idle_timeout_seconds as the key, and enter the number of seconds to wait, such as 600, as the value.

  4. Click Save.

You can set the same parameter when you add a model. If the form warns that some custom parameters may not be supported by the selected model, you can ignore the warning: Agent Mesh still applies the setting. A value of 0 or less uses the 300-second default. To set the same value in a declarative model resource, see Configuring Models with the CLI.

Enter stream_idle_timeout_seconds as an unquoted number, such as 600. Agent Mesh saves a quoted value, such as "600", without an error but ignores it.

Two other timeouts can end the request before a silent stream reaches the stream idle timeout:

  • When an agent delegates to a peer agent, the delegating agent receives no progress while the peer's stream is silent. Set the delegating agent's spec.additionalConfigurations.interAgentCommunication.requestTimeoutSeconds to a value greater than the stream idle timeout of the peer agent's model. The default is 120 seconds. For more information about the additionalConfigurations block, see Configure Settings Not Yet Modeled as Fields in Creating Agents with the CLI.

  • For an agent node in a workflow, the node timeout counts from when the node starts, so it must cover the agent's whole run, including a silent stream. In the workflow resource, set timeout on the node, or set spec.appConfig.default_node_timeout for nodes that set none. Both take a duration, such as 10m, and the default is 5m. For more information about workflow files, see Creating Workflows with the CLI (Early Access).

Setting requestTimeoutSeconds also changes the defaults of workflowRequestTimeoutSeconds and subTaskTimeoutSeconds. Both default to 600 seconds, but after you set requestTimeoutSeconds, they default to your value or 300 seconds, whichever is larger. To keep them at 600 seconds, set them explicitly.

Using a Model in an Agent

Agents choose a model in the agent editor. When you create or edit an agent, the Model picker lists every configuration from the Models page by its alias, and the agent uses whichever one you select for its reasoning. A new agent uses the general configuration by default. For the full agent workflow, see Creating Agents.

Define Models as Code Instead

The Agent Mesh UI is one way to manage model configurations; the sam CLI is the other. With declarative config you describe a model as YAML and reconcile it into Agent Mesh with sam config apply, the version-controllable path that suits GitOps and automation. To create the same model from the command line, see Configuring Models with the CLI.

Next Steps

You have model configurations that your agents can use. Most readers next want to put them to work: