Planning and Applying Changes

sam config plan and sam config apply are the two commands you run day to day: plan shows what would change, apply makes the change. This page covers both, the flags that shape an apply, and how to run them in CI. For pointing them at a target, see Targets and Authentication.

Preview Changes with Plan

sam config plan computes the difference between the manifest and the running Platform service and prints it. It changes nothing:

sam config plan -m manifest.yaml
agents/
  + release-notes-assistant  create + deploy
  - Orchestrator             delete (needs --prune)
2 agents: 1 create, 0 update, 1 delete, 0 unchanged

The plan diffs the Platform service against the manifest for each kind the manifest declares. A focused manifest, one that manages only a subset of resources, still shows every other agent of that kind, including any built-in agents, as a delete (needs --prune) row. These delete entries are informational: by default apply performs creates and updates but skips deletes, so the rest of the Platform service is left in place. Skipping deletes by default is what makes a focused manifest safe to apply. To preview the deletes that apply --prune would run, pass --prune to plan. The delete rows then read delete and count as pending changes, but plan still deletes nothing.

To see field-level detail for the resources that would be updated, add -v:

sam config plan -m manifest.yaml -v

Verbose output lists the changed fields per resource. Secret fields never print their values; a changed credential renders as (changed).

An update can remove stored settings that a resource file no longer sets. When an agent's file no longer sets toolsetConfigs or skillConfigs, apply removes the agent's stored toolset or skill settings, including settings saved in the Agent Mesh UI. When an entrypoint's file no longer sets publicUrl, apply removes the stored public URL. The apply command keeps any entrypoint values keys that the file leaves out. To keep a setting, add it back to the file before you run apply.

When an agent's file keeps some toolsetConfigs or skillConfigs entries and drops others, apply removes the dropped entries, and the plan row does not name the field. To see which toolset or skill names lose their settings, run sam config plan with -v.

When a file no longer sets toolsetConfigs, skillConfigs, or publicUrl at all, the plan row names the field, for example update + deploy (clears toolsetConfigs), and the plan prints a warning for the resource. When you pass --format json, the warning appears in diagnostics.

Apply the Configuration

sam config apply computes the same plan, then reconciles the Platform service in two phases. First it syncs configuration: it runs the creates and updates. Then it deploys the resources that run as services, such as agents, workflows, and entrypoints, bringing them online:

Diagram: Apply the Configuration

sam config apply -m manifest.yaml
agents/
  + release-notes-assistant  create + deploy
  - Orchestrator             delete (needs --prune)
2 agents: 1 create, 0 update, 1 delete, 0 unchanged

Applied agents:
  + release-notes-assistant  created
  - Orchestrator             skipped (use --prune to delete)
Deployments:
  * release-notes-assistant  deploy (deploy) completed

The skipped (use --prune to delete) line confirms that the resources the plan listed as deletes were left in place.

Flags That Shape an Apply

Flag Effect
--dry-run Compute the plan without changing the Platform service, the same result as sam config plan.
--no-deploy Run the configuration sync but skip the deployment phase, so a manifest can reconcile config without redeploying running services.
--prune Delete resources that exist on the Platform service but are absent from the manifest. Reserve this for a manifest that is the complete desired state.
--force Skip the --prune confirmation prompt. Required when --prune runs non-interactively, such as in CI.
--format Print the result as text (the default) or as one json document.
--no-build Skip building toolset and skill tools. On apply, a tool that has not been built is a hard error. On plan, an unbuilt tool is reported as [BUILD: skipped] rather than failing.
-v, --verbose Show the per-field diff for each resource that would be updated.

Deleting Resources

Because apply skips deletes by default, removing a resource from the manifest does not remove it from the Platform service. To make the manifest the authoritative, complete state, run with --prune:

sam config apply -m manifest.yaml --prune

--prune deletes every resource of a declared kind that is not in the manifest, after an interactive confirmation. If you decline the prompt, the command applies nothing and exits with status 1. Add --force to skip the prompt in a non-interactive pipeline.

--prune deletes resources, and it acts on the whole deployment, not on this repository. For each kind that the manifest declares, it deletes every resource that the manifest does not list, including the built-in agents and resources that another repository or the Agent Mesh UI created. Use --prune only when this repository is the only source of changes to the deployment and the manifest lists every resource of its kinds on the deployment, including the built-in agents.

CI and CD

Authenticate a pipeline with SAM_PLATFORM_TOKEN. For more information, see Targets and Authentication. A run is non-interactive when the CI environment variable is set to a value other than 0 or false, when standard input or standard error is not a terminal, or when you pass --no-interactive. A non-interactive run never prompts or opens a browser, and apply --prune fails unless you also pass --force or --dry-run.

The common pipeline runs plan on a pull request and apply on merge:

# On a pull request: preview only.
sam config plan -m manifests/prod.yaml

# On merge to the main branch: reconcile.
sam config apply -m manifests/prod.yaml --no-interactive

When the manifest is the complete desired state for its kinds, add --prune to the plan step and --prune --force to the apply step. Without --prune, the plan for a pull request that only removes resources marks each removal delete (needs --prune), as if the apply skips it. The apply on merge then deletes those resources.

For more information about what --prune deletes, see Deleting Resources.

Give the plan step the same token as the apply step. The Platform service never returns a stored secret, so when a resource file sets one, plan asks the Platform service whether the value changed, which requires the kind's update scope. For more information, see Comparing Secrets at Plan Time.

Pass --no-color or set the NO_COLOR environment variable to drop the ANSI color codes from the output of either command, which keeps captured CI logs readable.

An apply does not stop at the first failure: it continues through the remaining resources and prints a summary at the end. If any operation failed, the command exits with status 1, so a failed apply fails the pipeline step even though the other resources were reconciled.

Exit Codes and JSON Output

By default, sam config plan and sam config apply --dry-run exit with status 0 when the plan succeeds, regardless of whether changes are pending, and 1 when the command fails. Pass --detailed-exit-code to report pending changes in the exit status, so a pipeline step can fail when the Platform service no longer matches the manifest. With --detailed-exit-code, the exit status means:

Exit status Meaning
0 The plan succeeds and no changes are pending.
1 The command fails.
2 The plan succeeds and changes are pending.

The following example shows a scheduled job step that fails when the Platform service no longer matches the manifest:

sam config plan -m manifests/prod.yaml --detailed-exit-code

Deletes count as pending only when you pass --prune to plan or to apply --dry-run. An agent or entrypoint whose deployment is out of sync also counts, because apply redeploys it. Role-based access control (RBAC) changes do not count when the Platform service blocks RBAC writes, because apply skips them. The plan still lists them and prints a warning. An apply without --dry-run exits 0 on success and 1 on failure, and rejects --detailed-exit-code.

When the last deployment of an out-of-sync agent or entrypoint failed, its plan row reads out of sync — will re-deploy (last deploy failed). Running apply again retries the deployment but does not fix what made it fail. While the deployment keeps failing, plan --detailed-exit-code exits with status 2 and apply exits with status 1. To find the cause, read the error that apply prints for the resource, or the error field of its failed entry in results when you pass --format json. Fix the cause, then run apply again.

Pass --format json to print one JSON document on standard output instead of the text report. It contains:

  • changes: the kind, name, action, and source of each resource. The action is create, update, delete, redeploy, or unchanged.

  • summary: the number of resources with each action. It counts every entry in changes, including deletes in a run without --prune and RBAC changes that apply skips. To decide whether changes are pending, check the exit status in a run with --detailed-exit-code, not summary.

  • diagnostics: the warnings that the text report prints, each with a level (warning or error) and a message. When the run fails, the last entry has the error level and its message is the error.

  • results: one entry for each operation that apply attempts. The plan command omits this field, and an apply --dry-run attempts nothing, so the list is empty. The status is created, updated, deleted, deployed, skipped, or failed. A deployed, skipped, or failed entry may name the attempted operation in action, and a failed entry carries the error in error, including any message that the Platform service returns.

The changes and summary fields carry no field values, so no secret appears in them. A run that fails still prints the document and exits with status 1. When it fails before the plan finishes, such as on an authentication or manifest error, changes is empty. When some apply operations fail, results lists each failure. The exception is an apply that exits before it starts, for example because you pass --prune without --force in a non-interactive run. That apply writes only the error to standard error.

A Complete Pipeline

The following pipeline recipe works in any CI system:

  • On a pull request, the pipeline runs sam config plan and prints the plan in the job log. The pipeline writes nothing back to the pull request.

  • On merge to the main branch, the pipeline runs sam config apply --no-interactive, then sam eval run on the experiments you name. The job fails when an experiment's pass rate is below the threshold.

  • Only one apply runs at a time, through the CI system's own concurrency control.

  • The pipeline installs a pinned release of the Agent Mesh CLI (sam) and checks the downloaded archive against its SHA-256 checksum.

  • Every credential comes from the CI system's secret store.

The recipe is the same for a self-hosted deployment and for Agent Mesh Cloud. Only the URL and the API key change.

Prerequisites

The target deployment must meet these requirements:

Create the Pipeline's API Key

The pipeline authenticates as a system user with an API key in SAM_PLATFORM_TOKEN. For more information about system users, see Machine Entrypoints and System Users. Declare the system user and its role in the config repository, in their own manifest. A super administrator applies that manifest once with their own login, then creates the key. Both steps run outside the pipeline, because the key's owner must exist before the key and because only a signed-in super administrator can create a key.

Save the role under rbac/roles/, the system user under rbac/system-users/, and the manifest as manifests/rbac.yaml:

# ci-deployer.yaml: RBAC role
kind: rbacRole
name: ci-deployer
spec:
  scopes:
    - "*"
# ci-deployer.yaml: system user
kind: systemUser
name: ci-deployer
spec:
  roleNames:
    - ci-deployer
# rbac.yaml: manifest the super administrator applies
kind: manifest
name: rbac
target:
  url: https://sam.example.com
  auth:
    type: oauth
resources:
  rbacRoles:
    - ci-deployer
  systemUsers:
    - ci-deployer

A key with the * scope, the shorthand for *:*:*, has full control of the deployment, including RBAC, so store it as you would an administrator's password. For more information, see Applying the Full Configuration From a Pipeline.

Agents use the agent_builder category, workflows use workflow_builder, models use model_config, and Entrypoints use entrypoint. In a scope that a role or key holds, _ and * in the middle position are equivalent. List the scopes exactly as written in the following list, because they are the strings that Agent Mesh requires. For more information about the scope grammar, see Scope Format.

To limit the key, list only the scopes that the pipeline's manifest requires, in both the role and --scopes:

  • <category>:_:create, <category>:*:read, and <category>:*:update for each declared kind other than data sets, evaluators, and experiments, such as toolset:_:create, toolset:*:read, and toolset:*:update for toolsets

  • evaluation:_:create, evaluation:_:read, and evaluation:_:update when the manifest declares data sets, evaluators, or experiments

  • agent_builder:*:deploy, workflow_builder:*:deploy, and entrypoint:*:deploy for the agents, workflows, and Entrypoints that the manifest declares

  • skill:*:read, connector:*:read, and toolset:*:read when the manifest declares agents

  • agent_builder:*:read when the manifest declares workflows or Entrypoints, and workflow_builder:*:read when it declares Entrypoints

  • evaluation:_:delete when a data set's examples change, and evaluation:_:read and evaluation:_:invoke for sam eval run

A manifest that declares rbacRoles or systemUsers manages every custom role and system user on the deployment. A manifest that declares rbacRoles also manages every user's role assignments, including assignments made in the Agent Mesh UI. For each custom role, system user, and assignment that the manifest does not list, the plan shows delete (needs --prune). Do not apply the RBAC manifest with --prune unless it lists all of them.

To create the key, perform these steps:

  1. Unset SAM_PLATFORM_TOKEN and SAM_AUTH_TOKEN, because they override your login.

  2. Sign in with sam auth login and the deployment URL. Use the URL rather than a target name, so that the cached login matches the manifest's target.

  3. Apply the RBAC manifest.

  4. Create the key with sam api-key create. Standard output carries only the key, so send it directly to the CI system's secret store. Store it only when the command succeeds, so that a failed command cannot overwrite the secret with an empty value.

unset SAM_PLATFORM_TOKEN SAM_AUTH_TOKEN
sam auth login https://sam.example.com
sam config apply -m manifests/rbac.yaml
key=$(sam api-key create --url https://sam.example.com --user ci-deployer \
  --label "ci-deployer-$(date +%Y%m%d)" --scopes '*' --expires-in 90d) \
  && printf '%s' "$key" | gh secret set SAM_PLATFORM_TOKEN

The last line stores the key as a GitHub Actions secret. In another CI system, replace gh secret set SAM_PLATFORM_TOKEN with that system's command for setting a masked secret.

The value of --user is the system user's name, and the roles the system user holds must cover every scope in --scopes.

To rotate the key before it expires, perform these steps as a signed-in super administrator, with SAM_PLATFORM_TOKEN and SAM_AUTH_TOKEN unset:

  1. Create a key with the same sam api-key create command.

  2. Replace the CI secret with the new key.

  3. Find the old key's ID and delete it, because a system user can hold at most 10 active keys:

    sam api-key list --url https://sam.example.com --user ci-deployer
    sam api-key delete <old-key-id> --url https://sam.example.com --force
  4. Run the pipeline once, and confirm that sam api-key list shows a last-used time for the new key.

For more information, see Rotating a Key.

Install a Pinned CLI

Each job installs the Linux archive of the Agent Mesh CLI (solace-agent-mesh-<version>-cli-linux-amd64.tar.gz) from the CLI/ folder of the release on the Solace Product Portal. Mirror the archive to a location your runners can reach, and take its checksum from the file_checksum entry in the release's bill of materials, without the sha256: prefix. Do not build the Agent Mesh CLI from source in the pipeline. For more information about the release contents and verifying a download, see Obtain the Delivery Package. Save this script in the config repository as ci/install-sam.sh and make it executable:

#!/usr/bin/env bash
# Install the pinned sam CLI into .sam-cli/ after checking the archive's SHA-256 checksum.
set -euo pipefail
: "${SAM_CLI_VERSION:?set SAM_CLI_VERSION}" "${SAM_CLI_BASE_URL:?set SAM_CLI_BASE_URL}" "${SAM_CLI_SHA256:?set SAM_CLI_SHA256}"
archive="solace-agent-mesh-${SAM_CLI_VERSION}-cli-linux-amd64.tar.gz"
curl -fsSL -o "$archive" "${SAM_CLI_BASE_URL}/${archive}"
echo "${SAM_CLI_SHA256}  ${archive}" | sha256sum -c -
mkdir -p .sam-cli
tar -xzf "$archive" -C .sam-cli sam
rm "$archive"
.sam-cli/sam --version

Both plan and apply build the repository's toolsets, so the job requires each toolset's toolchain:

  • A Python toolset requires python3 and pip. The build targets the Python version that the Platform service reports. When the runner's Python version, operating system, or architecture differs from the ones that the Platform service reports, pip installs only prebuilt wheels, so any dependency published only as source fails to install.

  • A Go toolset requires Go.

  • A toolset with a build script requires whatever the script runs.

If your repository has no toolsets to build, any image with bash, curl, sha256sum, and tar is enough.

Variables and Secrets

The pipeline reads these variables and secrets:

Name Store as Value
SAM_URL Variable The Agent Mesh URL, for example https://sam.example.com.
SAM_CLI_VERSION Variable The Agent Mesh CLI release to install. Use the same version as the deployment.
SAM_CLI_BASE_URL Variable The URL of the directory that holds the mirrored archive.
SAM_CLI_SHA256 Variable The archive's SHA-256 checksum from the bill of materials, without the sha256: prefix.
SAM_EVAL_EXPERIMENTS Variable The experiments to run after an apply, separated by spaces.
SAM_EVAL_THRESHOLD Variable The minimum pass rate, from 0.0 to 1.0.
SAM_PLATFORM_TOKEN Secret The API key.

Also provide every ${VAR} that your resource files reference. Store secrets in the secret store and other values as variables. For example, the following model file references LLM_MODEL_NAME, LLM_API_BASE, and LLM_API_KEY:

# models/ci-general.yaml
kind: model
name: ci-general
spec:
  provider: openai
  modelName: ${LLM_MODEL_NAME}
  apiBase: ${LLM_API_BASE}
  authConfig:
    type: apikey
    api_key: ${LLM_API_KEY}

For more information about model files, see Configuring Models with the CLI. For more information about how substitution resolves these values, see Secrets and Variables.

Run the Jobs

Each job runs ci/install-sam.sh first, as described in Install a Pinned CLI. On a pull request, the plan job then runs the following command:

.sam-cli/sam config plan -m manifests/prod.yaml --url "$SAM_URL" --no-dotenv --no-interactive --no-color

On merge to the main branch, the apply job runs the following commands:

.sam-cli/sam config apply -m manifests/prod.yaml --url "$SAM_URL" --no-dotenv --no-interactive --no-color
.sam-cli/sam eval run $SAM_EVAL_EXPERIMENTS --url "$SAM_URL" --threshold "$SAM_EVAL_THRESHOLD"

Run the apply job under the CI system's concurrency control, so that only one apply runs at a time. Pass secrets only to the steps that run sam, so that other steps cannot read them. Many CI systems do not pass secrets to a pipeline for a pull request from a fork, so the plan job works only for branches in the repository.

The --no-dotenv flag prevents a .env file in the repository from supplying variable values. To load values from a file that the CI system writes to disk, add --env-file <path>. Values in that file override environment variables.

Do not set SAM_AUTH_TOKEN in the job, because sam eval run uses it in preference to SAM_PLATFORM_TOKEN.

The plan job runs unmerged code with the same key as the apply job, because plan requires the update scopes to compare stored secrets. Anyone who can push a branch to the repository can therefore use the key without a review. With the * scope, that includes granting themselves the super administrator role, which stays after the branch is deleted. Give push access only to people you would make super administrators, or narrow the key's scopes as described in Create the Pipeline's API Key.

Behavior to Plan For

A failed evaluation does not roll back the apply. The job fails, but the changes stay applied and deployed. To recover, revert the merge commit. The pipeline then applies the reverted files and runs the experiments again. The revert does not delete a resource that the merge created, because the pipeline runs without --prune. For more information, see Deleting Resources.

Set authConfig.type on every model. When a model's authConfig has no type, the Platform service stores it with no authentication type and does not treat its key as a secret.

Do not run sam config apply manually against a deployment that the pipeline manages. A manual apply skips review and the CI system's concurrency control, and the next pipeline run overwrites its changes.

Without -v, an update in the plan does not show which fields change. With -v, the plan shows the changed fields for models and agents only. For other kinds, read the diff of the pull request or merge request.

Most plans show changes even when no file has changed, because resources that Agent Mesh creates at installation, such as the built-in agents and the sample experiment, show delete (needs --prune). The pipeline therefore cannot use the plan to skip an apply when nothing has changed. To skip the apply when no configuration file changes, add a path filter to the apply job. List the directories that your manifest reads, the pipeline definition file, and ci/install-sam.sh. With a path filter, re-run a failed apply job yourself, because a later merge that changes no configuration file does not start an apply. A change to a variable or secret never starts a run, so re-run the last apply job to apply the new value.

A role file sets description at the top level, next to name. The sam config apply command rejects a description under spec. A role file with no description clears any description set in the Agent Mesh UI.

Related Topics

  • Exporting and Migrating covers sam config pull, which exports the running state and confirms that what you applied is what is running.

  • The Manifest covers which kinds a manifest declares, and therefore which kinds plan and apply consider.

  • Targets and Authentication covers the target and credentials these commands use.