SEMP API Architecture

This section explains the service-oriented, RESTful architecture of the SEMP API and provides some general SEMP operating information.

SEMP API Specification

Solace has used the OpenAPI Specification (specifically OpenAPI Specification v2.0) to define the SEMP API. This specification (formerly known as Swagger) provides a standard, language-neutral way of defining REST APIs, which gives developers a fully documented SEMP API specification so they can get up and running quickly. The API specification describes the various SEMP objects, how they are represented, and what URIs and HTTP method each object supports.

Swagger also provides a lot of tooling that make developers’ lives easier, including client library generation, which is explained below in Generating Client Libraries.

To obtain the SEMP API specification in a JSON format from an event broker, you can perform a GET on the following URI:

/SEMP/v2/config/spec

If you have an internet access, you can download the SEMP API specifications from the solace.com/downloads page.

You can also access the SEMP API Reference that covers SEMP's REST resources.

Generating Client Libraries

The SEMP API's use of the OpenAPI specification allows developers to easily generate client libraries in their favorite programming language. The Swagger tooling can export client libraries in over 25 different programming languages including:

Android, C#, Java, Javascript, Node, Python, Ruby, Scala, and Swift.

See Getting Started with SEMP for an example of how to generate a client library from a SEMP API specification and get going configuring your event broker.

Changes to the SEMP specification may affect the backwards compatibility of generated APIs. For details, see the SEMP API Reference

API Composition

SEMP is designed to support typical users and their configuration needs when interacting with event brokers. To help focus SEMP for its intended users, it is broken down into three separate APIs that each serve a different purpose.

API	Purpose	Base URI Path
Config	Configuration	`/SEMP/v2/config/`
Monitor	Monitoring	`/SEMP/v2/monitor/`
Action	Performing actions	`/SEMP/v2/action/`

A SEMP application could use more than one of these three APIs. However, it is suggested that common application use cases will map mainly to a single API.

The following are a few examples to help you get an idea of the three different APIs that make up SEMP.

Configuration relates to the read and write of administrative state on the event broker. This involves the creation and deletion of objects, and the reading and writing of their attributes. Examples of this are:

creating a queue
enabling a queue
checking if a queue is administratively-enabled

Monitoring relates to reading operational state on the event broker. This involves reading objects and reading the operational state of their attributes. Examples of this are:

checking if a client has bound to a queue
checking a queue’s stats
checking if the event broker is currently active for the message backbone

Action relates to manipulating non-configuration state on the event broker. Examples of this are:

deleting messages from a queue
clearing queue stats
reverting activity to an event broker's high-availability (HA) mate

SEMP Resources

SEMPv2 follows a conventional REST approach to resource modeling. Resources represent configuration objects of the event broker and are categorized as either objects or collections of objects.

Objects contain the following:

Attributes—scalar components of a resource
Child Resources—objects or collections.

Collections are homogeneous sets of objects

Collections are themselves addressable as a resource.

Resources are always nouns, with individual objects being singular and collections being plural. Objects within a collection are identified by an object ID (obj-id), which follows the collection name with the form collection-name/obj-id.

Example:

## Queue path pattern:
<base-path>/msgVpns/{msgVpnName}/queues/{queueName}

## Specific example:
/SEMP/v2/config/msgVpns/applicationA/queues/requestQ

These are simple cases where the object ID of the resource (here a Message VPN and a queue) is composed of a single part (commonly a name). Some objects in the event broker are uniquely identified by more than one string. In such a case the different strings are concatenated together with the "," character to form the object's ID.

Example:

## Bridge path pattern:
<base-path>/msgVpns/{msgVpnName}/bridges/{bridgeName},{bridgeVirtualRouter}

## Specific example:
/SEMP/v2/config/msgVpns/finance/bridges/ny-to-ldn,primary

In the example above, there is a collection of bridges within the finance Message VPN, one of which is called ny-to-ldn belonging to the primary virtual-router.

As a general rule, resource paths in the SEMP API are composable. That is, the client application, by following well understood patterns, can build the URI itself. In particular, SEMP does not make use of obfuscated or generated UUIDs, or some such scheme for generating unique URIs for each resource. The SEMP API Reference shows the resource URI for each event broker configuration resource.

Attribute Properties

Attributes of objects can have associated properties that dictate certain behaviors of SEMP when attributes are present in various requests. At a minimum, every object has one attribute. Attributes may have any (non-exclusively) of the following properties:

Property	Meaning
Identifying	The attribute is involved in unique identification of the object, and appears in its URI.
Required	The attribute must be provided in the request.
Read-Only	The attribute can only be read, not written. Read-only attributes may appear in POST and PUT/PATCH requests, but ignored as long as the attribute is not identifying.
Write-Only	The attribute can only be written, not read. This attribute will not be returned in GET requests.
Auto-Disable	Modifying this attribute while the object (or the relevant part of the object) is administratively enabled causes the event broker to temporarily disable one or more attributes to apply the change. This action may be service impacting.
Requires-Disable	The attribute cannot be changed while the object (or the relevant part of the object) is administratively enabled.
Deprecated	The attribute is deprecated and will be eliminated in the next SEMP version.

The SEMP API Reference identifies all attribute properties for each resource so that it is clear which attributes of an object have assigned properties.

SEMP Authentication and Security

SEMP supports two types of user authentication: Basic Authentication and OAuth authentication.

Basic Authentication

SEMP uses management username and password for user authentication. If authentication is successful, a session is created and the broker returns a cookie containing a session token. SEMP also supports both HTTP and HTTPS connections. For example:

curl -u "username:password" https://HOST:port/SEMP/v2/config/msgVpns

OAuth Authentication

To authenticate using OAuth, a SEMP client must include one or more OAuth tokens in the HTTP Authorization header as a bearer token. The bearer token in the Authorization header must be provided on every request.

In general, the iss claim in the ID token (for OpenID Connect) or access token (for OAuth 2.0), if present, is used by the event broker to identify which OAuth profile to use.

A specific OAuth profile can also be selected by adding ~base64(<issuer>)~ to the beginning of the bearer token. Base64 padding should not be used. For example, to use an OAuth profile called solace that has an issuer of https://www.solace.com with an access token:

Bearer ~aHR0cHM6Ly93d3cuc29sYWNlLmNvbQ~<access_token>

If a profile cannot be identified from the iss claim in the token, and no issuer prefix is provided in the Authorization header, the default profile is used.

Single token

In most case, a single OAuth token is used for authentication. The following format is used to pass a single token in the authentication header: Authorization: Bearer <token>.

Example:

Authorization: Bearer eyJhbGciOiJSUzI1NiIsImtpZCI6ImYwdDZrMUVyWVNXeDdCSmRzNHVkOU8wa2wzcEFxOHVTV3RTdnJHVTJaRmkiLCJ0eXAiOiJKV1QifQ.eyJhdWQiOiJTa2MxSmpsLyIsImVtYWlsIjoic29sbHlAZXhhbXBsZS5jb20iLCJleHAiOjE2MzY2NTA0MjMsImdyb3VwcyI6WyJPdHRlcnMiLCJNYW1tYWxzIl0sImlhdCI6MTYzNjY0NjgyMywiaXNzIjoiaHR0cHM6Ly9teS1vYXV0aC1wcm92aWRlci5leGFtcGxlLmNvbSIsInN1YiI6IlJpaEJDQmszIn0.oIo9gDbjx3qW3Rg-IiKw4B67KIcJ6_l7dTdlI5inwI9lH84moyG_GM0cN1yFnjVioVjUN7h5Ok5ubHW2BaSaSifC1JjpDzFB55bHSA8ygjZaEtCWXCFDUbs-8yN6BWr7jhKbJ6Iffg285Sm5C2H4rXkBC4Z50bu5k2-hK8OVkF5hi0amOl7-K0S6UtiFkqPaGtru5JHhOJ6m8b7tIfB3YYwIT1xGtkqr_4AfQO3tFziUJFXtZ7-dRMlqQhsS2nO_kA18Aa_5RNyAdRx-HGq3W_DMcxlNj1E3VURrLhLx6s-mxAZSTpwIXWv_KTUblUjAc7gp-sYZFwrXdbDhx-4B3A

Multiple tokens

When an OpenID Connect provider issues both an ID token and an access token together, the ID token may contain a reduced number of claims. In such cases, the event broker may require both the ID token and the corresponding access token—authenticating the user through the ID token, and using the access token to retrieve additional claims from the OpenID Connect Userinfo endpoint for the username or group information. The following format is used to pass multiple tokens in the authentication header: Authorization: Bearer <id-token>/<access-token>.

Example:

Authorization: Bearer eyJhbGciOiJSUzI1NiIsImtpZCI6ImYwdDZrMUVyWVNXeDdCSmRzNHVkOU8wa2wzcEFxOHVTV3RTdnJHVTJaRmkiLCJ0eXAiOiJKV1QifQ.eyJhdF9oYXNoIjoiMG5CUGZPUm5uX0l0Z2RTTUNsR2dHUSIsImF1ZCI6IktpQWVIMWtuIiwiZXhwIjoxNjM2NjUwNzYyLCJpYXQiOjE2MzY2NDcxNjIsImlzcyI6Imh0dHBzOi8vbXktb2F1dGgtcHJvdmlkZXIuZXhhbXBsZS5jb20iLCJzdWIiOiJPMjlWTmtvKyJ9.ZMQge6iyjmduO4IY0TaBB3ynqX1NeAmiMfYqZdsPn4bFSs9A0J7aAsc_iuJQcOQrZjEtv-3OXhrAsulTcmLYFCHUTlhWR5oA1eC0cbK1d3gYDeSFo6YUrxlGM9cRYQbr_0fG6VbRc5ivRyowAqmursnk5TVbje4H_Jpug2qnZVrYspZkixqlIkZN6nEbT2Uxzqypvo1o5y1H19uq1t5SF8KKsA9Y22Pgq88mpGg8U27Ov_CY0NKGh_U5KMRVrlQ1X84E0SmpJSWosM9L5Gy7nYGuGhvVGH91eGwxh24L0lpy6HcQkCS7wCiLseVntWcIvu5BUo4mMPCqvQDmg47I7A/zOvubUTO7ig+u2+2NWAibm2U5Tg/pKE4JcwTCwM+qNw=

SEMP Sessions

When a client makes its first SEMPv2 request, it must supply a username and password using HTTP Basic authentication, or an OAuth token or tokens using HTTP Bearer authentication.

When HTTP Basic authentication is used, the broker returns a cookie containing a session key. The client can omit the username and password from subsequent requests, because the broker now uses the session cookie for authentication instead. When the session expires or is deleted, the client must provide the username and password again, and the broker creates a new session.

There are a limited number of session slots available on the broker. The broker returns 529 No SEMP Session Available if it is not able to allocate a session.

If certain attributes—such as a user's password—are changed, the broker automatically deletes the affected sessions. However, changes in external user configuration data stored on a RADIUS or LDAP server do not trigger the broker to delete the associated session(s), therefore you must do this manually, if required.

The session is deleted if:

the client logs out
the session times out due to inactivity
the session reaches its maximum lifetime
the client's user credentials are altered on the broker. This includes password updates, username changes, and user deletion. It does not include access level changes. If the user credentials change, the event broker deletes all sessions for that user.
the broker's auth-type configuration changes (for example, from Radius to LDAP, or from one LDAP profile to another). If the auth-type configuration changes, the event broker deletes all sessions.
an admin user deletes the client's session(s) via SEMPv2 or the CLI
the broker restarts

When the session terminates for any reason, the associated token immediately becomes invalid and cannot be used again. The client must authenticate again to create a new session cookie.

A client can retrieve its current session information using the /about/user endpoint, delete its own session using the /about/user/logout endpoint. A client with appropriate permissions can also manage all sessions using the /sessions endpoint.

Sessions are not created when authenticating with an OAuth token or tokens using HTTP Bearer authentication. If a session cookie is provided, it is ignored.

For information about configuring sessions, see Managing SEMP Sessions.

Sessions are also used by Broker Manager. For details, see Broker Manager Sessions.

Role-Based Access Control

SEMP supports role-based access control. When executing commands against an event broker, the SEMP user will be authenticated as an event broker management user (equivalent to a CLI user) and receive the access level and scope assigned to that management user. The following is a brief summary of role-based access control on event brokers. For more details refer to Management & Shell Users, which provides full details of the rules and how they work together to provide access control.

There are two command scopes supported on the event broker:

Global
Message VPN

There are four user access levels in the event broker:

none
read-only
read-write
admin

Each attribute of a SEMP Resource is assigned a minimum access scope and user access level.

A management user is then assigned access levels for both the global level and for one or more message VPNs. This forms the management user's role-based access control. For example:

A system-wide event broker administrator would be assigned a global access level of admin, which allows that user to read and modify any attribute of any resource in the API, whether it is a global-scoped or Message VPN-scoped attribute.
A Message VPN administrator would be assigned a global access level of none and a message VPN access level of read-write for the Message VPN the administrator is responsible for.

There are many more examples of how you can effectively use the scope and access level for different types of users. For more examples see the Solace documentation referenced above.

In general, in the SEMP API all resources under /msgVpns/<msgVpnName>/... are part of the Message VPN scope. All other top-level resources form part of the global scope. The /msgVpns/<msgVpnName> is one resource where the scope of attributes is mixed. Some attributes require global scope and some require Message VPN-level scope. In all cases, the SEMP API Reference makes it clear which scope and access level is required for each SEMP resource.

The PUT method respects attribute scope levels, and updates every attribute in a resource. Attributes that are not specified in the body of a PUT request are reset to their default values. However, if a user does not have sufficient role based access scope to modify an attribute, then the attribute will not be returned to its default value. In effect, some more powerful user has chosen a different default and the attribute is ignored, but the PUT request will still be successful. This makes the SEMP API useable by SEMP users with global scope and message VPN scope with consistent behavior.

Rate Limiting

Currently, SEMP does not enforce a rate limit. It is recommended that users avoid aggressively polling the event broker. In general, the average number of SEMP requests made to an event broker should be less than 10 requests/sec.

Paging, Filtering, Sorting

Currently paging, filtering, and customizing returned data fields are supported. Sorting is not yet supported.

Paging

Paging is supported for the GET method when retrieving collections. A collection might contain hundreds, thousands, or even millions of items. Therefore, they must be paged to be manageable. Paging is controlled by two query parameters whose usage is explained below.

Parameter	Meaning
count=number	Include in response at most this many objects.
cursor=list-of-exprs	Current position in a paged GET request.

A SEMP application can limit the number of items returned from a GET of a collection by using the count query parameter.

The valid range that an application can ask for count is 1 to an undefined upper bound at least as great as the default value of 10. Each collection’s actual count upper bound MAY differ, and requests for count greater than this upper bound will be capped at the actual upper bound. Given, then, that there is no guaranteed way for an application to request all items in a collection in a single GET request, applications should always be written to handle pagination unless they know for certain that there are at most 10 items in the collection.

Although a page size of up to 100 is supported SEMP v2, using a maximum page size of more than 10 items may result in requests taking several seconds before they return any results. Therefore, it is currently recommended that you use a maximum page size of 10 items in SEMP v2.

Retrieving further pages is controlled by the cursor query parameter. This parameter provides the event broker with sufficient context to retrieve the next set of objects in the collection. A GET request which was not completely satisfied due to a count limit being reached includes, in the returned metadata, a paging field with two sub-fields:

The cursorUri is a URI equivalent to the just processed request, but with the cursor and count query parameters filled in to GET the next set of objects from the collection.
The cursorQuery string is just the value to provide to the cursor query parameter. This is useful for clients built using generated SDK libraries, which do not directly operate on URIs.

The absence of the paging field in a response indicates that either the full collection has been returned or that you have retrieved the last page of a collection.

In the current version of SEMP, it is possible to retrieve a collection and receive a partially-filled or even empty page mid response. This is particularly common when object filtering is used. Applications working with paging should continue to retrieve the next page until the paging field is absent in the response to confirm they have retrieved the full response. It is expected that this behavior will be improved in a subsequent version of SEMP to eliminate the possibility of retrieving empty pages in the middle of responses.

Returning Selected Data Fields

The query parameter select=<attr-list> is used to include in the response only selected attributes of the object, or exclude from the response selected attributes of the object. Use this query parameter to limit the size of the returned data for each returned object, return only those fields that are desired, or exclude fields that are not desired. The <attr-list> parameter is defined in Augmented BNF (ABNF) syntax as:

attr-list  = attr-name 0*254( "," attr-name ) ; comma-sep list of attribute names (0 to 255)
attr-name  = attr-str 0*63( "." attr-str )    ; attribute name, possibly nested (up to depth 64)
attr-str   = 1*128 ( nnnd-octet )             ; attribute string (length from 1 to 128)
nnnd-octet = %x01-2D / %x2E-FF                ; any non-NUL, non-DOT octet

If the list contains attribute names that are not prefaced by "-", only those attributes are included in the response. If the list contains attribute names that are prefaced by "-", those attributes are excluded from the response.

If the list contains both types:

all attributes that are prefaced by "-" must follow all attributes that are not prefaced by "-"
the difference of the first set of attributes and the second set of attributes is returned in the response

If the list is empty (select=), all attributes are returned.

Nested attribute names are supported by concatenating them together with the "." character.

The <attr-str> supports wildcard matching with the "*" character in the conventional manner (zero or more characters). The character "?", with typical meaning of matching a single character, is not supported.

Some additional rules:

The attributes in question are those in the returned data and links fields only, not the meta field.
Nesting makes the attribute unique. For example, select=-abc does not exclude attribute foo.abc.
Including or excluding an attribute with nested attributes includes or excludes all the nested attributes. For example, select=foo includes both foo.abc and foo.xyz.
Wildcard matching is done within an <attr-str>, not <attr-name>. For example, select=*bc does not include attribute foo.abc, whereas select=foo.*bc does.
The select parameter is applied after the where parameter.

Filtering

The query parameter where=<expr-list> is used to filter which objects will be returned from a collection, where <expr-list> is defined in ABNF as:

expr-list = attr-expr 1*63( "," attr-expr )       ; comma-sep list of attt-exprs (1 to 64)
attr-expr = attr-name expr-op string              ; comparison operation
expr-op   = "==" / "!=" / ">" / ">=" / "<" / "<=" ; comparison operator
string    = 1* 128 ( nn-octet )                   ; any string (length from 1 to 128)
nn-octet  = %x01-FF                               ; any non-NUL octet

When a GET request includes a where query parameter, only objects whose content matches all the given expressions will be returned. In the case that a given <string> needs to include any of the ",=!<>" characters, they MUST be escaped with the URI %xx notation to avoid any special meaning, consistent with Reserved Characters.

Some additional rules:

The attributes in question are those in the returned data field only, not the links or meta fields.
Wildcards are not allowed in <attr-name>.
The EQ and NE operators perform a string comparison, where <string> MAY include a "*" wildcard. In this case, the value of <attr-name> MUST be a string-based type.
The GT, GE, LT, LE operators perform a numeric comparison, with <string> interpreted as a number (specifically, interpreted as defined by the built-in Python int(<string>,0) function), and the value of <attr-name> MUST be a number-based type.
The where parameter is applied before the select parameter.

Collection Counts

Counts are supported for the GET method when retrieving the following collections:

/SEMP/v2/monitor/msgVpns/<name>/clients
- clients/*/rxFlows
- clients/*/transactedSessions
- clients/*/txFlows
/SEMP/v2/monitor/msgVpns/<name>/queues
- queues/<name>/msgs
- queues/<name>/txFlows
/SEMP/v2/monitor/msgVpns/<name>/topicEndpoints
- topicEndpoints/<name>/msgs
- topicEndpoints/<name>/txFlows
/SEMP/v2/monitor/msgVpns/<name>/transactions

When you issue a GET request for one of the supported collections, the total number of items is returned in the count field in the metadata of the GET response.

For example, to retrieve the total number of queues in the default Message VPN, you could issue the following request:

curl -X GET solace:solace 192.0.2.1:8080/SEMP/v2/monitor/msgVpns/default/queues

The response from the broker would then contain the number of queues in the count field. In this case a single queue exists in the Message VPN.

...    
    "meta":{
        "count":1,
        "request":{
            "method":"GET",
            "uri":"http://192.0.2.1:8080/SEMP/v2/monitor/msgVpns/default/queues"
        },
        "responseCode":200
    }

For the rxFlows, transactedSessions and txFlows collections, counts are supported only for all clients, with an asterisk used to represent all. When an asterisk is used, only the metadata section is returned in the response. In all other cases, the use of an asterisk is not allowed.

If the request includes any filtering, in other words if it includes a where query parameter, the count field is not included in the response.

Retrieving Message Counts from Queues or Replay Logs

In addition to counts of objects in a collection, counts of messages (which are child collections) are supported for the GET method when retrieving the following collections:

/SEMP/v2/monitor/msgVpns/<name>/queues
/SEMP/v2/monitor/msgVpns/<name>/topicEndpoints
/SEMP/v2/monitor/msgVpns/<name>/replaylogs

When you issue a GET request for a collection, the objects in the child collection are returned in the collections section of the GET response. Counts for supported child collections are included in the count field in the collections section.

For example, to retrieve the total number of messages in each queue in the default Message VPN, you could issue the following request:

curl -u "solace:solace" "192.0.2.1:8080/SEMP/v2/monitor/msgVpns/default/queues?count=100&select=queueName,msgs.count"

The response from the broker would then contain the number of messages in the count field in the collections section. In this case, there are two queues with one queue containing 50 messages and the other containing 75 messages. If there were more queues (up to 100) this request would also capture them, since it increases the page size to 100 (the maximum supported).

When polling objects (especially queues), it is important to use a select clause to limit the content of your request to only what you need. For example, if you only want the number of messages in each queue, you should use select=queueName,msgs.count. Doing so can dramatically reduce the amount of time the broker needs to service the request, and reduce the likelihood of interfering with the performance of other broker services.

{
    "collections":[
        {
            "msgs":{
                "count":50
            }
        },
        {
            "msgs":{
                "count":75
            }
        }
    ],
    "data":[
        {
            "queueName":"q1"
        },
        {
            "queueName":"q2"
        }
    ],
    "links":[
        {},
        {}
    ],
    "meta":{
        "count":2,
        "request":{
            "method":"GET",
            "uri":"http://192.168.132.24:8080/SEMP/v2/monitor/msgVpns/default/queues?count=100&select=queueName,msgs.count"
        },
        "responseCode":200
    }
}

Since the queue name is returned in the data section and the msgs count is returned in the collections section, you must use the index of the queue in the data section to determine the corresponding msgs count in the collections section. In the previous example, you can determine that q1 has 50 messages because q1 is the first entry in the data section and the 50 msgs count is the first entry in the collections section.

Transactions

SEMP does not support transactions, as it is not possible to group together multiple SEMP requests and guarantee their success or failure as a complete unit.

However, any individual SEMP POST, PUT, or PATCH request will complete successfully or not at all. For example, a PATCH to a queue, changing three attributes, where the change to the third attribute fails will leave the queue without the first two attributes changed; all three changes must succeed, otherwise the object will be left in its original state. However, this support is still not transactional in that an event broker restart midway through the handling of an individual request may result in the request only being partially satisfied. Concurrent modifications of Solace objects by CLI or legacy SEMP users is another way that a request may fail and leave a request partially satisfied. SEMP v2 users should avoid concurrent modification of objects by CLI or legacy SEMP users.

Any individual SEMP GET request which returns multiple objects, is guaranteed to complete successfully or not at all. For instance, when doing a GET of a collection, partial collection results will not be returned if there is a failure while processing the GET. Similarly, a GET of an individual object returns all the state of that object, or none.

Provide feedback