SEMP API Architecture
This section explains the service-oriented, RESTful architecture of the SEMP API and provides some general SEMP operating information.
SEMP API Specification
Solace has used the OpenAPI Specification (specifically OpenAPI Specification v2.0) to define the SEMP API. This specification (formerly known as Swagger) provides a standard, language-neutral way of defining REST APIs, which gives developers a fully documented SEMP API specification so they can get up and running quickly. The API specification describes the various SEMP objects, how they are represented, and what URIs and HTTP method each object supports.
Swagger also provides a lot of tooling that make developers’ lives easier, including client library generation, which is explained below in Generating Client Libraries.
To obtain the SEMP API specification in a JSON format from an event broker, you can perform a GET on the following URI:
/SEMP/v2/config/spec
If you have an internet access, you can download the SEMP API specifications from the solace.com/downloads page.
You can also access the SEMP API Reference that covers SEMP's REST resources.
Generating Client Libraries
The SEMP API's use of the OpenAPI specification allows developers to easily generate client libraries in their favorite programming language. The Swagger tooling can export client libraries in over 25 different programming languages including:
- Android, C#, Java, Javascript, Node, Python, Ruby, Scala, and Swift.
See Getting Started with SEMP for an example of how to generate a client library from a SEMP API specification and get going configuring your event broker.
Changes to the SEMP specification may affect the backwards compatibility of generated APIs. For details, see the SEMP API Reference
API Composition
SEMP is designed to support typical users and their configuration needs when interacting with event brokers. To help focus SEMP for its intended users, it is broken down into three separate APIs that each serve a different purpose.
API | Purpose | Base URI Path |
---|---|---|
Config |
Configuration |
/SEMP/v2/config/
|
Monitor |
Monitoring |
|
Action |
Performing actions |
|
A SEMP application could use more than one of these three APIs. However, it is suggested that common application use cases will map mainly to a single API.
The following are a few examples to help you get an idea of the three different APIs that make up SEMP.
- Configuration relates to the read and write of administrative state on the event broker. This involves the creation and deletion of objects, and the reading and writing of their attributes. Examples of this are:
- creating a queue
- enabling a queue
- checking if a queue is administratively-enabled
- Monitoring relates to reading operational state on the event broker. This involves reading objects and reading the operational state of their attributes. Examples of this are:
- checking if a client has bound to a queue
- checking a queue’s stats
- checking if the event broker is currently active for the message backbone
- Action relates to manipulating non-configuration state on the event broker. Examples of this are:
- deleting messages from a queue
- clearing queue stats
- reverting activity to an event broker's high-availability (HA) mate
SEMP Resources
SEMPv2 follows a conventional REST approach to resource modeling. Resources represent configuration objects of the event broker and are categorized as either objects or collections of objects.
- Objects contain the following:
- Attributes—scalar components of a resource
- Child Resources—objects or collections.
- Collections are homogeneous sets of objects
- Collections are themselves addressable as a resource.
Resources are always nouns, with individual objects being singular and collections being plural. Objects within a collection are identified by an object ID (obj-id
), which follows the collection name with the form collection-name/obj-id
.
Example:
## Queue path pattern: <base-path>/msgVpns/{msgVpnName}/queues/{queueName} ## Specific example: /SEMP/v2/config/msgVpns/applicationA/queues/requestQ
These are simple cases where the object ID of the resource (here a Message VPN and a queue) is composed of a single part (commonly a name). Some objects in the event broker are uniquely identified by more than one string. In such a case the different strings are concatenated together with the "," character to form the object's ID.
Example:
## Bridge path pattern: <base-path>/msgVpns/{msgVpnName}/bridges/{bridgeName},{bridgeVirtualRouter} ## Specific example: /SEMP/v2/config/msgVpns/finance/bridges/ny-to-ldn,primary
In the example above, there is a collection of bridges within the finance
Message VPN, one of which is called ny-to-ldn
belonging to the primary
virtual-router.
As a general rule, resource paths in the SEMP API are composable. That is, the client application, by following well understood patterns, can build the URI itself. In particular, SEMP does not make use of obfuscated or generated UUIDs, or some such scheme for generating unique URIs for each resource. The SEMP API Reference shows the resource URI for each event broker configuration resource.
Attribute Properties
Attributes of objects can have associated properties that dictate certain behaviors of SEMP when attributes are present in various requests. At a minimum, every object has one attribute. Attributes may have any (non-exclusively) of the following properties:
Property | Meaning |
---|---|
Identifying |
The attribute is involved in unique identification of the object, and appears in its URI. |
Required |
The attribute must be provided in the request. |
Read-Only |
The attribute can only be read, not written. Read-only attributes may appear in POST and PUT/PATCH requests, but ignored as long as the attribute is not identifying. |
Write-Only |
The attribute can only be written, not read. This attribute will not be returned in GET requests. |
Auto-Disable |
Modifying this attribute while the object (or the relevant part of the object) is administratively enabled causes the event broker to temporarily disable one or more attributes to apply the change. This action may be service impacting. |
Requires-Disable |
The attribute cannot be changed while the object (or the relevant part of the object) is administratively enabled. |
Deprecated |
The attribute is deprecated and will be eliminated in the next SEMP version. |
The SEMP API Reference identifies all attribute properties for each resource so that it is clear which attributes of an object have assigned properties.
SEMP Authentication and Security
SEMP supports two types of user authentication: Basic Authentication and OAuth authentication.
Basic Authentication
SEMP uses management username and password for user authentication. If authentication is successful, a session is created and the broker returns a cookie containing a session token. SEMP also supports both HTTP and HTTPS connections. For example:
curl -u "username:password" https://HOST:port/SEMP/v2/config/msgVpns
OAuth Authentication
To authenticate using OAuth, a SEMP client must include one or more OAuth tokens in the HTTP Authorization header as a bearer token. The bearer token in the Authorization header must be provided on every request.
In general, the iss
claim in the ID token (for OpenID Connect) or access token (for OAuth 2.0), if present, is used by the event broker to identify which OAuth profile to use.
A specific OAuth profile can also be selected by adding ~base64(<issuer>)~ to the beginning of the bearer token. Base64 padding should not be used. For example, to use an OAuth profile called solace
that has an issuer of https://www.solace.com
with an access token:
Bearer ~aHR0cHM6Ly93d3cuc29sYWNlLmNvbQ~<access_token>
If a profile cannot be identified from the iss
claim in the token, and no issuer prefix is provided in the Authorization header, the default profile is used.
Single token
In most case, a single OAuth token is used for authentication. The following format is used to pass a single token in the authentication header: Authorization: Bearer <token>
.
Example:
Authorization: Bearer eyJhbGciOiJSUzI1NiIsImtpZCI6ImYwdDZrMUVyWVNXeDdCSmRzNHVkOU8wa2wzcEFxOHVTV3RTdnJHVTJaRmkiLCJ0eXAiOiJKV1QifQ.eyJhdWQiOiJTa2MxSmpsLyIsImVtYWlsIjoic29sbHlAZXhhbXBsZS5jb20iLCJleHAiOjE2MzY2NTA0MjMsImdyb3VwcyI6WyJPdHRlcnMiLCJNYW1tYWxzIl0sImlhdCI6MTYzNjY0NjgyMywiaXNzIjoiaHR0cHM6Ly9teS1vYXV0aC1wcm92aWRlci5leGFtcGxlLmNvbSIsInN1YiI6IlJpaEJDQmszIn0.oIo9gDbjx3qW3Rg-IiKw4B67KIcJ6_l7dTdlI5inwI9lH84moyG_GM0cN1yFnjVioVjUN7h5Ok5ubHW2BaSaSifC1JjpDzFB55bHSA8ygjZaEtCWXCFDUbs-8yN6BWr7jhKbJ6Iffg285Sm5C2H4rXkBC4Z50bu5k2-hK8OVkF5hi0amOl7-K0S6UtiFkqPaGtru5JHhOJ6m8b7tIfB3YYwIT1xGtkqr_4AfQO3tFziUJFXtZ7-dRMlqQhsS2nO_kA18Aa_5RNyAdRx-HGq3W_DMcxlNj1E3VURrLhLx6s-mxAZSTpwIXWv_KTUblUjAc7gp-sYZFwrXdbDhx-4B3A
Multiple tokens
When an OpenID Connect provider issues both an ID token and an access token together, the ID token may contain a reduced number of claims. In such cases, the event broker may require both the ID token and the corresponding access token—authenticating the user through the ID token, and using the access token to retrieve additional claims from the OpenID Connect Userinfo endpoint for the username or group information. The following format is used to pass multiple tokens in the authentication header: Authorization: Bearer <id-token>/<access-token>
.
Example:
Authorization: Bearer eyJhbGciOiJSUzI1NiIsImtpZCI6ImYwdDZrMUVyWVNXeDdCSmRzNHVkOU8wa2wzcEFxOHVTV3RTdnJHVTJaRmkiLCJ0eXAiOiJKV1QifQ.eyJhdF9oYXNoIjoiMG5CUGZPUm5uX0l0Z2RTTUNsR2dHUSIsImF1ZCI6IktpQWVIMWtuIiwiZXhwIjoxNjM2NjUwNzYyLCJpYXQiOjE2MzY2NDcxNjIsImlzcyI6Imh0dHBzOi8vbXktb2F1dGgtcHJvdmlkZXIuZXhhbXBsZS5jb20iLCJzdWIiOiJPMjlWTmtvKyJ9.ZMQge6iyjmduO4IY0TaBB3ynqX1NeAmiMfYqZdsPn4bFSs9A0J7aAsc_iuJQcOQrZjEtv-3OXhrAsulTcmLYFCHUTlhWR5oA1eC0cbK1d3gYDeSFo6YUrxlGM9cRYQbr_0fG6VbRc5ivRyowAqmursnk5TVbje4H_Jpug2qnZVrYspZkixqlIkZN6nEbT2Uxzqypvo1o5y1H19uq1t5SF8KKsA9Y22Pgq88mpGg8U27Ov_CY0NKGh_U5KMRVrlQ1X84E0SmpJSWosM9L5Gy7nYGuGhvVGH91eGwxh24L0lpy6HcQkCS7wCiLseVntWcIvu5BUo4mMPCqvQDmg47I7A/zOvubUTO7ig+u2+2NWAibm2U5Tg/pKE4JcwTCwM+qNw=
SEMP Sessions
When a client makes its first SEMPv2 request, it must supply a username and password using HTTP Basic authentication, or an OAuth token or tokens using HTTP Bearer authentication.
When HTTP Basic authentication is used, the broker returns a cookie containing a session key. The client can omit the username and password from subsequent requests, because the broker now uses the session cookie for authentication instead. When the session expires or is deleted, the client must provide the username and password again, and the broker creates a new session.
There are a limited number of session slots available on the broker. The broker returns 529 No SEMP Session Available
if it is not able to allocate a session.
If certain attributes—such as a user's password—are changed, the broker automatically deletes the affected sessions. However, changes in external user configuration data stored on a RADIUS or LDAP server do not trigger the broker to delete the associated session(s), therefore you must do this manually, if required.
The session is deleted if:
- the client logs out
- the session times out due to inactivity
- the session reaches its maximum lifetime
- the client's user credentials are altered on the broker. This includes password updates, username changes, and user deletion. It does not include access level changes. If the user credentials change, the event broker deletes all sessions for that user.
- the broker's
auth-type
configuration changes (for example, from Radius to LDAP, or from one LDAP profile to another). If theauth-type
configuration changes, the event broker deletes all sessions. - an admin user deletes the client's session(s) via SEMPv2 or the CLI
- the broker restarts
When the session terminates for any reason, the associated token immediately becomes invalid and cannot be used again. The client must authenticate again to create a new session cookie.
A client can retrieve its current session information using the /about/user
endpoint, delete its own session using the /about/user/logout
endpoint. A client with appropriate permissions can also manage all sessions using the /sessions
endpoint.
Sessions are not created when authenticating with an OAuth token or tokens using HTTP Bearer authentication. If a session cookie is provided, it is ignored.
For information about configuring sessions, see Managing SEMP Sessions.
Sessions are also used by Broker Manager. For details, see Broker Manager Sessions.
Role-Based Access Control
SEMP supports role-based access control. When executing commands against an event broker, the SEMP user will be authenticated as an event broker management user (equivalent to a CLI user) and receive the access level and scope assigned to that management user. The following is a brief summary of role-based access control on event brokers. For more details refer to Management & Shell Users, which provides full details of the rules and how they work together to provide access control.
There are two command scopes supported on the event broker:
- Global
- Message VPN
There are four user access levels in the event broker:
- none
- read-only
- read-write
- admin
Each attribute of a SEMP Resource is assigned a minimum access scope and user access level.
A management user is then assigned access levels for both the global level and for one or more message VPNs. This forms the management user's role-based access control. For example:
- A system-wide event broker administrator would be assigned a global access level of admin, which allows that user to read and modify any attribute of any resource in the API, whether it is a global-scoped or Message VPN-scoped attribute.
- A Message VPN administrator would be assigned a global access level of none and a message VPN access level of read-write for the Message VPN the administrator is responsible for.
There are many more examples of how you can effectively use the scope and access level for different types of users. For more examples see the Solace documentation referenced above.
In general, in the SEMP API all resources under /msgVpns/<msgVpnName>/...
are part of the Message VPN scope. All other top-level resources form part of the global scope. The /msgVpns/<msgVpnName>
is one resource where the scope of attributes is mixed. Some attributes require global scope and some require Message VPN-level scope. In all cases, the SEMP API Reference makes it clear which scope and access level is required for each SEMP resource.
The PUT method respects attribute scope levels, and updates every attribute in a resource. Attributes that are not specified in the body of a PUT request are reset to their default values. However, if a user does not have sufficient role based access scope to modify an attribute, then the attribute will not be returned to its default value. In effect, some more powerful user has chosen a different default and the attribute is ignored, but the PUT request will still be successful. This makes the SEMP API useable by SEMP users with global scope and message VPN scope with consistent behavior.
Rate Limiting
Currently, SEMP does not enforce a rate limit. It is recommended that users avoid aggressively polling the event broker. In general, the average number of SEMP requests made to an event broker should be less than 10 requests/sec.
Paging, Filtering, Sorting
Currently paging, filtering, and customizing returned data fields are supported. Sorting is not yet supported.
Paging
Paging is supported for the GET method when retrieving collections. A collection might contain hundreds, thousands, or even millions of items. Therefore, they must be paged to be manageable. Paging is controlled by two query parameters whose usage is explained below.
Parameter | Meaning |
---|---|
count=number |
Include in response at most this many objects. |
cursor=list-of-exprs |
Current position in a paged GET request. |
A SEMP application can limit the number of items returned from a GET of a collection by using the count
query parameter.
The valid range that an application can ask for count
is 1 to an undefined upper bound at least as great as the default value of 10. Each collection’s actual count
upper bound MAY differ, and requests for count
greater than this upper bound will be capped at the actual upper bound. Given, then, that there is no guaranteed way for an application to request all items in a collection in a single GET request, applications should always be written to handle pagination unless they know for certain that there are at most 10 items in the collection.
Although a page size of up to 100 is supported SEMP v2, using a maximum page size of more than 10 items may result in requests taking several seconds before they return any results. Therefore, it is currently recommended that you use a maximum page size of 10 items in SEMP v2.
Retrieving further pages is controlled by the cursor
query parameter. This parameter provides the event broker with sufficient context to retrieve the next set of objects in the collection. A GET request which was not completely satisfied due to a count
limit being reached includes, in the returned metadata, a paging
field with two sub-fields:
- The
cursorUri
is a URI equivalent to the just processed request, but with thecursor
andcount
query parameters filled in to GET the next set of objects from the collection. - The
cursorQuery
string is just the value to provide to thecursor
query parameter. This is useful for clients built using generated SDK libraries, which do not directly operate on URIs.
The absence of the paging
field in a response indicates that either the full collection has been returned or that you have retrieved the last page of a collection.
In the current version of SEMP, it is possible to retrieve a collection and receive a partially-filled or even empty page mid response. This is particularly common when object filtering is used. Applications working with paging should continue to retrieve the next page until the paging
field is absent in the response to confirm they have retrieved the full response. It is expected that this behavior will be improved in a subsequent version of SEMP to eliminate the possibility of retrieving empty pages in the middle of responses.
Returning Selected Data Fields
The query parameter select=<attr-list>
is used to include in the response only selected attributes of the object, or exclude from the response selected attributes of the object. Use this query parameter to limit the size of the returned data for each returned object, return only those fields that are desired, or exclude fields that are not desired. The <attr-list>
parameter is defined in Augmented BNF (ABNF) syntax as:
attr-list = attr-name 0*254( "," attr-name ) ; comma-sep list of attribute names (0 to 255) attr-name = attr-str 0*63( "." attr-str ) ; attribute name, possibly nested (up to depth 64) attr-str = 1*128 ( nnnd-octet ) ; attribute string (length from 1 to 128) nnnd-octet = %x01-2D / %x2E-FF ; any non-NUL, non-DOT octet
If the list contains attribute names that are not prefaced by "-", only those attributes are included in the response. If the list contains attribute names that are prefaced by "-", those attributes are excluded from the response.
If the list contains both types:
- all attributes that are prefaced by "-" must follow all attributes that are not prefaced by "-"
- the difference of the first set of attributes and the second set of attributes is returned in the response
If the list is empty (select=
), all attributes are returned.
Nested attribute names are supported by concatenating them together with the "." character.
The <attr-str>
supports wildcard matching with the "*" character in the conventional manner (zero or more characters). The character "?", with typical meaning of matching a single character, is not supported.
Some additional rules:
- The attributes in question are those in the returned
data
andlinks
fields only, not themeta
field. - Nesting makes the attribute unique. For example,
select=-abc
does not exclude attributefoo.abc
. - Including or excluding an attribute with nested attributes includes or excludes all the nested attributes. For example,
select=foo
includes bothfoo.abc
andfoo.xyz
. - Wildcard matching is done within an
<attr-str>
, not<attr-name>
. For example,select=*bc
does not include attributefoo.abc
, whereasselect=foo.*bc
does. - The
select
parameter is applied after thewhere
parameter.
Filtering
The query parameter where=<expr-list>
is used to filter which objects will be returned from a collection, where <expr-list>
is defined in ABNF as:
expr-list = attr-expr 1*63( "," attr-expr ) ; comma-sep list of attt-exprs (1 to 64) attr-expr = attr-name expr-op string ; comparison operation expr-op = "==" / "!=" / ">" / ">=" / "<" / "<=" ; comparison operator string = 1* 128 ( nn-octet ) ; any string (length from 1 to 128) nn-octet = %x01-FF ; any non-NUL octet
When a GET request includes a where
query parameter, only objects whose content matches all the given expressions will be returned. In the case that a given <string>
needs to include any of the ",=!<>" characters, they MUST be escaped with the URI %xx notation to avoid any special meaning, consistent with Reserved Characters.
Some additional rules:
- The attributes in question are those in the returned
data
field only, not thelinks
ormeta
fields. - Wildcards are not allowed in
<attr-name>
. - The EQ and NE operators perform a string comparison, where
<string>
MAY include a "*" wildcard. In this case, the value of<attr-name>
MUST be a string-based type. - The GT, GE, LT, LE operators perform a numeric comparison, with
<string>
interpreted as a number (specifically, interpreted as defined by the built-in Pythonint(<string>,0)
function), and the value of<attr-name>
MUST be a number-based type. - The
where
parameter is applied before theselect
parameter.
Collection Counts
Counts are supported for the GET method when retrieving the following collections:
-
/SEMP/v2/monitor/msgVpns/<name>/clients
-
clients/*/rxFlows
-
clients/*/transactedSessions
-
clients/*/txFlows
-
-
/SEMP/v2/monitor/msgVpns/<name>/queues
queues/<name>/msgs
-
queues/<name>/txFlows
-
/SEMP/v2/monitor/msgVpns/<name>/topicEndpoints
-
topicEndpoints/<name>/msgs
-
topicEndpoints/<name>/txFlows
-
-
/SEMP/v2/monitor/msgVpns/<name>/transactions
When you issue a GET request for one of the supported collections, the total number of items is returned in the count
field in the metadata of the GET response.
For example, to retrieve the total number of queues in the default Message VPN, you could issue the following request:
curl -X GET solace:solace 192.0.2.1:8080/SEMP/v2/monitor/msgVpns/default/queues
The response from the broker would then contain the number of queues in the count
field. In this case a single queue exists in the Message VPN.
... "meta":{ "count":1, "request":{ "method":"GET", "uri":"http://192.0.2.1:8080/SEMP/v2/monitor/msgVpns/default/queues" }, "responseCode":200 }
For the rxFlows, transactedSessions and txFlows collections, counts are supported only for all clients, with an asterisk used to represent all. When an asterisk is used, only the metadata section is returned in the response. In all other cases, the use of an asterisk is not allowed.
If the request includes any filtering, in other words if it includes a where
query parameter, the count
field is not included in the response.
Retrieving Message Counts from Queues or Replay Logs
In addition to counts of objects in a collection, counts of messages (which are child collections) are supported for the GET method when retrieving the following collections:
-
/SEMP/v2/monitor/msgVpns/<name>/queues
-
/SEMP/v2/monitor/msgVpns/<name>/topicEndpoints
-
/SEMP/v2/monitor/msgVpns/<name>/replaylogs
When you issue a GET request for a collection, the objects in the child collection are returned in the collections
section of the GET response. Counts for supported child collections are included in the count
field in the collections
section.
For example, to retrieve the total number of messages in each queue in the default Message VPN, you could issue the following request:
curl -u "solace:solace" "192.0.2.1:8080/SEMP/v2/monitor/msgVpns/default/queues?count=100&select=queueName,msgs.count"
The response from the broker would then contain the number of messages in the count
field in the collections
section. In this case, there are two queues with one queue containing 50 messages and the other containing 75 messages. If there were more queues (up to 100) this request would also capture them, since it increases the page size to 100 (the maximum supported).
When polling objects (especially queues), it is important to use a select clause to limit the content of your request to only what you need. For example, if you only want the number of messages in each queue, you should use select=queueName,msgs.count
. Doing so can dramatically reduce the amount of time the broker needs to service the request, and reduce the likelihood of interfering with the performance of other broker services.
{ "collections":[ { "msgs":{ "count":50 } }, { "msgs":{ "count":75 } } ], "data":[ { "queueName":"q1" }, { "queueName":"q2" } ], "links":[ {}, {} ], "meta":{ "count":2, "request":{ "method":"GET", "uri":"http://192.168.132.24:8080/SEMP/v2/monitor/msgVpns/default/queues?count=100&select=queueName,msgs.count" }, "responseCode":200 } }
Since the queue name is returned in the data
section and the msgs
count is returned in the collections
section, you must use the index of the queue in the data
section to determine the corresponding msgs
count in the collections
section. In the previous example, you can determine that q1
has 50 messages because q1
is the first entry in the data
section and the 50 msgs
count is the first entry in the collections
section.
Transactions
SEMP does not support transactions, as it is not possible to group together multiple SEMP requests and guarantee their success or failure as a complete unit.
However, any individual SEMP POST, PUT, or PATCH request will complete successfully or not at all. For example, a PATCH to a queue, changing three attributes, where the change to the third attribute fails will leave the queue without the first two attributes changed; all three changes must succeed, otherwise the object will be left in its original state. However, this support is still not transactional in that an event broker restart midway through the handling of an individual request may result in the request only being partially satisfied. Concurrent modifications of Solace objects by CLI or legacy SEMP users is another way that a request may fail and leave a request partially satisfied. SEMP v2 users should avoid concurrent modification of objects by CLI or legacy SEMP users.
Any individual SEMP GET request which returns multiple objects, is guaranteed to complete successfully or not at all. For instance, when doing a GET of a collection, partial collection results will not be returned if there is a failure while processing the GET. Similarly, a GET of an individual object returns all the state of that object, or none.