SEMP API Architecture

This section explains the service-oriented, RESTful architecture of the SEMP API and provides some general SEMP operating information.

SEMP API Specification

Solace has used the OpenAPI Specification (specifically OpenAPI Specification v2.0) to define the SEMP API. This specification (formerly known as Swagger) provides a standard, language-neutral way of defining REST APIs, which gives developers a fully documented SEMP API specification so they can get up and running quickly. The API specification describes the various SEMP objects, how they are represented, and what URIs and HTTP method each object supports.

Swagger also provides a lot of tooling that make developers’ lives easier, including client library generation, which is explained below in Client Library Options.

To obtain the SEMP API specification in a JSON format from an event broker, you can perform a GET on the following URI:

/SEMP/v2/config/spec

If you have an internet access, you can download the SEMP API specifications from the solace.com/downloads page.

You can also access the SEMP API Reference that covers SEMP's REST resources.

Client Library Options

The SEMP API's use of the OpenAPI specification allows developers to easily generate client libraries in their favorite programming language. The Swagger tooling can export client libraries in over 25 different programming languages including:

  • Android, C#, Java, Javascript, Node, Python, Ruby, Scala, and Swift.

See Getting Started with SEMP for an example of how to generate a client library from a SEMP API specification and get going configuring your event broker.

Changes to the SEMP specification may affect the backwards compatibility of generated APIs. For details, see the SEMP API Reference

API Composition

SEMP is designed to support typical users and their configuration needs when interacting with event brokers. To help focus SEMP for its intended users, it is broken down into three separate APIs that each serve a different purpose.

API Purpose Base URI Path
Config Configuration /SEMP/v2/config/

Monitor

Monitoring

/SEMP/v2/monitor/

Action Performing actions

/SEMP/v2/action/

A SEMP application could use more than one of these three APIs. However, it is suggested that common application use cases will map mainly to a single API.

The following are a few examples to help you get an idea of the three different APIs that make up SEMP.

  • Configuration relates to the read and write of administrative state on the event broker. This involves the creation and deletion of objects, and the reading and writing of their attributes. Examples of this are:
    • creating a queue
    • enabling a queue
    • checking if a queue is administratively-enabled
  • Monitoring relates to reading operational state on the event broker. This involves reading objects and reading the operational state of their attributes. Examples of this are:
    • checking if a client has bound to a queue
    • checking a queue’s stats
    • checking if the event broker is currently active for the message backbone
  • Action relates to manipulating non-configuration state on the event broker. Examples of this are:
    • deleting messages from a queue
    • clearing queue stats
    • reverting activity to an event broker's high-availability (HA) mate

Resources

SEMPv2 follows a conventional REST approach to resource modeling. Resources represent configuration objects of the event broker and are categorized as either objects or collections of objects.

  • Objects contain the following:
    • Attributes—scalar components of a resource
    • Child Resources—objects or collections.
  • Collections are homogeneous sets of objects
    • Collections are themselves addressable as a resource.

Resources are always nouns, with individual objects being singular and collections being plural. Objects within a collection are identified by an obj-id, which follows the collection name with the form collection-name/obj-id.

Example:

## Queue path pattern:
<base-path>/msgVpns/{msgVpnName}/queues/{queueName}
	 
## Specific example:
/SEMP/v2/config/msgVpns/applicationA/queues/requestQ

These are simple cases where the obj-id of the resource (here a Message VPN and a queue) is composed of a single part (commonly a name). Some objects in the event broker are uniquely identified by more than one string. In such a case the different strings are concatenated together with the “,” character to form the object’s obj-id.

Example:

## Bridge path pattern:
<base-path>/msgVpns/{msgVpnName}/bridges/{bridgeName},{bridgeVirtualRouter}
	 
## Specific example:
/SEMP/v2/config/msgVpns/finance/bridges/ny-to-ldn,primary

In the example above, there is a collection of bridges within the finance Message VPN, one of which is called ny-to-ldn belonging to the primary virtual-router.

As a general rule, resource paths in the SEMP API are composable. That is, the client application, by following well understood patterns, can build the URI itself. In particular, SEMP does not make use of obfuscated or generated UUIDs, or some such scheme for generating unique URIs for each resource. The SEMP API Reference shows the resource URI for each event broker configuration resource.

Attribute Properties

Attributes of objects can have associated properties that dictate certain behaviors of SEMP when attributes are present in various requests. At a minimum, every object has one or more identifying attributes. Attributes may have any (non-exclusively) of the following properties:

Property

Meaning

Identifying

Attribute is involved in unique identification of the object, and appears in its URI.

Required

Attribute must be provided in the request.

Read-Only

Attribute can only be read, not written. Read-only attributes may appear in POST and PUT/PATCH requests, but ignored as long as the attribute is not identifying.

Write-Only

Attribute can only be written, not read. This attribute will not be returned in GET requests.

Requires-Disable

To change the attribute, an ancestor (typically its parent) needs to be disabled. See below Requires-Disable for more details.

Deprecated

Attribute is deprecated and will be eliminated in the next SEMP version.

The SEMP API Reference identifies all attribute properties for each resource so that it is clear which attributes of an object have assigned properties.

Requires-Disable

Requires-disable attributes are attributes that require their parent (or possibly some older ancestor) to be disabled in order to change its value. This is a common pattern in Solace PubSub+ software object management, and it is generally used when an attribute change cannot be applied to the running system immediately (for example, it is involved in connection establishment). Rather than allowing the change, and potentially causing some future service interruption when conditions change (for example, a connection bounce), the object (or its ancestor) must first be disabled to make obvious to the operator that the change may be service affecting. Temporally, the attribute change and the service interruption are aligned.

This attribute is provided solely for the purpose of documenting a restriction that the event broker will enforce. It is up to the SEMP application to inspect and enable/disable an object as needed to change a requires-disable attribute.

Authentication and Security

SEMP uses management user name and password for user authentication. SEMP also supports both HTTP and HTTPS connections. For example:

curl -u "username:password" https://HOST:port/SEMP/v2/config/msgVpns

Role-Based Access Control

SEMP supports role-based access control. When executing commands against an event broker, the SEMP user will be authenticated as an event broker management user (equivalent to a CLI user) and receive the access level and scope assigned to that management user. The following is a brief summary of role-based access control on event brokers. For more details refer to Management & Shell Users, which provides full details of the rules and how they work together to provide access control.

There are two command scopes supported on the event broker:

  • Global
  • Message VPN

There are four user access levels in the event broker:

  • none
  • read-only
  • read-write
  • admin

Each attribute of a SEMP Resource is assigned a minimum access scope and user access level.

A management user is then assigned access levels for both the global level and for one or more message VPNs. This forms the management user's role-based access control. For example:

  • A system-wide event broker administrator would be assigned a global access level of admin, which allows that user to read and modify any attribute of any resource in the API, whether it is a global-scoped or Message VPN-scoped attribute.
  • A Message VPN administrator would be assigned a global access level of none and a message VPN access level of read-write for the Message VPN the administrator is responsible for.

There are many more examples of how you can effectively use the scope and access level for different types of users. For more examples see the Solace documentation referenced above.

In general, in the SEMP API all resources under /msgVpns/<msgVpnName>/... are part of the Message VPN scope. All other top-level resources form part of the global scope. The /msgVpns/<msgVpnName> is one resource where the scope of attributes is mixed. Some attributes require global scope and some require Message VPN-level scope. In all cases, the SEMP API Reference makes it clear which scope and access level is required for each SEMP resource.

:  The PUT method respects attribute scope levels, and updates every attribute in a resource. Attributes that are not specified in the body of a PUT request are reset to their default values. However, if a user does not have sufficient role based access scope to modify an attribute, then the attribute will not be returned to its default value. In effect, some more “powerful” user has chosen a different default and the attribute is ignored, but the PUT request will still be successful. This makes the SEMP API useable by SEMP users with global scope and message VPN scope with consistent behavior.

Rate Limiting

Currently, SEMP does not enforce a rate limit. It is recommended that users avoid aggressively polling the event broker. In general, the average number of SEMP requests made to an event broker should be less than 10 requests/sec.

Paging, Filtering, Sorting

Currently paging, filtering, and customizing returned data fields are supported. Sorting is not yet supported.

Paging

Paging is supported for the GET method when retrieving collections. A collection might contain hundreds, thousands, or even millions of items. Therefore, they must be paged to be manageable. Paging is controlled by two query parameters whose usage is explained below.

Parameter Meaning
count=number Include in response at most this many objects.
cursor=list-of-exprs Current position in a paged GET request.

A SEMP application can limit the number of items returned from a GET of a collection by using the “count” query parameter.

The valid range that an application can ask for count is 1 to an undefined upper bound at least as great as the default value of 10. Each collection’s actual count upper bound MAY differ, and requests for count greater than this upper bound will be capped at the actual upper bound. Given, then, that there is no guaranteed way for an application to request all items in a collection in a single GET request, applications should always be written to handle pagination unless they know for certain that there are at most 10 items in the collection.

Although a page size of up to 100 is supported SEMP v2, using a maximum page size of more than 10 items may result in requests taking several seconds before they return any results. Therefore, it is currently recommended that you use a maximum page size of 10 items in SEMP v2.

Retrieving further pages is controlled by the cursor query parameter. This parameter provides the event broker with sufficient context to retrieve the next set of objects in the collection. A GET request which was not completely satisfied due to a count limit being reached includes, in the returned metadata, a paging field with two sub-fields:

  • The cursorUri is a URI equivalent to the just processed request, but with the cursor and count query parameters filled in to GET the next set of objects from the collection.
  • The cursorQuery string is just the value to provide to the cursor query parameter. This is useful for clients built using generated SDK libraries, which do not directly operate on URIs.

The absence of the paging field in a response indicates that either the full collection has been returned or that you have retrieved the last page of a collection.

:  In the current version of SEMP, it is possible to retrieve a collection and receive an empty page mid collection. This is particularly common when object filtering is used. Applications working with paging should continue to retrieve the next page until the paging field is absent in the response to confirm they have retrieved the full collection. It is expected that this behavior will be improved in a subsequent version of SEMP to eliminate the possibility of retrieving empty pages in the middle of collections.

Returning Selected Data Fields

The query parameters select=<attr-list> and xselect=<attr-list> are used to change the data attributes returned in the JSON content, where <attr-list> is defined in Augmented BNF (ABNF) syntax as:

attr-list  = attr-name 0*254( "," attr-name ) ; comma-sep list of attribute names (0 to 255)
attr-name  = attr-str 0*63( "." attr-str )    ; attribute name, possibly nested (up to depth 64)
attr-str   = 1*128 ( nnnd-octet )             ; attribute string (length from 1 to 128)
nnnd-octet = %x01-2D / %x2E-FF                ; any non-NUL, non-DOT octet

When a GET, POST, PUT, or PATCH request includes a select query parameter, data attributes which match the given comma-separated list of attribute names will be returned in the JSON message body (rather than what would have otherwise been returned). Nested attribute names are supported by concatenating them together with the “.” character.

The <attr-str> supports wildcard matching with the “*” character in the conventional manner (zero or more characters). The character “?”, with typical meaning of matching a single character, is not supported.

The xselect query parameter does just the opposite—only attributes other than the listed attributes names are returned. Both parameters MAY be provided at the same time, and the intersection of the two sets of attributes (those that would be included by select and not excluded by xselect) will be returned.

:  The xselect query parameter is not currently supported for the first release of SEMP v2.

Some additional rules:

  • The attributes in question are those in the returned data and links fields only, not the meta field.
  • Nesting makes the attribute unique. For example, xselect=abc does not exclude attribute foo.abc.
  • Including or excluding an attribute with nested attributes includes or excludes all the nested attributes. For example, xselect=foo excludes both foo.abc and foo.xyz.
  • Wildcard matching is done within an <attr-str>, not <attr-name>. For example, xdata=*bc does not exclude attribute foo.abc, whereas xdata=foo.*bc does.
  • The select and xselect parameters are applied after the where parameter.

Filtering

The query parameter where=<expr-list> is used to filter which objects will be returned from a collection, where <expr-list> is defined in ABNF as:

expr-list = attr-expr 1*63( "," attr-expr )       ; comma-sep list of attt-exprs (1 to 64)
attr-expr = attr-name expr-op string              ; comparison operation
expr-op   = "==" / "!=" / ">" / ">=" / "<" / "<=" ; comparison operator
string    = 1* 128 ( nn-octet )                   ; any string (length from 1 to 128)
nn-octet  = %x01-FF                               ; any non-NUL octet

When a GET request includes a where query parameter, only objects whose content matches all the given expressions will be returned. In the case that a given <string> needs to include any of the “,=!<>” characters, they MUST be escaped with the URI %xx notation to avoid any special meaning, consistent with Reserved Characters.

Some additional rules:

  • The attributes in question are those in the returned data field only, not the links or meta fields.
  • Wildcards are not allowed in <attr-name>.
  • The EQ and NE operators perform a string comparison, where <string> MAY include a “*” wildcard. In this case, the value of <attr-name> MUST be a string-based type.
  • The GT, GE, LT, LE operators perform a numeric comparison, with <string> interpreted as a number (specifically, interpreted as defined by the built-in Python int(<string>,0) function), and the value of <attr-name> MUST be a number-based type.
  • The where parameter is applied before the select and xselect parameters.

Collection Counts

Counts are supported for the GET method when retrieving the following collections:

  • /SEMP/v2/monitor/msgVpns/<name>/clients

    • clients/*/rxFlows

    • clients/*/transactedSessions

    • clients/*/txFlows

  • /SEMP/v2/monitor/msgVpns/<name>/queues

    • queues/<name>/msgs
    • queues/<name>/txFlows

  • /SEMP/v2/monitor/msgVpns/<name>/topicEndpoints

    • topicEndpoints/<name>/msgs

    • topicEndpoints/<name>/txFlows

  • /SEMP/v2/monitor/msgVpns/<name>/transactions

When you issue a GET request for one of the supported collections, the total number of items is returned in the count field in the metadata of the GET response.

For example, to retrieve the total number of queues in the default Message VPN, you could issue the following request:

curl -X GET solace:solace 192.0.2.1:8080/SEMP/v2/monitor/msgVpns/default/queues

The response from the broker would then contain the number of queues in the count field. In this case a single queue exists in the Message VPN.

...    
    "meta":{
        "count":1,
        "request":{
            "method":"GET",
            "uri":"http://192.0.2.1:8080/SEMP/v2/monitor/msgVpns/default/queues"
        },
        "responseCode":200
    }

For the rxFlows, transactedSessions and txFlows collections, counts are supported only for all clients, with an asterisk used to represent all. When an asterisk is used, only the metadata section is returned in the response. In all other cases, the use of an asterisk is not allowed.

If the request includes any filtering, in other words if it includes a where query parameter, the count field is not included in the response.

Retrieving Message Counts from Queues or Replay Logs

In addition to counts of objects in a collection, counts of messages (which are child collections) are supported for the GET method when retrieving the following collections:

  • /SEMP/v2/monitor/msgVpns/<name>/queues

  • /SEMP/v2/monitor/msgVpns/<name>/topicEndpoints

  • /SEMP/v2/monitor/msgVpns/<name>/replaylogs

When you issue a GET request for a collection, the objects in the child collection are returned in the collections section of the GET response. Counts for supported child collections are included in the count field in the collections section.

For example, to retrieve the total number of messages in each queue in the default Message VPN, you could issue the following request:

curl -u "solace:solace" "192.0.2.1:8080/SEMP/v2/monitor/msgVpns/default/queues?count=100&select=queueName,msgs.count"

The response from the broker would then contain the number of messages in the count field in the collections section. In this case, there are two queues with one queue containing 50 messages and the other containing 75 messages. If there were more queues (up to 100) this request would also capture them, since it increases the page size to 100 (the maximum supported).

When polling objects (especially queues), it is important to use a select clause to limit the content of your request to only what you need. For example, if you only want the number of messages in each queue, you should use select=queueName,msgs.count. Doing so can dramatically reduce the amount of time the broker needs to service the request, and reduce the likelihood of interfering with the performance of other broker services.

{
    "collections":[
        {
            "msgs":{
                "count":50
            }
        },
        {
            "msgs":{
                "count":75
            }
        }
    ],
    "data":[
        {
            "queueName":"q1"
        },
        {
            "queueName":"q2"
        }
    ],
    "links":[
        {},
        {}
    ],
    "meta":{
        "count":2,
        "request":{
            "method":"GET",
            "uri":"http://192.168.132.24:8080/SEMP/v2/monitor/msgVpns/default/queues?count=100&select=queueName,msgs.count"
        },
        "responseCode":200
    }
}

Since the queue name is returned in the data section and the msgs count is returned in the collections section, you must use the index of the queue in the data section to determine the corresponding msgs count in the collections section. In the previous example, you can determine that q1 has 50 messages because q1 is the first entry in the data section and the 50 msgs count is the first entry in the collections section.

Transactions

SEMP does not support transactions, as it is not possible to group together multiple SEMP requests and guarantee their success or failure as a complete unit.

However, any individual SEMP POST, PUT, or PATCH request will complete successfully or not at all. For example, a PATCH to a queue, changing three attributes, where the change to the third attribute fails will leave the queue without the first two attributes changed; all three changes must succeed, otherwise the object will be left in its original state. However, this support is still not transactional in that an event broker restart midway through the handling of an individual request may result in the request only being partially satisfied. Concurrent modifications of Solace objects by CLI or legacy SEMP users is another way that a request may fail and leave a request partially satisfied. SEMP v2 users should avoid concurrent modification of objects by CLI or legacy SEMP users.

Any individual SEMP GET request which returns multiple objects, is guaranteed to complete successfully or not at all. For instance, when doing a GET of a collection, partial collection results will not be returned if there is a failure while processing the GET. Similarly, a GET of an individual object returns all the state of that object, or none.