Queues and Topic Endpoints

PubSub+ event brokers support the following endpoint typess for receiving, storing, and delivering Guaranteed messages:

Both types of endpoints are visible only on the event broker. Client applications connect to the event broker that has the endpoint that it wants to consume messages from. There is no propagation of the existence of endpoints from one event broker to another via control or routing protocols, so there is no need for endpoints to have unique names throughout a network.

Queues are significantly more flexible than topic endpoints and are the recommended approach for most applications. The use of topic endpoints should be restricted to JMS applications.

Endpoint Durability

Queues and topic endpoints can be either durable or temporary, except for Partitioned Queues, which can only be durable.

Durable Endpoints

Durable queues and topic endpoints are provisioned objects on the event broker that have a life span independent of a particular client session. They also survive an event broker restart and are preserved as part of the event broker configuration for backup and restoration purposes. Administrators can provision durable queues and topic endpoints through the Solace CLI, SEMP, or PubSub+ Broker Manager. Client applications can dynamically provision durable endpoints through the Solace messaging APIs.

A durable endpoint has an associated access type which determines how messages are delivered to bound consumer flows. For more information, see Queue Access Types and Topic Endpoints.

Temporary Endpoints

Temporary queues and topic endpoints are dynamically created and destroyed by client applications. Temporary queues and topic endpoints are typically used as temporary destinations for service requests.

When a client application has a connected session, it can provision a temporary queue or topic endpoint and create a consumer flow to bind to it. These temporary queues and topic endpoints are non-durable because they only last as long as the client’s session is connected. Only a single consumer can bind to a non-durable endpoint, and there is no support for multiple consumers or non-exclusive access.

The naming of the temporary destinations is controlled by the application. On a BIND request from the client, a name is provided. If the destination does not exist, one is created.

When a client dynamically creates a queue, it is configured by those endpoint properties and provision flags that the client may provide with a create API function or method. Any other endpoint parameters are then configured with the values used for endpoints provisioned by an administrator through the Solace CLI. By default, the system defaults are used. However, it is also possible to use CLI-provisioned queues and topic endpoints with custom values, and those values will be applied to any new client-created queues and topic endpoints.

A non-durable queue or topic endpoint can be dynamically created if:

Topic Subscriptions

You can add one or more topic subscriptions to a durable endpoint so that Guaranteed messages published to matching topics are delivered to that endpoint.

Any Guaranteed messages published to topics that match subscriptions associated with endpoints are delivered to those endpoints. Error indications are returned to the publisher if the message can't be delivered to one or more endpoints for any reason. That is, the feedback to the publisher is identical to that provided when the messages are published directly to the endpoint.

As illustrated in the following diagram, topic subscriptions allow a single message published to a topic to be delivered to a combination of topic endpoints, queues, and consuming clients with matching Direct messaging topic subscriptions. For information on how a message’s delivery mode can be changed to deliver a message to a client’s topic subscription, refer to Topic Matching and Message Delivery Modes.

Diagram showing the concepts described in the surrounding text.

The mechanism for adding subscriptions depends on the type of endpoint:

Topic Subscription Exceptions

In addition to topic subscriptions, you can add one or more topic subscription exceptions to a durable queue so that Guaranteed messages published to matching topics are not delivered to the queue. Topic subscription exceptions are not supported for topic endpoints.

Using topic subscription exceptions allows you to easily exclude specific topics from the set of topics delivered to a the queue by adding a leading "!" character to the topic subscription. For example, if you add a topic subscription "animals/f*" and a topic subscription exception "!animals/fox" to a queue, guaranteed messages published to "animals/frog" are delivered to the queue; however, messages published to "animals/fox" are discarded.

Topic subscription exceptions are enabled by default, and always take precedence over subscriptions regardless of the order in which the subscriptions and the exceptions are configured. To enable or disable this functionality on event brokers, refer to System-Level Subscription Exception Configuration.

To configure subscription exceptions on a queue, see Adding Topic Subscription Exceptions to Queues.

Endpoint Permissions and Access Control

The flow chart below shows the layers of access control that are used to determine whether clients may bind to and browse or consume messages from endpoints.

The first layer is a configuration in the associated client profile that determines whether the client is permitted to receive Guaranteed messages. If this is allowed, then the endpoint permissions are checked.

The endpoint permission rules are:

  • The owner has full access to an endpoint. The owner of an endpoint is configurable. In the case of dynamically‑created endpoints, the default owner is the client username of the client that created the endpoint.
  • CLI-created queues and topic endpoints can only be deleted through the CLI.
  • Each endpoint has a “Permission All” parameter that determines what access control clients other than the owner of an endpoint have. By default, the “Permission All” parameter has a permission level of none. However, permissions can be explicitly specified when the endpoint is initially created through the CLI or dynamically through a messaging API. The permission level can also be changed through the CLI.

The permissions available at the following levels are:

  • none—no access.
  • read-only—allows the messages only to be read; they cannot be removed or consumed from the message spool.
  • consume—allows messages to be browsed and consumed from the message spool.
  • modify-topic—allows the topics assigned to endpoints to be modified. Modify-topic also implicitly includes same permissions as consume.
  • delete—allows deleting queues or topic endpoints. Delete also implicitly includes same permissions as modify-topic.

Selectors

Selectors are filters that clients can apply when they bind to endpoints. They enable clients to filter which messages they are interested in receiving, as determined by the messages’ header field and property values. A selector is a string with a maximum of 2,000 bytes that uses a conditional expression syntax that is a subset of SQL92.

As shown in the diagram below, selectors can filter messages from queues and topic endpoints.

  • For queues: Any messages that do not match the client’s selector string are not delivered to the requesting client, but remain in the queue.

    Selectors are not supported for Partitioned Queues.

  • For topic endpoints: Any messages that do not match the selector are not delivered to the requesting client, but are removed from the topic endpoint.

    Selectors on topic endpoints run as background processes to avoid slowing down publishing pipelines. It is possible to see the number of messages in a topic endpoint shrink as the selector is applied. For example, if 100 messages are published that match the associated topic of a topic endpoint, then these 100 messages are spooled into the topic endpoint. The associated selector starts testing these 100 new messages, and any messages that the selector rejects are removed from the topic endpoint. If there is a selector associated with a topic endpoint, then the selector must filter the messages prior to delivery to the flow. If either the topic or selector changes, then all messages are removed from the topic endpoint before applying the new topic and selector.

Client applications can also specify a selector string when browsing a queue so that the client only browses messages that match the selector.

Diagram illustrating the concepts described in the surrounding text.

Performance Considerations When Using Selectors

While selectors are fully supported, the best performance is typically achieved using topic matching without selectors. In many cases, it’s possible to include the message properties that would be selected upon within the topic hierarchy. Topic subscriptions and topic subscription exceptions can then be used to match against the relevant levels within the topic to implement selector expression-like matching.

Topic subscriptions and topic subscription exceptions support wildcards and prefix matching within levels. With wildcards, exceptions, and the ability to specify multiple subscriptions or exceptions for a queue or topic endpoint, it’s possible to implement matching operations like AND, OR, and NOT.

Consideration must also be given to performance when using selectors to retrieve messages from deep queues. The Solace event broker needs to retrieve each of the queue’s messages, potentially from disk, to evaluate its contents against the consumer’s selector expression. If the queue is very deep, it may take several seconds before the first matching message is found. Large messages also take longer to retrieve and evaluate against selector expressions. These challenges are eliminated when using topic subscriptions for filtering, because the filtering occurs before the event broker adds the messages into the queues.

In general, you should avoid using selectors whenever possible