The Solace router supports two types of endpoints for receiving, storing, and delivering Guaranteed messages:
Note: Endpoints are of local router significance only and are not a network‑wide concept insofar as a consumer must connect to the router that has the endpoint. There is no propagation through router-to-router control/routing protocols of the existence of endpoints. Thus, there is no enforcement of endpoint name uniqueness throughout a network by the routers.
A queue acts as both a destination that clients can publish messages to and as an endpoint that clients can bind consumers to and consume messages from. A queue is typically used in a point-to-point (PTP) messaging environment.
Although many consumers can bind to a queue, an individual message spooled to a queue can only be consumed by a single consumer.
Note: It is also possible to add topic subscriptions to a queue so messages published to matching topics are delivered to the queue. (For more information, refer to Adding Topic Subscriptions to Queues.) Therefore, it is also possible to use queues in a publish and subscribe (Pub/Sub) model.
Queues are significantly more flexible than topic endpoints and are the recommended approach for most applications. The use of topic endpoints should be restricted to JMS applications.
A queue has an access type, which determines how messages are delivered when multiple consumer flows are bound to it. Queues can be assigned one of the following access types:
- exclusive—Only one consumer can receive a message at any one time, while additional consumers may be connected as standby. That is, only one Flow can be active. Only the first consumer to bind can receive messages. If the first consumer disconnects, the second consumer receives data, and so on. An exclusive queue always delivers messages in the order they are received.
- non‑exclusive—Multiple consumers can bind to a non-exclusive queue. Each consumer is serviced in round‑robin fashion. This provides load-balancing; however, if a connection fails, then unacknowledged messages are delivered to another consumer with the re-delivered flag set. In this way, messages can be delivered to consumers out of order.
- The access type can only be changed for a durable queue and only when client access to that durable queue is disabled through the shutdown Queue CONFIG command.
- The access type assigned to a queue does not affect the ability of clients to browse the message on a queue with a queue browser.
You can add one or more topic subscriptions to a durable queue so that Guaranteed messages published to matching topics are also delivered to and spooled by the queue.
As shown below, this allows a single message published to a topic to be delivered to a combination of topic endpoints, one or more queues, or even clients with matching Direct Messaging topic subscriptions. (For information on how a message’s delivery mode can be changed to deliver a message to a client’s topic subscription, refer to Topic Matching and Message Delivery Modes.)
Possible Routing of a Message According to its Topic
Any Guaranteed messages published to topics that match subscriptions associated with queues are delivered to those queues. Error indications are returned to the publisher if the message cannot be delivered to one or more queues for any reason (that is, the feedback to the publisher is identical to that provided when the messages are published directly to the queues).
Topic subscription to queue mappings are applicable to both durable and non‑durable queues. Deletion of a queue for any reason results in the deletion of all topic subscription to queue mappings for that queue.
- It is also possible for published Guaranteed messages to be spooled to a queue if it is assigned a topic subscription that matches the message’s topic destination. For more information, refer to Adding Topic Subscriptions to Queues
- A client cannot remove topic subscriptions from a queue that an administrator has added through the CLI or SEMP.
A topic endpoint attracts messages published to a topic for which the topic endpoint has a matching topic subscription. The topic subscription for the topic endpoint is specified in the client request to bind a Flow to that topic endpoint. Only a single client can bind to and consume messages from a topic endpoint. (If another client bind request is received when a client is already bound to a topic endpoint, the request is rejected.)
Topic endpoints can be used in a pub/sub model. They are equivalent to durable topic subscriptions in Java Message Service (JMS).
Queues and topic endpoints can be either durable or temporary.
Durable queues and topic endpoints are provisioned objects on the router that have a life span independent of a particular client session. They also survive a router restart and are preserved as part of the router configuration for backup and restoration purposes. Administrators can provision durable queues and topic endpoints through the Solace router CLI, SEMP, or SolAdmin; client applications can dynamically provision durable endpoints through the Solace messaging APIs.
Temporary queues and topic endpoints are dynamically created and destroyed by client applications. Temporary queues and topic endpoints are typically used as temporary destinations for service requests.
When a client application has a connected session, it can provision a temporary queue or topic endpoint and create a consumer flow to bind to it. These temporary queues and topic endpoints are non-durable because they only last as long as the client’s session is connected. Only a single consumer may bind to a non-durable endpoint, and there is no support for multiple consumers or non-exclusive access.
The naming of the temporary destinations is controlled by the application. On a BIND request from the client, a name is provided. It the destination does not exist, one is created.
When a client dynamically creates a queue, it is configured by those endpoint properties and provision flags that the client may provide with a create API function or method. Any other endpoint parameters are then configured with the values used for endpoints provisioned by an administrator through the Solace router CLI. By default, the system defaults are used. However, it is also possible to use CLI-provisioned queues and topic endpoints with custom values, and those values will be applied to any new client-created queues and topic endpoints.
A non-durable queue or topic endpoint can be dynamically created if:
- the client is associated with a profile that permits clients to create endpoints (refer to Allowing Guaranteed Endpoint Create).
- the maximum number of endpoints available for the Message VPN that the client has connected has not been reached (refer to Configuring the Max Number of Endpoints).
The flow chart below shows the layers of access control that are used to determine whether clients may bind to and browse or consume messages from endpoints.
Access Control Layers Used for Consuming Clients
The first layer is a configuration in the associated client profile that determines whether the client is permitted to receive Guaranteed messages. If this is allowed, then the endpoint permissions are checked.
The endpoint permission rules are:
- The owner has full access to an endpoint. The owner of an endpoint is configurable. In the case of dynamically‑created endpoints, the default owner is the client username of the client that created the endpoint.
- CLI-created queues and topic endpoints can only be deleted through the CLI.
- Each endpoint has a “Permission All” parameter that determines what access control clients other than the owner of an endpoint have.
Note: By default, the “Permission All” parameter has a permission level of
none. However, permissions can be explicitly specified when the endpoint is initially created through the CLI or dynamically through a messaging API. The permission level can also be changed through the CLI.
The permissions available at the following levels are:
- none—no access.
- read-only—allows the messages only to be read; they cannot be removed or consumed from the message spool.
- consume—allows messages to be browsed and consumed from the message spool.
- modify-topic—allows the topics assigned to endpoints to be modified. Modify-topic also implicitly includes same permissions as consume.
- delete—allows deleting queues or topic endpoints. Delete also implicitly includes same permissions as modify-topic.
Client applications using Solace enterprise APIs can create a queue browser in a session to look at messages spooled on a queue in order of oldest to newest without consuming them. That is, browsing messages returns the full content of messages, complete with all message headers and payloads, but those browsed messages are not removed from the message spool.
Note: When you browse a queue that has an active consumer, it is possible that the browser will not receive all messages published to the queue because the consumer can receive and acknowledge messages before they are delivered to the browser.
The figure below shows how a Browser Flow returns messages from the queue, without consuming them so that clients with established Flows can still consume them. In this example, a selector string is also used so that the client application only browses messages that match that selector. (Refer to Selectors for more information.)
Selectors are filters that clients can apply when they bind to endpoints. They enable clients to filter which messages they are interested in receiving, as determined by the messages’ header field and property values. A selector is a string up to a maximum of 1,023 bytes that uses a conditional expression syntax that is a subset of SQL92.
As shown in the figure below, selectors can filter messages from queues and topic endpoints.
- For queues, any messages that do not match the client’s selector string are not delivered to the requesting client, but remain in the queue.
- For topic endpoints, any messages that do not match the selector are not delivered to the requesting client, but are removed from the topic endpoint.
Selectors on topic endpoints are a background process to avoid slowing down publishing pipelines. It is possible to see the number of messages in a topic endpoint shrink as the selector is applied. For example, if 100 messages are published which match the associated topic of a topic endpoint, then these 100 messages are spooled into the topic endpoint. The associated selector starts testing these 100 new messages, and any messages that the selector rejects are removed from the topic endpoint. If there is a selector associated with a topic endpoint, then the selector must filter the messages prior to delivery to the flow.
Note: If either the topic or selector changes, then all messages are removed from the topic endpoint before applying the new topic and selector.
Client applications can also specify a selector string when browsing a queue so that the client only browses messages that match the selector.
Guaranteed Message Selectors
By default, Guaranteed messages are removed from a durable endpoint's message spool and discarded when:
- the number of redelivery attempts for a message exceeds the Max Redelivery value for the original destination endpoint;
- or a message’s TTL value has been exceeded and the endpoint is configured to respect message TTL expiry times.
However, messages that are flagged as DMQ-eligible by the publishing client can be sent to a DMQ assigned to the endpoint rather than be discarded.
Any durable queue on the same Message VPN as the endpoint that the messages were spooled to can be assigned as that endpoint's DMQ. Although durable endpoints are assigned a default DMQ (
#DEAD_MSG_QUEUE), every durable endpoint can be assigned a specific DMQ. Therefore, there can be multiple DMQs per Message VPN.
Note: If an endpoint's assigned DMQ does not exist, the Guaranteed messages will be discarded even if they are DMQ-eligible. In addition, although it is the default DMQ, a management user must manually create the
The following are use-cases for how using different endpoints within a Message VPN associated with different DMQs might be advantageous:
When messages are delivered to an endpoint through topic-to-queue mapping, and that message is subsequently moved to the default DMQ used by multiple endpoints, there is no way for an application servicing the DMQ to know which endpoint the message came from. Having a DMQ that is used only by a single endpoint would solve this problem.
- It is possible to deliberately use message TTLs in conjunction with the DMQs to achieve a delayed delivery of messages. If client applications bind to a DMQ rather than a "regular" endpoint, they will only consume expired messages that were moved a DMQ. In this use-case, applications don’t want the messages from multiple endpoints converging on a single DMQ.
For information, see Setting Dead Message Queues.
If a queue is assigned a max-spool-usage value of 0, that queue can only spool the last message that it received. With this spooling behavior, the queue is acting as a Last Value Queue.
A client publishing Guaranteed messages can apply a topic subscription to a Last Value Queue so that it attracts all the messages that the client publishes. This allows that client to use the Last Value Queue to accurately determine the very last Guaranteed message that it successfully published.
This could be beneficial, for example, if an application failure occurs after a message has been published. In this case, if the client application does not receive an acknowledgment from the router for the message, it does not know if the published message was lost entirely, or if the message was received but just the acknowledgment was lost. If a Last Value Queue is used, when the client reconnects and rebinds to the Last Value Queue, it can determine what was the last successfully published message, and it can continue publishing from where it left off without creating duplicate messages or losing messages.
To configure a queue can be configured to behave as a Last Value Queue, you can set the max spool usage value for the queue to 0 (refer to Configuring Max Spool Usage Values).
Note: Solace recommends as a best practice to NOT use this feature with topic endpoints.
When there is more than one publisher for a given topic, the publisher should be identified in published topic, which can by wildcarded by subscribing applications. For example, assign the Last Value Queue for each publishing client a topic subscription of the form
Pub1/> for the first publisher,
Pub2/> for the second publisher, and so on). The clients can then publish messages to topics of the form
uniquePubId/some/hierarchical/topic/. For example,
Pub1 might publish to
Pub2 might publish to
Pub2/price/equities/apple, and so on.
This allows clients to specifically subscribe to their own Last Value Queue, but other clients, who also want to receive messages on those topics, can use a topic subscription of the form
*/some/hierarchical/topic. For example,
- Message selectors are not supported with Last Value Queues.
- When the show queue User EXEC command is used to display the configuration and status information for a Last Value Queue, and no clients are bound to that queue, the output might show that multiple messages are spooled even though, in fact, only the most current message is spooled.