Using MQTT

The Message Queuing Telemetry Transport (MQTT) protocol is a lightweight, open protocol that can be used for Machine to Machine (M2M) and Internet of Things (IoT) use cases.

Solace PubSub+ event brokers support OASIS Standard MQTT v3.1.1 and MQTT v5.0. This allows client applications to inter-operate with PubSub+ event brokers without relying on Solace-specific APIs or custom software development.

MQTT Concepts

If you haven't used MQTT and Solace PubSub+ together before, you might want to look at the MQTT section of the Overview: How Apps Interact with PubSub+ Messaging Components page. It'll take you on a tour of the PubSub+ event broker's messaging components, and explain at a high level how they make data move from producers to the broker, and from the broker to consumers, with MQTT.

MQTT Sessions

An MQTT client connects to a specific Message VPN on an event broker, using a dedicated MQTT port configured for that Message VPN. The MQTT client connection also requires an MQTT session on the server.

An MQTT session object is a virtual representation of an MQTT client connection that exists as a managed object on an event broker. An MQTT session holds the state of an MQTT client (that is, it contains a client’s QoS 0 and QoS 1 topic filter sets and any undelivered QoS 1 messages).

An MQTT session can be created:

• Automatically when a client successfully connects to the event broker.
• By an administrator using the Broker Manager, Solace CLI, SEMP, or SolAdmin. Although the MQTT specification does not require administrator-provisioned MQTT sessions to be supported, they are allowed, and they provide more flexibility for application development. For more information about administrator-provisioned MQTT sessions, see Managing MQTT Sessions.

MQTT Session Durability

Depending on the flags a client provides when it connects to the broker, the corresponding MQTT session may or may not persist after its client becomes disconnected. A session that survives a client disconnect is referred to as durable. Durable sessions are restored after a restart and survive HA failovers, preserving the client ID, topic filters, and undelivered QoS1 messages of the client.

The behavior of sessions differs depending on the whether the client uses MQTT v3.1.1 or MQTT v5.0:

• MQTT v3.1.1: When a connecting v3.1.1 client provides a CleanSession=1 flag for the MQTT session, the client's existing MQTT session is discarded (if it exists) and any related information (subscription sets and undelivered messages) are not persisted after the client disconnects. That is, the flag ensures that the session "starts clean", that it "cleans up" after itself, and that no information is stored on the event broker after the client disconnects. This is true even if the session was administratively provisioned (through CLI or SEMP).

  If the client provides a CleanSession=0 flag, the MQTT session is persisted on the event broker (that is, it is a durable session). The client can reconnect to this durable session later.

• MQTT v5.0: When a connecting v5.0 client provides a CleanStart=1 flag for the MQTT session, the client's existing MQTT session is discarded (if it exists). That is, the flag ensures that the session "starts clean."

  Independent of this, the client may provide a Session Expiry Interval to control how long the session survives after its client becomes disconnected. Until the session expires, it is persisted (i.e., durable) on the event broker, and the client can reconnect to it.

An MQTT session can always be deleted by an administrator through the Solace CLI, SEMP, SolAdmin, or using Broker Manager.

Quality of Service Levels

MQTT publish messages and topic filters are assigned separate quality of service (QoS) levels, which determine the level of guarantee applied to message delivery. The MQTT QoS levels are:

• QoS 0—At most once delivery. No response is required by the receiver (whether it is an event broker or a subscriber), and no retry attempts are made if the message is not delivered.

  The Solace equivalent to QoS 0 is a message delivery mode of Direct.

• QoS 1—At least once delivery. This level ensures that the message is delivered at least once. In a QoS 1 exchange, the receiver (whether it is an event broker or a subscriber) is required to send an acknowledgment of the message to indicate that it has been received.

  The Solace equivalent to QoS 1 is a message delivery mode of Guaranteed. For more information, see Guaranteed Messages.

  An event broker requires a Guaranteed messaging configuration to provide QoS 1 service. If an event broker is not configured for Guaranteed messaging, all QoS 1 and QoS 2 MQTT topic filters are downgraded to QoS 0. Additionally, QoS 1 and QoS 2 messages are accepted by the event broker, but they are delivered as QoS 0 messages. For more information on how to configure an event broker for Guaranteed Messaging, see Guaranteed Messaging Configuration.

• QoS 2—Exactly once delivery. Solace converts published QoS 2 messages to QoS 1.

The following figure shows how different QoS levels may be applied to the same message. From a publisher to an event broker, an MQTT publish message uses the QoS level assigned by the message publisher. From the event broker to the message subscriber, an MQTT publish message uses the minimum of the QoS of the matching subscription (filter) and the QoS with which the message was originally published.

QoS Levels Applied During Message Delivery

Payload Handling When Message Types Change

This section discusses how the payloads of MQTT publish messages received by the event broker (that is, ingress messages) are handled when they are subsequently delivered as different message types (either as SMF or REST messages) to non‑MQTT clients that have matching topic subscriptions. It also discusses how the message payloads of received SMF or REST message are handled when they are subsequently delivered as MQTT publish messages to consuming MQTT clients with matching topic filters.

MQTT Ingress, SMF Egress

When an event broker receives an MQTT publish message from a client, the received message's payload, which is a sequence of bytes, is encapsulated in the resulting egress SMF message as a binary attachment.

MQTT Ingress, REST Egress

For MQTT v3.1.1, when an event broker receives an MQTT publish message from a client, the received message's payload is delivered with an HTTP Content-Type of "application/octet‑stream".

MQTT v5.0 can have its own content type and payload format indicator properties. If the received MQTT v5.0 message has a content type property, the delivered HTTP message will have a Content-Type matching that. If the received MQTT v5.0 message has no content type property, but does have a payload format indicator property of 1, the delivered HTTP message will have a Content-Type of "text/plain".

SMF Ingress, MQTT Egress

An SMF message contains the following:

• SMF Header (mandatory) and parameters (optional)
• XML message payload (optional)
• binary attachment (optional)
• binary-encoded Metadata (optional)

Either the SMF message's XML message payload or binary attachment, but not both, can be used for the payload for the MQTT publish message. In addition:

• If the SMF message contains only a binary attachment, the following occurs:
  • If there is no binary metadata, then the binary attachment is copied into the payload field of the MQTT publish message.
  • If there is binary metadata that describes the format of the binary attachment, the data of the specified type is copied into the payload field of the MQTT publish message.

    Solace enterprise messaging APIs support the ability to carry structured data types (SDTs), such as maps and streams, in the binary attachment of the message or as user-defined message header fields. However, these SDTs cannot be used by MQTT clients. Therefore, they are not included in the MQTT publish message.

• If the SMF message contains only an XML message payload, it is copied into the payload field of the MQTT publish message.

  For MQTT v5.0, if the SMF message doesn't have a Content-Type, the MQTT message will have a Content-Type of "text/xml" and a payload format indicator of 1.

• If the SMF message contains both a binary attachment and an XML message payload, neither is sent—regardless of their content.
• For MQTT v3.1.1, custom user properties and userdata properties are not copied to MQTT publish message.

  For MQTT v5.0, SMF user properties are converted to text if possible, then put in the user properties of the MQTT publish message. If an SMF user property cannot be converted to text, the entire message is discarded as unformattable.

If the original SMF message contained a payload, but the process described above results in an MQTT publish message with no payload, the MQTT publish message is still delivered to the MQTT client even though it contains no payload. In this case, the message is noted as unformattable in the MQTT Session statistics.

REST Ingress, MQTT Egress

For MQTT v3.1.1, when an event broker receives a REST message, its payload, which consists of the data transmitted after the message's HTTP headers, is delivered in its entirety in an MQTT publish message to an MQTT client. The particular Content-Type of the published message is not significant.

For MQTT v5.0, the HTTP Content-Type is converted to the MQTT content type. Certain HTTP Content-Type values also result in an MQTT v5.0 payload format indicator of 1.

Will Messages

When connecting to an event broker, MQTT clients may specify a "last will and testament" (or simply "will") message. A will message is stored with the MQTT session information that is allocated for the client, and is sent if an MQTT client is disconnected from the event broker unexpectedly.

An MQTT v3.1.1 will message consists of a topic, QoS level, and message body. An MQTT v5.0 will message may additionally contain a payload format, message expiry, content type, response topic, correlation data, and user properties. Will messages allow other interested MQTT clients to detect unexpected connection loss.

An event broker does not broadcast will messages when an MQTT session is terminated due to an event broker restart, high availability (HA) event broker failover, Message VPN shutdown, or Guaranteed messaging shutdown.

Retained Messages

Retained messages are used when publishers send state messages on an irregular and/or infrequent basis. Published messages which have their retain bit set are stored by the Solace messaging network and delivered to subscribing clients when they subscribe to a topic that matches the retained message’s published topic. This allows subscribers to be notified immediately of the most recent state upon subscribing.

For more information about configuring Solace PubSub+ event brokers to support retained messages, see Managing MQTT Retained Messages.

Support for MQTT Retained Messages is a Controlled Availability (CA) feature. Please contact Solace to find out if this feature is supported for your use case.

MQTT Retain Caches

In Solace PubSub+ event brokers, retained messages are preserved in special Cache Clusters called MQTT retain caches. In other words, PubSub+ Cache is the underlying mechanism used to store, manage, and select which retained messages to publish as clients add topic filters.

MQTT Retain Cache Ancillary Objects

When you create an MQTT retain cache, the event broker creates the following ancillary PubSub+ Cache objects.

Object Type	Name	Notes
Distributed Cache	#retain-cache-<router-name>	Each event broker (or pair, in HA deployments) constitutes its own distributed cache. For more information about distributed caches, refer to Distributed Caches.
Cache Cluster	#retain-cache-cluster	Every cache cluster deployed as a retain cache shares the same name. For more information about cache clusters, refer to Cache Clusters.
Cache Instance (Primary)	#retain-cache-instance-<router-name>-primary	Every event broker with a retain cache runs a retain cache instance. The primary instance runs on the primary router in an HA pair. For more information about cache instances, refer to PubSub+ Cache Instances.
Cache Instance (Backup)	#retain-cache-instance-<router-name>-backup	The backup cache instance is like the primary cache instance, except it runs on the backup router if redundancy is enabled.

Shared Subscriptions

Solace PubSub+ event brokers support OASIS Standard MQTT v5.0 shared subscriptions for both v3.1.1 and v5.0 clients. You can use shared subscriptions to load balance large volumes of client data across multiple instances of back end data center applications. Each instance of the data center application joins the shared subscription, and the volume of data matching the shared subscription is randomly distributed between the application instances.

Consider the following when implementing shared subscriptions:

• Only QoS 0 shared subscriptions are supported. If a shared subscription with a higher QoS is requested, it is downgraded to QoS 0.

• Shared subscriptions are supported only on non-durable sessions (corresponding to v3.1.1 clients that connect with CleanSession=1 or v5.0 clients that connect with SessionExpiry=0). A request to add a shared subscription to a durable MQTT session is rejected by the event broker.
• In the OASIS Standard MQTT v5.0, a CONNACK property indicates whether an event broker supports shared subscriptions. Because CONNACK properties are not present in MQTT v3.1.1, the event broker cannot transmit this information to v3.1.1 clients. Solace PubSub+ event brokers support shared subscriptions as of release 9.1.
• Because allowing indiscriminate access to shared subscriptions is a potential security issue, you can specify which clients are able to use shared subscriptions. For more information, see Allowing Shared Subscriptions.
• If the broker is configured to use an ACL to control what clients can subscribe to, it validates only the {filter} portion of a shared subscription.
• In MNR or DMR deployments with subscription exporting enabled, shared subscriptions are exported by default. In these cases, the event broker treats each exported subscription as a separate entity. In other words, clients connected to different nodes are not load balanced as a single group. To prevent the event broker from exporting subscriptions, start the subscription (topic filter) with $noexport. For example, $noexport/$share/{ShareName}/{filter}.

Video: Shared Subscriptions Using MQTT

In this video, Leah Robert talks about Shared Subscriptions and demonstrates how to set them up using MQTT.