Using MQTT

The Message Queuing Telemetry Transport (MQTT) protocol is a lightweight, open protocol that can be used for Machine to Machine (M2M) and Internet of Things (IoT) use cases.

The Solace Messaging Platform, as of SolOS version 7.1.1, supports this OASIS (Organization for the Advancement of Structured Information Systems) standard protocol (version 3.1.1). This support allows client applications to inter-operate with the Solace Messaging Platform without relying on Solace-specific APIs or custom software development.

Like the Solace Message Format (SMF) protocol used by the Solace Messaging Platform, MQTT is a topic-based client/server publish/subscribe protocol that enables client applications to exchange messages without direct knowledge of how those messages will be delivered to other interested clients.

Clients can publish messages to defined topics, and they may use topic filters (that is, topic subscriptions) to receive messages that are published to those matching topics. All topics are UTF-8 strings, which consist of one or more topic levels that are separated by forward slash “/” characters. Separating topic levels using slashes creates a hierarchy of information for organizing topics.

Note:  The SMF and MQTT protocols use similar topic syntax, therefore SMF and MQTT messaging applications can be used together in the same messaging network. That is, when an intersecting topic hierarchy is used, MQTT clients can receive messages published by non-MQTT clients (for example, SMF or REST clients), and non-MQTT clients can receive messages published by MQTT clients. However, there are some differences in topic syntax and usage between SMF and MQTT that must be considered before deploying MQTT applications in a network alongside SMF applications. For more information, refer to Topic Support and Syntax.

MQTT clients connect to an MQTT server (in this case, a Solace router), which is responsible for maintaining their subscription sets and directing published messages to clients that have matching topic filters. (Note that topic filters function the same as topic subscriptions in the Solace Messaging Platform, so for consistency, they will be referred to as topic subscriptions throughout this document.)

An MQTT client connection to a specific Message VPN on a Solace router is made through a dedicated MQTT port configured for the Message VPN that they are connecting to. The MQTT client connection also requires an MQTT session on the server. An MQTT session is represented as a managed object in SolOS, and it contains the session state for the MQTT client (that is, its subscriptions and messages).

Note:   

  • The Solace implementation of MQTT complies with the 3.1.1 MQTT protocol specification. Solace provides an annotated version of the specification (Contact Us ׀ Support ׀ Blog ׀ solace.com) that highlights any deviations, limitations, or choices made in the “SHOULD” and “MAY” clauses of the protocol specification for the Solace implementation. It is strongly recommended that network architects and programmers should review this document.
  • MQTT is not supported on the Solace 3230 appliances or Solace appliances that use a Network Acceleration Blade (NAB) model NAB-0401EM.

MQTT Sessions

An MQTT session object is a virtual representation of an MQTT client connection that exists as a managed object on a Solace router. An MQTT session holds the state of an MQTT client (that is, it is used to contain a client’s QoS 0 and QoS1 subscription sets and any undelivered QoS 1 messages).

An MQTT session can be created:

  • Automatically when a client successfully connects to the Solace router.
  • By an administrator using the Solace CLI, SEMP, or SolAdmin. Although the MQTT specification does not require administrator‑provisioned MQTT sessions to be supported, they are allowed, and they provide more flexibility for application development.

Note:  MQTT sessions should not be confused with Solace sessions (that is, non‑MQTT sessions).

MQTT Session Persistence

When a connecting client provides a CLEAN=1 flag for the MQTT session, the client’s MQTT session and any related information (subscription sets and undelivered messages) are not persisted after the client disconnects. That is, the flag ensures that the session “cleans up” after itself and no information is stored on the router after the client disconnects. This is true even if the session was administratively provisioned (through CLI or SEMP). If the client provides a CLEAN=0 flag, the MQTT session is persisted on the router, which means that the client’s client ID, topic subscriptions, QoS levels, and undelivered messages are all maintained (that is, they are not cleaned up). The client may then reconnect to the persisted session later.

An MQTT session can be deleted:

  • automatically when a client that created the MQTT session with a CLEAN=1 flag disconnects
  • when a client creates a new MQTT session with a CLEAN=1 flag and the same session identifier as the previous session
  • manually by administrators through the Solace CLI, SEMP, or SolAdmin

Quality of Service Levels

MQTT publish messages and topic subscriptions are assigned separate quality of service (QoS) levels, which determine the level of guarantee applied to message delivery. The MQTT QoS levels are:

  • QoS 0—At most once delivery. No response is required by the receiver (whether it is a Solace router or a subscriber), and no retry attempts are made if the message is not delivered.
  • The Solace equivalent to QoS 0 is a message delivery mode of Direct.

  • QoS 1—At least once delivery. This level ensures that the message is delivered at least once. In a QoS 1 exchange, the receiver (whether it is a Solace router or a subscriber) is required to send an acknowledgment of the message to indicate that it has been received.
  • The Solace equivalent to QoS 1 is a message delivery mode of Guaranteed (that is, non‑persistent or persistent). For more information, refer to Working With Guaranteed Messages.

  • A Solace router requires a Guaranteed messaging configuration to provide QoS 1 service. If a router is not configured for Guaranteed messaging, all QoS1 and QoS 2 MQTT topic subscriptions will be downgraded to QoS 0. Additionally, QoS 1 and QoS 2 messages will be accepted by the router, but they will be delivered as QoS 0 messages. For more information on how to configure a Solace router for Guaranteed Messaging, refer to Managing Guaranteed Messaging.
  • QoS 2—Exactly once delivery. Solace converts published QoS 2 messages to QoS 1.

The following figure shows how different QoS levels may be applied to the same message. From a publisher to a Solace router, an MQTT publish message uses the QoS level assigned by the message publisher. From the router to the message subscriber, an MQTT publish message uses the QoS level set by the matching topic subscription for the consuming client.

QoS Levels Applied During Message Delivery
QoS Levels Applied During Message Delivery

When there is a topic subscription match for an MQTT publish message, a consuming client will receive the message with the lowest possible QoS level for its subscription. For example, if a message is published with a QoS of 1, and it matches the client’s QoS 0 subscription, it will be delivered with a QoS of 0. However, because MQTT uses a one-to-many pub/sub messaging model, that message could also be delivered to a matching QoS 1 topic subscription with a QoS of 1.

Queues for QoS1 Suscriptions

An MQTT session that has QoS 1 topic subscriptions must be associated with a durable queue to hold those subscriptions and any undelivered messages that are attracted by those QoS1 subscriptions.

When an MQTT session is automatically created by a client, a queue is created when the first QoS1 subscription is added. If the MQTT session was created with CLEAN=1 CONNECT, the queue is deleted along with the MQTT session when the client disconnects, but if the MQTT session was created with CLEAN=0 CONNECT, the queue will remain after the client disconnects, and will only be deleted by administrative action.

When an MQTT session is administratively created, a queue is not created automatically for the MQTT session. Queues for administratively-created MQTT sessions must be manually created and deleted.

The configuration parameters given to an MQTT queue depends on whether it was created automatically on a client‑created MQTT session, or if it was created by an administrative action:

  • client‑created MQTT sessions—the queue uses the same configuration values used for standard dynamic client‑created queues. (For information on the default values that are used, refer to Configuring Default Values for Client-Created Endpoints.)
  • Note:   

    • Unlike standard dynamically-created endpoints, an MQTT client cannot pass in custom endpoint properties and provision flags.
    • If the copy‑from‑on-create command is used to specify a CLI‑provisioned queue or topic endpoint with custom values, those values will also be applied to MQTT queues for client‑created MQTT sessions.

  • administratively created MQTT sessions—the queue uses the default configuration values for an queues provisioned through the Solace CLI with the exception that it is enabled (no shutdown) when created. For information on the default values used for queues provisioned by an administrator through the Solace CLI, refer to Configuring Queues.

MQTT Topics

As a publish/subscribe messaging protocol, MQTT relies on a hierarchical topics. Clients can publish messages on specific topics, and clients can receive published messages that match their current topic subscriptions.

MQTT topics are UTF-8 strings, which consist of one or more topic levels that are separated by forward slash “/” characters. Separating topic levels using slashes creates a hierarchy of information for organizing topics.

Connected clients can publish MQTT messages to defined topics, and they may use topic filters (that is, topic subscriptions) to receive messages that are published to those matching topics.

Payload Handling When Message Types Change

This section discusses how the payloads of MQTT publish messages received by the Solace router (that is, ingress messages) are handled when they are subsequently delivered as different message types (either as SMF or REST messages) to non‑MQTT clients that have matching topic subscriptions. It also discusses how the message payloads of received SMF or REST message are handled when they are subsequently delivered as MQTT publish messages to consuming MQTT clients with matching topic subscriptions.

MQTT Ingress, SMF Egress

When a Solace router receives an MQTT publish message from a client, the received message’s payload, which is a sequence of bytes, is encapsulated in the resulting egress SMF message as a binary attachment.

MQTT Ingress, REST Egress

When a Solace router receives an MQTT publish message from a client, the received message’s payload is delivered with a Content-Type of application/octet‑stream.

SMF Ingress, MQTT Egress

An SMF message may contain binary data and XML data payloads, and in some cases, user-defined and Solace-defined message header fields.

When a Solace router receives an SMF publish message from a client for which there is a matching MQTT topic subscription, the payload of the message is processed before it is sent to the MQTT client as an MQTT publish message.

The SMF message’s XML message payload or binary attachment can be used for the payload for the MQTT publish message, but not both.

  • If the SMF message contains only a binary attachment, the following occurs:
    • If there is no binary metadata, then the binary attachment is copied into the payload field of the MQTT publish message.
    • If there is binary metadata, which describes the format of the binary attachment, (that is, a text or binary data) the data of the specified type is copied into the payload field of the MQTT publish message.
  • Note:  Solace enterprise messaging APIs support the ability to carry structured data types (SDTs), such as maps and streams, in the binary attachment of the message or as user-defined message header fields. However, these SDTs cannot be used by MQTT clients. Therefore, they are not included in the MQTT publish message.

  • If the SMF message contains only an XML message payload, it will be copied into the payload field of the MQTT publish message.
  • If the SMF message contains both a binary attachment and an XML message payload, neither is sent—regardless of their content.
  • Custom user properties and userdata properties are not copied to MQTT publish message.

Note:  If the original SMF message contained a payload, but the process described above results in an MQTT publish message with no payload, the MQTT publish message is still delivered to the MQTT client even though it contains no payload. In this case, the message is noted as unformattable in the MQTT Session statistics.

REST Ingress, MQTT Egress

When a Solace router receives a REST message, its payload, which consists of the data transmitted after the message’s HTTP headers, is delivered in its entirety in an MQTT publish message to an MQTT client. The particular Content-Type of the published message is not significant.

Will Messages

When connecting to a Solace router, MQTT clients may specify a “last will and testament” (or simply “will”) message. A will message is stored with the MQTT session information that is allocated for the client, and it will be sent if an MQTT client is disconnected from the Solace router unexpectedly.

A will message consists of a topic, QoS level, and message body. Will messages allow other interested MQTT clients about unexpected connection loss.

Note:   

  • A Solace router will not broadcast will messages when an MQTT session is terminated due to a router restart, high availability (HA) router failover, Message VPN shutdown, or Guaranteed messaging shutdown.
  • Retained will messages are not supported by the Solace MQTT implementation. If included in a message, the “Will Retain” flag is ignored.