MQTT Protocol | MQTT in Depth | MQTT vs CoAP

Message Queue in MQTT Protocol

A Message Queueing system receives, stores and forwards messages. Those actions tend to vary heavily from one implementation or messaging protocol to another.

The core component of a queueing system is, of course, a data structure called queue. The underlying theory of a queueing system is quite complex, so let’s assume for now that a queue is some sort of a database. While most general purpose databases have powerful query capabilities such as the SQL query language, a queue in it’s most basic form only knows two operations, push and pop.

The Push operation inserts a new element to the queue, the Pop operation reads the oldest element from the queue and deletes it afterwards. This is also called the FIFO model, first in first out. In certain cases the Pop operation should always read the newest element instead of the oldest. Such a datastructure is called a stack, while the model is called LIFO, last in first out.

A message queueing system could implement more advanced queuing operations like message priority support or ways to automatically delete old messages.

A message queue is a perfect addition to the Publish/Subscribe pattern because it backs the decoupling principle. For this reason most Publish/Subscribe protocol implementations rely on a message broker that also implements a message queuing system.

MQTT 5 | MQTT 5.0 | MQTT Protocol 5

The MQTT version 5 was released last year and a lot of people have been blogging and tweeting and talking about the merits of this protocol update. And there are tons of improvements! Here’s a list of some of them:

• Metadata via user properties: the ability to add extra information which can be inspected without having to parse and inspect the payload.
• Negative acknowledgements which makes it a lot easier for a client to understand what’s happening.
• Request/Response interactions.
• Provisioning features where brokers can tell clients to (re)connect to other broker instances.
• Fine grained retained messages handling.
• Differentiate received messages for which the subscriptions overlap using subscription identifiers.
• Enhanced authentication mechanism.
• Shared subscriptions to easily handle massive fan-in scenarios.
• Topic aliases which saves band-with when publishing messages.
• Use Message expiration to prevent clients from receiving stale/irrelevant information.

All above mention features are sorely needed. Some more than others. Negative acknowledgements allow for much easier debugging cycles when trying to understand why some client can’t connect, publish or subscribe to a specific topic by telling the client what actually happened. Shared subscriptions are needed to overcome the limitations of a single subscriber receiving more messages than it can handle. Topic aliases are great for those clients where the up-link is expensive and saving as many bits as possible is paramount.

Is it still lightweight?

MQTT 5 is same as MQTT 3.1.1. The only new packet is the AUTH packet which can be used to support SASL type authentication schemes. The rest flow as usual and the only case where more packets are flowing compared to MQTT 3.1.1 is to support the new features such as negative acknowledgements where earlier the client would just be disconnected.

The biggest change on the protocol level is the addition of properties which (for most) packets are optional. Properties are a list of key-value pairs which is used to support many of the new features introduced in the protocol. For instance topic aliases, message expiration intervals, enhanced authentication methods and data, and metadata (user properties) are all implemented as specific properties on the various packets.

On the wire this means that packets can contain extra optional data that only has to be parsed if it is there. The additional extra computational overhead is therefore to check if optional data is present or not, and if so, to parse it.

The extra work the parser has to do is extremely little and would not be noticeable on small scale MQTT deployments and even on large scale deployments it would not be expected to have any significant impact.

So MQTT 5 basically provides a lot of new features while making sure the computation and band-with cost is effectively zero if they aren’t used!

Broker and client library perspective

Compliant client libraries and brokers have to implement all these new features in all their glory and complexity. Fortunately most people don’t have to worry about this as a bunch of great MQTT 5 compliant client libraries are already in development! For instance the Eclipse Paho C, Go, and Java clients are well under way and the highly popular Mosquitto broker has also announced MQTT 5 support by August 2018.