Today's Deep-Dive: Mosquitto
Ep. 274

Today's Deep-Dive: Mosquitto

Episode description

The Deep Dive explores the infrastructure behind the Internet of Things (IoT), focusing on Eclipse Mosquito and the MQTT protocol. Mosquito is an open-source message broker that implements MQTT, designed for efficient, lightweight communication between IoT devices. The protocol is crucial for low-power sensors and embedded systems, offering minimal data packets and reliability over unreliable networks. The broker manages communication between devices, ensuring security and scalability. Mosquito supports various versions of MQTT and is compatible with multiple platforms, making it accessible for developers. It is widely used in industrial settings for real-time data collection and predictive maintenance, as well as in smart grids for energy management. The deep dive also discusses the decision-making process for choosing between open-source, cloud, and enterprise versions of Mosquito based on project needs and scale. Additionally, it highlights the community support and resources available for learning and development.

Gain digital sovereignty now and save costs

Let’s have a look at your digital challenges together. What tools are you currently using? Are your processes optimal? How is the state of backups and security updates?

Digital Souvereignty is easily achived with Open Source software (which usually cost way less, too). Our division Safeserver offers hosting, operation and maintenance for countless Free and Open Source tools.

Try it now for 1 Euro - 30 days free!

Download transcript (.srt)
0:00

Welcome back to the Deep Dive. If you've ever sort of wondered how all those tiny

0:04

sensors,

0:05

you know, smart meters, things on factory floors, how they all talk to each other

0:08

efficiently

0:10

without completely draining their batteries or hogging bandwidth, well, you're

0:15

about to get the

0:15

inside track. Yeah, today we're really focusing on the infrastructure that makes

0:19

the whole

0:19

Internet of Things tick. We're talking about Eclipse Mosquito and the protocol is

0:25

built on MQTT.

0:26

It's basically the messaging backbone for, well, almost all modern connected

0:30

systems,

0:31

and we're going to try and make it really clear why it's so vital. Exactly. We're

0:35

going straight

0:35

into the architecture, explaining what a broker actually does, how this lightweight

0:39

messaging works,

0:40

and why Mosquito, this specific open source software, has become, well, pretty much

0:44

the

0:45

standard. Okay, but before we jump into all the technical bits, we really want to

0:49

thank the

0:49

supporter of this Deep Dive, Safe Server. Safe Server specializes in hosting this

0:54

kind of essential

0:55

software and also supporting digital transformation projects. They help you manage

0:59

and scale up complex

1:00

infrastructure, just like what we're discussing today. If you want to learn more

1:03

about how they

1:04

could help it out, just visit www.safeserver.de. Yeah, thanks, Safe Server. Okay,

1:10

so let's start

1:11

with the basics. Our source material defines Eclipse Mosquito pretty simply. It's

1:16

an open

1:17

source message broker, its main job. Implementing the MQTT protocol, it handles the

1:21

modern versions,

1:22

5.0, but also the older ones like 3.1.1 and 3.1, which is important for

1:27

compatibility. Right,

1:28

and MQTT, message queuing, telemetry, transport, we need to really underline the

1:32

why here. Why have

1:33

a whole separate protocol just for IoT stuff? Why not just use, I don't know,

1:36

regular web protocols?

1:37

That's a great question. It really comes down to efficiency. Standard web

1:40

communication like HTTP,

1:42

it's fine for websites for complex data, but it eats up bandwidth, and it has a lot

1:47

of overhead

1:48

that a tiny battery-powered sensor just can't afford. MQTT, on the other hand, is

1:53

designed from

1:54

the ground up to be incredibly lightweight. We're talking minimal data packets

1:57

designed to work even

1:58

over flaky, unreliable networks. That low overhead is precisely why it's perfect

2:03

for internet of

2:04

things messaging, you know, those low power sensors, mobile apps, embedded

2:07

computers, microcontrollers,

2:08

all that stuff. Okay, so MQTT is the efficient language, and Mosquito is. Is it

2:12

like the travel

2:13

controller, the post office managing the whole conversation? What exactly does the

2:18

central

2:18

MQTT broker do? That's a good analogy. It's the central nervous system, really. The

2:24

broker manages

2:25

absolutely all the communication between clients, the devices sending data, and

2:30

other clients or

2:31

applications that need that data. It has to sit in the middle. That's key. It means

2:36

the publisher,

2:36

the thing sending the data, and the subscriber, the thing receiving it, they never

2:40

even need to

2:41

know the other one exists. That simplifies scaling and security massively. Okay, so

2:45

walk me through

2:46

it. A message comes into Mosquito. What happens step by step? Right. So first, the

2:51

broker gets

2:51

the message from a publisher. Step one is checking. Is this client actually allowed

2:56

to publish on this

2:57

topic? Authorization. Then, and this is really important, it queues the messages.

3:02

But it does

3:03

this according to the requested quality of service, the QoS level. QoS. Right. That's

3:08

something

3:09

beginners might skim over. What does that mean in practical terms for someone

3:13

listening? It's all

3:14

about reliability. How sure do you need to be that the message got through? QoS

3:17

basically has three

3:18

levels. Zero, one, and two. Level zero is kind of fire and forget. Fastest uses the

3:24

least resources,

3:25

but no guarantee it arrives. Level one guarantees it arrives at least once. Might

3:29

get duplicates,

3:30

but it will get there. And level two guarantees it arrives exactly once. No losses,

3:33

no duplicates.

3:34

Most reliable, but also the most overhead. So the broker's job is to handle all

3:39

that complexity to

3:40

enforce those guarantees, making sure data is reliable, even if the network hiccups.

3:44

Then

3:45

once it knows the message is okay and queued correctly, it figures out which

3:49

authorized

3:50

subscribers are interested in that specific data and sends it out securely. Got it.

3:54

So the broker

3:56

handles the mechanics of security, the reliability rules, but the really clever

3:59

part seems to be how

4:00

it routes the data, this decoupling you mentioned. Can you break down the publish-subscribe

4:05

model or

4:06

pub-sub that makes this work so well, especially compared to, say, direct

4:10

communication? Yeah,

4:11

this is the big architectural idea. Think about traditional communication. If your

4:16

temperature

4:16

sensor needed to send its reading to three different dashboards, right, the sensor

4:22

itself

4:22

would have to manage three separate connections. It needs to know the addresses,

4:26

handle potential

4:27

failures for each one. That sounds complicated and probably drains the battery

4:30

faster. Exactly,

4:31

and it's fragile. If one dashboard is offline, the sensor might have issues. With

4:36

pub-sub,

4:37

it's totally different. The publisher, let's stick with the temperature sensor,

4:41

sends its data only to the broker, just one connection, and it tags the data with a

4:45

specific

4:46

topic, something like factory line one temperature. The sensor honestly doesn't

4:50

care who, if anyone

4:51

is listening. Okay, so the broker gets that message, sees the factory line one

4:56

temperature

4:57

topic, and then it does the work of finding anyone who's raised their hand, so to

5:01

speak,

5:01

saying, I'm interested in that topic. Precisely. Those are the subscribers. Maybe

5:06

one is a live

5:06

dashboard. Another might be a system logging historical data. A third could be an

5:11

analytics

5:12

engine looking for anomalies. They all just tell the broker, hey, I want messages

5:16

on factory line

5:17

one temperature. The broker then pushes the message out to all of them. The sensor

5:21

never knew they

5:22

existed. That sounds way simpler to set up and manage, especially when you add more

5:26

devices or

5:27

listeners. And Mosquito itself makes getting started easy, right? The sources

5:31

mention some

5:32

handy tools for developers. Oh, yeah, definitely. If you're coding, say, an

5:36

embedded device,

5:36

Mosquito provides a C library, which is very efficient. But for just quick testing,

5:41

debugging,

5:42

or even simple scripts, the command line tools are fantastic. Mosquito Pub to

5:46

publish a message,

5:47

and Mosquito Sub to subscribe and see messages. I remember using those all the time

5:51

on Raspberry

5:52

Pis or small Linux boxes just to quickly check if things were connected and talking.

5:56

You can

5:56

literally send and receive data in seconds from the command line. Super useful for

6:01

validation.

6:02

Okay, so it's flexible. Let's talk about getting it installed. Accessibility. Since

6:07

Mosquito is

6:08

open source and runs on tiny things like a Raspberry Pi right up to big servers,

6:12

presumably it's easy to get hold of. Yeah, that wide platform support is a huge

6:16

reason for its

6:17

popularity. The sources confirm it's pretty much everywhere. If you're on a Mac,

6:21

you can just use

6:21

homebrew.brew installmosquito on Windows. Simple installers by 64 and by 86

6:28

versions. And for

6:29

Linux users, Raspberry Pi OS, Ubuntu, Debian, whatever, it's usually right there in

6:34

the standard

6:34

package repositories. Or you can use things like snap packages or specific PPAs if

6:39

you need the

6:40

absolute latest version. So generally no complex builds or hunting for weird

6:43

dependencies. You just

6:45

install it like any other standard package. Pretty much, yeah. Makes setup really

6:48

straightforward.

6:49

And what about testing? If someone's just learning or wants to test their client

6:52

code without setting

6:53

up their own broker right away. Ah, this is brilliant. The Mosquito Project itself

6:57

runs a

6:57

public test MQTT server. It's at test.mosquito.org. It's a fantastic resource,

7:03

completely free to use

7:04

for testing purposes. That's incredibly useful. Saves a lot of hassle initially.

7:08

What sort of

7:08

connections does it support? Can you test secure connections on it? Yep. Covers all

7:13

the main bases

7:13

you need for real-world testing. You can connect using plain, unencrypted MQTT,

7:18

which is good for

7:19

initial tests. But critically, it also supports MQTT secured over TLS encryption.

7:25

And for web

7:25

applications, it supports MQTT over web sockets, both plain and also secured with TLS.

7:31

So you can

7:32

test embedded devices, server applications, web front ends, all against that one

7:36

public broker.

7:37

Okay. So we understand the tech, how it works, how to get it. Let's connect it to

7:40

the real world.

7:41

Where is this technology actually making a difference? What are some big use cases?

7:46

Well, the sources really highlight its impact in particularly industrial settings.

7:50

Let's take

7:50

manufacturing. It's often used there as a core data transmission solution. Think

7:54

about collecting

7:55

sensor data, temperature, pressure, vibration levels from maybe hundreds of

8:00

machines on a

8:01

factory floor in real time. Right. Moving data off the machines and into systems

8:05

that can analyze it,

8:06

replacing older, maybe less flexible systems. Exactly. It allows manufacturers to

8:10

gather all

8:11

this high frequency data very efficiently. And the big payoff there is enabling

8:15

things like

8:15

predictive maintenance. Instead of just running a machine until it breaks, you

8:19

analyze the vibration

8:20

patterns, the temperature trends, and the system can predict a likely failure

8:24

before it happens.

8:26

You get an alert, schedule maintenance during planned downtime, and avoid those

8:29

costly,

8:30

unexpected stops. That's huge for efficiency. Makes sense. And the other major area

8:35

mentioned

8:36

was infrastructure, something about energy management. Yes, absolutely vital area.

8:41

Think

8:41

about smart grids. Mosquito and MQTT are often used to aggregate huge volumes of

8:46

data, but typically

8:47

small individual messages from thousands or even millions of smart meters across a

8:52

city or region.

8:53

This lets the energy providers see electricity usage patterns in near real time.

8:58

They can spot

8:58

disruptions or outages faster, optimize how power is distributed across the grid,

9:03

and generally keep

9:04

things stable and efficient. Okay, that paints a clear picture. So if someone

9:07

listening is thinking,

9:08

right, I need this, and they're moving beyond just testing, they need to try as a

9:13

broker. We've

9:14

focused on the open source mosquito, but the sources mention other types too. How

9:19

do you choose the

9:20

right tool for the job? That's a key decision, and it really depends on your needs,

9:25

scale,

9:25

features, how critical the application is. So first, you have the open source

9:29

mosquito that

9:30

we've been discussing. It's free, incredibly flexible, because you can even modify

9:33

the code

9:34

if needed, and has strong community support. Great for testing, prototyping, and

9:39

many small

9:40

to medium-sized applications. Cost is minimal. Okay, option one, free and flexible.

9:45

What's next?

9:45

Next up, you've got the cloud brokers. Think services like AWS IoT Core, Google

9:51

Cloud IoT,

9:52

or Azure IoT Hub. These are fully managed services run by the big cloud providers.

9:56

The advantage, massive instant scalability. You don't worry about managing servers

10:01

or uptime.

10:01

They handle it all. Ideal for very large scale, often global deployments. But of

10:07

course,

10:08

you pay for that managed service and convenience. Right, the pay-as-you-go managed

10:12

route.

10:12

And the third type, you mentioned something like a ProEdition.

10:15

Yeah, so there are also enterprise or ProEditions, like the ProEdition for

10:19

Mosquito offered by Cedalo. These are commercial versions built on the open source

10:23

core,

10:24

but with added features specifically for mission critical demanding applications.

10:28

This is where you get things like built-in high availability clustering, making

10:33

sure the broker

10:33

keeps running. Even if one server fails, you often get more advanced security

10:37

features,

10:38

better monitoring, maybe REST APIs for easier integration with other systems.

10:42

And crucially, you usually get dedicated professional support. If something goes

10:46

wrong at 3 a.m.,

10:47

you have someone to call. So the decision comes down to factors like,

10:51

do I absolutely need 99.999% uptime? How many messages per second am I handling? Do

10:57

I need

10:57

enterprise-level security integrations and official support? If the answer starts

11:01

being yes to those,

11:02

especially around high availability and support, that's when you look beyond the

11:06

free version

11:07

towards cloud or enterprise options. And ensuring things like TLS and proper access

11:11

control lists

11:11

are probably non-negotiable at that point. Exactly. It's about matching the tool to

11:16

the

11:16

risk and scale of the project. Okay, this has been a really insightful look at Mosquito

11:20

and MQTT.

11:21

So, to quickly recap, Eclipse Mosquito is that versatile open source message broker.

11:27

It implements

11:28

MQTT, the lightweight protocol that's really essential for efficient communication

11:31

and the

11:32

internet of things, enabling that publish-subscribe model. And if you want to learn

11:36

more or get

11:37

involved, the community is very active. You can report bugs or suggest features on

11:41

GitHub. There

11:42

are mailing lists and forums for asking questions and getting help from other users.

11:46

And remember,

11:47

for those bigger, more critical deployments, companies like Sadello offer

11:50

professional support,

11:51

consulting, and even training, like their free MQTT academy, to help you build

11:56

robust solutions.

11:57

Yeah, and that choice we discussed, sticking with open source versus moving to a

12:01

commercial

12:02

or cloud solution, that really is a major crossroads for many growing IoT projects.

12:07

It brings up an interesting question for you, the listener, to maybe think about.

12:12

If you're using the free, open source Mosquito right now, what specific requirement

12:17

might trigger

12:17

the need to move? Is it needing truly persistent queues for millions of messages if

12:22

the broker

12:23

restarts? Is it needing geographic clustering for resilience? What's the breaking

12:28

point

12:28

that makes a commercial solution mandatory for your specific, large-scale, mission-critical

12:34

deployment?

12:34

That's definitely something to consider as systems grow. Food for thought.

12:38

And with that, we once again want to thank Safe Server for supporting this deep

12:41

dive into digital

12:42

infrastructure. You can find out more about their hosting and digital

12:45

transformation services at

12:46

deep dive.

12:46

deep dive.