Welcome to the deep dive the place where we unlock the secrets of emerging tech and
Download transcript (.srt)
0:00
0:04
0:08
0:10
0:16
0:16
0:23
0:27
0:28
0:31
0:36
0:37
0:41
0:45
0:47
0:51
0:52
0:58
0:59
1:03
1:05
1:09
1:10
1:14
1:19
1:25
1:31
1:31
1:35
1:38
1:42
1:44
1:48
1:51
1:54
1:58
2:00
2:05
2:06
2:10
2:11
2:15
2:16
2:21
2:21
2:25
2:27
2:31
2:36
2:39
2:45
2:46
2:51
2:53
2:59
3:02
3:06
3:09
3:13
3:18
3:19
3:23
3:27
3:31
3:37
3:41
3:42
3:46
3:49
3:50
3:52
3:57
3:59
4:03
4:04
4:08
4:08
4:12
4:13
4:18
4:19
4:24
4:29
4:30
4:32
4:35
4:40
4:44
4:49
4:52
4:53
4:57
5:00
5:01
5:06
5:08
5:14
5:17
5:18
5:23
5:27
5:32
5:34
5:38
5:39
5:42
5:48
5:51
5:55
5:56
6:00
6:05
6:06
6:10
6:12
6:16
6:17
6:21
6:22
6:27
6:29
6:32
6:37
6:42
6:46
6:48
6:55
6:57
7:01
7:04
7:09
7:10
7:13
7:17
7:19
7:23
7:29
7:31
7:35
7:39
7:41
7:43
7:47
7:49
7:54
7:56
7:58
8:03
8:04
8:06
8:10
8:15
8:17
8:20
8:20
8:24
8:28
8:30
8:35
8:40
8:40
8:46
8:47
8:52
8:54
8:58
9:03
9:04
9:08
9:12
9:12
9:17
9:18
9:23
9:28
9:32
9:36
9:42
9:42
9:46
9:48
9:52
9:55
9:57
10:01
10:02
10:05
10:07
10:11
10:13
10:17
10:22
10:23
10:27
10:28
10:33
10:35
10:40
10:45
10:50
10:51
10:55
11:00
11:01
11:03
11:07
11:08
11:11
11:14
11:14
Really figure out why it matters to you today. We're tackling something that I
think a lot of people are thinking about absolutely
We're talking about personal assistance, but with a huge twist one that doesn't spy
on you
We are diving into Leon AI right if you've ever wanted that convenience
You know of a digital helper, but without that feeling that some big company is
listening in on your living room
This is for you. It's a very common feeling these days
It is we're talking about an open source totally private and self-hosted virtual
assistant
But before we get into why Leon is sometimes called a virtual brain
We really want to thank the supporter of this deep dive of course this show is
brought to you by safe server
So if you're looking to host software just like Leon and you need some support for
your digital transformation
Safe server is there for you. You can find all the info at www.safeserver.de. Okay,
so let's unpack this
We've all used the big-name digital assistant. Yeah, but they all work on a trade.
You give them your data for convenience
That's the deal. Yeah, our mission today is to figure out what makes Leon
fundamentally different
we want to focus on its its core idea of ownership and
The really cool modular architecture behind it and that's the hook right there
Leon is designed to be your virtual brain, but and this is the critical part
This brain can live entirely on your server on my server not theirs. Exactly. That's
the whole shift
It's an assistant that does stuff when you ask it to whether you're talking or just
typing you control the entire relationship
So functionally it's similar to what we already know, but the whole infrastructure
is flipped upside down completely inverted
So I lead with open source. What is that MIT license, which I know the creator
chose. What does that actually mean for a regular person?
It means freedom and it means speed of development
The MIT license is basically the most permissive one out there. So no strings
attached pretty much
It signals that the goal is growth its contribution with almost no restrictions.
Anyone can see the code
They can change it use it for their own stuff commercial or not and then contribute
back
So you're not just a user you're part of a community that actually owns the tool
and that idea of ownership
Feeds directly into its main selling point, which is privacy. Yes, that is the
central pillar
This is where Leon goes head-to-head with all the concerns about data tracking from
commercial AI because it's on my server
Exactly, you are in complete control of your data
No conversation goes to the third-party server unless you explicitly set it up that
way and the source material really highlights this
You can configure Leon to work while being completely offline. Wait, okay offline
mode. That's a total game-changer
You're talking about sensitive business data or just conversations in your own home
that gives you complete
Digital sovereignty. Mm-hmm. But here's the question then if I host it myself am I
sacrificing all the power of you know
Massive cloud computers and that's the exact challenge Leon is built to solve
It uses its modularity to stay powerful while keeping you in control
The whole goal is to automate stuff and make your virtual life easier
Right, so it uses this really flexible structure to manage all that complexity
right there locally
So tell me about that structure. It's not just one big program
Is it it's built on a system of modules they call skills, correct?
Think of Leon's core as like the central nervous system
Yeah, it's the conductor the skills are the individual musicians in the orchestra
I like that analogy each skill is a tiny self-contained module that does one
specific thing
It could check the weather manage your calendar control your smart lights
Anything so if I want Leon to learn something new, I don't have to update the
entire system
I just add a new skill precisely
It makes it super scalable without making the core program bloated or fragile the
structure lets anyone
You know any developer out there create their own skills and then share them like
an app score for abilities
That's a great way to put it the creators even say there is only one core to rule
them all
To keep it all consistent and the plan is to build a dedicated skill registry
platform
So like NPM for JavaScript or pip for Python exactly like that, but just for Leon's
automation skills
It'll make finding and installing new abilities super simple. That's really elegant
But let's look behind the curtain for a second the repository the actual code has a
bunch of different parts
What are those other things doing?
Like why does a personal assistant need a TCP server?
Yeah, that gets a little technical but it's all about performance
The different parts of Leon need to talk to each other and they need to do it fast
You have the main server the skills the web app for the interface and the hot work
No, that's always listening for you to say its name. So they're all separate
processes
They are and the TCP server is like the internal switchboard
it just makes sure data gets passed between all those different parts reliably and
quickly and
Since so many AI tools are written in Python. They also needed a dedicated Python
bridge to translate because it translate
Yeah, it lets the JavaScript core talk seamlessly with all the Python AI libraries
They need to use that paints a much clearer picture. It's like a distributed system
in a box
So speaking of cutting-edge tools, what's the current state of development?
Read the dev branch was going through some major changes around June
2024 it is the biggest shift is the integration of what are called foundation
models. Hmm. Okay for a beginner
What does that mean? Think of foundation models as those huge general purpose AI
brains
Like the ones that power the most famous chat bots you hear about
Leon is starting to use those but and this is key in a hybrid approach hybrid
So it's not just using that massive AI brain for everything
No, because that would be really slow and heavy for a self-hosted assistant their
hybrid model is smart
It balances the power of those big models for complex tasks
Like say summarize this long email for me with simpler faster techniques for basic
commands like turn on the light
Exactly for something simple like that. It uses a much lighter almost instant
classification method
It only calls in the big guns the foundation models when it actually needs that
extra brainpower
So you get the best of both worlds speed for simple things and power for complex
ones
Optimal speed and accuracy without sacrificing that local first efficiency. Okay.
Here's where it gets really interesting for me
The AI toolbox itself
How does Leon get its ears and mouth in a way that still respects my privacy, right?
We're talking about three core AI concepts here NLP or natural language processing
which is for understanding what you mean the brain part the brain part then TTS or
text-to-speech for its voice and
STT speech-to-text for listening to you and I the user get to choose which tech I
want for all of those you get total control
Absolute control this modularity means you can choose what you're comfortable with
if you want the absolute best performance and the clearest voice
You can connect it to cloud services like Google cloud AWS or IBM Watson, but if I'm
doing this whole thing for privacy
I'd want the local options. Are they any good? Are they robust enough?
They're getting better all the time. And yes, there are strong offline options for
text-to-speech
You have tools like CMU flight for the listening part speech to text
They support Koki STT and more are on the way. What's the trade-off then? The trade-off
is usually a little bit of speed and
Maybe the voice doesn't sound quite as natural as the big cloud ones
But for people who are serious about privacy the peace of mind you get from running
it all locally is well
It's often worth that small dip in quality
It's just amazing how the flexibility is built into every single layer from the
core right down to the voice engine
Now open source projects like this live and die by their community for someone
listening who's getting excited about this
How is it all maintained and how can they get involved?
Leon is really driven by that idea that the more skills he has the more skillful he
becomes the author
Louie grenard is big on community
he encourages people to join their discord channel to share ideas or you know,
Even contribute code and building something this complex has to take a ton of time.
It does it's mostly a spare time project
So what's the plan for sustainability? Yeah, make sure it doesn't just you know
fade out
That's a super important point for any big open source project
The source material notes that at the end of the day sustainability is key. The
author has bills to pay, right?
We all do we all do so sponsoring the project lets him and other core contributors
Dedicate more real focused time to it instead of just grabbing a few hours here and
there
Financial support helps turn a passion project into well a full-time dedicated job.
That makes perfect sense
Okay, let's get practical for anyone listening. Who's ready to install this virtual
brain on their own machine
What do they need to get started? It's actually not as scary as it sounds
You just need a few standard things installed first. This is for Linux Mac OS or
Windows
You need a modern version of node.js and its package manager NPM
Okay node.js and NPM got it and we always recommend using the latest stable
versions
It just helps avoid headaches down the line. So once that's on my machine, what's
the first Leon specific step?
The first step is to install the Leon command line interface or CLI
You just open your terminal and type NPM install global at Leon ICLEI
That one command gives you the main tool to manage Leon
Okay, then to actually download all the files and set up your assistant
You run a really simple kind of cool command Leon create birth Leon create birth. I
like that
Yeah, it downloads everything gets the whole structure ready for beginners
We definitely say stick to the stable version for now
Especially with the big changes we talked about happening on the development branch
your tip and then to actually fire it up you just run Leon start the server will
boot up and then you just open a
Web browser and go to HTTP dot localhost dot one three three seven, and that's it.
That's it
You should see the web interface ready for you to configure skills and start
talking to your own private AI
They've really tried to make self hosting less intimidating. This has been a really
fascinating deep dive
Leon AI is clearly this powerful privacy first and
Super customizable assistant that's built by and for its community. Yeah, it's more
than just an alternative
It really is. Yeah, it feels like it's demanding a higher standard for how we think
about our data
And that kind of leads us to a final provocative question for you to talk about if
we connect the rise of these
open-source self hosted AIs like Leon to the bigger picture
How will this shift change what we expect from technology?
We'll self hosting tools like this eventually make us see those big centralized
assistance as being you know
Fundamentally broken because they're missing the most important feature
Privacy that is something to mull over as you decide what tech you want in your
life and where you want to draw that line
On your own digital sovereignty
Thank you again to safe server for supporting this deep dive into open-source
architecture
You can learn more about how they can help of your software hosting and digital
transformation at www safe server dot de
We will, for another Deep Dive.
We will, for another Deep Dive.