Today's Deep-Dive: Windmill
Ep. 246

Today's Deep-Dive: Windmill

Episode description

This deep dive discusses Windmill, an open-source developer platform designed to transform scripts into powerful, scalable internal tools. It highlights Windmill’s ability to turn simple scripts into web hooks, robust workflows, and interactive user interfaces, drastically speeding up development by up to 10 times. The platform supports multiple programming languages and automatically generates UIs based on script parameters, making it accessible even to non-technical users. Windmill consolidates various development tasks, eliminating the need for complex front-end development and infrastructure management. It features a low-code app editor for building rich data-intensive applications and supports various triggers like cron schedules, webhooks, and CLI commands. The architecture of Windmill, built with Rust, Postgres, and modern runtimes, ensures high performance and security. It is easy to self-host, with minimal setup requirements, and is designed to scale from local development to large Kubernetes clusters. Windmill is fully open-source under the AGPL v3 license, making it accessible for internal use without commercial licensing. The platform empowers users to automate complex tasks, reduce infrastructure sprawl, and enable cross-team collaboration, ultimately accelerating development and enhancing productivity.

Gain digital sovereignty now and save costs

Let’s have a look at your digital challenges together. What tools are you currently using? Are your processes optimal? How is the state of backups and security updates?

Digital Souvereignty is easily achived with Open Source software (which usually cost way less, too). Our division Safeserver offers hosting, operation and maintenance for countless Free and Open Source tools.

Try it now for 1 Euro - 30 days free!

Download transcript (.srt)
0:00

Welcome to the deep dive where we cut through the noise and get straight to the

0:03

insights you need

0:04

Have you ever found yourself writing, you know a handy script maybe to automate a

0:09

report or manage some data

0:11

Oh, absolutely all the time and then you wish it could be more than just well a

0:14

script like maybe it had a button or

0:16

Proper web interface. Yeah, or could easily talk to other services

0:19

Exactly without you having to suddenly become like a full stack developer overnight.

0:24

Uh-huh. That's the dream

0:25

Well today we're diving into something that might just make that dream a reality

0:30

But before we unpack this exciting solution just quick word from our supporter

0:34

Safe server cares about the hosting of this software and they are here to support

0:38

you in your digital transformation

0:40

You can find more info at www safe server dot DE

0:44

Okay, so our mission for you today is really a shortcut a shortcut to understanding

0:50

how this

0:51

Powerful open source platform can literally transform how you build and automate

0:55

things and we're really talking about turning those one-off scripts

0:58

Those little helpers into genuinely powerful scalable internal tools, right?

1:04

Even if you're maybe just starting out in the world of development or you know

1:08

infrastructure

1:08

We've pulled insights directly from its github repository and the official windmill

1:13

website. So we've got the ground truth here

1:15

Okay, let's get into it. We're talking about windmill for someone new to this space

1:19

Maybe just looking for like smarter ways to automate their work. What is windmill

1:25

exactly? Why should it grab their attention?

1:27

Okay. So what's really fascinating here is that windmill is fundamentally an open-source

1:31

developer platform think of it as your

1:34

Comprehensive toolkit. It lets you take those everyday scripts

1:38

You write could be Python for data bash for servers

1:41

Maybe type script write the usual suspect exactly and it lets you instantly

1:45

transform them into incredibly useful things like web hooks or

1:49

Robust workflows or even you know fully interactive user interfaces Wow

1:53

It's core promise really is to drastically speed up your development

1:58

They talk about helping you write and deploy software up to like 10 times faster 10

2:02

times

2:03

that's a big claim it is but it's also about running it with high reliability and

2:07

Crucially deep observability. So, you know what's going on

2:11

Observability always important definitely and for people familiar with other tools

2:17

in this space

2:18

Maybe things like retool or temporal pipe dream super blocks

2:22

Windmill positions itself as a really compelling open source alternative. Okay, so

2:28

if I'm getting this right

2:29

it's not just another automation tool, it's more like a a

2:32

Developer superpower for building internal stuff. That's a good way to put it. Yeah.

2:38

So for you listening right now

2:39

Maybe you're looking to streamline tasks build internal apps

2:43

But you don't want to get bogged down in weeks of setup or figuring out complex

2:47

front-end stuff

2:48

Yeah, this sound like it's seriously lowers that barrier to entry exactly. Is that

2:52

where the 10x speed claim comes?

2:53

Yeah, just removing all those traditional hurdles precisely. Yeah that 10x speed

2:57

isn't just about like raw code execution being faster, though

3:00

It is fast. It's really about eliminating the friction friction

3:03

Okay, think about all the time you normally spend building basic ui's setting up

3:07

API's managing deployments trying to connect different services

3:10

Yeah, the plumbing right windmill consolidates a lot of that

3:14

It kind of takes away the need to manually stitch together all these separate bits

3:19

Scripting scheduling API's UI and lets you just focus on the logic the problem you

3:24

actually want to solve that is a powerful promise

3:26

Okay, this is where it gets really interesting for me

3:29

How does windmill actually do this? How does it work?

3:33

What are the main concepts the building blocks someone needs to grasp to turn their

3:37

scripts into these tools?

3:39

Well, it breaks down into a few core

3:42

Very flexible components. Yeah at its heart windmill is all about letting you

3:46

define minimal generic scripts minimal and generic

3:50

Oh, when I say generic, I mean really polyglot

3:52

You can write these scripts in a huge range of languages Python typescript go bash

3:57

SQL even things like PHP rust

4:00

C-shop was quite a list it is and you can write them right inside their web IDE

4:04

Which is super handy or if you prefer your own setup, you can just sync them

4:08

directly with your github repo

4:09

Okay, so you write the script then what then comes one of the really clever bits

4:13

Especially if you aren't a fan of front-end development, uh-huh guilty as charged

4:18

sometimes right so windmill

4:21

Automatically parses your scripts parameters the inputs it needs and it generates a

4:26

basic front-end UI for it automatically automatically

4:29

Imagine you write a Python script. It takes maybe a user's email and a message

4:33

string

4:33

Okay, and poof windmill just gives you a simple web form with input fields for that

4:38

email and message

4:39

It instantly turns your code into a shareable tool that even non-technical people

4:44

can use that's fascinating

4:45

So I just write my script to find the inputs

4:48

Yeah, and windmill kind of figures out the UI doesn't guess input types like this

4:52

should be text. This should be a number exactly

4:55

Yeah, it intelligently infers the types. It gives you sensible defaults and you can

4:59

tweak them if needed. Okay, cool

5:00

So once I have these individual smart scripts, what's the next step? How do I build

5:04

something more complex, right?

5:06

So you've got these building blocks. The next step is chaining them together

5:09

You can link your own scripts or even pull in community shared scripts from

5:13

something they call windmill hub

5:15

Ah a community aspect too. Yeah, and you combine these into flows

5:20

think of flows like a

5:22

Sophisticated recipe or workflow diagram. Okay, technically they're directed a cyclic

5:28

graphs or DAGs

5:29

Which is just a fancy way of saying it's a sequence of steps that run in the right

5:33

order

5:34

No loops going backward, you know

5:36

Like you mix ingredients before you bake the cake got it makes sense and this lets

5:40

you build really powerful automations

5:42

It has built-in stuff for proper error handling automatic retries if a step is flaky

5:46

Loops for processing lists of things conditional branches like if this then do that

5:52

all the logic you need

5:53

It's a real workflow orchestration basically

5:56

Absolutely, and then beyond those auto-generated ui's for single scripts if you

6:00

need something more polished or custom for say a whole process

6:03

There's a low code app editor loco to yeah, it's drag-and-drop

6:08

You get over 60 built-in component buttons tables charts inputs

6:13

And you can visually build quite rich data intensive applications right on top of

6:18

your scripts and flows

6:19

You can go from just a script to a pretty decent internal app surprisingly fast

6:24

That's quite a spectrum from simple script UI to low-code apps it really is and

6:29

finally these scripts and flows

6:30

They don't just sit there waiting for you to click a button. They can be triggered

6:33

the automation part exactly

6:36

You can run them on standard cron schedules like run this every hour

6:40

They can be triggered by web hooks and get this every script and flow automatically

6:44

gets its own web hook URL generated

6:46

Oh, that's huge for integrations totally or you can trigger them via a command line

6:50

interface the CLI or even integrate them with things

6:53

Like slack or incoming emails. It's designed to fit into how you already work. That

6:58

is incredible

6:58

Okay, so let me recap you write a bit of code in your favorite language

7:01

Windmill instantly gives it a UI like a button

7:05

Uh-huh, you can then string those buttons those scripts together into complex flows

7:09

to automate whole processes

7:11

Or even build a full application around them using the low code editor

7:15

Oh all without becoming a front-end wizard or an orchestration expert. You've got

7:19

it. That's the core value proposition

7:21

Simplifying all that complexity. It really does sound like it simplifies things

7:25

Especially for someone who's maybe strong on scripting but less so on the UI or

7:29

infrastructure side

7:30

But it makes me wonder with all that power and flexibility

7:33

What's actually driving it? What's under the hood making it tick? How are they

7:38

getting that speed and

7:39

Importantly security. Yeah excellent question because the architecture underneath

7:45

is actually a big differentiator for windmill

7:47

They really pride themselves on being quote the fastest self-hostable workflow

7:51

engine out there fastest

7:53

Yeah, they published benchmarks showing it can be significantly faster up to 13

7:57

times faster

7:58

They claim than alternatives like say Apache airflow 13 times Wow

8:03

And that applies apparently to both lightweight really quick tasks and those long-running

8:07

heavy background jobs

8:09

That raw speed means your automations just run more efficiently tasks finish faster

8:14

Less waiting fewer resources tied up. So what's the secret sauce there? What's the

8:18

tech stack?

8:19

Okay delving into the tech stack. The back end is primarily written in rust rust

8:24

known for performance and safety

8:26

Exactly using rust gives them that speed memory safety reliability fewer unexpected

8:32

crashes

8:32

They use Postgres a rock-solid database for storage standard robust choice

8:37

Yep, and the front end the UI you interact with is built with svelte, which is

8:42

known for creating really fast responsive interfaces

8:45

Okay, and for running the scripts themselves. That's key to for JavaScript

8:50

They use things like denna core which leverages Google's v8 engine for TypeScript

8:54

They use bun and Dino again very modern fast runtimes and for other languages like

8:59

Python or go

9:00

They use standard battle tested runtimes

9:02

This choice of modern high-performance components is absolutely crucial to how they

9:06

achieve that speed and flexibility and that polyglot support

9:09

You mentioned Python go TypeScript whatever being able to just use what you know

9:13

best sounds like a huge productivity win

9:15

How does it handle dependencies like if my Python script needs specific libraries?

9:20

Yeah, that's managed really smoothly to when you define a script

9:23

You can specify its dependencies right there like a requirements dot txt for Python

9:28

or a package dot JSON for node and TypeScript

9:31

Okay, and windmill just handles it

9:33

It sets up the right environment make sure those dependencies are installed and

9:36

available when the script runs

9:38

You're not messing around with virtual environments or NPM install on the server

9:42

yourself. Oh nice abstraction

9:44

Exactly. It takes care of that setup letting you focus just on your code and this

9:48

makes it really easy to mix languages in a flow

9:51

Because each script runs in its own kind of defined isolated environment isolation

9:57

Yeah, that brings up security

9:59

Especially with internal tools potentially touching sensitive systems or data. How's

10:04

that handled? Yeah, absolutely critical

10:06

They take it very seriously script executions are sandboxed using something called

10:11

Google's ends jail

10:12

Sandboxed like contained precisely sandboxing means each script runs in its own

10:17

little isolated mini environment like a secure container

10:20

So if one script messes up or has some weird vulnerability, it can't easily affect

10:24

other scripts or the main windmill system

10:26

That's vital. It really is. It's the same kind of tech that fly.io uses for example

10:31

for their production environments

10:32

It's considered multi-tenant grade secure. Okay, and what about secrets API keys

10:37

passwords?

10:38

Good point for sensitive data like credentials windmill uses an encryption key

10:43

specific to each workspace to encrypt those secrets before storing them and they

10:48

also strongly recommend as a best practice

10:50

Encrypting the entire Postgres database itself for an extra layer. That's actually

10:54

what they do for their own cloud offering Wow

10:57

Okay, that sounds like a really solid approach to both performance and security

11:01

So let's bring it back to the practical side for someone listening may be thinking.

11:05

Okay, this sounds cool

11:06

I want to try it, but maybe they're intimidated by server setup

11:09

How easy is it to actually get started to just experiment or run a small internal

11:14

project, right?

11:16

And this is another area where windmill really shines, especially for learners or

11:19

smaller teams

11:20

It's designed to be incredibly easy to self-host easy self-hosting. Tell me more

11:25

for a quick start

11:25

Seriously, you can deploy the whole thing with just three configuration files using

11:29

Docker compose. It's literally a single command

11:32

No way. Yeah, you just need Docker running on your machine Mac Windows Linux

11:38

Whatever you grab the Docker dash compose dot AML a caddy file for networking and

11:43

in dot ENV file for settings from

11:46

Their github repo. Okay, then you just run Docker compose up DE in your terminal

11:51

and a voila as they say

11:53

It's up and running available right there at HTTP local host. That's it

11:57

That's basically it for a basic local setup

12:00

The default admin login is admin at windmill dev with the password change obviously

12:05

change that password

12:06

But you can log in and start building flows like within minutes

12:09

That is genuinely minimal setup for what sounds like maximum impact. It really

12:13

lowers the barrier now for more serious or production

12:16

Setups. It also supports kubernetes deployment using Helm charts and you can even

12:19

run it from pre-compiled binaries if you don't want to use Docker

12:21

So it scales up to oh, yeah

12:23

It's designed to scale from you know

12:25

Your laptop or a single virtual machine all the way up to large kubernetes clusters

12:30

managing potentially hundreds or even a thousand worker nodes

12:33

It can grow with your needs and for local development if I want to code scripts in

12:37

my own editor

12:38

They've got you covered there too. There's a powerful command line interface the CLI

12:43

and also a VS code extension

12:46

these let you sync scripts directly from files on your computer or your github repos

12:50

and

12:50

Test and run them locally against your windmill instance. It integrates into a

12:54

typical developer workflow quite nicely

12:57

Okay, one last crucial question on the practical side

12:59

Licensing it's open source. You said yes fully open source under the AGPL v3

13:05

license AGPL v3

13:07

Yeah, what does that mean for someone using it internally good question?

13:11

AGPL basically means if you modify windmill and then distribute that modified

13:15

version to others like offering it as a service

13:17

You generally need to share your modifications under the same AGPL license. It

13:23

ensures the core stays open

13:25

However, they also have what they call the community addition

13:28

This is what you get from the Docker images or the github binary releases

13:33

It actually includes some extra proprietary features built on top of the open

13:38

source core

13:39

ah, so there's a bit more in the easy-to-install versions exactly and

13:43

Importantly this community edition is free to use internally within your own

13:48

organization

13:48

Although there might be certain quotas on some of the advanced proprietary features

13:52

Okay, so if we're just building internal tools for my company, I'm generally good

13:56

to go

13:57

absolutely the main time you'd need a commercial license is if you are planning to

14:01

take parts of windmill and like

14:02

re-expose them as a core feature of your own commercial product that you sell to

14:06

others

14:07

But for simply using windmill internally to run your automations build your ui's

14:12

use its API's

14:13

No commercial license needed. It makes it super accessible for learning

14:18

Experimenting and just running your internal operations

14:22

It really sounds like they've put a ton of thought into lowering that barrier to

14:26

entry even though it's pretty comprehensive platform

14:29

That's a massive plus for you know, the learner

14:33

We talked about somebody who wants to experiment and build powerful stuff without

14:37

getting tangled in a massive scary setup

14:39

Definitely being able to go from zero to building flows in minutes is a game

14:42

changer

14:43

Okay, so let's zoom out again for you are curious listener hearing all this

14:47

What's the biggest takeaway? Why should windmill be on your radar right now?

14:52

What kind of real tangible impact could have on your journey with automation or

14:56

building internal tools?

14:57

Yeah, connecting it back to the bigger picture. I think windmill offers huge value

15:01

in a few key ways

15:02

Yes, like we said, it just significantly accelerates your development process the

15:06

speed factor again, right?

15:07

Imagine that critical report pipeline that used to take a week of coding testing

15:12

Maybe some basic UI with windmill because the UI is instant and the workflow part

15:17

is built in

15:17

Maybe that's a day or two instead. That's a serious difference

15:21

It is and it helps reduce that feeling of infrastructure sprawl where you have

15:26

scripts here scheduled tasks

15:27

There may be a separate UI tool it consolidates the one platform to rule them all

15:32

kind of maybe not rule but certainly simplify

15:35

Second that ability to auto generate you eyes and also build low code apps is

15:41

genuinely transformative

15:43

I think so it means people who aren't developers can easily interact with and

15:47

trigger potentially complex automations that you've built

15:50

It breaks down those silos between say

15:52

engineering and marketing or sales

15:55

Empowerment for other teams exactly. I've seen scenarios where like a simple button

16:00

on a windmill app

16:01

Let's the sales team generate their own custom reports instantly without needing to

16:05

file a ticket with engineering that saves everyone time and hassle

16:08

I can definitely see the value there and finally the open source aspect. That means

16:12

total transparency. You can see the code

16:14

there's a community contributing improvements and

16:17

You have the ultimate flexibility to self-host it customize it own your data own

16:23

your operations full control full control

16:25

Which is just invaluable, especially today

16:27

It's really about empowering you the user to build smarter faster and with much

16:32

more control over your technology stack

16:35

Okay

16:35

So for your final thought today something to chew on after this deep dive

16:38

Think about your own work your own projects or processes

16:42

What complex or slow or just annoying manual task?

16:45

Could you imagine turning into a one-click button or a fully automated workflow

16:49

using something like windmill?

16:51

Consider the possibilities that open up when scripting automation and UI building

16:56

aren't these separate difficult hurdles anymore

16:58

But they become integrated superpowers right at your fingertips. What could you

17:03

build then?

17:03

We really hope this deep dive into windmill has given you some valuable insights

17:07

and maybe spark some ideas

17:09

Until next time keep exploring and once again a huge

17:13

Thank you to safe server for supporting this deep dive and helping with your

17:16

You can find more information at www.safeserver.de

17:16

You can find more information at www.safeserver.de