Systemd 250 Released(lwn.net)
lwn.net
Systemd 250 Released
https://lwn.net/Articles/879739/
203 comments
There is so much truth to what you say. I used Linux exclusively from January 2000 until April 2012. A lot has changed in that time, but out of all the changes the three I'm most thankful for are:
(1) Xorg replacing XFree86 and making video "just work"
(2) Pulse Audio replacing OSS and making audio "just work"
(3) systemd replacing the dog's breakfast of scripting madness each distro built from scratch instead of collaborating
It amazes me that the same person was responsible for leading 2 of the 3 changes.
I for one welcome an increasingly unified operating system core, instead of the random competing and overlapping components we used to have. Sure they worked 97% of the time, but the other 3% when they didn't could be a royal PITA. Especially when two waring components wasted time pointing blame at each other instead of focusing on solving end user issues.
(1) Xorg replacing XFree86 and making video "just work"
(2) Pulse Audio replacing OSS and making audio "just work"
(3) systemd replacing the dog's breakfast of scripting madness each distro built from scratch instead of collaborating
It amazes me that the same person was responsible for leading 2 of the 3 changes.
I for one welcome an increasingly unified operating system core, instead of the random competing and overlapping components we used to have. Sure they worked 97% of the time, but the other 3% when they didn't could be a royal PITA. Especially when two waring components wasted time pointing blame at each other instead of focusing on solving end user issues.
I don't know about (2). Alsa "just worked" when pulse audio was released and PA did not "just work" for like 8 years after the first release.
Seems like software history revisionism to me.
Seems like software history revisionism to me.
While PA in the beginning certainly wasn't manna from heaven, it still was a godsend; pulseaudio exposed all the inconsistencies, flaws and straight out bugs in the alsa driver implementations.
The reason so many people had issues with PA was not PA but the brokenness of the underlying system that PA exposed for the first time when trying to use all these drivers in a systematic way and build functionality on top of it. It took years of fixing all the drivers and PA - unfortunately - got a lot of undeserved flak.
PA is the reason that Linux has a halfway decent audio subsystem, being able to deal with all the modern devices; thank you, PA devs.
I find it hilarious that people are festive about PA being replaced by Pipewire - if PA hadn't paved the way, the Linux driver landscape would not be in such a good shape today and Pipewire would 'fail' in exactly the same way that PA 'failed.
The reason so many people had issues with PA was not PA but the brokenness of the underlying system that PA exposed for the first time when trying to use all these drivers in a systematic way and build functionality on top of it. It took years of fixing all the drivers and PA - unfortunately - got a lot of undeserved flak.
PA is the reason that Linux has a halfway decent audio subsystem, being able to deal with all the modern devices; thank you, PA devs.
I find it hilarious that people are festive about PA being replaced by Pipewire - if PA hadn't paved the way, the Linux driver landscape would not be in such a good shape today and Pipewire would 'fail' in exactly the same way that PA 'failed.
I'm grateful we got a reasonable system out of PA eventually but damn it took time and I agree it is today better than what we had with ALSA. But as I remember it audio sucked with ALSA until it didn't and then we had about two good years of audio before PA came about. Then we had 8~ years of crap again and yes a lot of it was definitely due to really bad drivers and masked inconsistencies in the past.
That's not my point tho, my point is that PA didn't sweep in and save us, in fact it at first pushed us back A LOT. And claiming otherwise is a disservice to ourselves when we consider additional radical change. Such as Pipewire and and Wayland.
That's not my point tho, my point is that PA didn't sweep in and save us, in fact it at first pushed us back A LOT. And claiming otherwise is a disservice to ourselves when we consider additional radical change. Such as Pipewire and and Wayland.
> in fact it at first pushed us back A LOT
That argument I don't quite follow. Would you consider rephrasing it? My question is; how did PA set back Linux sound if it did expose all those flaws in the audio driver in the first place?
That argument I don't quite follow. Would you consider rephrasing it? My question is; how did PA set back Linux sound if it did expose all those flaws in the audio driver in the first place?
Not everyone is a power user. At this time I was in university and Linux went from a reasonable choice to an unreasonable due to sound not working again.
If you care about real world usage, PA set us back years when Ubuntu made it default.
If you care about real world usage, PA set us back years when Ubuntu made it default.
> Linux went from a reasonable choice to an unreasonable due to sound not working again.
But that's the crux, isn't it? Linux sound was not not-working because of PA. Linux sound was not working because it was fundamentally broken with drivers that had wildly different behaviors. Everyone was hacking around broken drivers all the time, I distinctly remember how I had to recompile the kernel back then to get some custom-fixes for my setup in because the driver was so kaput.
Only with the introduction of PA was there finally a way to test and evaluate the drivers in a systematic fashion.
So, in short, my point is this; any software that - in this broken state of Linux audio - tried to introduce a software that made available the features the drivers proclaimed would have run into exactly the same problem as PA. Because PA didn't introduce the problem, PA made it visible.
It's like blaming the doctor for diagnosing the illness.
So, while your experience was that sound suddenly wasn't working anymore because Ubuntu configured the default setup to use PA, for lots of people that sound never worked this was fine - because their drivers weren't broken.
But that's the crux, isn't it? Linux sound was not not-working because of PA. Linux sound was not working because it was fundamentally broken with drivers that had wildly different behaviors. Everyone was hacking around broken drivers all the time, I distinctly remember how I had to recompile the kernel back then to get some custom-fixes for my setup in because the driver was so kaput.
Only with the introduction of PA was there finally a way to test and evaluate the drivers in a systematic fashion.
So, in short, my point is this; any software that - in this broken state of Linux audio - tried to introduce a software that made available the features the drivers proclaimed would have run into exactly the same problem as PA. Because PA didn't introduce the problem, PA made it visible.
It's like blaming the doctor for diagnosing the illness.
So, while your experience was that sound suddenly wasn't working anymore because Ubuntu configured the default setup to use PA, for lots of people that sound never worked this was fine - because their drivers weren't broken.
Alsa is fine, if you needed absolutely no features whatsoever. This weird glamorization of alsa as sufficient or at all adequate feels like itcs true, for only rhe most basic basic basic of users.
Anyone who has ever plugged in a usb headset or connected a bluetooth headset or plugged an hdmi output into their laptop knows the value of pulseaudio. It's role as a modular, pluggable, controllable audio intermediary layer is invaluable: it has a great control panel that lets you say, ok app, start sending your output somewhere else now. Even if you only have one soundcard ever: it let you adjust per app volume! Utterly basic need. To propose that users ought have been fine without these basic capabilities is insanity. Alsa was insanity.
Alsa was a dead, impotent, lifeless husk. It let apps output audio to a device, if you knew to try various hardcoded strings like hw:1.0, hw:2.0, hw:2.1. But that was all alsa was good for. One couldnt open a control panel & change the volume of an app. One couldnt send an app to a new output (bluetooth, hdmi, usb audio). Trying to get a sound card to pick between various output modes, for optical output for example, was hell: i spent literally days getting optical audio going. Everything required careful scripting in a shitty awful .alsarc dsl to do anything good.
Pulseaudio made it all much easier.
I have no clue why anyone would valorize alsa. It was incredible finnicky, super hard to configure. It feels like certain sects just want to spread poison against linux & freedesktop. There have always been reactive, outraged camps. Eternally dismayed at progress, at better. I just can't take seriously the thought that alsa was an appropriate & adequate tool for end users, in any way though. It's ridiculous: nothing worked without careful careful delicate scripting & trial by error, and the system was utterly static, unable to respond to change. Alsa was not an end user system, unlike pulseaudio: it was an api for software. Pulseaudio changed the game. I am so tired lf hearing pretenses that alsa was ok or enough. It never was amything remotely adequate. The nonsense hate has to die. Progress was required. I saw it work flawlessly on many systems even in the earliest days. It wasn't & still usually isn't required. I don't see what anyone could be asking for. It just seems like a need to sow doubt.
Anyone who has ever plugged in a usb headset or connected a bluetooth headset or plugged an hdmi output into their laptop knows the value of pulseaudio. It's role as a modular, pluggable, controllable audio intermediary layer is invaluable: it has a great control panel that lets you say, ok app, start sending your output somewhere else now. Even if you only have one soundcard ever: it let you adjust per app volume! Utterly basic need. To propose that users ought have been fine without these basic capabilities is insanity. Alsa was insanity.
Alsa was a dead, impotent, lifeless husk. It let apps output audio to a device, if you knew to try various hardcoded strings like hw:1.0, hw:2.0, hw:2.1. But that was all alsa was good for. One couldnt open a control panel & change the volume of an app. One couldnt send an app to a new output (bluetooth, hdmi, usb audio). Trying to get a sound card to pick between various output modes, for optical output for example, was hell: i spent literally days getting optical audio going. Everything required careful scripting in a shitty awful .alsarc dsl to do anything good.
Pulseaudio made it all much easier.
I have no clue why anyone would valorize alsa. It was incredible finnicky, super hard to configure. It feels like certain sects just want to spread poison against linux & freedesktop. There have always been reactive, outraged camps. Eternally dismayed at progress, at better. I just can't take seriously the thought that alsa was an appropriate & adequate tool for end users, in any way though. It's ridiculous: nothing worked without careful careful delicate scripting & trial by error, and the system was utterly static, unable to respond to change. Alsa was not an end user system, unlike pulseaudio: it was an api for software. Pulseaudio changed the game. I am so tired lf hearing pretenses that alsa was ok or enough. It never was amything remotely adequate. The nonsense hate has to die. Progress was required. I saw it work flawlessly on many systems even in the earliest days. It wasn't & still usually isn't required. I don't see what anyone could be asking for. It just seems like a need to sow doubt.
Alsa just worked but had limitations. Some apps grabbed sound and you could not play sounds from other apps. Often, one app could play sound at a time. It depended on whether your sound card could handle several streams IIRC. Each major desktop environment had its own sound server to work around these limitations (KDE's aRts, Gnome's ESD).
PulseAudio let us have a common stack instead. There were bugs (PulseAudio was probably a bit buggy at first, and also hit audio driver bugs because of the new ways of using them). The transition was rough and PulseAudio is not perfect too but is it there for reasons and solves many issues.
Now it is being replaced by PipeWire, probably for the best (more efficient, can replace JACK too, making sound management much easier on Linux), but PipeWire leverages experience from PulseAudio.
I lived the transition to PulseAudio and it went well for me and probably a lot (the majority?) of people. Some things could probably have been handled better, especially the (too early?) transition. I think PulseAudio still is/was a good thing.
PulseAudio let us have a common stack instead. There were bugs (PulseAudio was probably a bit buggy at first, and also hit audio driver bugs because of the new ways of using them). The transition was rough and PulseAudio is not perfect too but is it there for reasons and solves many issues.
Now it is being replaced by PipeWire, probably for the best (more efficient, can replace JACK too, making sound management much easier on Linux), but PipeWire leverages experience from PulseAudio.
I lived the transition to PulseAudio and it went well for me and probably a lot (the majority?) of people. Some things could probably have been handled better, especially the (too early?) transition. I think PulseAudio still is/was a good thing.
Consider me in the same camp. When I set up audio on my linux pc back in the day, the only thing I had to do was enable the alsa daemon to restore volume levels at boot and everything just worked. Then later Pulsaudio was kinda forced on me and I didn't (and still don't) understand why it was needed, it was something about allow multiple applications outputting audio at the same time but that worked just fine with alsa? Plus it didn't work well: under load audio would stutter which was extremely annoying as I listened to music quite frequently while working on my computer, a problem I did not have with alsa. So there I was with a shiny AMD Athlon 64 3500+ that couldn't even play mp3's straight.
The thing is, you used to be able to fix something and have it stay fixed. You used to be able to find a config that worked - which, sure, might have required some manual experimentation and customization - but then you could back it up and keep it.
Maybe pulseaudio and systemd only breaks 2% of the time rather than 3% of the time. But when they do break, you can't understand what's happened and you can't fix them, and even if they worked yesterday they won't necessarily work today. If that was tradeoff I was happy with I'd use windows or OSX.
Maybe pulseaudio and systemd only breaks 2% of the time rather than 3% of the time. But when they do break, you can't understand what's happened and you can't fix them, and even if they worked yesterday they won't necessarily work today. If that was tradeoff I was happy with I'd use windows or OSX.
> you used to be able to fix something and have it stay fixed
No, I used to patch bash scripts, patch the deb package, deploy to prod, then have to do it all over again when the init script changed upstream in the distribution. Now I just have my configuration management change the options I need in the unit file and never touch it again, so now I sleep while unattended-upgrades works for me.
No, I used to patch bash scripts, patch the deb package, deploy to prod, then have to do it all over again when the init script changed upstream in the distribution. Now I just have my configuration management change the options I need in the unit file and never touch it again, so now I sleep while unattended-upgrades works for me.
No idea what you are talking about. Pulseaudio nor systemd have wild crazy "it just stopped running" issues, like you suggest.
If something critical launched by your init system isnt coming up, I'd way way rather be using systemd. There's know wwys to inspect, check out what ran, what didnt, see what dependencies are failing. Check the common logging system. In the dark old days it was a nightmare of different daemons, each with their own custom init scripts, & little common reporting. Every problem was unique & required unique diagnosis.
I dont get what the complaints are. Things work great now. There's great diagnostic tools. Most of them with json/machine readible output. Distros sometimes do break various daemons but systemd and pulseaudio are pretty much bulletproof, and if they worked yesterday, i absolutely have faoth they'll work today. I detest so much that this kind of casual easy convenient Fear Uncertainty g Doubt shade throwing persists, that we so casually malign while making no assertions that could be refuted. There's a spectre of doubt that sabotages the good, that seeds disbelief, and it's never justified, never backed up by anything concrete or real or debateable: it's all just this haunting image that it's not going to work. It's seditious.
If something critical launched by your init system isnt coming up, I'd way way rather be using systemd. There's know wwys to inspect, check out what ran, what didnt, see what dependencies are failing. Check the common logging system. In the dark old days it was a nightmare of different daemons, each with their own custom init scripts, & little common reporting. Every problem was unique & required unique diagnosis.
I dont get what the complaints are. Things work great now. There's great diagnostic tools. Most of them with json/machine readible output. Distros sometimes do break various daemons but systemd and pulseaudio are pretty much bulletproof, and if they worked yesterday, i absolutely have faoth they'll work today. I detest so much that this kind of casual easy convenient Fear Uncertainty g Doubt shade throwing persists, that we so casually malign while making no assertions that could be refuted. There's a spectre of doubt that sabotages the good, that seeds disbelief, and it's never justified, never backed up by anything concrete or real or debateable: it's all just this haunting image that it's not going to work. It's seditious.
That’s no good reason to ditch a superior solution. A modern combustion engine is pretty much impossible to fix with your ordinary tooling/knowledge, but it doesn’t make me want to use some old, weak/inefficient one instead.
Also, I could not do anything with a bug inside firefox, the kernel and a litany of other complex programs (mind that they are complex due to essential complexity) so I don’t see system booting an exception to that.
Also, I could not do anything with a bug inside firefox, the kernel and a litany of other complex programs (mind that they are complex due to essential complexity) so I don’t see system booting an exception to that.
Can you elaborate? Systemd also uses text based config files... how did you calculate the "breaks" percentage? I've been using Fedora with Systemd since a few years now. Nothing that broke was Systemd's fault in any way.
Yeah. Lennart Poettering is one of the unsung heros of modern Linux - and the most maligned one, in my opinion.
While I get that some people are uncomfortable with breaking with traditions, I absolutely do not get all the hostility directed towards this craftsman programmer.
While I get that some people are uncomfortable with breaking with traditions, I absolutely do not get all the hostility directed towards this craftsman programmer.
> (3) systemd replacing the dog's breakfast of scripting madness each distro built from scratch instead of collaborating
Which goes completely against what Linux stands for - collaboration. The "scripting madness" as you call it was actually completely transparent - if the scripting was bad, you were free to fix it. But now you have to issue a bug report when systemd does something stupid and it's not even your fault to Lennart and his gang only to get it dismissed with "no a bug, won't fix".
Also, Pulse Audio only "just works" because the distribution maintainers do that for you. It's still a stinking pile of "don't no how, but it just works".
Keep telling yourself that Lennart does a great job. Stockholm syndrom much?
Which goes completely against what Linux stands for - collaboration. The "scripting madness" as you call it was actually completely transparent - if the scripting was bad, you were free to fix it. But now you have to issue a bug report when systemd does something stupid and it's not even your fault to Lennart and his gang only to get it dismissed with "no a bug, won't fix".
Also, Pulse Audio only "just works" because the distribution maintainers do that for you. It's still a stinking pile of "don't no how, but it just works".
Keep telling yourself that Lennart does a great job. Stockholm syndrom much?
Administering hundreds of linux servers is my job and has been for the past 20 years. Fixing bash scripts left and right was a nightmare. I love bash, but not for this. The right tool for the job and all that. Systemd made my job la lot easier. So yes, I do think Lennart does a great job.
> Stockholm syndrom much?
Yes, but not how you'd imagine. I spent years as a hard core Linux fanboy. Advocating for Linux. Declaring Linux the future when many thought Windows would obviously eliminate all competition. Volunteering for install fests and wrestling various distros onto unwilling hardware.
I also spent 9 years teaching and writing training materials for RHEL, SUSE, & Ubuntu. As a result I dug deeper into core system details than most. (How many remember SUSE's attempt to achieve faster boot times by introducing an optional alternative to normal init that dynamically generated a makefile to execute init scripts in parallel? That was one of many strange rabbit holes I got to explore.)
Thankfully, I've gotten over it. I still prefer to keep Linux near the center of my career, but I've been able to adapt as "the cloud" has rendered my old school sysadmin skills less relevant. Occasionally I call on them to rescue a pet, but today I find "herding cattle" and going "serverless" much more enjoyable.
And you?
Yes, but not how you'd imagine. I spent years as a hard core Linux fanboy. Advocating for Linux. Declaring Linux the future when many thought Windows would obviously eliminate all competition. Volunteering for install fests and wrestling various distros onto unwilling hardware.
I also spent 9 years teaching and writing training materials for RHEL, SUSE, & Ubuntu. As a result I dug deeper into core system details than most. (How many remember SUSE's attempt to achieve faster boot times by introducing an optional alternative to normal init that dynamically generated a makefile to execute init scripts in parallel? That was one of many strange rabbit holes I got to explore.)
Thankfully, I've gotten over it. I still prefer to keep Linux near the center of my career, but I've been able to adapt as "the cloud" has rendered my old school sysadmin skills less relevant. Occasionally I call on them to rescue a pet, but today I find "herding cattle" and going "serverless" much more enjoyable.
And you?
> Which goes completely against what Linux stands for - collaboration.
Have a look at the contributor stats[0] for systemd and then repeat that with a straight face.
> Lennart and his gang
Also known as the Linux Plumbers[1]. You know, the guys that actually tie everything together and that enjoy the well-deserved trust of most distributions and users.
> dismissed with "no a bug, won't fix".
Usually with very good technical reasons.
> The "scripting madness" as you call it was actually completely transparent - if the scripting was bad, you were free to fix it.
You're still free to fix it; systemd is open-source.
[0] https://github.com/systemd/systemd/graphs/contributors
[1] https://www.linuxplumbersconf.org/
Have a look at the contributor stats[0] for systemd and then repeat that with a straight face.
> Lennart and his gang
Also known as the Linux Plumbers[1]. You know, the guys that actually tie everything together and that enjoy the well-deserved trust of most distributions and users.
> dismissed with "no a bug, won't fix".
Usually with very good technical reasons.
> The "scripting madness" as you call it was actually completely transparent - if the scripting was bad, you were free to fix it.
You're still free to fix it; systemd is open-source.
[0] https://github.com/systemd/systemd/graphs/contributors
[1] https://www.linuxplumbersconf.org/
As someone who has learned Linux/Unix just the past few months using a systemd based distro, I find it a breeze to use. Great documentation, FOSS, and really easy to pick up. I just don’t understand why there is so much dislike thrown around when that energy can be focused elsewhere on truly bad behavior/software. And from my understanding, there are distros out there that don’t use systemd for those who dislike it that much. It’s really perplexing to me as someone newer to the community.
I'm an old school unixhead who's nonetheless made his peace with systemd because, honestly, it makes lots of things work way better than most previously available solutions did.
Thing is, it also forcibly changed a bunch of things by introducing defaults that were not at all what people expected. They might have been better overall, but it still caused some nasty surprises.
Let me try and explain a bit more concretely -
An easy example (not a hypothetical one, btw): if you have a physical server with multiple hard drives and one of them doesn't start, old school unix would boot everything it -could- boot anyway. SystemD changed that behaviour to "if a filesystem doesn't come up, and you don't set an option to tell systemd it's ok if that doesn't happen, don't finish booting".
Now, imagine you're somebody who has a physical server somewhere that has a big cache disk that you use a cheap drive for because if it fails, well, you've lost some cache space. Now imagine your server used to run a pre-systemd setup, and when you upgraded it to a systemd using version of the same distro you left all your settings as-is because everything seemed to be working fine. Now ... imagine that drive dies during a reboot ... and so the server doesn't finish booting, and so you can't even ssh in to it to figure out what's going on, and the only way to figure it out is to get physically in front of the console.
If that server is four hours' drive away, you might not be very impressed.
I still, overall, think systemd is pretty nice. But while some of the people annoyed with it are annoyed with it on a purely aesthetic/principle sort of basis, there are definitely some decisions the authors made that caused -very- surprising results for people who had been running servers for years already, and some of the dislike being thrown around is very much understandable.
Thing is, it also forcibly changed a bunch of things by introducing defaults that were not at all what people expected. They might have been better overall, but it still caused some nasty surprises.
Let me try and explain a bit more concretely -
An easy example (not a hypothetical one, btw): if you have a physical server with multiple hard drives and one of them doesn't start, old school unix would boot everything it -could- boot anyway. SystemD changed that behaviour to "if a filesystem doesn't come up, and you don't set an option to tell systemd it's ok if that doesn't happen, don't finish booting".
Now, imagine you're somebody who has a physical server somewhere that has a big cache disk that you use a cheap drive for because if it fails, well, you've lost some cache space. Now imagine your server used to run a pre-systemd setup, and when you upgraded it to a systemd using version of the same distro you left all your settings as-is because everything seemed to be working fine. Now ... imagine that drive dies during a reboot ... and so the server doesn't finish booting, and so you can't even ssh in to it to figure out what's going on, and the only way to figure it out is to get physically in front of the console.
If that server is four hours' drive away, you might not be very impressed.
I still, overall, think systemd is pretty nice. But while some of the people annoyed with it are annoyed with it on a purely aesthetic/principle sort of basis, there are definitely some decisions the authors made that caused -very- surprising results for people who had been running servers for years already, and some of the dislike being thrown around is very much understandable.
That's a fantastic explanation, thanks! Yep, I can definitely understand the dislike of software coming with defaults that break commonly understood precedent.
ILOM/iDRAC/etc largely solved that sort of thing, but there was always a chance that grub config became corrupted eg when upgrading kernels, leaving you with a 4-hour drive. Pid 1 is critical, and it's ridiculous that it took until systemd for it to stop being something that was just hacked together instead of a well-supported (if surprising at time) software package. It was a real "cobbler's son has no shoes" moment when I first realized back in the RHEL 4 days that the /etc/init.d/ scripts I was looking at were the actual implementation and not some abstraction.
> SystemD changed that behaviour to "if a filesystem doesn't come up, and you don't set an option to tell systemd it's ok if that doesn't happen, don't finish booting".
I still have to find a way to let it digest sshfs filesystems that may hang at boot without stopping the whole boot process.
I still have to find a way to let it digest sshfs filesystems that may hang at boot without stopping the whole boot process.
Systemd by itself is alright, the problem for me is the larger movement from "Linux is about choice" to tightly integrated, opinionated tools. Examples:
- Gnome started to depend on systemd features. Now non-systemd systems become second class citizens.
- You used to be able to freely swap out window managers, taskbars, and so on, because everything was working with common standards. In the modern world, everything is integrated in your wayland compositor, and if you want to change the way window decorations look, you will also have to change everything else. Systemd is similar, it wants to take over so many responsibilities (task scheduling, DNS, session management), and if you want to swap out one part it is going to be difficult.
- Pulseaudio, NetworkManager, Polkit: Written in the same spirit, in part from the same people. Work great if everything works as expected, if they break it is a nightmare to figure out why. But code depends on them, so they become de facto standard on desktop linux.
- Gnome started to depend on systemd features. Now non-systemd systems become second class citizens.
- You used to be able to freely swap out window managers, taskbars, and so on, because everything was working with common standards. In the modern world, everything is integrated in your wayland compositor, and if you want to change the way window decorations look, you will also have to change everything else. Systemd is similar, it wants to take over so many responsibilities (task scheduling, DNS, session management), and if you want to swap out one part it is going to be difficult.
- Pulseaudio, NetworkManager, Polkit: Written in the same spirit, in part from the same people. Work great if everything works as expected, if they break it is a nightmare to figure out why. But code depends on them, so they become de facto standard on desktop linux.
Linux has never been about choice. There was choice because no two people could agree on a set of protocols and standards. And slowly we're converging on a small set of decent (not perfect!) ideas volunteers can focus their efforts on. I honestly hope the choice meme dies because it is so toxic.
My Linux workstation and servers are not something I want to tinker with, I want solidity and stability and a fragmented user space ecosystem is not the solution.
My Linux workstation and servers are not something I want to tinker with, I want solidity and stability and a fragmented user space ecosystem is not the solution.
"linux" (as in: Unixoid distributions for desktop systems) has always been about choice. Debian's update-alternatives, the whole FreeDesktop project, ICCCM standards. The theming community, gnome-look.org, kde-look.org, and so on.
If I want the stabilty and flexibility of Linux (the kernel) and the GNU command line tools, and the flexibility to choose a UX paradigm that I prefer and visuals that I like, I go to desktop Linux. In that sense, choice is the USP of linux (linux as in: desktop distribution). If I loose the flexibilty, I go back to Windows or Mac. Opinionated ("not fragmented") systems like Gnome only work if you're fully on board with their choices. If not, then there is really no benefit for me over just sticking with windows.
Now, on a server the situation is of course different, and stability is paramount. I like for example that services are declarative and not imperative in systemd. But on a desktop, I don't really care.
If I want the stabilty and flexibility of Linux (the kernel) and the GNU command line tools, and the flexibility to choose a UX paradigm that I prefer and visuals that I like, I go to desktop Linux. In that sense, choice is the USP of linux (linux as in: desktop distribution). If I loose the flexibilty, I go back to Windows or Mac. Opinionated ("not fragmented") systems like Gnome only work if you're fully on board with their choices. If not, then there is really no benefit for me over just sticking with windows.
Now, on a server the situation is of course different, and stability is paramount. I like for example that services are declarative and not imperative in systemd. But on a desktop, I don't really care.
> "linux" (as in: Unixoid distributions for desktop systems) has always been about choice.
Again, no. You call it choice, I call it chaos, it was anarchy, and that's the default state for any loosely-coupled systems. There is no manifesto, no underlying philosophy that decided that yes, Linux is about choice. Don't conflate lack of organisation with choice.
Linux has always been about creating a libre UNIX-like kernel for x86 processors. And converging toward the same set of libre techonologies isn't against the Linux spirit.
I still don't understand why people keep harping on about choice. I've used Linux full time since 2001, and the anarchy has just made a libre desktop an utopia because everyone wants every software to support every single option, with no understanding that the number of bugs scales exponentially with the number of choices.
Again, no. You call it choice, I call it chaos, it was anarchy, and that's the default state for any loosely-coupled systems. There is no manifesto, no underlying philosophy that decided that yes, Linux is about choice. Don't conflate lack of organisation with choice.
Linux has always been about creating a libre UNIX-like kernel for x86 processors. And converging toward the same set of libre techonologies isn't against the Linux spirit.
I still don't understand why people keep harping on about choice. I've used Linux full time since 2001, and the anarchy has just made a libre desktop an utopia because everyone wants every software to support every single option, with no understanding that the number of bugs scales exponentially with the number of choices.
> I still don't understand why people keep harping on about choice.
Because they don't want the choices other people made for them.
Because they don't want the choices other people made for them.
Fragmentation is not a skippable problem though. And especially when you depend on lower level services, choice without a standard interface will create an exponential explosion of should-be-supported APIs, which is terrible for an already thin community.
> Linux has never been about choice. There was choice because no two people could agree on a set of protocols and standards. And slowly we're converging on a small set of decent (not perfect!) ideas volunteers can focus their efforts on. I honestly hope the choice meme dies because it is so toxic.
So everyone who uses or develops alternative init systems like runit and openrc and alternative DEs like KDE and package managers like apk and pacman should just stop and work on and use systemd, GNOME, and flatpak instead because that is the One True Way of using Linux on desktop?
Quite a power trip attitude I'd say. I can't imagine why I would use Linux if choices I don't agree with were shoved down my throat.
So everyone who uses or develops alternative init systems like runit and openrc and alternative DEs like KDE and package managers like apk and pacman should just stop and work on and use systemd, GNOME, and flatpak instead because that is the One True Way of using Linux on desktop?
Quite a power trip attitude I'd say. I can't imagine why I would use Linux if choices I don't agree with were shoved down my throat.
Wayland doesn’t mandate anything, it’s a goddamn protocol to put a rectangular window’s content to the screen. It is also freely extendable and has a very sane feature discovery in-built so basically any additional functionality can be created through a simple API (and many such exist already)
But can you run a remote application from another machine on your own machine? This is my biggest obstacle with Wayland. Xorg did it perfectly.
Yes you can with proper compression and usual streaming. X is no longer network transparent anyways (everyone uses proper GPU-acceleration with a litany of bitmaps so it’s not as easy as serialize drawing commands)
So if I SSH into my remote machine, with the appropriate forwarding agents and wayland in place I can run evolution?
Ok I will give that a go.
That reminds me when will Wayland be able to support disparate displays say one big display on 4k with 30 bit colour and two others on QHD and 24 bit colour?
Ok I will give that a go.
That reminds me when will Wayland be able to support disparate displays say one big display on 4k with 30 bit colour and two others on QHD and 24 bit colour?
Disclaimer: I _still_ like systemd.
My only real big pet peeve with Systemd is the documentation is somewhat lacking..., like it's all there in the manual, but its laid out poorly and makes it hard to understand. The interaction between unit files and dependencies is also quite convoluted when you start digging into how things are ordered. There's a ton of unit file options that enable weird conditional behavior. The moment you need to start comprehending the difference between Wants vs. Requires and not hooking into running when multi-user.target is reached (the most common situation) you are in for a world of trial-by-error testing.
Systemd is great for starting simple background services on simple desktop and server environments. It becomes a real pain in the ass once you get beyond that.
My only real big pet peeve with Systemd is the documentation is somewhat lacking..., like it's all there in the manual, but its laid out poorly and makes it hard to understand. The interaction between unit files and dependencies is also quite convoluted when you start digging into how things are ordered. There's a ton of unit file options that enable weird conditional behavior. The moment you need to start comprehending the difference between Wants vs. Requires and not hooking into running when multi-user.target is reached (the most common situation) you are in for a world of trial-by-error testing.
Systemd is great for starting simple background services on simple desktop and server environments. It becomes a real pain in the ass once you get beyond that.
> The moment you need to start comprehending the difference between Wants vs. Requires and not hooking into running when multi-user.target is reached (the most common situation) you are in for a world of trial-by-error testing.
And the worst is that the tooling to test is very limited ( non-existent). I had a problem with conflicting/circular ndependencies ( my service had to be launched before networking but after dbus, which is just impossible because dbus needs networking itself), which worked 9 times out of 10. It was nearly impossible to debug, i had to use the graph tool that shows you visually how much time each service took to start, and completely by chance i noticed that dbus started after networking, which led me to look into it.
And the worst is that the tooling to test is very limited ( non-existent). I had a problem with conflicting/circular ndependencies ( my service had to be launched before networking but after dbus, which is just impossible because dbus needs networking itself), which worked 9 times out of 10. It was nearly impossible to debug, i had to use the graph tool that shows you visually how much time each service took to start, and completely by chance i noticed that dbus started after networking, which led me to look into it.
[deleted]
If forcibly "updating" my system so that it doesn't boot doesn't count as "truly bad behavior/software", I don't know what does. The difference between systemd and outright ransomware is only a matter of degree (frankly I'd prefer paying a bit of cash to having to learn a new set of config files and admin commands because someone decided they didn't like the old ones).
(I don't particularly blame him for the specific non-booting bug, but I do blame him for the arrogance. Everyone makes mistakes, but most people learn from them and develop a bit of humility. Pottering continues to break people's computers every couple of years)
(I don't particularly blame him for the specific non-booting bug, but I do blame him for the arrogance. Everyone makes mistakes, but most people learn from them and develop a bit of humility. Pottering continues to break people's computers every couple of years)
> Become the internet's villain.
This is not about the problems systemd solves, but how they solve them and how they act while solving them.
This is not about the problems systemd solves, but how they solve them and how they act while solving them.
not to mention the new problems it introduces
The most important of them being flamewars for the past ten years stirred up by those who have nothing better to do.
could you please elaborate on the new problems?
Take the best coder you know, take their bugs-per-line-of-code rate, and multiply it by 713,364 (systemd *.h *.c)
Since all software has bugs, that doesn't make it a new problem now does it?
”All software has bugs, but some are more buggy than others.”
Systemd is exceptionally voluminous, fast moving, feature-laden (which is part of why it is so exciting to so many people), and buggy.
Systemd is exceptionally voluminous, fast moving, feature-laden (which is part of why it is so exciting to so many people), and buggy.
systemd had way more bugs and crinkles for an init system when it first started, and an init system is critical software.
Everything is buggy the newer it is, your point being?
Linux itself was mostly unusable when it first started. And a kernel is even more important.
Linux itself was mostly unusable when it first started. And a kernel is even more important.
> Everything is buggy the newer it is, your point being?
There's a distinction between labeling it as a hobby project and something ready for production.
Linux started as a hobby OS, and being buggy is OK in that regard.
systemd is way more buggy than it should when it's deemed production ready. This is my point.
I'm using Linux for a long time, and I know the meanings of unstable, buggy and experimental; as well as production ready.
I don't hate or I'm not against systemd, I'm against the attitude behind it.
There's a distinction between labeling it as a hobby project and something ready for production.
Linux started as a hobby OS, and being buggy is OK in that regard.
systemd is way more buggy than it should when it's deemed production ready. This is my point.
I'm using Linux for a long time, and I know the meanings of unstable, buggy and experimental; as well as production ready.
I don't hate or I'm not against systemd, I'm against the attitude behind it.
Is Lennart/Red Hat meant to be the internet's villain? Perhaps that's the view of a small minority.
I for one am very grateful for Lennart/Red Hat's vast contributions. Leading modern Linux.
I for one am very grateful for Lennart/Red Hat's vast contributions. Leading modern Linux.
Be aware that "modern" has a very negative connotation for a lot of people now.
“Offering some extra functionality” is a novel way of describing how systemd has expanded since being anointed as everyone’s init-replacement.
Systemd broke my ability to login, once. That was fun.
Systemd broke my ability to login, once. That was fun.
More like replace a working system with a more unreliable one with stability problems (and prioritize more features instead of fixing it) and more resource usage, then push for its adoption, then complain when you get hate.
Now yes, it gives you more functionality, but it was a pain until it got there. Systemd was just Pulseaudio all over again (except now in init, so if it breaks your whole system can't boot).
Now, yes. initd is jurassic. Systemd got stable eventually and has gone beyond it. And this is more complicated than it seems. But still, it's the attitude that made things hard.
Now yes, it gives you more functionality, but it was a pain until it got there. Systemd was just Pulseaudio all over again (except now in init, so if it breaks your whole system can't boot).
Now, yes. initd is jurassic. Systemd got stable eventually and has gone beyond it. And this is more complicated than it seems. But still, it's the attitude that made things hard.
There is no reason to be so dramatic. In the end you are free to use any init system you want.
> In the end you are free to use any init system you want.
Not really
https://wiki.gentoo.org/wiki/Hard_dependencies_on_systemd
Not really
https://wiki.gentoo.org/wiki/Hard_dependencies_on_systemd
Honestly this list is way shorter than I would have expected before clicking, and most of it is very niche stuff (not on the scale of "the entirety of GNOME", for example). Also, some of the entries have notes like "likely usable without systemd if someone puts in the work".
Good catch, for reference I'm surprised they fixed this (which was my main worry):
> For versions 3.28 and earlier, the GNOME desktop environment contained components that required systemd-logind[5] (among them, notably, its display manager, GDM). However, as of version 3.30, Gentoo's packaging of GNOME allows it to work once again with sysvinit + OpenRC as the init system
> For versions 3.28 and earlier, the GNOME desktop environment contained components that required systemd-logind[5] (among them, notably, its display manager, GDM). However, as of version 3.30, Gentoo's packaging of GNOME allows it to work once again with sysvinit + OpenRC as the init system
I was happy ignoring systemd (and no, not depending on broken debian/redhat rc.d scripts) till my distro gave update that due to GNOME 3.8 they are forced to switch everything to systemd instead of keeping it optional like before.
It became a question of "are you going to have GNOME 3.8 or later, or not"
It became a question of "are you going to have GNOME 3.8 or later, or not"
Not really, I currently run Gnome 3.40 without needing systemd on my machine. How? I use Gentoo.
Gentoo devs are sometimes a bit dedicated. Sincerely, long time user of Gentoo :)
You make a valid observation. However I doubt that most Systemd haters are champions of commercial bloat. If I had to generalize, I'd assume that they dislike both.
Half of systemd's problems are coming from human aspects rather than technical.
I'm too tired to write them all over again, but here's a rundown, from me.
- https://news.ycombinator.com/item?id=25611149
- https://news.ycombinator.com/item?id=27651567
I'm too tired to write them all over again, but here's a rundown, from me.
- https://news.ycombinator.com/item?id=25611149
- https://news.ycombinator.com/item?id=27651567
Personally I use Gentoo with OpenRC. It's basically the opposite of bloat.
Yeah, it's about whom to target. Open source overlords are hard to satisfy, to be honest.
systemd is a cuckoo bird that found the perfect nest, init. Not-a-hero.
I just want to pre-empt the inevitable bitching about systemd to say I think it’s awesome and incredibly useful and consistent and powerful and I’d take it any day over the mess of stuff that filled systemd’s duties before.
Sure it isn’t perfect, but what software is?
And let me ask a question, if systemd is crap, why does every major distro use it? If it was truly crap then surely the distro builders would use the alternative?
Sure it isn’t perfect, but what software is?
And let me ask a question, if systemd is crap, why does every major distro use it? If it was truly crap then surely the distro builders would use the alternative?
> And let me ask a question, if systemd is crap, why does every major distro use it? If it was truly crap then surely the distro builders would use the alternative?
Because it's tightly coupled to everything (particularly to the few profitable parts of linux - most open-source is maintained on a shoestring, so it's pretty cheap to take control over development of even vital low-level infrastructure). This is how systemd bypassed the bikeshedding problem of there being half a dozen replacement inits but no clear consensus, with the result that everything relied on a lowest common denominator. But it's toxic for the long term, because it undermines the whole point of using linux if you can't swap out pieces or maintain your own forks of individual components.
Because it's tightly coupled to everything (particularly to the few profitable parts of linux - most open-source is maintained on a shoestring, so it's pretty cheap to take control over development of even vital low-level infrastructure). This is how systemd bypassed the bikeshedding problem of there being half a dozen replacement inits but no clear consensus, with the result that everything relied on a lowest common denominator. But it's toxic for the long term, because it undermines the whole point of using linux if you can't swap out pieces or maintain your own forks of individual components.
> But it's toxic for the long term, because it undermines the whole point of using linux if you can't swap out pieces or maintain your own forks of individual components.
You can, though? And many distros do by default.
Don’t like systemd-boot? You can replace it with grub.
Don’t like systemd-networkd? You can replace it with networkctl. Want to use it but prefer to handle Wireguard tunnels with wg-quick? Fine!
Don’t like systemd-resolved? You can replace it with dnsmasq.
Don’t like systemd-cryptsetup? systemd won’t care if some other system takes care of your LUKS partitions.
Etc, etc.
You can, though? And many distros do by default.
Don’t like systemd-boot? You can replace it with grub.
Don’t like systemd-networkd? You can replace it with networkctl. Want to use it but prefer to handle Wireguard tunnels with wg-quick? Fine!
Don’t like systemd-resolved? You can replace it with dnsmasq.
Don’t like systemd-cryptsetup? systemd won’t care if some other system takes care of your LUKS partitions.
Etc, etc.
That’s a short list of other packages (none of which I think are part of a base Linux install, correct me if I’m wrong) that have decided to depend on systemd?
Where's the "hard dependency on libc" list? Or "hard dependency on Bash"?
It's ridiculous to imagine that no program is allowed to depend on SystemD.
It's ridiculous to imagine that no program is allowed to depend on SystemD.
> Where's the "hard dependency on libc" list?
There are multiple libc implementations. libc independence is important and valuable. You'll probably find a "hard dependency on glibc" list in the documentation of e.g. Alpine.
> Or "hard dependency on Bash"?
Ubuntu will have a list from when they switched to ash as /bin/sh. Again, multiple implementations and independence are important, and something the linux community generally cares about.
There are multiple libc implementations. libc independence is important and valuable. You'll probably find a "hard dependency on glibc" list in the documentation of e.g. Alpine.
> Or "hard dependency on Bash"?
Ubuntu will have a list from when they switched to ash as /bin/sh. Again, multiple implementations and independence are important, and something the linux community generally cares about.
> There are multiple libc implementations
You could make another SystemD implementation if you really wanted. The point is that there are plenty of APIs - even ones with a single implementation - that nobody bats an eye about programs having a "hard dependency" on, but suddenly for SystemD it's apparently a big issue?
It's bullshit technical excuses to hide the real reasons people object to SystemD, which are more embarrassing.
You could make another SystemD implementation if you really wanted. The point is that there are plenty of APIs - even ones with a single implementation - that nobody bats an eye about programs having a "hard dependency" on, but suddenly for SystemD it's apparently a big issue?
It's bullshit technical excuses to hide the real reasons people object to SystemD, which are more embarrassing.
> You could make another SystemD implementation if you really wanted.
No you can't, because they don't offer stable interfaces as a matter of deliberate policy.
No you can't, because they don't offer stable interfaces as a matter of deliberate policy.
That seems to be completely untrue: https://systemd.io/PORTABILITY_AND_STABILITY/
That's changed then, now that the competition has been killed off; at the time certainly logind and the cgroup integration had no stable interface.
But no systemd distro is saying “we want to dump systemd but we can’t because it’s too tightly Coupled” are they?
Or maybe the maintainers are I don’t know do you have links to show this is the case?
Systemd is used by all major linux flavors because it’s the best solution for the problems it solves.
The distro maintainers are too smart to be stuck with systemd against their will.
And it’s linux …. make a better solution, if it’s truly better it’ll be used.
Or maybe the maintainers are I don’t know do you have links to show this is the case?
Systemd is used by all major linux flavors because it’s the best solution for the problems it solves.
The distro maintainers are too smart to be stuck with systemd against their will.
And it’s linux …. make a better solution, if it’s truly better it’ll be used.
That's a bit of revisionist history.
Systemd was minority pretty much till GNOME 3.8 introduced super-hard dependencies on it through an interface that was supposedly documented (Ubuntu sunk a bunch of money trying to keep alternative implementation running).
For many a distro, it became a question - "do we include GNOME, or do we drop it?" and that was impetus to move to systemd
Systemd was minority pretty much till GNOME 3.8 introduced super-hard dependencies on it through an interface that was supposedly documented (Ubuntu sunk a bunch of money trying to keep alternative implementation running).
For many a distro, it became a question - "do we include GNOME, or do we drop it?" and that was impetus to move to systemd
This is simply not true[0]. To quote:
> This is really unfortunate, but GNOME 3.8 does not require logind. I discussed the non-dependency of logind+systemd on #gentoo-desktop and why they thought different. Apparently GDM 3.8 assumes that an init system will also clean up any processes it started. This is what systemd does, but OpenRC didn’t support that.
Systemd was adopted because it was (and still is) simply superior in many aspects to previously available process managers, because application developers had it up to here to maintain n different init scripts for n different distributions and distribution packagers didn't want to have to write custom init scripts for each application anymore.
As an example, check out the discussion around the adoption of systemd in Debian[1].
[0] https://blogs.gnome.org/ovitters/2013/09/25/gnome-and-logind...
[1] https://www.debian.org/vote/2019/vote_002
> This is really unfortunate, but GNOME 3.8 does not require logind. I discussed the non-dependency of logind+systemd on #gentoo-desktop and why they thought different. Apparently GDM 3.8 assumes that an init system will also clean up any processes it started. This is what systemd does, but OpenRC didn’t support that.
Systemd was adopted because it was (and still is) simply superior in many aspects to previously available process managers, because application developers had it up to here to maintain n different init scripts for n different distributions and distribution packagers didn't want to have to write custom init scripts for each application anymore.
As an example, check out the discussion around the adoption of systemd in Debian[1].
[0] https://blogs.gnome.org/ovitters/2013/09/25/gnome-and-logind...
[1] https://www.debian.org/vote/2019/vote_002
It might be because people did report on in trying to run GNOME 3.8 without logind, and reported massive faults?
And no, not in "GDM doesn't clean up after itself", which was given as explanation for the infamous "systemd now defaults to destroying everything in a session on logout" change few years later. (Which GDM could require anyway earlier, but apparently somehow it wasn't enough)
There is a lot of good ideas that went into systemd though, and I'd agree that a lot of anger is in how badly they are implemented (the only reason I do not complain more about systemctl is because upstart managed to be even worse). In my copious free time, I've been tinkering at alternative, but even if I disagree at the format of systemd service files, it includes a parser for them.
And no, not in "GDM doesn't clean up after itself", which was given as explanation for the infamous "systemd now defaults to destroying everything in a session on logout" change few years later. (Which GDM could require anyway earlier, but apparently somehow it wasn't enough)
There is a lot of good ideas that went into systemd though, and I'd agree that a lot of anger is in how badly they are implemented (the only reason I do not complain more about systemctl is because upstart managed to be even worse). In my copious free time, I've been tinkering at alternative, but even if I disagree at the format of systemd service files, it includes a parser for them.
> It might be because people did report on in trying to run GNOME 3.8 without logind, and reported massive faults?
If you by 'it' mean the fact that the distributions switched to systemd, I would be declined to discount the speculation that 'it was because people did report in trying to run GNOME 3.8 without logind [...]'. Do you honestly think that Fedora, Arch or any of the other distributions would not try to evaluate other strategies of getting GNOME 3.8 to run? Replacing the process manager and PID0 is not an endeavor chosen lightly - it is a massive undertaking that every distribution would have weight very carefully. "Oh, there are people having trouble running GNOME 3.8 without systemd present" seems not to be enough reason to go through all the pain of switching to systemd.
It's rather that systemd offered something that systemd offered something that the preceding mess of shell scripts and conventions didn't offer; consistency and reliability, as much as you might doubt that. Have a look at the Debian general resolution with regards to systemd and the provided arguments by the proponents[0].
> I've been tinkering at alternative
I would be very glad if your alternative implementation were even modestly successful. And I'd be thoroughly thankful if it managed to replace systemd.
Not because I think systemd is badly implemented (I've yet to see any actual evidence in that regard from any detractor) but because systemd needs competition and I would prefer to see some well established and stable protocols between the individual parts of systemd. That will only happen if said protocols get also provided by alternative implementations.
With regards to you re-using the systemd configuration format; that's probably a smart move. Systemd is at this point a well established fact and anyone trying to best it at its game will have to make is as easy to migrate as possible. Hurdles like a new file format might well be too much for an indifferent user.
[0] https://www.debian.org/vote/2019/vote_002
If you by 'it' mean the fact that the distributions switched to systemd, I would be declined to discount the speculation that 'it was because people did report in trying to run GNOME 3.8 without logind [...]'. Do you honestly think that Fedora, Arch or any of the other distributions would not try to evaluate other strategies of getting GNOME 3.8 to run? Replacing the process manager and PID0 is not an endeavor chosen lightly - it is a massive undertaking that every distribution would have weight very carefully. "Oh, there are people having trouble running GNOME 3.8 without systemd present" seems not to be enough reason to go through all the pain of switching to systemd.
It's rather that systemd offered something that systemd offered something that the preceding mess of shell scripts and conventions didn't offer; consistency and reliability, as much as you might doubt that. Have a look at the Debian general resolution with regards to systemd and the provided arguments by the proponents[0].
> I've been tinkering at alternative
I would be very glad if your alternative implementation were even modestly successful. And I'd be thoroughly thankful if it managed to replace systemd.
Not because I think systemd is badly implemented (I've yet to see any actual evidence in that regard from any detractor) but because systemd needs competition and I would prefer to see some well established and stable protocols between the individual parts of systemd. That will only happen if said protocols get also provided by alternative implementations.
With regards to you re-using the systemd configuration format; that's probably a smart move. Systemd is at this point a well established fact and anyone trying to best it at its game will have to make is as easy to migrate as possible. Hurdles like a new file format might well be too much for an indifferent user.
[0] https://www.debian.org/vote/2019/vote_002
Fedora and Arch are literally the early adopters of systemd, long before GNOME 3.8. GNOME 3.8 made hard dependencies on logind because it was default in Fedora - Arch similarly was working full tilt on implementing all proposed things from systemd long before GNOME 3.8 release (I know, I had lost my only on-hand system due to one of those changes applying badly and making it unbootable)
As for supporting configuration format - it's an obvious move, though more as "import" rather than only source of truth (different internal data models), and the format... is not exactly good (mostly due to lack of sane defaults that start to hit you once you wander beyond functionality that can be summarised as "SysV initd plus dependencies")
As for supporting configuration format - it's an obvious move, though more as "import" rather than only source of truth (different internal data models), and the format... is not exactly good (mostly due to lack of sane defaults that start to hit you once you wander beyond functionality that can be summarised as "SysV initd plus dependencies")
> GNOME 3.8 made hard dependencies on logind because it was default in Fedora
Looks like we're still in disagreement about this. As far as I can tell, there was never a 'hard' dependency of GNOME 3.8 on logind; logind was an optional dependency[0].
> beyond functionality that can be summarised as "SysV initd plus dependencies"
I would attribute that to the lack of similar "service manager" functionality in other "boot managers". That comes down to the hesitancy of these "boot managers" to embrace the role of "service managers". There's tons of functionality that the systemd project (not necessarily the pid0 binary) introduced to make services run consistently and reliably. They paved the way in that regard and we're still waiting for contenders to catch up.
[0] https://mail.gnome.org/archives/desktop-devel-list/2013-Marc...
Looks like we're still in disagreement about this. As far as I can tell, there was never a 'hard' dependency of GNOME 3.8 on logind; logind was an optional dependency[0].
> beyond functionality that can be summarised as "SysV initd plus dependencies"
I would attribute that to the lack of similar "service manager" functionality in other "boot managers". That comes down to the hesitancy of these "boot managers" to embrace the role of "service managers". There's tons of functionality that the systemd project (not necessarily the pid0 binary) introduced to make services run consistently and reliably. They paved the way in that regard and we're still waiting for contenders to catch up.
[0] https://mail.gnome.org/archives/desktop-devel-list/2013-Marc...
Why was it chosen by debian multiple times with the most democratic process out there?
It was forced through by breaking the rules of the process and overruling the committee that should have made that decision. Many old-school Debian maintainers quit in disgust at that point, so if subsequent votes have been in systemd's favour then that's probably why.
Some source would be interesting, because it is just blind words otherwise. Also, Arch and several other distros made the choice independently. I am fairly sure Red Hat is not some evil entity kidnapping the families of maintainers making them switch to systemd..
No rules were broken.[0]
[0] https://www.debian.org/vote/2019/vote_002
[0] https://www.debian.org/vote/2019/vote_002
> And let me ask a question, if systemd is crap, why does every major distro use it?
If Big Mac is a crappy food, why billions are served with it?
If Big Mac is a crappy food, why billions are served with it?
Or maybe you should questions yourself as to why you are so angry about what others like
If others enjoy eating garbage, does that obligate me to like it, too?
Yes, pretty much so. If there is a dish everybody likes but you say it is "garbage" then you are just wrong and an arrogant snob.
A 'Big Mac' is objectively not nutritious, and cheap meat though.
I like systemd.
I like systemd.
> And let me ask a question, if systemd is crap, why does every major distro use it? If it was truly crap then surely the distro builders would use the alternative?
If Windows is crap, why does every system builder ship the majority of their machines with it? Why bother with a different desktop than Windows? Just because it's popular doesn't mean it's good.
Building a distro is a lot of work already, and avoiding systemd takes a lot of effort these days. Plus, there seems to be a lot of people who like systemd, so if you want curmudgeons like me (well, probably not me, I'm going to FreeBSD instead; I understand there may be some sort of fancy rc replacement in the works there, but I presume it won't break the things I expect to work, and it won't subsume half of base) and people who like systemd, then you've got to do twice the work, and everybody will ask you why you're pushing against the tide.
If Windows is crap, why does every system builder ship the majority of their machines with it? Why bother with a different desktop than Windows? Just because it's popular doesn't mean it's good.
Building a distro is a lot of work already, and avoiding systemd takes a lot of effort these days. Plus, there seems to be a lot of people who like systemd, so if you want curmudgeons like me (well, probably not me, I'm going to FreeBSD instead; I understand there may be some sort of fancy rc replacement in the works there, but I presume it won't break the things I expect to work, and it won't subsume half of base) and people who like systemd, then you've got to do twice the work, and everybody will ask you why you're pushing against the tide.
> And let me ask a question, if systemd is crap, why does every major distro use it?
Won't find systemd in Gentoo, Slackware or Alpine. Also it is a little interesting, just a little, that whatever imperative systemd has to exist is entirely absent in every SysV and BSD, and yet variants of SysV and BSD and thier hybrids still exist, somehow, no worse for the wear.
Won't find systemd in Gentoo, Slackware or Alpine. Also it is a little interesting, just a little, that whatever imperative systemd has to exist is entirely absent in every SysV and BSD, and yet variants of SysV and BSD and thier hybrids still exist, somehow, no worse for the wear.
systemd-networkd will now automatically configure routes to addresses specified in AllowedIPs=. This feature can be controlled via RouteTable= and RouteMetric= settings in [WireGuard] or [WireGuardPeer] sections.
I really hope apt will notify users about this when upgrading.
sd-boot gained the ability to change screen resolution during boot-time, by hitting the "r" key. This will cycle through available resolutions and save the last selection.
Overall I learned to stop worrying and love the systemd, especially systemd-networkd. The maintainers can be impressively responsive in acknowledging and fixing reported issues.
Reading this the following comes to mind: Wow systemd breaks important things and solves non-problems, awesome, I love systemd!
Honestly, moving my VPN management from wg-quick+ip-route+iptables to systemd-networkd was a great experience and made things so much easier and predictable. Wish I had done it way sooner. Similarly, after having spent way too much time fiddling with IP and routing in other ways (OpenWRT comes to mind), systemd-networkd made it so much more predictable and straight-forward.
Similar story with other components.
Having to read the release notes before upgrading to a new major version is a price I’m willing to pay.
Similar story with other components.
Having to read the release notes before upgrading to a new major version is a price I’m willing to pay.
Now if only there was a way to reliably test networkd setups...
Sincerely, someone who lost a week or two trying to get anywhere with a bit more complex setup and ended up going back to /etc/interfaces because it had a "show what commands will be run" option
Sincerely, someone who lost a week or two trying to get anywhere with a bit more complex setup and ended up going back to /etc/interfaces because it had a "show what commands will be run" option
> there was reasoning behind not doing that in the first place
What would the reasons be?
What would the reasons be?
SystemD has something of a reputation for being a bunch of arrogant young engineers making fundamental changes to the long established status quo in order to solve clever problems that range from tiny-minority-edge-case to non-existent.
It’s an absolute classic anti-pattern any industry veteran would recognise. Your new-grad engineer, top of their class and incredibly smart, spends two weeks rewriting a core system in Idris without telling anyone.
So accordingly, I was delighted to see the following:
> [files in] systemd-homed home areas are now internally owned by the "nobody" user and dynamically mapped to the UID used locally.
> This makes migrating home areas between different systems cheaper because recursively chown()ing file system trees is no longer necessary.
All that to save a chown -R if I screw my drive into a new PC (or rebuild the current PC around it)?
If I move it to a non systemD system, you’ve just added the need to chown -R all my files because the new OS says they are all owned by nobody.
I don’t even believe for one moment that a recursive chown of 100k files takes any appreciable amount of time. Certainly not when it’s a one off. (5s for 66k files on my crappy VPS just now.)
In any case, the nobody user is a blunt but useful instrument for running daemons as a user that does not own any files. I don’t really want some janky daemon to run as a user which has write access to /home/gorgoiler.
[I have some guacamole prepared, to spread on my inedible but soon to be eaten hat, the moment someone links me to a diff showing the author of this change as Alan Cox.]
[Update: The hat and condiment survive, author is one L Poettering: https://github.com/systemd/systemd/commit/cf5115f6e531c65bbb...
It’s an absolute classic anti-pattern any industry veteran would recognise. Your new-grad engineer, top of their class and incredibly smart, spends two weeks rewriting a core system in Idris without telling anyone.
So accordingly, I was delighted to see the following:
> [files in] systemd-homed home areas are now internally owned by the "nobody" user and dynamically mapped to the UID used locally.
> This makes migrating home areas between different systems cheaper because recursively chown()ing file system trees is no longer necessary.
All that to save a chown -R if I screw my drive into a new PC (or rebuild the current PC around it)?
If I move it to a non systemD system, you’ve just added the need to chown -R all my files because the new OS says they are all owned by nobody.
I don’t even believe for one moment that a recursive chown of 100k files takes any appreciable amount of time. Certainly not when it’s a one off. (5s for 66k files on my crappy VPS just now.)
In any case, the nobody user is a blunt but useful instrument for running daemons as a user that does not own any files. I don’t really want some janky daemon to run as a user which has write access to /home/gorgoiler.
[I have some guacamole prepared, to spread on my inedible but soon to be eaten hat, the moment someone links me to a diff showing the author of this change as Alan Cox.]
[Update: The hat and condiment survive, author is one L Poettering: https://github.com/systemd/systemd/commit/cf5115f6e531c65bbb...
> SystemD has something of a reputation for being a bunch of arrogant young engineers making fundamental changes to the long established status quo in order to solve clever problems that range from tiny-minority-edge-case to non-existent.
With all due respect, but this is bullshit. The devs that started and are working on systemd are clearly veterans of linux system programming and understand quite well the ins and outs of initializing a linux machine. Unless by some chance you're a gray beard that was working on posix in the eighties, I would give the title of veteran to Lennart and the gang.
With all due respect, but this is bullshit. The devs that started and are working on systemd are clearly veterans of linux system programming and understand quite well the ins and outs of initializing a linux machine. Unless by some chance you're a gray beard that was working on posix in the eighties, I would give the title of veteran to Lennart and the gang.
> SystemD has something of a reputation for being a bunch of arrogant young engineers making fundamental changes to the long established status quo in order to solve clever problems that range from tiny-minority-edge-case to non-existent.
Now, I wonder what could be the reason for that? Maybe because some people just trow out unsubstantiated accusations and hearsay as matter-of-fact? Maybe. Who knows.
> If I move it to a non systemD system, you’ve just added the need to chown -R all my files because the new OS says they are all owned by nobody.
Looks like you completely misunderstood the use case. Congratulation.
Now, I wonder what could be the reason for that? Maybe because some people just trow out unsubstantiated accusations and hearsay as matter-of-fact? Maybe. Who knows.
> If I move it to a non systemD system, you’ve just added the need to chown -R all my files because the new OS says they are all owned by nobody.
Looks like you completely misunderstood the use case. Congratulation.
systemd-homed is not intended for your single-user desktop. Don't use it.
What is it intended for? Certainly not large scale user groups backed by an LDAP server, as those would have consistent uid/gid so setting everything to be owned by nobody wouldn't make sense technically...
What is the use case?
What is the use case?
Portable home directories. Think pen drives.
In a cloud setting, I guess having user /home dirs on separate volumes from the root disk and having them relocatable is a reasonable analogue. The real issue is that much like the pen drive thing, absolutely no-one actually does this.
No one? Some people were cared enough about this use case to push it through, so I don't think it's no one. Minority maybe, but not no one.
What place do these niche solutions have in the main project?
In the old days you would write a little script to set up the mapping for your USB drive, once it is plugged in. Linux provides the UID mapping feature, udev gives you a place hook your script, and you glue it together the way you want.
If your solution was particularly high quality it might get included as a package in the OS. Someone else might write a different solution that gets packaged alongside yours. Maybe they scratch the same itch in two different ways? Maybe theirs is just better?
Systemd on the other hand is judge, jury and package maintainer. What they decide upon is what gets included. They are the cathedral, where once there was a bazaar.
In the old days you would write a little script to set up the mapping for your USB drive, once it is plugged in. Linux provides the UID mapping feature, udev gives you a place hook your script, and you glue it together the way you want.
If your solution was particularly high quality it might get included as a package in the OS. Someone else might write a different solution that gets packaged alongside yours. Maybe they scratch the same itch in two different ways? Maybe theirs is just better?
Systemd on the other hand is judge, jury and package maintainer. What they decide upon is what gets included. They are the cathedral, where once there was a bazaar.
I guess the use case is for when you want to migrate home to a different install. At the same time assuming you can't just adjust passwd to make uids work.
So I guess there is none.
So I guess there is none.
This week I have just learned that systemd has an alternative scheduler in addition to cron.
At this rate systemd will probably become another new OS on top of Linux similar to Android.
At this rate systemd will probably become another new OS on top of Linux similar to Android.
You mean systemd timers?
https://wiki.archlinux.org/title/Systemd/Timers
They were always there, and they work much better than cron in my experience (you're not forced to use cron expressions because it supports things like human-readable time formats; they can randomize startup times, which is useful for backups; they can put additional constraints (like "start only if network is available"); and they can actually be controlled from a single place (systemctl list-timers) instead of cron files that can be spread across multiple configs and directories in /etc/cron*, and pretty much every single user profile).
It's a tiny slice of functionality and it makes a lot of sense to keep it in the service manager.
Try harder next time.
https://wiki.archlinux.org/title/Systemd/Timers
They were always there, and they work much better than cron in my experience (you're not forced to use cron expressions because it supports things like human-readable time formats; they can randomize startup times, which is useful for backups; they can put additional constraints (like "start only if network is available"); and they can actually be controlled from a single place (systemctl list-timers) instead of cron files that can be spread across multiple configs and directories in /etc/cron*, and pretty much every single user profile).
It's a tiny slice of functionality and it makes a lot of sense to keep it in the service manager.
Try harder next time.
What I really like is that systemd can tell me when the next execution will happen. One typo in the awful crontab format and your job will never execute with no way of testing it.
My favorite feature of systemd timers is that they get triggered when the time is set by NTP on boot. Combine that with a system without an rtc and a timer to reboot the system every week.
It just works.
Another one that can't be done easily with cron: when your system boots and an execution was missed, systemd timers can be told to run it
Cron has all these features:
- single place control: crontab -e
- additional constraints: systemctl status && scirpt.sh
- randomize startup times: sleep ${RANDOM:0:MAX} && script.sh
The nice thing is that it integrates with the existing infrastructure (shell scripts). This means you don't have to add code to cron to add features.
I agree with the 10s of files in /etc/cron.daily, etc. But those are there for convenience (you can even remove them from /etc/crontab)
- single place control: crontab -e
- additional constraints: systemctl status && scirpt.sh
- randomize startup times: sleep ${RANDOM:0:MAX} && script.sh
The nice thing is that it integrates with the existing infrastructure (shell scripts). This means you don't have to add code to cron to add features.
I agree with the 10s of files in /etc/cron.daily, etc. But those are there for convenience (you can even remove them from /etc/crontab)
There are some cool features in systemd that are not easy to do in cron.
Example: I use restic backup. One timer runs every 24h or on the next boot if there where more than 24h since the last backup as this is a laptop. Also it waits some random amount so that not all script run at the same time after boot. If a backup fails (eg remote server down) it retires every 15min, but no more than one hour before I get an Mail with a detailed report.
All really easy in systemd. No scripting required.
Example: I use restic backup. One timer runs every 24h or on the next boot if there where more than 24h since the last backup as this is a laptop. Also it waits some random amount so that not all script run at the same time after boot. If a backup fails (eg remote server down) it retires every 15min, but no more than one hour before I get an Mail with a detailed report.
All really easy in systemd. No scripting required.
Redhat made the scripting for you. I don't think it's that complicated of a script (although it would be easier with a retry command)
Yes systemd might be slightly easier, but that's because they put a lot of money into systemd, not because systemd is a better architecture. It's nice that they open sourced it (although it does give them a competitive edge), but we can't act like what used to be there isn't as good.
Someone could have packaged utility wrapper scripts in much less time than it did to add those features to systemd, and these wrapper script would be more widely usable (by init scripts, cronjobs, etc)
sleep $random
for i in 1 2 3 4; do
backup && exit 0
sleep 15m
done
logger "backup failed"
exit 1
Yes systemd might be slightly easier, but that's because they put a lot of money into systemd, not because systemd is a better architecture. It's nice that they open sourced it (although it does give them a competitive edge), but we can't act like what used to be there isn't as good.
Someone could have packaged utility wrapper scripts in much less time than it did to add those features to systemd, and these wrapper script would be more widely usable (by init scripts, cronjobs, etc)
No, you are maybe 80% there.
- Your logger does not send a mail with the restic logs to me.
- How do I check that the backup is newer than 24h? I have to make sure the last backup succeeds.
- Do I have to touch a file in /var for every backup script? When do I touch it? After the backup run, before? What if I shutdown my laptop during the backup?
These corner cases take exponentially more time to get right than the first 80%.
"Someone could have packaged utility wrapper scripts in much less time than it did to add those features to systemd"
But nobody has, and I am exhausted to hear how easy it is without systemd without offering a non-hypothetical solution. You don't like systemd, then do not use it and move on. There are distributions without it.
These corner cases take exponentially more time to get right than the first 80%.
"Someone could have packaged utility wrapper scripts in much less time than it did to add those features to systemd"
But nobody has, and I am exhausted to hear how easy it is without systemd without offering a non-hypothetical solution. You don't like systemd, then do not use it and move on. There are distributions without it.
Well it's not a hypothetical solution, the script is probably about the same length as the timer config.
I used logger because it's better practice to log and for the logger to send email, you can send email using the mail command or msmtp. The timing was omitted because it is trivial and didnt serve to illustrate the point, just do backup && touch /var/lib/lastbackup && exit 0, with a if at the start of the script like if [ -x "$(find -mtime -1 /var/lib/lastbackup)" ]
The mid-backup shutdown is a property of the backup script, you'll have to consider that even using systemd.
Sadly there are not many distros without systemd. Everyone is strained in resources and Redhat is pumping money into the ecosystem, so it's really easy to take over most of it. Like I said I don't dislike systemd, I think it adds value, but it's just not the best solution since it does not integrate with others well.
EDIT:
>You don't like systemd, then do not use it and move on.
But this is the issue with systemd: you can't. Want gnome? needs dbus, which needs logind which is systemd. Need libvirt? also needs dbus. Need Y? Needs systemd-tmpfs, which packages the whole systemd as a dependency. Same thing for systemd-udev.
The interlock is so brutal most distros are forced to switch because they can't handle the dev burden.
I used logger because it's better practice to log and for the logger to send email, you can send email using the mail command or msmtp. The timing was omitted because it is trivial and didnt serve to illustrate the point, just do backup && touch /var/lib/lastbackup && exit 0, with a if at the start of the script like if [ -x "$(find -mtime -1 /var/lib/lastbackup)" ]
The mid-backup shutdown is a property of the backup script, you'll have to consider that even using systemd.
Sadly there are not many distros without systemd. Everyone is strained in resources and Redhat is pumping money into the ecosystem, so it's really easy to take over most of it. Like I said I don't dislike systemd, I think it adds value, but it's just not the best solution since it does not integrate with others well.
EDIT:
>You don't like systemd, then do not use it and move on.
But this is the issue with systemd: you can't. Want gnome? needs dbus, which needs logind which is systemd. Need libvirt? also needs dbus. Need Y? Needs systemd-tmpfs, which packages the whole systemd as a dependency. Same thing for systemd-udev.
The interlock is so brutal most distros are forced to switch because they can't handle the dev burden.
Yesterday I learned that it has its own alternative to NTP: systemd-timesyncd. Quite a big dose of NIH.
Solving integration problems well is often harder than implementing functionality.
I second that but unfortunately they implemented only the basic functionality. I quote https://unix.stackexchange.com/a/464729
> systemd-timesyncd does no clock discipline: the clock is not trained or compensated, and internal clock drift over time is not reduced. It has rudimentary logic to adjust poll interval but without disciplining the host will end up with uneven time forever as systemd-timesyncd pushes or pulls at whatever interval it thinks the near-term drift requires. It also can't assess the quality of the remote time source. You're unlikely to get accuracy much greater than 100ms. This is sufficient for simple end user devices like laptops, but it could definitely cause problems for distributed systems that want greater time precision.
> systemd-timesyncd does no clock discipline: the clock is not trained or compensated, and internal clock drift over time is not reduced. It has rudimentary logic to adjust poll interval but without disciplining the host will end up with uneven time forever as systemd-timesyncd pushes or pulls at whatever interval it thinks the near-term drift requires. It also can't assess the quality of the remote time source. You're unlikely to get accuracy much greater than 100ms. This is sufficient for simple end user devices like laptops, but it could definitely cause problems for distributed systems that want greater time precision.
This release should have my small fix for a pet peeve: if any local filesystem failed to fsck, the system used to go into emergency mode even if the fsck entry was marked "nofail" or "noauto". It contradicted intuition and documentation. It came up rarely but was frustrating to have the system broken until you were physically there, even if you tried to avoid this exact behavior.
Now the system boots anyway, just as it would if the filesystem was entirely missing.
Now the system boots anyway, just as it would if the filesystem was entirely missing.
Being able to seal credentials for services like TLS keys to your TPM is extremely dope. I welcome this feature a lot.
minor issue, but anyone know how breaking changes are managed?
between ubuntu16 and ubuntu20 some unit file parameters have changed. I find it rather troubling that we can't maintain consistency.
between ubuntu16 and ubuntu20 some unit file parameters have changed. I find it rather troubling that we can't maintain consistency.
The approach was always "fuck you" and it's bigger part of "lennart hate" than people like to admit...
I'm a Log4Shell survivor and I'm finding it hard to see them packing more and more features into this monolith.
It'll probably be fine, but I'll be Jack's complete lack of surprise when I'm taping servers back together after a DNS exploit gives some kids root access.
It'll probably be fine, but I'll be Jack's complete lack of surprise when I'm taping servers back together after a DNS exploit gives some kids root access.
I recently switched from Ubuntu to FreeBSD, and systemd-networkd is what I miss the most. Network settings in /etc/rc.conf totally feel like a bandaid on a bandaid on a bandaid.
> systemd-homed now makes use of UID mapped mounts for the home areas. If the kernel and used file system support it, files are now internally owned by the "nobody" user (i.e. the user typically used for indicating "this ownership is not mapped"), and dynamically mapped to the UID used locally on the system via the UID mapping mount logic of recent kernels. This makes migrating home areas between different systems cheaper because recursively chown()ing file system trees is no longer necessary.
I wish it was possible to pull a hard drive from one Linux system, and plug it into another, and mount the ext4/etc. partitions in a "removable" mode which ignores permissions and mismatched UIDs, and grants full access to the files without needing to be root.
I wish it was possible to pull a hard drive from one Linux system, and plug it into another, and mount the ext4/etc. partitions in a "removable" mode which ignores permissions and mismatched UIDs, and grants full access to the files without needing to be root.
You can set an ACL on the appropriate mount point to grant all users in an arbitrary group rwX permissions.
I cannot believe how big of a monstrosity systemd has become. It started out as an init system and was promoted as such and I started out liking it. Then the wording suddenly changed to "a suite of basic building blocks for a Linux system", then came the hardcoded DNS servers, then the shitload of bugs that has been ignored since about 2015. Now instead of fixing what's broken and making things more secure, more and more cruft keeps getting added.
> We used to sit around in the Unix Room saying, 'What can we throw out? Why is there this option?' It's often because there is some deficiency in the basic design — you didn't really hit the right design point. Instead of adding an option, think about what was forcing you to add that option. (Doug McIlroy).
Sigh!
> We used to sit around in the Unix Room saying, 'What can we throw out? Why is there this option?' It's often because there is some deficiency in the basic design — you didn't really hit the right design point. Instead of adding an option, think about what was forcing you to add that option. (Doug McIlroy).
Sigh!
I wish Systemd-networkd had native PPPoE support.
https://github.com/systemd/systemd/issues/481
Still, systemd is just great and it has made my life a lot easier.
https://github.com/systemd/systemd/issues/481
Still, systemd is just great and it has made my life a lot easier.
I'm curious if it's still falling back to Google DNS / NTP if not otherwise configured. That was a very bad choice even though I understand it'll only kick in under unusual circumstances.
> The static lease DHCP server logic in systemd-networkd may now serve IP addresses outside of the configured IP pool range for the server.
See, see, taking lessons from C++. So someone requested a bug, and they got it. How could that be useful?
See, see, taking lessons from C++. So someone requested a bug, and they got it. How could that be useful?
Suppose you have a network 192.168.0.0/24, with the router at 192.168.0.1, and you want the DHCP server to hand out dynamic addresses from 192.168.0.50 to 192.168.0.240 to clients, and hand out static IPs from 192.168.0.2 to 192.168.0.49 to specific identified devices via static leases. Those static IPs are outside the configured IP range for the DHCP server, intentionally.
DHCP protocol actively avoids specifying a policy - that is for the system administrator. It defines three 'mechanisms' for allocation [0]:
[0] https://datatracker.ietf.org/doc/html/rfc2131#section-1
* automatic == permanent address
* dynamic == finite (time-limited) and possibly re-assignable to a different client
* manual == assigned by system administrator (and simply conveyed by DHCP)
[0] https://datatracker.ietf.org/doc/html/rfc2131#section-1
I think there’s a misunderstanding. The [DHCPServer] section is tied to a [Network] unit. The [Network] unit has an address range. The [DHCPServer] section has options to reserve a range for dynamic leases. There is also [DHCPServerStaticLease] sections to assign static leases. However, previous versions of networkd did not allow the static lease to be outside of the [DHCPServer] dynamic range. Now, the static lease can be outside the dynamic range but within the network range. So, pretty much what any other standard DHCP server can do.
Systemd is a matryoshka doll of bugs.
How could that be useful?
How could that be useful?
If homed makes it into the major distributions I'm done with mainstream Linux.
genuine question: what's so bad about homed to jump ship? can't you just disable it and not look back?
What do you mean "jump ship"? There are plenty of Linux distributions that still let you choose.
Genuine question though: why didn't Lennart and his gang just fork some Linux distribution for their ideas? Why invade everywhere else instead? Anybody ask them that?
Genuine question though: why didn't Lennart and his gang just fork some Linux distribution for their ideas? Why invade everywhere else instead? Anybody ask them that?
I use Devuan, systemd-free fork of Debian, no complaints so far.
Congratulations to the systemd team.
I still won't use systemd, and to the extent of my ability I will fight the trend of the Linux ecosystem hard-depending on it. But it's a bit silly to be anything but grateful for open source and choice.
I still won't use systemd, and to the extent of my ability I will fight the trend of the Linux ecosystem hard-depending on it. But it's a bit silly to be anything but grateful for open source and choice.
I applaud your ability to simultaneously dislike systemd and acknowledge the achievements of it's developers without the need to vilify them.
Still no default-deny option in systemd.
At this point any system that is running this should not be called "UNIX derived" or "UNIX like" as systemd has subsumed all of the pre-existing architecture around system startup and management. Not arguing if that is a good or bad thing just saying we should stop putting systemd managed systems in the 'UNIX' column just like we don't put Windows NT systems in the UNIX column.
I'd just like to interject for a moment. What you're referring to as Linux, is in fact, systemd/Linux, or as I’ve recently taken to calling it, systemd plus Linux. Linux is not an operating system unto itself, but rather another free component of a fully functioning systemd system made useful by the systemd init system, IPC layer, and other vital system components comprising a full OS as defined by POSIX.
Many computer users run a modified version of the systemd system every day, without realizing it. Through a peculiar turn of events, the version of systemd which is widely used today is often called "Linux", and many of its users are not aware that it is basically the systemd system, developed by Red Hat. There really is a Linux, and these people are using it, but it is just a part of the system they use.
Linux is the kernel: the program in the system that allocates the machine’s resources to the other programs that you run. The kernel is an essential part of an operating system, but useless by itself; it can only function in the context of a complete operating system. Linux is normally used in combination with the systemd operating system: the whole system is basically systemd with Linux added, or systemd/Linux. All the so-called "Linux" distributions are really distributions of systemd/Linux.
(/s)
Many computer users run a modified version of the systemd system every day, without realizing it. Through a peculiar turn of events, the version of systemd which is widely used today is often called "Linux", and many of its users are not aware that it is basically the systemd system, developed by Red Hat. There really is a Linux, and these people are using it, but it is just a part of the system they use.
Linux is the kernel: the program in the system that allocates the machine’s resources to the other programs that you run. The kernel is an essential part of an operating system, but useless by itself; it can only function in the context of a complete operating system. Linux is normally used in combination with the systemd operating system: the whole system is basically systemd with Linux added, or systemd/Linux. All the so-called "Linux" distributions are really distributions of systemd/Linux.
(/s)
Okay, that made my wife come in to see what I was laughing about.
I don’t follow how a new startup/system management architecture makes it “not UNIX”. We do put OS X in the UNIX column, after all. POSIX and all that.
https://en.wikipedia.org/wiki/Unix_philosophy
Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface.
AFAIK none of these describes systemd.
Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface.
AFAIK none of these describes systemd.
That seems to describe systemd extremely well. Lots of little tools like networkd, homed, journald... that do one thing, do it well and work together. And using text+time stamps.
AFAIK the systemd parts don't have well-defined interfaces. At least I've never seen any independent replacement of any of those small parts, as opposed to replacement of whole systemd.
Do you browse the web like `curl | render-http`? UNIX philosophy is not a law. For complex problem domains separating it into multiple parts is not always possible and only makes the whole thing more complex. System init is quite a complex problem domain.
I browse the web using Firefox, i.e., extendable browser with a huge number of plugins. Unix philosophy is not a low. It's a rule helping to make computing understandable, robust and extendable.
I don’t see why something that is derived from UNIX has to have to the UNIX philosophy.
Here’s a thought I haven’t evaluated fully: can the same thing be said about the Linux kernel?
At the end of the day I think some of the major points of criticisms of systemd (moving fast and breaking things for example) are valid, however I still see it as a net gain. Fortunately there are distros like Devuan, OpenWRT and Alpine for those who want something different.
Here’s a thought I haven’t evaluated fully: can the same thing be said about the Linux kernel?
At the end of the day I think some of the major points of criticisms of systemd (moving fast and breaking things for example) are valid, however I still see it as a net gain. Fortunately there are distros like Devuan, OpenWRT and Alpine for those who want something different.
Would anyone care to explain to someone who isn't 1337 in the inner workings of Linux, why systemd makes modern Linux not very UNIXY anymore and why everyone seems to hate it to death?
It used to be process 1 was init, and it forked/exec'd everything else typically from /etc/rc (this elides over a lot) -its the first binary the kernel runs. Everything else stems from the birth of user process number 1.
Fork/exec is the lightweight process inheriting of open i/o and memory state with copy-on-write which made Unix a joy to work with. It's a delightful mapping of kernel and userspace into complex inherited tree of processes sharing that origin state (or not. You can close it all down, change things.)
systemd is a complex mega binary which parses control files and subsumes several functions. It's big. It's not small like init. It's not fork/exec simple it forks control processes and monitors children and does a lot of work some daemons did themselves (getty, inetd) but others didn't do. And you get to monitor things. It's not unlike launchctl on OSX or other platforms. And after all /etc/rc is now 5 or more/etc/rcX.d/ complex substates of states to look for scripts, SAT solver complex dependency stuff between semi arcane shellscripts... blah.
PRO: everything is now much more consistent
CON: greybeards like me hate it (beardless btw but meh) because it's complex
Fork/exec is the lightweight process inheriting of open i/o and memory state with copy-on-write which made Unix a joy to work with. It's a delightful mapping of kernel and userspace into complex inherited tree of processes sharing that origin state (or not. You can close it all down, change things.)
systemd is a complex mega binary which parses control files and subsumes several functions. It's big. It's not small like init. It's not fork/exec simple it forks control processes and monitors children and does a lot of work some daemons did themselves (getty, inetd) but others didn't do. And you get to monitor things. It's not unlike launchctl on OSX or other platforms. And after all /etc/rc is now 5 or more/etc/rcX.d/ complex substates of states to look for scripts, SAT solver complex dependency stuff between semi arcane shellscripts... blah.
PRO: everything is now much more consistent
CON: greybeards like me hate it (beardless btw but meh) because it's complex
consistency ftw: https://news.ycombinator.com/item?id=29670751
gimme open-rc or gimme death
actually, sysv was fine too
open-rc solves the parallel-startup desire for people who want a faster boot and doesn't hide logs in binary formats which can only be interrogated with specialised tools - my primary reason for leaving Debian-based systems that I'd used for 16 years was having systemd forced on me and trying to deal with how to read good old system logging, not to mention 5-minute shutdown times, slower boot on the same machine than with Ubuntu's upstart or with open-rc.
gimme open-rc or gimme death
actually, sysv was fine too
open-rc solves the parallel-startup desire for people who want a faster boot and doesn't hide logs in binary formats which can only be interrogated with specialised tools - my primary reason for leaving Debian-based systems that I'd used for 16 years was having systemd forced on me and trying to deal with how to read good old system logging, not to mention 5-minute shutdown times, slower boot on the same machine than with Ubuntu's upstart or with open-rc.
You want MXLinux - Debian with just enough systemd to work in a modern environment but with SysV init. https://mxlinux.org
> not to mention 5-minute shutdown times
5 minutes? Try total hang on shutdown, and often not starting up. That was my systemd experience. I traced it down to a NFS mount that no amount of tinkering would mount reliably pn boot. Granted that was several years ago, but soured me on systemd.
> not to mention 5-minute shutdown times
5 minutes? Try total hang on shutdown, and often not starting up. That was my systemd experience. I traced it down to a NFS mount that no amount of tinkering would mount reliably pn boot. Granted that was several years ago, but soured me on systemd.
Tried FreeBSD? Very consistent and has POLA, the principle of Least astonishment. The proper good stuff Dell Isilon and NetApp are relying on.
Because using journalctl is so bad.. Hell, you can even forward to a text-based logger. And it’s not like it is some proprietary format either
My Debian came with rsyslog enabled by default which writes the journald logs to disk as plain text.
Like someone already answered systemd started as init replacement "to make Linux boot faster" - that was PR pitch. But then it consumed many system services like logging, time, logging in, hardware detection, network, mounting, audio, GNOME, etc etc. That alone is enough to clearly state systemd is bullshit. Everything is now interdependend and it is realy hard to replace some parts of it like eg. udev - decade old story ?
Yes, some things was improved but it could be done in sane way - making things by design replecable, as it was always in UNIX way. But systemd is intentionally self absorbing, more - looks like it is intentionally layer above Linux. Now what we can do when one day Pottering or anyone from that IBM/Intel/MS cabal will merge Linux kernel uncompatible change ? And make Windows kernel "working flawless" - for some time of course. And with build in tracing. Or login right licensing ? :) That can be easily done today. And what plain users will tell ? "Works for me, what you talking about ??" Remember, some idiots already tried to sell plant seeds that you needed to re-buy every year. Business unchecked becoming evil.
Btw. about that "boot fast" :> Now it is dropped, of course. Also making system gooing down almoust instant is broken for years now.
My prediction is that in some time facepalm will be most common reaction when someone hears "systemD".
Yes, some things was improved but it could be done in sane way - making things by design replecable, as it was always in UNIX way. But systemd is intentionally self absorbing, more - looks like it is intentionally layer above Linux. Now what we can do when one day Pottering or anyone from that IBM/Intel/MS cabal will merge Linux kernel uncompatible change ? And make Windows kernel "working flawless" - for some time of course. And with build in tracing. Or login right licensing ? :) That can be easily done today. And what plain users will tell ? "Works for me, what you talking about ??" Remember, some idiots already tried to sell plant seeds that you needed to re-buy every year. Business unchecked becoming evil.
Btw. about that "boot fast" :> Now it is dropped, of course. Also making system gooing down almoust instant is broken for years now.
My prediction is that in some time facepalm will be most common reaction when someone hears "systemD".
Systemd was never meant for “making init faster” - it was meant to making it correct and maintainable. That’s another added benefit that actually getting the boot dependency ordering correctly allows for parallelism beside many other benefits which will make it faster.
Also, your listed “features” are bullshit: logging in, audio, GNOME, etc have absolutely nothing to do with systemd. Even logging can still be forwarded to any log tool you would like, but having it implemented inside is a sane choice — you have to start taking logs when there is not even a file system available. You know how was it solved in the pre-systemd days? It was simply ignored!
System init is a very complex problem, systemd only gives you the essential complexity for the task.
Also RE poweroff speed: there is a single flag you can toggle to either kill lingering processes (which is usually not enabled because some hacky tools like tmux likes to live in the background and systemd killing it is bad) or wait a bit before. So yet again, systemd does the correct thing while before these were simply killed.
Also, your listed “features” are bullshit: logging in, audio, GNOME, etc have absolutely nothing to do with systemd. Even logging can still be forwarded to any log tool you would like, but having it implemented inside is a sane choice — you have to start taking logs when there is not even a file system available. You know how was it solved in the pre-systemd days? It was simply ignored!
System init is a very complex problem, systemd only gives you the essential complexity for the task.
Also RE poweroff speed: there is a single flag you can toggle to either kill lingering processes (which is usually not enabled because some hacky tools like tmux likes to live in the background and systemd killing it is bad) or wait a bit before. So yet again, systemd does the correct thing while before these were simply killed.
UNIX, TCP/IP, C libraries, web browsers and HTML… all are interoperable technologies. If your site works in browser X, you can switch to browser Y and it still works. If your code compiles for x86 Linux, it’ll run on arm Linux too.
SystemD is not an interoperable technology. It insists on wrapping itself around what was already there, in the guise of making it better while also making it impossible to use something else.
Abstract example: imagine systemd-plated manages access to crockery in your kitchen, providing neat features like being able to quickly pick out plates by colour (the old system never did that!). However, if you take a plate for yourself instead of asking systemd for it then systemd will take the plate out of your hands, put it back in the stack, and say “error: plate not requested via systemd”. Now imagine everything in your kitchen has to go through systemd. Now imagine that systemd is one of the biggest and most rapidly changing machines in your kitchen that breaks a lot. You’d be pretty annoyed if you ever did anything weird enough to encounter these bugs. If you only ever want a blue plate, you’re fine.
Real example: Any day now we’ll be unable to use plain LXC on Linux because systemd (1) assumes you only manage containers via systemd and (2) clobbers any changes you make outside systemd, using the lxc-* tools.
SystemD is not an interoperable technology. It insists on wrapping itself around what was already there, in the guise of making it better while also making it impossible to use something else.
Abstract example: imagine systemd-plated manages access to crockery in your kitchen, providing neat features like being able to quickly pick out plates by colour (the old system never did that!). However, if you take a plate for yourself instead of asking systemd for it then systemd will take the plate out of your hands, put it back in the stack, and say “error: plate not requested via systemd”. Now imagine everything in your kitchen has to go through systemd. Now imagine that systemd is one of the biggest and most rapidly changing machines in your kitchen that breaks a lot. You’d be pretty annoyed if you ever did anything weird enough to encounter these bugs. If you only ever want a blue plate, you’re fine.
Real example: Any day now we’ll be unable to use plain LXC on Linux because systemd (1) assumes you only manage containers via systemd and (2) clobbers any changes you make outside systemd, using the lxc-* tools.
Linux via Unix: Do one thing, and do it well. https://en.wikipedia.org/wiki/Unix_philosophy
Systemd: Not that.
See https://news.ycombinator.com/item?id=29670980 for an example.
Systemd: Not that.
See https://news.ycombinator.com/item?id=29670980 for an example.
I really don’t think having timers is the best example of this. systemd at it’s core is a resource monitor — a tool that sets up and babysits different types of things — processes, mounts, sockets, scheduled tasks, devices, etc. and manages the dependencies that exist between them. It fits right in with the other stuff.
The startup scripts that were in use before systemd did one thing but they certainly didn't do it well.
* dbus — This is part of a broader change in Linux userspace from text/byte streams to a message bus of structure data similar to COM. It’s incredibly powerful the things you can accomplish with it but it’s sysadmin/power user unfriendly if you expect the interface to system services to be “call a binary from bash.” systemd uses dbus for basically everything which makes it really really easy to program while simultaneously erasing every sysadmins muscle memory and ability to solve every program with more bash.
* lack of composability — This is what is really meant by the “do one thing” mantra. Plenty of UNIXY tools are bloated but they don’t feel like it because they can be called as filters. systemd is actually quite modular but the pieces aren’t designed to be standalone. Like how in Ansible is everything is a plug-in but it’s cumbersome to just run a single module.
* “better is worse” — one of the weirder faults of systemd is that it’s thoughtfully designed, handles edge cases gracefully, and does things the right way. Kinda odd to paint this as a fault but it’s the reason people like bash over powershell; Unix tools were mostly made by people solving their own problems and so are completely inconsistent, full of sharp edges, odd syntax and parsers, have plenty of unimplemented features but are fiercely useful. systemd is a tool not forged in battle and is designed by people trying to write good software over software that solves an immediate need. This isn’t to say that systemd isn’t useful — it’s genuinely transformative but software written out of frustration tends to be designed differently and it shows.
* lack of composability — This is what is really meant by the “do one thing” mantra. Plenty of UNIXY tools are bloated but they don’t feel like it because they can be called as filters. systemd is actually quite modular but the pieces aren’t designed to be standalone. Like how in Ansible is everything is a plug-in but it’s cumbersome to just run a single module.
* “better is worse” — one of the weirder faults of systemd is that it’s thoughtfully designed, handles edge cases gracefully, and does things the right way. Kinda odd to paint this as a fault but it’s the reason people like bash over powershell; Unix tools were mostly made by people solving their own problems and so are completely inconsistent, full of sharp edges, odd syntax and parsers, have plenty of unimplemented features but are fiercely useful. systemd is a tool not forged in battle and is designed by people trying to write good software over software that solves an immediate need. This isn’t to say that systemd isn’t useful — it’s genuinely transformative but software written out of frustration tends to be designed differently and it shows.
> why everyone seems to hate it to death
This isn't universal. For service management systemd is awesome, and has been way more enjoyable to use than any other system (Ubuntu's upstart, Solaris's SMF, FreeBDS, and old-school linux init). As a sysadmin my only issues with systemd have been occasional patches that required rebooting, but these have practically disappeared in the last few years.
This isn't universal. For service management systemd is awesome, and has been way more enjoyable to use than any other system (Ubuntu's upstart, Solaris's SMF, FreeBDS, and old-school linux init). As a sysadmin my only issues with systemd have been occasional patches that required rebooting, but these have practically disappeared in the last few years.
Basically, systemd is a big binary that manages the system.
That used to be many discrete programs and shell scripts and files.
There is a lot of philosophy to consider from there.
That used to be many discrete programs and shell scripts and files.
There is a lot of philosophy to consider from there.
On my laptop, systemd consists of 60 different small binaries. Looks pretty UNIXy to me (maybe even too much so).
That is what I get for using mobile in the evening.
I really should have expanded on that post, and at a minimum removed "discrete programs" from it, the distinction being the behavior in shell scripts and such and sequential behavior at boot time.
I really should have expanded on that post, and at a minimum removed "discrete programs" from it, the distinction being the behavior in shell scripts and such and sequential behavior at boot time.
>Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new "features".
It's a bit of a monolith. People will say it's not, because it's made out of a bunch of smaller components, but when those components are that closely interlinked it stops mattering if they live in different code bases.
It also has a pretty wide scope. PID 1 is responsible for things like dealing with zombie/orphan processes and it's generally a good idea to keep it as minimal as possible, and keep the attack surface as small as possible. Systemd handles things like socket activation, logging, scheduling, etc. It handles all this in more or less one process, which means that when/if that process crashes or has security concerns it's a much bigger deal. It even has it's own docker-equivalent!
Really you want to keep PID 1 as minimal and secure as possible.
>Expect the output of every program to become the input to another, as yet unknown, program. Don't clutter output with extraneous information. Avoid stringently columnar or binary input formats. Don't insist on interactive input.
They've made it hard to work with generic linux tools. For example they have their own custom log format that isn't text based. This means that linux tools like find or grep won't work on systemd logs. Generally systemd has replacements for those tools, like you can still grep but you've got to learn the specific systemd ways of actually getting the output and all that (note that a lot of distros seem to ignore systemd's logging provisions and just log to files like normal, for a bunch of stuff).
It's pretty complicated to reason about, one "unit" can be a bunch of different files spread out all over the filesystem, and it can be hard to get a good view of all the aspects that effect a unit without resorting to systemd-specific tools to figure out the relationships between things. For example if you want to edit a unit file the suggested way to edit a unit file is with something like `sudo systemctl edit ssh` or possible `sudo systemctl edit ssh --full` depending on the situation. Spreading files all over the place like that sort of betrays their enterprise roots, it makes some sense when you're working with very large teams but just having all the until files in one location (/etc/init.d) would be a lot more readable and manageable. Also having one file per unit, instead of having to have separate file for units and their dependencies (things like sockets) would be nice.
Another problem is that they broke chroot, which makes dealing with embedded systems that use systemd a pain if you're not using a systemd based host to develop on. They sort of force you to use systemd's docker equivalent instead of the more common chroot if you're dealing with systemd-based embedded systems. That means you'd better be running a systemd based host system as well.
---
Also I think some of us feel like it's being forced on us. Redhat/IBM funds or has control over a lot of the unix ecosystem, so when we see things like Gnome not working on BSD any more because it has a hard dependency on systemd, well it starts to look like an intentional.
Things like the systemd decision to randomly kill off screen/tmux sessions on user logout leave a really bad taste in some people's mouth. Honestly a fair bit of the dislike is just repeated instances like that, and repeatedly ignoring feedback as to why that's actually a pretty big problem, and generally being unwilling to compromise with the broader community.
https://www.reddit.com/r/programming/comments/4ldewx/systemd...
It's a bit of a monolith. People will say it's not, because it's made out of a bunch of smaller components, but when those components are that closely interlinked it stops mattering if they live in different code bases.
It also has a pretty wide scope. PID 1 is responsible for things like dealing with zombie/orphan processes and it's generally a good idea to keep it as minimal as possible, and keep the attack surface as small as possible. Systemd handles things like socket activation, logging, scheduling, etc. It handles all this in more or less one process, which means that when/if that process crashes or has security concerns it's a much bigger deal. It even has it's own docker-equivalent!
Really you want to keep PID 1 as minimal and secure as possible.
>Expect the output of every program to become the input to another, as yet unknown, program. Don't clutter output with extraneous information. Avoid stringently columnar or binary input formats. Don't insist on interactive input.
They've made it hard to work with generic linux tools. For example they have their own custom log format that isn't text based. This means that linux tools like find or grep won't work on systemd logs. Generally systemd has replacements for those tools, like you can still grep but you've got to learn the specific systemd ways of actually getting the output and all that (note that a lot of distros seem to ignore systemd's logging provisions and just log to files like normal, for a bunch of stuff).
It's pretty complicated to reason about, one "unit" can be a bunch of different files spread out all over the filesystem, and it can be hard to get a good view of all the aspects that effect a unit without resorting to systemd-specific tools to figure out the relationships between things. For example if you want to edit a unit file the suggested way to edit a unit file is with something like `sudo systemctl edit ssh` or possible `sudo systemctl edit ssh --full` depending on the situation. Spreading files all over the place like that sort of betrays their enterprise roots, it makes some sense when you're working with very large teams but just having all the until files in one location (/etc/init.d) would be a lot more readable and manageable. Also having one file per unit, instead of having to have separate file for units and their dependencies (things like sockets) would be nice.
Another problem is that they broke chroot, which makes dealing with embedded systems that use systemd a pain if you're not using a systemd based host to develop on. They sort of force you to use systemd's docker equivalent instead of the more common chroot if you're dealing with systemd-based embedded systems. That means you'd better be running a systemd based host system as well.
---
Also I think some of us feel like it's being forced on us. Redhat/IBM funds or has control over a lot of the unix ecosystem, so when we see things like Gnome not working on BSD any more because it has a hard dependency on systemd, well it starts to look like an intentional.
Things like the systemd decision to randomly kill off screen/tmux sessions on user logout leave a really bad taste in some people's mouth. Honestly a fair bit of the dislike is just repeated instances like that, and repeatedly ignoring feedback as to why that's actually a pretty big problem, and generally being unwilling to compromise with the broader community.
https://www.reddit.com/r/programming/comments/4ldewx/systemd...
Should macOS and Solaris also not be considered "UNIX derived" or "UNIX like" because of their use of launched and SMF?
[deleted]
Agreed, also no judgement here as well.
I propose: "UNIX-ish" or Sysd type Linux.
Shit, what am I thinking!
GNU LINUX was the older discussion and development.
SYSTEMD LINUX is likely most applicable nomenclature, given current development.
I propose: "UNIX-ish" or Sysd type Linux.
Shit, what am I thinking!
GNU LINUX was the older discussion and development.
SYSTEMD LINUX is likely most applicable nomenclature, given current development.
Well UNIX is "a pun on multics", so how about EUNICS? The S can stand for systemd.
Lol
I used to care a lot more. Frankly, what I do now is make sure I can use an OS and that I am depending on more portable software where I can. I find myself in Windows, OSX, and Linux land, maybe now I should just say Systemd Linux land and pretty much have something other than the OS to worry about.
Ideally, the OS gives me few worries.
I used to care a lot more. Frankly, what I do now is make sure I can use an OS and that I am depending on more portable software where I can. I find myself in Windows, OSX, and Linux land, maybe now I should just say Systemd Linux land and pretty much have something other than the OS to worry about.
Ideally, the OS gives me few worries.
Well the name System D implies a manifest destiny to conquer System V (i.e. UNIX) because D=500 and V=5 in Roman numerals. Why else would they choose the name? Also you should call it GNU/Systemd.
Come up with a commercial solution to a simple problem. Ex: file sharing or password management. Make it slow, insecure, privacy violating, lose people's data. Make millions, become a hero.