Evaluating FreeBSD Current for Production Use(klarasystems.com)
klarasystems.com
Evaluating FreeBSD Current for Production Use
https://klarasystems.com/articles/evaluating-freebsd-current-for-production-use/
33 comments
While I have used FreeBSD before on my desktops (starting with version 8.1) and love the design of the system along with the UNIX feel, I do not really see how it can become popular with the WiFi stack in shambles (my current Intel Wireless device gets half of the speed it does on other OS) and other quirks. I feel that a lot more of the donations should go to working on FreeBSD device support if we ever to dream of it going popular.
Despite their proclaimed support of open source, Intel wifi can be a mess on Linux too. Most distributions work well, but Gentoo for one loads the incorrect firmware blob and I am forced to try to create a custom installer image in a VM after emerging the linux-firmware package.
The iwlwifi driver just landed in FreeBSD 13.1-RELEASE, and I have yet to have any luck with it there, either. Neither Gentoo nor FreeBSD seems to want to load the correct cc-a0-66.ucode blob for my AX200 chip. I think the downstream developers can only bear so much blame; Intel could surely do a better job helping to clear up edge cases like this.
The iwlwifi driver just landed in FreeBSD 13.1-RELEASE, and I have yet to have any luck with it there, either. Neither Gentoo nor FreeBSD seems to want to load the correct cc-a0-66.ucode blob for my AX200 chip. I think the downstream developers can only bear so much blame; Intel could surely do a better job helping to clear up edge cases like this.
Question: If you use the FreeBSD Ports collection for the services you run, how do you deal with Ports' quarterly releases? One reason I haven't used FreeBSD much is because of the fear of doing bulk updates every 90 days, and having to fix random things that might break due to software upgrades.
For a home server I have no problem with this, but if it's something I want to leave in production and have security updates for, I'm more at-ease using Debian or Ubuntu with their 3- and 5-years' worth of support.
Is there something I'm missing about using FreeBSD in production, or is anything outside of the base system just supposed to be more hands-on?
For a home server I have no problem with this, but if it's something I want to leave in production and have security updates for, I'm more at-ease using Debian or Ubuntu with their 3- and 5-years' worth of support.
Is there something I'm missing about using FreeBSD in production, or is anything outside of the base system just supposed to be more hands-on?
FreeBSD has two repositories of packages, quarterly and latest.
$ cat /usr/local/etc/pkg/repos/FreeBSD.conf
FreeBSD: { enabled: no } # disable the default config
FreeBSD-latest: {
url: "pkg+http://pkg.FreeBSD.org/${ABI}/latest",
#url: "pkg+http://pkg.FreeBSD.org/${ABI}/quarterly",
mirror_type: "srv",
signature_type: "fingerprints",
fingerprints: "/usr/share/keys/pkg",
enabled: yes
}I'm aware of the Latest branch, but its shorter release cycle would be even less-suited for production usage (for people who are into Very Serious Business™).
I was talking about Quarterly since it seems to the most stable way to get packages not in the base system.
I was talking about Quarterly since it seems to the most stable way to get packages not in the base system.
Ports are rolling release. Simply run 'portsnap fetch update' and you'll have the latest available in /usr/ports just one 'portmaster -a' (upgrade) away.
Same goes for packages (pkg), they are rolling release.
No idea where you're getting this 90 days release stuff from :) (Perhaps there's a LTS 'release' for ports? I don't think many people use it if there is. Trust me, you're safe enough with rolling ports).
Same goes for packages (pkg), they are rolling release.
No idea where you're getting this 90 days release stuff from :) (Perhaps there's a LTS 'release' for ports? I don't think many people use it if there is. Trust me, you're safe enough with rolling ports).
See the following. There's a 'latest' branch and a 'quarterly' branch.
https://wiki.freebsd.org/Ports/QuarterlyBranch https://pkg.freebsd.org/FreeBSD:13:amd64/ https://svnweb.freebsd.org/ports/branches/
That said, I don't have any problems with rolling-release. I've ran stuff like openSUSE Tumbleweed for years, and back in my dabbling days, plenty of Gentoo and Arch.
My issue is that for non-personal and longer-term deployments, FreeBSD has never been a contender for me because of having to keep on top of Ports upgrades. For example for a project at work, I wouldn't want to return to a FreeBSD deployment months later to add a feature, and then have to update all my packages on that system.
I understand chances are the upgrades would likely be hitch-free, or would only require a little bit of fixing around, but I still really don't want to hold my breath every 3 months.
https://wiki.freebsd.org/Ports/QuarterlyBranch https://pkg.freebsd.org/FreeBSD:13:amd64/ https://svnweb.freebsd.org/ports/branches/
That said, I don't have any problems with rolling-release. I've ran stuff like openSUSE Tumbleweed for years, and back in my dabbling days, plenty of Gentoo and Arch.
My issue is that for non-personal and longer-term deployments, FreeBSD has never been a contender for me because of having to keep on top of Ports upgrades. For example for a project at work, I wouldn't want to return to a FreeBSD deployment months later to add a feature, and then have to update all my packages on that system.
I understand chances are the upgrades would likely be hitch-free, or would only require a little bit of fixing around, but I still really don't want to hold my breath every 3 months.
As a young SWE interested in IT (which is how I got interested in programming in the first place, was doing sysadmin things as a teenager like running an IRC leaf), older IT wizards told me to never mix ports and packages. And furthermore, not to use ports as they lead to fundamental breakages.
https://www.freebsd.org/security/#sup
Unlike some debian-ish distros I don't think there's any security-update-only repo for ports/packages to track. Meanwhile freebsd-update(8) handles security updates for the base system.
Under the current support model, each major version’s stable branch
is explicitly supported for 5 years, while each individual point release
is only supported for three months after the next point release.
Unlike some debian-ish distros I don't think there's any security-update-only repo for ports/packages to track. Meanwhile freebsd-update(8) handles security updates for the base system.
Yeah, I love FreeBSD's base system's release model. It feels like a complete system, and each release is supported for a long time.
Same with systems like Ports, and in particular, NetBSD's pkgsrc. I've personally been using pkgsrc on my MacBook (and sometimes on Linux too) for years, and have really enjoyed using it as a cross-platform system to add my own little packages that'll work across macOS/Linux/NetBSD (I use NetBSD on my Pi). And I have no problems doing upgrades every 3 months, since I want my shiny new software.
So I definitely see and appreciate Ports/pkgsrc's benefits - but I still feel uneasy using stuff like FreeBSD+Ports on non-personal projects, like for projects at work.
Same with systems like Ports, and in particular, NetBSD's pkgsrc. I've personally been using pkgsrc on my MacBook (and sometimes on Linux too) for years, and have really enjoyed using it as a cross-platform system to add my own little packages that'll work across macOS/Linux/NetBSD (I use NetBSD on my Pi). And I have no problems doing upgrades every 3 months, since I want my shiny new software.
So I definitely see and appreciate Ports/pkgsrc's benefits - but I still feel uneasy using stuff like FreeBSD+Ports on non-personal projects, like for projects at work.
Until pkg(8) came around ports were one of my least favorite parts of FreeBSD. I like(d) pkgsrc even less.
Part of it was the simplistic support for binary packages that felt entirely like an unwanted afterthought. Compile everything from source just doesn't scale especially if you want to use complex, slow-to-compile projects like gcc or firefox.
The other part is that make(1) is SLOW and simply does not scale to the size of something like the ports tree. It's also a bit tedious to use out of your ports tree and the documentation was kinda meh. There were a number of hacks layered on top that sort of eased the pain (I want to say portupgrade?, I never got into portmaster) but always felt really brittle and things were always getting out of sync. Even pkg(8) is a bit funky (and I don't particularly like the reliance on srv records no matter how elegant a solution it may seem).
> I still feel uneasy using stuff like FreeBSD+Ports on non-personal projects, like for projects at work.
I would never use ports in a production environment without some detailed justification. The binary packages are the way to go. Building your own ports (as in your own software not building your own copy of an existing port) is also a bit of a trial by fire thing involving a lot of hair pulling. Dunno what the update quarterly stuff is about tho, I run pkg update weekly to keep on top of things – but I track the latest repo not the quarterly one.
Honestly one of the few things I miss about Linux is the Debian package ecosystem. But as I learned with fink on OSX it's not just the infrastructure that matters.
Part of it was the simplistic support for binary packages that felt entirely like an unwanted afterthought. Compile everything from source just doesn't scale especially if you want to use complex, slow-to-compile projects like gcc or firefox.
The other part is that make(1) is SLOW and simply does not scale to the size of something like the ports tree. It's also a bit tedious to use out of your ports tree and the documentation was kinda meh. There were a number of hacks layered on top that sort of eased the pain (I want to say portupgrade?, I never got into portmaster) but always felt really brittle and things were always getting out of sync. Even pkg(8) is a bit funky (and I don't particularly like the reliance on srv records no matter how elegant a solution it may seem).
> I still feel uneasy using stuff like FreeBSD+Ports on non-personal projects, like for projects at work.
I would never use ports in a production environment without some detailed justification. The binary packages are the way to go. Building your own ports (as in your own software not building your own copy of an existing port) is also a bit of a trial by fire thing involving a lot of hair pulling. Dunno what the update quarterly stuff is about tho, I run pkg update weekly to keep on top of things – but I track the latest repo not the quarterly one.
Honestly one of the few things I miss about Linux is the Debian package ecosystem. But as I learned with fink on OSX it's not just the infrastructure that matters.
>pkg
pkg-ng is indeed a godsend, and I think I just got PTSD thinking about what Ports management was like before it. Currently getting flashbacks to the salad of utilities I used to use from ports-mgmt/.
>make
Yes, the Makefile infrastructure for both Ports/pkgsrc is clunky and slow, and the documentation is also terrible too. Took quite a bit of suffering to learn the dark art of properly authoring packages in pkgsrc.
For what it's worth, there's also the Gentoo Prefix project, but I haven't ever been able to use it because the bootstrap for it has failed every time I tried it on macOS or Linux.
At least for pkgsrc using it is as simple as: $ git clone https://github.com/netbsd/pkgsrc --depth 1 $ cd ~/pkgsrc/bootstrap && ./bootstrap --unprivileged $ cd ~/pkgsrc/misc/tmux && ~/pkg/bin/bmake install $ ~/pkg/bin/tmux
As flawed and janky as pkgsrc is, it has at least worked consistently for me on macOS/Linux for years.
>compiling from source
Yeah, I agree. Compiling everything from source and using it in production is for insane people. Should have clarified that when I said "Ports/pkgsrc", I meant using official binary pacakges, and not necessarily building from source.
>latest and quarterly
The point of quarterly releases is to serve as a base for stable package versions, backported security fixes, and as a chance to fix package runtime/compilation issues before being generally available - something harder to do if you track bleeding-edge packages, where things can shift under people's feet.
It's also useful when building from source, like I do with pkgsrc for macOS, because lots of random things will break if there's too large of a rift between what's installed and what's in bleeding-edge pkgsrc. In these cases, you'd usually have to do mass rebuilds, and/or deal with dependency hell (eg, pkg X you have installed needs need <=libsomething-1336, but pkg Y in pkgsrc needs >=libsomething-1337).
I use the packages installed via pkgsrc pretty much daily on my MacBook, and it'd be really annoying having things possibly breaking after an update just a week apart, especially if I'm in a rush and need to get something done for work. So using a quarterly branch is useful, since it's a cadence I can stomach, and there's a higher likelihood that the devs had more time to polish the packages in that release.
>Fink
Never heard of this before. I'll give it a gander sometime soon.
pkg-ng is indeed a godsend, and I think I just got PTSD thinking about what Ports management was like before it. Currently getting flashbacks to the salad of utilities I used to use from ports-mgmt/.
>make
Yes, the Makefile infrastructure for both Ports/pkgsrc is clunky and slow, and the documentation is also terrible too. Took quite a bit of suffering to learn the dark art of properly authoring packages in pkgsrc.
For what it's worth, there's also the Gentoo Prefix project, but I haven't ever been able to use it because the bootstrap for it has failed every time I tried it on macOS or Linux.
At least for pkgsrc using it is as simple as: $ git clone https://github.com/netbsd/pkgsrc --depth 1 $ cd ~/pkgsrc/bootstrap && ./bootstrap --unprivileged $ cd ~/pkgsrc/misc/tmux && ~/pkg/bin/bmake install $ ~/pkg/bin/tmux
As flawed and janky as pkgsrc is, it has at least worked consistently for me on macOS/Linux for years.
>compiling from source
Yeah, I agree. Compiling everything from source and using it in production is for insane people. Should have clarified that when I said "Ports/pkgsrc", I meant using official binary pacakges, and not necessarily building from source.
>latest and quarterly
The point of quarterly releases is to serve as a base for stable package versions, backported security fixes, and as a chance to fix package runtime/compilation issues before being generally available - something harder to do if you track bleeding-edge packages, where things can shift under people's feet.
It's also useful when building from source, like I do with pkgsrc for macOS, because lots of random things will break if there's too large of a rift between what's installed and what's in bleeding-edge pkgsrc. In these cases, you'd usually have to do mass rebuilds, and/or deal with dependency hell (eg, pkg X you have installed needs need <=libsomething-1336, but pkg Y in pkgsrc needs >=libsomething-1337).
I use the packages installed via pkgsrc pretty much daily on my MacBook, and it'd be really annoying having things possibly breaking after an update just a week apart, especially if I'm in a rush and need to get something done for work. So using a quarterly branch is useful, since it's a cadence I can stomach, and there's a higher likelihood that the devs had more time to polish the packages in that release.
>Fink
Never heard of this before. I'll give it a gander sometime soon.
Were it my responsibility I wouldn't worry too much about using FreeBSD binary packages in production. The way ports are set up, typically you'd have to go out of your way to install a bleeding edge version of something. If you're relying on something (e.g. ruby, python) with a bunch of commonly used versions you can alias the default to your preferred version. This may make audit compliance more difficult but not by much (especially considering how laborious compliance is anyhow).
At this point my biggest qualms about FreeBSD in production aren't really around the ports themselves but more around how much stuff has become Linux only (e.g. dockerify everything).
At megacorp we had to use an old version of CentOS (ugh) and the whole "keep updating ancient versions of third party software" thing came with plenty of caveats and frustrations. Regardless of the operating system it's not an approach I'd recommend.
Edit: Fink is the OSX port of dpkg more or less.
At this point my biggest qualms about FreeBSD in production aren't really around the ports themselves but more around how much stuff has become Linux only (e.g. dockerify everything).
At megacorp we had to use an old version of CentOS (ugh) and the whole "keep updating ancient versions of third party software" thing came with plenty of caveats and frustrations. Regardless of the operating system it's not an approach I'd recommend.
Edit: Fink is the OSX port of dpkg more or less.
I cannot say much with any confidence because I started revisiting FreeBSD after a long (like 15 years! :) ) absence. However, from what I see there are ways to mitigate issues that might occur. For example, doing a ZFS snapshot before doing a package update can be one thing to do. I think if you have a cluster of machines, doing a canary update with the option to rollback sounds like something that would ease the mind. I think also the tooling will allow you to control any bulk updating. This is in fact what I was starting to play with today - new port updates are available, how do I do selective updates? It's possible, but some reading of the handbook and/or man pages will get to that answer probably.
> I wish FreeBSD was more popular. I run it on my personal servers for more than a decade non-stop. ... About a year ago I migrated one FreeBSD to OpenBSD...
This is exactly how I feel about OpenBSD. While part of me is happy that it's a niche OS, another part of me wishes that it had more support by being a more mainstream OS.
This is exactly how I feel about OpenBSD. While part of me is happy that it's a niche OS, another part of me wishes that it had more support by being a more mainstream OS.
For me the main showstopper was a system requirements doc. Last time i tried instalation failed in a VM with max number of inodes reached. I do not want to try ZFS. I want to do a quick try to see if it plays well. Can anyone recommend the minimum disk size to install the port collection and to compile for example an LXDE or KDE ?
For me it's daily driver. Though not current but release.Just updated to 13.1-RELEASE yesterday.
For who isn't into FreeBSD, release is like debian stable and current is the upcoming dev release. Like debian sid but not rolling
For who isn't into FreeBSD, release is like debian stable and current is the upcoming dev release. Like debian sid but not rolling
Doesn't the notion of using anything "bleeding edge" sort of go against the fundamental law of production? Doesn't development sound like a much better fit for getting to know Current and the future of FreeBSD? I wouldn't speak for anyone else, but my ideal production environment is one where no one even thinks about the operating system, because it wasn't broken before, still isn't broken, and is expected to keep on not being broken; it just does its thing. Because whatever production is doing that is critical, it isn't system operating, it is something else, some big data manipulation, database something, or application crunching or generating something or people using the systems to produce. Maybe others have been more fortunate, but I've never had much success fixing something that isn't broken.
What is the downside to using Stable, or maybe even... Stable from two whole number versions ago? Honestly, adding features to production is, as a grey beard once said to me, paraphrasing, as useless as nailing jelly to a bull, or something along those lines.
What is the downside to using Stable, or maybe even... Stable from two whole number versions ago? Honestly, adding features to production is, as a grey beard once said to me, paraphrasing, as useless as nailing jelly to a bull, or something along those lines.
I once broke FreeBSD Current.
I was porting a zfs on linux commit to allow mounting filesystems in parallel, instead of copying the zfs-on-linux code I decided to read it and write it by hand. I didn’t test the patch… it was merged and was quickly reverted because it broke the boot sequence for many people.
great learning experience!
great learning experience!
Sounds like FreeBSD has a process problem if such a patch got merged.
Starting around 6.x, -CURRENT got ridiculously reliable (the infamously problematic 5.x release series informed the releng process a lot).
By late 00s, if you were running FreeBSD and pushing it to its limits in terms of performance or features, you'd probably be running current and have at least one committer or active contributor on the team, upstreaming as many business-specific improvements as possible while keeping track of incoming commit stream.
By late 00s, if you were running FreeBSD and pushing it to its limits in terms of performance or features, you'd probably be running current and have at least one committer or active contributor on the team, upstreaming as many business-specific improvements as possible while keeping track of incoming commit stream.
I used FreeBSD for about a decade and lately use Linux which has honestly just been a docker host for me. Does anyone on FreeBSD miss docker? Are jails a close subsitute?
Coming from a Linux background but experimenting heavily with jails lately, I think jails are more like LXD/LXC containers than docker.
While most jails usually start the init system inside the jail, you can set the "entrypoint" of a jail to just one app and run only that in the jail. That would be a little closer to a typical docker container.
The tooling around jails is not like docker. There is no central "hub" with different images centered around specific usecases, you mostly have to roll your own. There have been various efforts to improve that, but I haven't had the impression that I could rely on anything that isn't in FreeBSD base when it comes to jails. Too much abandoned wrappers and helpers out there.
While most jails usually start the init system inside the jail, you can set the "entrypoint" of a jail to just one app and run only that in the jail. That would be a little closer to a typical docker container.
The tooling around jails is not like docker. There is no central "hub" with different images centered around specific usecases, you mostly have to roll your own. There have been various efforts to improve that, but I haven't had the impression that I could rely on anything that isn't in FreeBSD base when it comes to jails. Too much abandoned wrappers and helpers out there.
Isn't the runj backend for containerd supposed to pave the way for native docker on jails?
Does Docker run as well on the BSDs as it does on Linux? and those Docker-ecosystem tools like the ones that scan your docker images for size optimizations?
The infra my projects work on are all heavily based on docker so that’s a hard requirement.
The infra my projects work on are all heavily based on docker so that’s a hard requirement.
It does not. You would have to run a VM like OSX and Windows do. Devs have not wanted to build in standardized ergonomic wrappers around jails like Docker, or some native answer to Docker compatibility itself, and you'll get talked down to in the user forums any time it's brought up ("use jails instead"). Though there are tools (iocage, Bastille, etc.) that make it easier to manage jails in a loosely similar way. https://github.com/sadaszewski/focker is an attempt to get the pieces together like Docker and docker-compose using some YAML, jails, and ZFS.
The title this HN is post wrong, it should be "Evaluating FreeBSD CURRENT for Production Use" as used in the original article.
The capitalization is important, since it makes it very clear it's talking about the CURRENT branch.
The capitalization is important, since it makes it very clear it's talking about the CURRENT branch.
Out of curiosity, as someone who doesn’t use FreeBSD what does “Current” (not all caps) imply instead?
It means the 'current' latest development code, not a release branch.
When FreeBSD cuts a release the code is handed over to a releng team (e.g on GitHub you'll see releng/13.1 atm) who handle security fix back ports and suchlike. Development then continues on master (or whatever the svn equiv is) until they cut another release.
So, this would be building from master. Or main. Whatever. Instead of using a release branch
:}
When FreeBSD cuts a release the code is handed over to a releng team (e.g on GitHub you'll see releng/13.1 atm) who handle security fix back ports and suchlike. Development then continues on master (or whatever the svn equiv is) until they cut another release.
So, this would be building from master. Or main. Whatever. Instead of using a release branch
:}
To clarify a bit: The release engineering team owns the branch during the release process, then hands it over to the security team.
(Of course there's always cooperation between the two teams -- the security officer has to get permission to commit to the release branch prior to the release but in practice that conversation consists of "we have a security advisory affecting the upcoming release" "go ahead". Theoretically the answer might come back "please wait a few hours because we're in the middle of builds right now" though.)
(Of course there's always cooperation between the two teams -- the security officer has to get permission to commit to the release branch prior to the release but in practice that conversation consists of "we have a security advisory affecting the upcoming release" "go ahead". Theoretically the answer might come back "please wait a few hours because we're in the middle of builds right now" though.)
I'm not sure. The intent seemed clear without the capitalization. (Though I've been a FreeBSD user for over 25 years.)
[deleted]
I took conservative approach and I always run -RELEASE version, not even -STABLE. I'm glad Klara is evaluating -CURRENT branch, good read.
Btw, once - several years ago I bought an new system and network card was not supported in the -RELEASE branch, so I took the driver from -CURRENT, compiled and loaded as kernel module. Worked flawlessly :)