Linux 5.1 Arrives(zdnet.com)
zdnet.com
Linux 5.1 Arrives
https://www.zdnet.com/article/linux-5-1-arrives/
44 comments
It also has the Spectre/Meltdown mitigations, which happen to significantly decrease performance.
Whats the current status of the situation between the ZFS-on-Linux project and Linux 5.1+?
current git works fine - 5.1 compat fixes just landed - https://github.com/zfsonlinux/zfs/commits/master
How is ZFS in general vs btrfs?
ZFS is basically killed by Oracle right?
ZFS is basically killed by Oracle right?
ZFS is OK I suppose, but it isn't native. So you have to jump through hoops to use it, just like Nvidia binary graphics drivers.
BTRFS works well. I've used it for a NAS since 2012. It started with two mirrored 2 TB WD RED drives. Since then it's gone through an expansion to 6 drives in RAID 10 and two drive failures and replacements. Pretty decent performance and never a serious problem.
I've also used it on two laptops with SSD. It had some weird edge cases there using Docker with a lot of snapshots and running out of space for metadata. Unlike some people, I found it pretty straight-forward to recover by adding more space using either a USB drive or reusing my swap partition.
BTRFS snapshots are easy to do and great for backups and Docker image layers.
I felt the need to write this because your comment got three or four anti-BTRFS responses, and BTRFS has always been good to me.
BTRFS works well. I've used it for a NAS since 2012. It started with two mirrored 2 TB WD RED drives. Since then it's gone through an expansion to 6 drives in RAID 10 and two drive failures and replacements. Pretty decent performance and never a serious problem.
I've also used it on two laptops with SSD. It had some weird edge cases there using Docker with a lot of snapshots and running out of space for metadata. Unlike some people, I found it pretty straight-forward to recover by adding more space using either a USB drive or reusing my swap partition.
BTRFS snapshots are easy to do and great for backups and Docker image layers.
I felt the need to write this because your comment got three or four anti-BTRFS responses, and BTRFS has always been good to me.
I agree, BTRFS has always been good to me as well and I use it day to day, however out of all the vague negativity in the comments below, this has real merit:
> ... raid5,6 had serious issues a while back - the write hole problem still exists for btrfs.
it does indeed. BTRFS should not be used on a RAID: https://btrfs.wiki.kernel.org/index.php/RAID56
> ... raid5,6 had serious issues a while back - the write hole problem still exists for btrfs.
it does indeed. BTRFS should not be used on a RAID: https://btrfs.wiki.kernel.org/index.php/RAID56
It shouldn't be used for RAID5 or 6. RAID0, 1 and 10 are fine.
No, OpenZFS is alive and thriving and we use it in production on many Centos systems.
btrfs is a awful. We used it for a brief time on a production system with vanilla mirrored drives. Absolutely horrible. I will never use btrfs for anything ever again. I've been running unix systems for 30 years; even the darkest days of UFS was not that bad.
btrfs is a awful. We used it for a brief time on a production system with vanilla mirrored drives. Absolutely horrible. I will never use btrfs for anything ever again. I've been running unix systems for 30 years; even the darkest days of UFS was not that bad.
Could you elaborate a bit on what was awful about it?
Sure. Our small ~500GB volume (on an OpenSuse system) became essentially read-only with ~50GB of free space on it. The error code was no free space. We clearly didn't have the "toolkit" needed to deal with such an issue. The box was a secured system, so we couldn't add an additional device easily (non-keyboard USB devices were disabled in the kernel, for example). Also, the read speed off the volume became terribly slow...it would've taken days to get the data off the device. We tried many commands to try to unbork the volume, with no luck. Even some of the btrfs commands were taking hours to run and became unkillable. We had to reboot the box several times...
We had a 24 hr old copy of the data and we were able to lift the few very important modified files off the device. So we reformatted with XFS and were able to restore the volume to a working state.
Fortunately, the failure happened late on a Friday so we had the weekend to put it all back together again.
I have only hatred for btrfs. Complete crap. Only our business competitors should use it ;-) We only chose to use it on this small system just so that we could get meaningful experience with it. We also had it running on another system where the next weekend we took emergency downtime and removed btrfs from it as well.
I've been using ZFS in production since it was released in Solaris 10 in 2006. It isn't perfect either. We did once hit a performance degradation which was fixed a few days later with a kernel patch. But that was 10 years ago. We never lost any data or sleep over it. We no longer have any Solaris boxes; they were all converted to Centos with ZFS. The main active ZFS pools are all 6TB of mirrored NVMe storage on systems with 2TB of ram and 48 core of INTC Platinum cpu's. Works like a charm.
We had a 24 hr old copy of the data and we were able to lift the few very important modified files off the device. So we reformatted with XFS and were able to restore the volume to a working state.
Fortunately, the failure happened late on a Friday so we had the weekend to put it all back together again.
I have only hatred for btrfs. Complete crap. Only our business competitors should use it ;-) We only chose to use it on this small system just so that we could get meaningful experience with it. We also had it running on another system where the next weekend we took emergency downtime and removed btrfs from it as well.
I've been using ZFS in production since it was released in Solaris 10 in 2006. It isn't perfect either. We did once hit a performance degradation which was fixed a few days later with a kernel patch. But that was 10 years ago. We never lost any data or sleep over it. We no longer have any Solaris boxes; they were all converted to Centos with ZFS. The main active ZFS pools are all 6TB of mirrored NVMe storage on systems with 2TB of ram and 48 core of INTC Platinum cpu's. Works like a charm.
If you fill a ZFS volume too far you will start encountering a lot of difficulties just as you do with btrfs. This is an edge case that you really should see coming if you've worked with either on real machines.
> ZFS is basically killed by Oracle right?
Uh, no? OpenZFS is being actively developed and maintained, albeit not by Oracle.
Uh, no? OpenZFS is being actively developed and maintained, albeit not by Oracle.
Being not by Oracle is a feature; not a bug.
Broad general statements are always difficult but FWIW:
- btrfs is fine if you want checksumming for your external HDD with some backup stuff that is mostly sequential i/o.
- For everything else - especially anything that goes beyond a single disk use ZFS. btrfs RAID1 is not really raid1 but rather oddness of pid decides which drive to read from, raid5,6 had serious issues a while back - the write hole problem still exists for btrfs. ZFS stripe, mirror, raidz are well understood and work flawless afaik. btrfs also does strange things when replacing disks, like going read-only when one disk fails - at least it used to be that way.
- performance is a mixed bag but broadly speaking for anything that is not sequential io on an empty disk btrfs is slower than other filesystems[1], for ZFS it depends can be slower or faster than ext4 depending on a lot of settings...
- ZFS tooling is far superior to btrfs - I've used both a lot and you won't go back to btrfs tooling once you've used ZFS.
- btrfs snapshots degrade performance with rising snapshot count. ZFS works fine.
- ZFS has native encryption with zfs send / recv support in the upcoming 0.8.0. So you can do encrypted differential backups this way.
- btrfs quota was for a long time broken and error-prone, not sure what the current status is but it never worked correctly for me.
- ZFS lacks cp --reflink and linux pagecache integration as well as sendfile() on Linux and the ZFS ARC is independed of the pagecache - that has some consequences for e.g. mmap().
- ZVOLs are ZFS only you can create block devices as ZFS datasets i.e. zfs create -V 10G rpool/xfs && mkfs.xfs /dev/zvol/rpool/xfs && mount /mnt /dev/zvol/rpool/xfs can be useful for ceph, running docker on xfs volumes in container
- btrfs works fine inside containers so you can run Docker in LXD/LXC - ZFS is not there yet - overlay2 support is also missing.
- no io-scheduling via cgroups for ZFS afaik because the Linux queue subsystem is not used...
- Last but not least: ZFS has ZIL & L2ARC. ZIL is huge for spinning disks because sync() writes are first written to an ssd storage and later async and sequential to the hdd. TL;DL: apt update and databases are fast even on spinning disks. L2ARC is an SSD-Cache but it's not so effective in my experience (low hitrate for typical hosting stuff)
- Personally I will avoid using btrfs if I can.
1: https://www.phoronix.com/scan.php?page=article&item=freebsd-... it's a little bit apples/oranges because ZFS Freebsd != Linux ZFS but IMHO it shows that the architecture/design is sound.
- btrfs is fine if you want checksumming for your external HDD with some backup stuff that is mostly sequential i/o.
- For everything else - especially anything that goes beyond a single disk use ZFS. btrfs RAID1 is not really raid1 but rather oddness of pid decides which drive to read from, raid5,6 had serious issues a while back - the write hole problem still exists for btrfs. ZFS stripe, mirror, raidz are well understood and work flawless afaik. btrfs also does strange things when replacing disks, like going read-only when one disk fails - at least it used to be that way.
- performance is a mixed bag but broadly speaking for anything that is not sequential io on an empty disk btrfs is slower than other filesystems[1], for ZFS it depends can be slower or faster than ext4 depending on a lot of settings...
- ZFS tooling is far superior to btrfs - I've used both a lot and you won't go back to btrfs tooling once you've used ZFS.
- btrfs snapshots degrade performance with rising snapshot count. ZFS works fine.
- ZFS has native encryption with zfs send / recv support in the upcoming 0.8.0. So you can do encrypted differential backups this way.
- btrfs quota was for a long time broken and error-prone, not sure what the current status is but it never worked correctly for me.
- ZFS lacks cp --reflink and linux pagecache integration as well as sendfile() on Linux and the ZFS ARC is independed of the pagecache - that has some consequences for e.g. mmap().
- ZVOLs are ZFS only you can create block devices as ZFS datasets i.e. zfs create -V 10G rpool/xfs && mkfs.xfs /dev/zvol/rpool/xfs && mount /mnt /dev/zvol/rpool/xfs can be useful for ceph, running docker on xfs volumes in container
- btrfs works fine inside containers so you can run Docker in LXD/LXC - ZFS is not there yet - overlay2 support is also missing.
- no io-scheduling via cgroups for ZFS afaik because the Linux queue subsystem is not used...
- Last but not least: ZFS has ZIL & L2ARC. ZIL is huge for spinning disks because sync() writes are first written to an ssd storage and later async and sequential to the hdd. TL;DL: apt update and databases are fast even on spinning disks. L2ARC is an SSD-Cache but it's not so effective in my experience (low hitrate for typical hosting stuff)
- Personally I will avoid using btrfs if I can.
1: https://www.phoronix.com/scan.php?page=article&item=freebsd-... it's a little bit apples/oranges because ZFS Freebsd != Linux ZFS but IMHO it shows that the architecture/design is sound.
ZFS on Linux is where the majority of ZFS innovation is these days. Btrfs should be permanently deleted except for a sole copy in a museum somewhere as an example of a historical mistake negatively affecting millions of people.
Why is everyone shitting on btrfs in this thread?
used it for an Hadoop cluster after I've read an interview with the author that it's ready for production and Suse talking that it's the premier filesystem for their enterprise distribution. At that time ZFS was at 0.6.x and the new ARC stuff was not there and no dkms for Ubuntu and I was afraid that nobody could handle that... A few month later I'm obsessed with reading any commit in master and the mailing list, because things like undeletable files, hard lockups of the whole cluster and manual reboots in rescue-image and running btrfsck and stuff like that started to took too much time. That beeing said no problems after 4.8 or so, besides out of space on 20gb used with lots of files in a 16tb raid-10 btrfs array but that was minor in comparision to the other problems...
Might be unfair, but I've learned my lesson there. I decided to follow the spectactle from a large distance.
Might be unfair, but I've learned my lesson there. I decided to follow the spectactle from a large distance.
Those who have used it have many battle scars that lead to us never ever trusting it again.
Apparently because quite a few people have had bad experiences with it? When filesystems go bad, they really go bad, so it makes sense that these users would feel motivated to share their experiences.
Perhaps it's a vocal minority, but does it matter? Isn't the error rate what's most important? I've seen ZFS horror stories too, but not nearly as many, and that says something at least.
Disclaimer: I've never used btrfs. I quite like ZFS.
Perhaps it's a vocal minority, but does it matter? Isn't the error rate what's most important? I've seen ZFS horror stories too, but not nearly as many, and that says something at least.
Disclaimer: I've never used btrfs. I quite like ZFS.
btrfs is a dumpster truck on fire and everyone serious about their data has switched to ZoL or is holding out in hopes that bcachefs won't be a similar clusterfsck (which shouldn't be too hard to do).
Btrfs has such a bad rep right now, but because such comments like yours lack any specifics I don't know if I should worry if all my systems with a simple btrfs root partition are going to spontaneously corrupt one day. Can anyone give concrete examples of what's wrong with btrfs?
Lilo doesn't support it. Other than that I have no problem with it. I think it had some problems with being declared "prod ready" by some vendors with some certain kernel version, and then a bunch of redhat users went off and tried it with ancient kernels and lost data.
To me, this is just ext4 all over again. That filesystem was _terrible_ in the early days. Ext4 is pretty ok now, just slow...
To me, this is just ext4 all over again. That filesystem was _terrible_ in the early days. Ext4 is pretty ok now, just slow...
HAMMER2 is the outside contender. See https://en.wikipedia.org/wiki/HAMMER2 (summary) and http://gitweb.dragonflybsd.org/dragonfly.git/blob_plain/HEAD... (design document)
It's BSD licensed, among other useful characteristics.
It's BSD licensed, among other useful characteristics.
I don’t think the kernel people have reverted their ZoL-hostile changes, if that’s what you’re asking about.
That said: ZoL works with Linux 5.0, but AFAIU that’s due to ZoL falling back to less optimal code. I know it works because I run both on my laptop.
That said: ZoL works with Linux 5.0, but AFAIU that’s due to ZoL falling back to less optimal code. I know it works because I run both on my laptop.
> The first of these is Linux now supports persistent memory as RAM. Sure, non-volatile memory (NVM) isn't as fast as good old RAM, but on newer systems it gives you the option of expanding your memory.
That sounds like swap. What's the difference?
That sounds like swap. What's the difference?
Swap has to get paged into real RAM when there is a page fault (presumably evicting something else). I'm guessing that this is actually mapping the NVM directly into the process, so individual loads and stores write directly to the device instead of actual RAM.
That seems like an implementation detail done for the performance of prioritizing the use of the faster device. If you need something that's in storage/swap, chances are you're going to need it repeatedly and so it's a performance boost to "swap" that with whatever is being used least in RAM.
From a functional point of view, it's still really just expanding your memory with a persistent storage device.
> I'm guessing that this is actually mapping the NVM directly into the process, so individual loads and stores write directly to the device instead of actual RAM.
If this were the only difference, then it would be just swap without its main optimization and for no noticeable benefit. I doubt this is all there is to it.
From a functional point of view, it's still really just expanding your memory with a persistent storage device.
> I'm guessing that this is actually mapping the NVM directly into the process, so individual loads and stores write directly to the device instead of actual RAM.
If this were the only difference, then it would be just swap without its main optimization and for no noticeable benefit. I doubt this is all there is to it.
Traditional swap is more complicated than that. The kernel can't map it directly as RAM, it has to be paged to and from the backing store and RAM - doing so is not just an optimization.
I think that this article explains it somewhat: https://lwn.net/Articles/777212/
Basically there's Intel Optane DIMM which costs less than DRAM and offers good enough performance, so you can use Optane with DRAM as a cache and those patches provide proper Linux support for that scheme.
Basically there's Intel Optane DIMM which costs less than DRAM and offers good enough performance, so you can use Optane with DRAM as a cache and those patches provide proper Linux support for that scheme.
Isn't swap basically expanded memory while using DRAM as cache? It seems like this could be just a different way of operating that cache.
I wonder if this new feature would be used with swapon/swapoff or if it would need a different interface.
I wonder if this new feature would be used with swapon/swapoff or if it would need a different interface.
Intel have a fork of Redis that uses Optane storage instead of RAM. The end result is a persistent DB that can operate at the speed of Redis.
> Basically there's Intel Optane DIMM
These chips are extraordinary, when the prices finally go down to a reasonable level it will be another revolution in both persistent (speed) and volatile (size) memory.
These chips are extraordinary, when the prices finally go down to a reasonable level it will be another revolution in both persistent (speed) and volatile (size) memory.
SSD on DIMM.
So, a RAM Drive?
https://en.wikipedia.org/wiki/RAM_drive
https://en.wikipedia.org/wiki/RAM_drive
RAM drive used RAM as a storage device. This is the other way around. Storage used as RAM. Drive RAM, if you will.
No. RAM drives use normal, volatile RAM. Power down and it all goes away. SSD is non-volatile. Power down and it stays.
DIMM is a form factor and specific bus for getting data from storage on one side of the slot to CPU on the other. This is putting a different type of storage on the other side of that bus.
DIMM is a form factor and specific bus for getting data from storage on one side of the slot to CPU on the other. This is putting a different type of storage on the other side of that bus.
[deleted]
https://kernelnewbies.org/LinuxChanges more detailed changelog
Oh goodie, I get a new kernel version to compile repeatedly for the next few days.
If any of you are successfully rocking Linux on an LG Gram (especially the 17Z990), and you have decent ACPI support, I'd loooooooove to see your kernel config file. Please.
If any of you are successfully rocking Linux on an LG Gram (especially the 17Z990), and you have decent ACPI support, I'd loooooooove to see your kernel config file. Please.
Posting your current config would allow others to see if their own is relevantly different. Sorry I can't help though, I'll be building this for an arm chromebook, not a pc, so my configs will be nothing helpful.
Hm storage as RAM. Sounds like what IBM AS/400 was doing like 30 years ago for mini-frame machines.
A lot of the current server trends are basically reinventing wheels that were already spinning on mainframes multiple decades ago.
Those things were rock solid as well...
One of the coolest new features was just on HN:
Efficient IO with io_uring, the new async I/O interface in Linux 5.1
https://news.ycombinator.com/item?id=19843464
Personally I'm most excited for "Safe signal delivery in presence of PID reuse". I think the kernel/syscall boundary has so many cracks when it comes to race-conditions. It really shows how much has changed with respect to concurrency.
Personally I'm most excited for "Safe signal delivery in presence of PID reuse". I think the kernel/syscall boundary has so many cracks when it comes to race-conditions. It really shows how much has changed with respect to concurrency.