A modular, extensible DIY NAS(blog.georgovassilis.com)
blog.georgovassilis.com
A modular, extensible DIY NAS
https://blog.georgovassilis.com/2020/04/01/building-the-perfect-cheap-diy-nas/
112 comments
> Moral of the story: If you are depending on certain hardware features, you need to have a cold spare available with identical hardware and firmware. If you cannot afford one, it's better to either use a software implementation, have a really good service contract with the vendor or just let a VM provider take care of your infrastructure.
+1 for this. Hardware raid absolutely has its place, but if you expect to be able to rebuild an array without restoring from a backup, you absolutely need to have spare parts that are tested in advance.
Also, RAID is not a backup. RAID provides high(er) availability. You need to have backups still!
+1 for this. Hardware raid absolutely has its place, but if you expect to be able to rebuild an array without restoring from a backup, you absolutely need to have spare parts that are tested in advance.
Also, RAID is not a backup. RAID provides high(er) availability. You need to have backups still!
>Also, RAID is not a backup. RAID provides high(er) availability. You need to have backups still!
Absolutely true. In this case we underestimated the workload caused by the service tag and firmware version issues and the fact that the new controller wanted to check and rebuild the array, when the disks themselves should have been fine. In hindsight it might have been faster to reinstall the OS and services plus restore the backups.
Absolutely true. In this case we underestimated the workload caused by the service tag and firmware version issues and the fact that the new controller wanted to check and rebuild the array, when the disks themselves should have been fine. In hindsight it might have been faster to reinstall the OS and services plus restore the backups.
I always thought the rise of RansomWare would put a nail in the ol' "RAID as poor-man's backup" coffin, but guess not!
> Also, RAID is not a backup.
You are absolutely right, but there are use-cases where backup is too expensive, so investing into robustness and availability is the only option left.
Backups give you a third thing: immutability.
You can have a nice redundant storage for local performance and resilience, even geographic replication for resilience against a single site loss, but you need snapshots of some kind to protect against replicated destructive updates.
You can have a nice redundant storage for local performance and resilience, even geographic replication for resilience against a single site loss, but you need snapshots of some kind to protect against replicated destructive updates.
>backup is too expensive, so investing into robustness and availability is the only option left.
No please don't say something like that if you love your data. Raid is for the availability of your data, a Backup is the secure storage/versioning of you data, two completely different things.
No please don't say something like that if you love your data. Raid is for the availability of your data, a Backup is the secure storage/versioning of you data, two completely different things.
My love for my data has a price tag :-) If I afford backup with this amount, I need to cut down on other expenses in my life.
It might help separating your data into different tiers of "love", and deciding on your backup strategies from there.
For example, my irreplaceable photos (probably around 3TB worth) get packed into year-based TAR files, then uploaded to Glacier Deep Archive. It's only around $3 USD a month.
I won't have to care about retrieval cost for the moment, because I think I'll only need to touch it in the event of a catastrophe.
Most media files and Linux ISOs don't get backed up at all.
For example, my irreplaceable photos (probably around 3TB worth) get packed into year-based TAR files, then uploaded to Glacier Deep Archive. It's only around $3 USD a month.
I won't have to care about retrieval cost for the moment, because I think I'll only need to touch it in the event of a catastrophe.
Most media files and Linux ISOs don't get backed up at all.
Have a look at some used LTO4 tape-drives ~40$ they can store up to 1.6TB per Tape (compressed) and are the perfect thing to offline your data.
Thanks. I looked at the specifications; it's not clear how that would connect to the server. All SATA ports are taken and USB is too slow (estimated 72 hours) for a full backup and would require 6-8 tape changes.
Used PCIe Fiber card or cloud-something like Tarsnap or Backblaze?
BTW: For Tape it's REALLY important to have a steady Dataflow, too slow like USB is a big NONO, you NEED something like SAS/SATA or Fibre. The tape can but should not stop and wait for new data to be written.
EDIT: Well then look at LTO5 (double cap)...LTO6 is probably still too expensive.
BTW: For Tape it's REALLY important to have a steady Dataflow, too slow like USB is a big NONO, you NEED something like SAS/SATA or Fibre. The tape can but should not stop and wait for new data to be written.
EDIT: Well then look at LTO5 (double cap)...LTO6 is probably still too expensive.
Good, cheap, fast. Pick two.
This is the reason I now use a 3rd party RAID controller on my HP Microserver Gen 8 - I have 3 RAID cards with the same hardware and firmware, and tested that the RAID config can be imported from one to the other.
I ran into the same (type of) problem years ago.
I think I had a raid card with two mirrored drives. Thing is when one drive failed, I couldn't just pull the data off because the remaining drive could only be accessed by that raid controller.
I think I had a raid card with two mirrored drives. Thing is when one drive failed, I couldn't just pull the data off because the remaining drive could only be accessed by that raid controller.
An untested disaster recovery or backup plan is a bad plan. Sorry you had to learn that the hard way.
Agreed that identical or confirmed working spare components are important.
I've migrated RAID arrays through several generations of Areca cards successfully and even tested back and forth between models. I like them.
Also remember RAID is not backup; keep a second copy of your data around (offsite if you're serious).
Agreed that identical or confirmed working spare components are important.
I've migrated RAID arrays through several generations of Areca cards successfully and even tested back and forth between models. I like them.
Also remember RAID is not backup; keep a second copy of your data around (offsite if you're serious).
IIRC the Linux kernel RAID infrastructure supports various on-disk data structures generated by various hardware RAID controllers, so probably it would be easier to recover from your situation in 2020.
I am not sure but I think that it is for various "soft RAIDs" like those that come with AHCI / SATA controllers (including integrated ones). True hardware RAID controlelr probably require more magic.
While I'm quite new to the DIY NAS, I have been running a FreeNAS instance for about a year and finally decided to switch to Ansible-NAS[1] about 6 months ago.
It has all the features I'd want and seamlessly supports ZFS as my filesystem without standing in the way (since it's all just Ansible playbooks)
[1] https://davidstephens.uk/ansible-nas/
[1] https://davidstephens.uk/ansible-nas/
Hey, thanks for the shout :-)
> I hear and read conflicting testimonies about ZFS which supposedly handles data corruption better and will give it a try at some point. For now, I use md5sum to create and verify checksums of my backup .tar.gz archives and parchive to create error recovery files. I’ve written about parchive before and remain to this day a dedicated fan.
I’m very surprised you went through the trouble of building such an amazing NAS and skipped ZFS! Definitely give it a try. It’s especially useful for combatting disk rot. Not to mention how easy it is to setup and use.
I’m very surprised you went through the trouble of building such an amazing NAS and skipped ZFS! Definitely give it a try. It’s especially useful for combatting disk rot. Not to mention how easy it is to setup and use.
It definitely sounds overwhelming at first but once you get the gist of it it is quite easy to manage. I even somehow managed to migrate my FreeBSD pool to OpenZFS without any major hurdles.
Thanks. I haven't tried ZFS so far for historical reasons: the post describes the latest evolution of my NAS; in fact, there were several hardware versions before with the hard disks being transplanted (and later upgraded) from the previous to the next. The filesystem structures are over a decade old, having been upgraded from ext2 to ext3 and now ext4.
The next thing I will try out is dm-integrity.
The next thing I will try out is dm-integrity.
Why not also have a look at btrfs?
I bookmarked a ZFS-inspired storage system for preserving data integrity, and dm-integrity has a role there:
https://insanity.industries/post/preserving-data-integrity/
Hacker News discussion: https://news.ycombinator.com/item?id=24372662
https://insanity.industries/post/preserving-data-integrity/
Hacker News discussion: https://news.ycombinator.com/item?id=24372662
Most filesystems that are designated as "modern" include checksumming, snapshots, cow, etc.
However, do some include support for CRC? That would allow not only detecting, but also correcting bitrot. I assume that's what Raid6 does, albeit less granular than per file, and more aimed at recovering a full file than a few bits?
Can anyone point me to a FS that supports it, or to the reason it's not done?
As for ZFS, I think I'll keep avoiding it until it is mainlined. I don't really want to play that game.
However, do some include support for CRC? That would allow not only detecting, but also correcting bitrot. I assume that's what Raid6 does, albeit less granular than per file, and more aimed at recovering a full file than a few bits?
Can anyone point me to a FS that supports it, or to the reason it's not done?
As for ZFS, I think I'll keep avoiding it until it is mainlined. I don't really want to play that game.
ZFS adds complexity and licensing issues into the mix. parchive is a good way to separate different layers of concerns to achieve similar end point.
I built a similar little machine a couple years ago, mostly for storing my big archived files, being a borg backup target for my other machines, and subsequently mirroring said backups offsite with rclone.
I went for a Chenbro SR301 case (I don't know what's changed in the still-available SR301 Plus model) + modest commodity parts over the HP or Supermicro turnkey boxes in a similar form-factor mostly because it takes standard components (mini-ITX boards, ATX PSU, standard fans... nothing I couldn't overnight from any major vendor for cheap) for easy replacement. It was also a little cheaper.
I did btrfs "RAID10" instead of RAID. Hardware RAID is always a hassle, btrfs is in-kernel, and it has all the nice modern data integrity features (two spindle rule, I have a timer that scrubs it periodically for to verify checksums, etc.) - I've been really pleased with that except for the lack of good automated monitoring/notification tools for btrfs.
Documentation I wrote at the time, more about the backup scheme than the machine at https://pappp.net/?p=1627 I'm still using that setup, eventually I'll need to swap some discs as capacities increase, but that machine has required zero fucking around since it was set up.
I went for a Chenbro SR301 case (I don't know what's changed in the still-available SR301 Plus model) + modest commodity parts over the HP or Supermicro turnkey boxes in a similar form-factor mostly because it takes standard components (mini-ITX boards, ATX PSU, standard fans... nothing I couldn't overnight from any major vendor for cheap) for easy replacement. It was also a little cheaper.
I did btrfs "RAID10" instead of RAID. Hardware RAID is always a hassle, btrfs is in-kernel, and it has all the nice modern data integrity features (two spindle rule, I have a timer that scrubs it periodically for to verify checksums, etc.) - I've been really pleased with that except for the lack of good automated monitoring/notification tools for btrfs.
Documentation I wrote at the time, more about the backup scheme than the machine at https://pappp.net/?p=1627 I'm still using that setup, eventually I'll need to swap some discs as capacities increase, but that machine has required zero fucking around since it was set up.
Been building DIY NAS servers for 15 years for my personal use. ZFS with initially OpenSolaris, then FreeBSD. Typically 5 to 7 drives in raidz2.
I found it very strange the guy is happy with & measured 100 Mbps throughput over the network. Any 2+ drive NAS from the last 5 years should be able to saturate a 1Gbps link.
I found it very strange the guy is happy with & measured 100 Mbps throughput over the network. Any 2+ drive NAS from the last 5 years should be able to saturate a 1Gbps link.
I think the OP means 100 MB/s and not 100 Mbps here
I wrote "100 mb/s" - I now realise capitalisation is important.
Well, there is also the LUKS encryption overhead which caps transfer rates from the RAID. I also run boinc (giving back to the community) which limits available CPU.
LUKS is capable of using AES-NI, and most CPUs are faster at AES-NI than two-lane NVMe drives at delivering that data. It should be absolutely no problem with SATA.
His CPU has no AES-NI, then it is a bit of a problem. But i was really astonished that a celeron supports ECC...quite nice ;)
https://ark.intel.com/content/www/de/de/ark/products/71074/i...
EDIT: He upgraded to a XEON..so forget all above.
https://ark.intel.com/content/www/de/de/ark/products/71074/i...
EDIT: He upgraded to a XEON..so forget all above.
What is the point to run RAID6 on 4 drives, instead of RAID10? With both, half the space is usable, but with RAID10 read speed is twice as high, and resilvering is twice as fast.
RAID6 makes sense for 5-8 drives.
RAID6 makes sense for 5-8 drives.
> What is the point to run RAID6 on 4 drives, instead of RAID10?
With RAID10, if the 'wrong' two disks fail, the array will go offline.
With RAID6, any two disks can fail and the array will still be online.
EDIT: The author has also demonstrated that they've gone through various growth permutations over the years. 4 disk RAID6 under md-raid has a supported growth mechanism, allowing you to add additional disks and have it restripe data automagically for you. It's not a good idea as far as performance is concerned, but for a home use nas that is primarily intended for streaming media, it's not a major issue, and enables you to grow to your heart's content (even if it is a bad idea, as I successfully proved growing from 4 disks to 16 over a few years, before respinning as multiple 6-disk arrays)
With RAID10, if the 'wrong' two disks fail, the array will go offline.
With RAID6, any two disks can fail and the array will still be online.
EDIT: The author has also demonstrated that they've gone through various growth permutations over the years. 4 disk RAID6 under md-raid has a supported growth mechanism, allowing you to add additional disks and have it restripe data automagically for you. It's not a good idea as far as performance is concerned, but for a home use nas that is primarily intended for streaming media, it's not a major issue, and enables you to grow to your heart's content (even if it is a bad idea, as I successfully proved growing from 4 disks to 16 over a few years, before respinning as multiple 6-disk arrays)
Actually, according to recent testing by ArsTechinca [0] performance with RAID the topic is more complicated.
The testing is focused on ZFS but, quoting from the conclusion: If you're looking for raw, unbridled performance it's hard to argue against a properly-tuned pool of ZFS mirrors. RAID10 is the fastest per-disk conventional RAID topology in all metrics, and ZFS mirrors beat it resoundingly—sometimes by an order of magnitude—in every category tested, with the sole exception of 4KiB uncached reads.
[0] https://arstechnica.com/gadgets/2020/05/zfs-versus-raid-eigh...
The testing is focused on ZFS but, quoting from the conclusion: If you're looking for raw, unbridled performance it's hard to argue against a properly-tuned pool of ZFS mirrors. RAID10 is the fastest per-disk conventional RAID topology in all metrics, and ZFS mirrors beat it resoundingly—sometimes by an order of magnitude—in every category tested, with the sole exception of 4KiB uncached reads.
[0] https://arstechnica.com/gadgets/2020/05/zfs-versus-raid-eigh...
> it's hard to argue against a properly-tuned pool of ZFS mirrors ... ZFS mirrors beat it resoundingly
Emphasis on 'properly-tuned'. This doesn't hold as true as the OP states for the defaults (which is what the majority of home users will be using)
Emphasis on 'properly-tuned'. This doesn't hold as true as the OP states for the defaults (which is what the majority of home users will be using)
A pool of mirrored pairs is effectively the ZFS equivalent of RAID10.
https://www.klennet.com/notes/2019-08-04-zfs-vs-RAID10.aspx
https://www.klennet.com/notes/2019-08-04-zfs-vs-RAID10.aspx
I’d urge anyone planning on building or buying a home server to try out Unraid. It’s not free or open source, but it’s only $130 iirc for the unlimited lifetime license and it Just Works^tm.
You install it on a USB stick, and the web interface has you up and running within like 2 minutes. The only hard rule is that your parity drives must be as large as or the same size as your data drives, but other than that you’re completely free to add or remove disks one, two, or ten at a time and all it takes is like 3 button clicks.
The community is large so there’s always someone to help should you run into trouble but in general the whole thing is pretty brainless. Click button, server work.
You install it on a USB stick, and the web interface has you up and running within like 2 minutes. The only hard rule is that your parity drives must be as large as or the same size as your data drives, but other than that you’re completely free to add or remove disks one, two, or ten at a time and all it takes is like 3 button clicks.
The community is large so there’s always someone to help should you run into trouble but in general the whole thing is pretty brainless. Click button, server work.
Unraid is getting popular on some DIY space but I feel that it's hobby-ish product. I astonished that it won't set root password and able to login by default. Write performance is terrible (by design) and "SSD cache mechanism" is made by crappy shell script. I agree that it still worth for some use case.
The whole point of the exercise is to have a device completely under your control. Closed source need not apply for security / trust reasons.
Otherwise, e.g. Synology produces excellent devices, etc.
Otherwise, e.g. Synology produces excellent devices, etc.
>Synology produces excellent devices,
Sorry but had quite the opposite experience even with the professional lines. The only NAS i trust is a Server (at least HP mini server) and ZFS. Stuff like unraid is for Windows mindsets, i don't trust HW-Raid and i especially don't trust Closedsource Software BS for stuff like that, exceptions are Enterprise stuff like EVA's or NetApp and EMC.
Sorry but had quite the opposite experience even with the professional lines. The only NAS i trust is a Server (at least HP mini server) and ZFS. Stuff like unraid is for Windows mindsets, i don't trust HW-Raid and i especially don't trust Closedsource Software BS for stuff like that, exceptions are Enterprise stuff like EVA's or NetApp and EMC.
Synology OS is a modified Linux and it uses mdraid for RAID. There is no hardware RAID, and very little closedsource bullshit.
This is how you'd mount a synology array in a Linux server: https://www.synology.com/en-us/knowledgebase/DSM/tutorial/St... - it's basically two terminal commands after all the preparation stuff.
I believe QNAP uses mdraid as well but I haven't had any experience with those.
This is how you'd mount a synology array in a Linux server: https://www.synology.com/en-us/knowledgebase/DSM/tutorial/St... - it's basically two terminal commands after all the preparation stuff.
I believe QNAP uses mdraid as well but I haven't had any experience with those.
>Synology OS is a modified Linux and it uses mdraid for RAID.
Yes and that raid destroyed itself many times which btw never happened to me on a self-installed Linux.
Yes and that raid destroyed itself many times which btw never happened to me on a self-installed Linux.
> Yes and that raid destroyed itself many times which btw never happened to me on a self-installed Linux.
I'm sorry you've had the misfortune of encountering such problems.
I've been using Synology/Xpenology (a fork of their GPL bootloader) for 8 years, and it's still going great for me.
I'm sorry you've had the misfortune of encountering such problems.
I've been using Synology/Xpenology (a fork of their GPL bootloader) for 8 years, and it's still going great for me.
They where all from that RS -Series i think we had about 15 of them from RS14XX to RS1219 and all of them destroyed their own Raid after around 2 years just like that, no firmware update before that or other things that could trigger such a crash.
I've been using Unraid - https://unraid.net/ - for the past few years now (after playing around with countless FreeNAS and custom setups).
I'm really impressed with how stable it is - never had any issues with it and no dataloss, even with a couple of hard drive failures along the way.
I know many of you will prefer to customise everything and take a deep dive into the way things are set up, but I really do enjoy the simplicity of Unraid.
I'm really impressed with how stable it is - never had any issues with it and no dataloss, even with a couple of hard drive failures along the way.
I know many of you will prefer to customise everything and take a deep dive into the way things are set up, but I really do enjoy the simplicity of Unraid.
I don't understand why anyone would do this instead of running FreeNAS. It comes with everything you need including ZFS pools.
I'm quite comfortable with my NAS running debian and using btrfs-based RAID to manage the disks. Setting it up has been quite straightforward and it's been running for the last 8 years without breaking a sweat. For SOHO applications is more than enough.
Because you can't easily change the size of the disks one at a time. How would you migrate from 4 x 4TB to 4 x 6TB disks by buying one disk at a time?
You replace each disk in turn, then grow the pool, right? You don't get the increased capacity until you're done, but still workable.
Workable, but not the point.
This is why I picked Synology as my nas. The Synology Hybrid Raid system let me just shove all my extra drives in the 4 bay NAS and still have parity protection.
Then I just slowly upgraded all drives to the current money/capacity optimum one or two at a time.
If I was made of money and could afford to upgrade all the drives in a pool in one go, I'd definitely go for ZFS too.
This is why I picked Synology as my nas. The Synology Hybrid Raid system let me just shove all my extra drives in the 4 bay NAS and still have parity protection.
Then I just slowly upgraded all drives to the current money/capacity optimum one or two at a time.
If I was made of money and could afford to upgrade all the drives in a pool in one go, I'd definitely go for ZFS too.
You don’t need to upgrade at once. You can upgrade gradually with the right config.
You gradually replace one disk at a time, resilver then repeat. ZFS automates most or all the work mentioned in this article and it comes with built-in bitrot detection.
Btw I have run a NAS similar to this machine for 6 years now using FreeNAS. It’s awesome.
Btw I have run a NAS similar to this machine for 6 years now using FreeNAS. It’s awesome.
So if I have 4x4TB drives and switch a drive so it's now 3x4TB + 1x6TB, and do the resilver, will the ZFS capacity be increased by 2TB?
No not until you have replaced them all.
In that case, I don't understand why you replied to my comment asking "How would you migrate from 4 x 4TB to 4 x 6TB disks by buying one disk at a time?". In case it wasn't clear, the buying one disk at a time aspect is important, because storage needs increase steadily over time, not all at once, so it makes financial sense to buy a higher capacity drive and replace a lower capacity one over a longer period of time, instead of replacing all drives at once and then have most of the extra storage sit there unused.
FreeNAS is great but last time I checked it didn't support the Opteron based microservers from HP. I don't know if this has changed in the past few years but it's why I went Linux on mine.
According to
https://www.servethehome.com/hpe-proliant-microserver-gen10-...
the 10+ seems to run freenas fine.
https://www.servethehome.com/hpe-proliant-microserver-gen10-...
the 10+ seems to run freenas fine.
Better range of hardware support in Linux than FreeBSD.
Also note that the FreeBSD implementation of ZFS has been replaced and ported from OpenZFS (formerly ZFSonLinux).
Also note that the FreeBSD implementation of ZFS has been replaced and ported from OpenZFS (formerly ZFSonLinux).
On paper. In practice FreeBSD has support for the great hardware you should be using anyway. The sketchy stuff, and the oddball only one person even has it in their personal museum Linux has better support for. (Also the latest and greatest linux will generally get support for first, but if it is worth using FreeBSD will get support)
I have plenty of experience with Ubuntu and wanted to reuse that. Learning FreeNAS just for this one machine I am building seems like an overkill.
Please note that modern microserver called HP Microserver Gen10 plus. Also to make Gen8 almost silent, it's necessary to utilize built-in fake-raid (not very convenient with Linux).
I don't think it is true: I have an almost silent gen8 and I'm not using the built-in raid
I think there is some spread of the fans (case and power supply) used in the gen8. My gen8 is ok for an office or a large room, but too loud for a study room. Unfortunately the fan has a special pinout and cannot be replaced easily with a quieter piece: https://www.youtube.com/watch?v=ZPn8tXP-ZjI
ah ok, because I can barely hear my fan (and I never changed it)
I checked with iLO and fan rotates at much lower speed with RAID configuration. I would be happy to run it without RAID, but it's too loud. I've found that solution on homeservers forum, so I guess, I'm not the only one.
I think they fixed that in a firmware upgrade.
Can you please post(or send me over email) your BIOS/iLO versions? I plan do do a NAS build with a Gen8 and it would greatly help me to know what versions don't have the fan noise issue. Thank you in advance!
J06 04/04/2019
I think there's been some work on the fan speen in recent BIOS/iLO versions. AFAIK HP currently recommend you put the controller in AHCI mode for use with linux nowadays. I haven't seen versions of their Linux fake-raid driver for a few years now. I think it's been retired.
Another issue I've seen people report as a reason for the increased fan speed were hdds which didn't report temperature properly to the controller so the controller didn't know whether the drives were cool or hot.
What BIOS/iLO versions are you running?
Another issue I've seen people report as a reason for the increased fan speed were hdds which didn't report temperature properly to the controller so the controller didn't know whether the drives were cool or hot.
What BIOS/iLO versions are you running?
My BIOS is J06 04/04/2019, iLO firmware 2.70 May 07 2019.
Latest B120i driver was released at 30 Apr 2020. I don't think it's retired yet, but they don't support RHEL 8, so I'm forced to stay on RHEL 7.
I can see that iLO firmware is not latest, but it's BIOS who's responsible for fan speed control and I think that I'm running latest BIOS.
That's not really about BIOS versions. When HDD runs under B120i fakeraid mode, server is aware about HDD temperature which is displayed in iLO. As long as this temperature is low, fans stay low. When HDD runs under AHCI mode, server is not aware about HDD temperature anymore, so it uses high fan speed to ensure that cooling is good enough even if drives are hot. That's how I interpret this whole situation.
Latest B120i driver was released at 30 Apr 2020. I don't think it's retired yet, but they don't support RHEL 8, so I'm forced to stay on RHEL 7.
I can see that iLO firmware is not latest, but it's BIOS who's responsible for fan speed control and I think that I'm running latest BIOS.
That's not really about BIOS versions. When HDD runs under B120i fakeraid mode, server is aware about HDD temperature which is displayed in iLO. As long as this temperature is low, fans stay low. When HDD runs under AHCI mode, server is not aware about HDD temperature anymore, so it uses high fan speed to ensure that cooling is good enough even if drives are hot. That's how I interpret this whole situation.
> Latest B120i driver was released at 30 Apr 2020. I don't think it's retired yet
Ok, I might have misunderstood the info posted at: http://downloads.linux.hpe.com/SDR/project/ubuntu-hpdsa/ which states: "NOTICE! The hpdsa driver is no longer being developed past the versions indicated. Do not upgrade the Ubuntu kernel or try to use this binary driver with Xenial. If you have a B-series Smart Array, it is advised to use the Linux md (software raid) driver, or upgrade to an H or P series hardware-based host bus adapter."
I thought the hpdsa driver was the same across distros, not unique to Ubuntu. Based on what you're telling me it seems that only the Ubuntu version was discontinued.
Ok, I might have misunderstood the info posted at: http://downloads.linux.hpe.com/SDR/project/ubuntu-hpdsa/ which states: "NOTICE! The hpdsa driver is no longer being developed past the versions indicated. Do not upgrade the Ubuntu kernel or try to use this binary driver with Xenial. If you have a B-series Smart Array, it is advised to use the Linux md (software raid) driver, or upgrade to an H or P series hardware-based host bus adapter."
I thought the hpdsa driver was the same across distros, not unique to Ubuntu. Based on what you're telling me it seems that only the Ubuntu version was discontinued.
I have 3x Microservers Gen8 and they are all dead silent.
You're _that_ louwrentius? I read your posts when I tried to replace the case fan!
I guess so...
What's the power use on this? I would have thought x86 is a terrible choice for a NAS, as it only does the one thing and you don't need much processing power to do it (and implicitly x86's advantage lays primarily in being 1: fast and 2: compatible with x86 binaries, not in being low-power).
I run a similar setup and run various things like Wireguard (so I can connect in), an HTTP server, Home Assistant for my house automations, automated nightly backups, etc etc. It's more than just data access.
Decent x86 desktop computer (or server that uses same platform like this Microserver) consumes around 10-20W at idle that isn't "terrible". It also provides great performance. Atom is better for power consumption. Arm isn't option now because there's no equivalent parts.
I upgraded to a Xeon processor because LUKS encryption was way too slow on the stock Celeron.
The problem with extensibility here is the fact that the enclosure is limited to four drives. Growing the capacity is quite hard but probably necessary considering the RAID 6 only gives you two HDDs worth of usable storage.
I have worked around that with external USB drives. Which is very slow :-(
Do you include your external drives in your RAID-6? That sounds dangerous.
No :-) The comment was about reshaping/expanding RAID. Since all 4 HDD bays are already taken, I usually connect an external HDD through USB for the reshaping.
I tried something similar after looking at what I really needed a NAS for: serving music and video to my home network, and via Wireguard VPN if needed. When I actually looked at the requirements in detail I set up a Raspberry Pi 4B with 2Gb memory, a 1Tb SSD connected via USB3/SATA and just installed Samba and MiniDLNA. Careful tweaking of the Samba config file ensured that it played nicely with all devices here, including Roku and Apple. It's simple, works well and is backed up to another, offline disk.
I'd love HP to release a Ryzen based MicroServer
Asus already has one (zen2) and Asrock is about to. I’d imagine HP is only a matter of time
Can you link to any or provide model names?
Asus one is PN50. 4300U - 4800U
Unsure about asrock. I think they maybe have pivoted to a non renoir cpu
Unsure about asrock. I think they maybe have pivoted to a non renoir cpu
The PN50 is not a miniserver: total storage is an NVMe slot and space for a 2.5" SSD. It's a mini-desktop, successor to their A500.
The A500 is very nice, though in practice it needs a USB3 hub sitting next to it.
The A500 is very nice, though in practice it needs a USB3 hub sitting next to it.
Thanks, this looks like a really small form factor PC, not a server/nas.
[deleted]
A NAS seems like a nice way to consolidate my photoalbums and have redundancy in terms of physical storage. But that whole cloud thingy is something I don't need, heck, I don't want!
I would say that a good router is the way to go and I think I should replace the one my ISP has given me. Can anyone point me in the direction for a guide to a hardware router and appropriate firmware, so I can be sure to control what the NAS (or any other device within the network) can connect to on "the internet"?
I would say that a good router is the way to go and I think I should replace the one my ISP has given me. Can anyone point me in the direction for a guide to a hardware router and appropriate firmware, so I can be sure to control what the NAS (or any other device within the network) can connect to on "the internet"?
[deleted]
> Can anyone point me in the direction for a guide to a hardware router and appropriate firmware, so I can be sure to control what the NAS (or any other device within the network) can connect to on "the internet"?
If you're looking to build something more powerful, OPNsense on an x86 box is the way to go. I also think that the interface is much more intuitive compared to OpenWRT's.
If you're looking to build something more powerful, OPNsense on an x86 box is the way to go. I also think that the interface is much more intuitive compared to OpenWRT's.
> The script is a bit more complex than that as device names are not stable and various error conditions need to be taken into account
I always use /dev/disk/by-id/* .
I always use /dev/disk/by-id/* .
So I had this QNAP TS451+ sitting and gathering dust after a move. I had no interest in dealing with the terrible QNAP OS on it and wanted to do something open source.
The only somewhat difficult part was preserving data from the old disks as QNAP has custom extensions to LVM or MDRAID which work nowhere else. So there were a few back and forth of removing a few disks from the pool, rebooting to a normal linux, formatting them without the custom garbage, booting back and copying over.
In the end I decided to go with NixOS as I've already used it on my laptop/desktop for about 5 years. I also wanted a modern CoW file system and went with bcachefs. Getting it working with NixOS takes some fiddling, especially if you also want encryption but it works. It now runs samba, matrix-synapse and soon maybe nextcloud. Not quite there yet with everything but quite happy that I went down this route.
The only thing missing from the HW side is an SSD, but the box has USB3 and UAS support so I might add an external one that bcachefs can use as a faster tier.
The only somewhat difficult part was preserving data from the old disks as QNAP has custom extensions to LVM or MDRAID which work nowhere else. So there were a few back and forth of removing a few disks from the pool, rebooting to a normal linux, formatting them without the custom garbage, booting back and copying over.
In the end I decided to go with NixOS as I've already used it on my laptop/desktop for about 5 years. I also wanted a modern CoW file system and went with bcachefs. Getting it working with NixOS takes some fiddling, especially if you also want encryption but it works. It now runs samba, matrix-synapse and soon maybe nextcloud. Not quite there yet with everything but quite happy that I went down this route.
The only thing missing from the HW side is an SSD, but the box has USB3 and UAS support so I might add an external one that bcachefs can use as a faster tier.
[deleted]
zfs works great, you have a lot of benefits on a nas like snapshots, scrubbing,...
Qnap OS Hero Edition [1], They said it will be coming to consumer version soon. Hopefully we will soon be able to buy a consumer NAS with ZFS without too much fiddling.
[1] https://www.qnap.com/quts-hero/en/
[1] https://www.qnap.com/quts-hero/en/
Read "DIY NAS" and was thinking...sure a HP Mini and Freenas or pure FreeBSD or Xigmanas and what do i see, Ubuntu...but that's probably just because of the Bitcoin mining....with a Celeron... :)
I upgraded to a Xeon because of LUKS encryption; Celeron doesn't have hardware support for encryption. Indeed I run a computationally heavy workload - boinc with protein folding and TB prevention :)
Ah nice...because of AES-NI! Being a longtime user/giver to World Community Grid.
A quick word of caution: RAID is not a backup (as already mentioned elsewhere in the comments). Even more so in home solutions whether they are DIY or off the shelf.
Why? Because storing a single copy of your data on a single local RAID array doesn't protect you from disasters such as fire, theft, accidental fluid spills, small children/pets, a power surge after a lightning strike, flooding, and so on. Basically, a lot of the stuff you'd fine in your home insurance policy.
RAID doesn't guarantee data integrity either. You lost data after an unintended deletion or corruption by some computer program? RAID won't help you get your data back any more then a single disk scenario would. If the original file got overwritten and mirrored across the array, and all you had was that one copy, you won't be able to go back to that original.
A more exceptional scenario would be a drive silently failing and RAID happily mirroring corrupted data across the entire array. Unless you use a filesystem that implements checksums - like ZFS - you wouldn't notice this unless it was too late. [1]
[1] https://superuser.com/questions/112683/does-raid-1-protect-a...
I'm not advising against RAID. I think there's merit in using RAID when it comes to convenience and availability. Like, your data remaining available when you experience a sudden disk failure, and not having to spend hours or days getting everything back on line again.
I'd say that RAID and backups are complementary.
So, what about backups then? The 3-2-1 backup strategy is a good start: 3 copies, 2 local, 1 physically remote. That could be as simple as copying your drives every week to a separate HD or SSD drive, and every month to another separate drive which is safely stored outside of your home, i.e. a family member, friend, or at the bank.
You could step it up a notch and choose to rsync your data across two connected drives, while rsync a third copy to specialized cloud storage service such as Borgbackup or Wasabi. Add in scripts that perform regular checksums control and reports the results daily and you've got a pretty solid solution.
The final step is looking at your data and differentiating between the stuff you absolutely can't afford to lose (e.g. family albums) and the stuff you may afford to lose (e.g. software downloads). The goal here is to calculate the costs associated with required storage volume and bandwidth, and land on an appropriate backup solution that matches your budget and your needs.
Finally, that's just safeguarding your data in the present and short term future. Neither RAID or backups are a long term preservation strategy. Which is a whole other can of worms, including such challenges as migrating obsolete data formats (Yeah, I have WordPerfect 5 and AutoCAD files in my own personal archive), or dealing with obsolete hardware I/O (Yeah, I have a bunch of old IDE drives in a cardboard box for which I had to buy an I/O converter).
Why? Because storing a single copy of your data on a single local RAID array doesn't protect you from disasters such as fire, theft, accidental fluid spills, small children/pets, a power surge after a lightning strike, flooding, and so on. Basically, a lot of the stuff you'd fine in your home insurance policy.
RAID doesn't guarantee data integrity either. You lost data after an unintended deletion or corruption by some computer program? RAID won't help you get your data back any more then a single disk scenario would. If the original file got overwritten and mirrored across the array, and all you had was that one copy, you won't be able to go back to that original.
A more exceptional scenario would be a drive silently failing and RAID happily mirroring corrupted data across the entire array. Unless you use a filesystem that implements checksums - like ZFS - you wouldn't notice this unless it was too late. [1]
[1] https://superuser.com/questions/112683/does-raid-1-protect-a...
I'm not advising against RAID. I think there's merit in using RAID when it comes to convenience and availability. Like, your data remaining available when you experience a sudden disk failure, and not having to spend hours or days getting everything back on line again.
I'd say that RAID and backups are complementary.
So, what about backups then? The 3-2-1 backup strategy is a good start: 3 copies, 2 local, 1 physically remote. That could be as simple as copying your drives every week to a separate HD or SSD drive, and every month to another separate drive which is safely stored outside of your home, i.e. a family member, friend, or at the bank.
You could step it up a notch and choose to rsync your data across two connected drives, while rsync a third copy to specialized cloud storage service such as Borgbackup or Wasabi. Add in scripts that perform regular checksums control and reports the results daily and you've got a pretty solid solution.
The final step is looking at your data and differentiating between the stuff you absolutely can't afford to lose (e.g. family albums) and the stuff you may afford to lose (e.g. software downloads). The goal here is to calculate the costs associated with required storage volume and bandwidth, and land on an appropriate backup solution that matches your budget and your needs.
Finally, that's just safeguarding your data in the present and short term future. Neither RAID or backups are a long term preservation strategy. Which is a whole other can of worms, including such challenges as migrating obsolete data formats (Yeah, I have WordPerfect 5 and AutoCAD files in my own personal archive), or dealing with obsolete hardware I/O (Yeah, I have a bunch of old IDE drives in a cardboard box for which I had to buy an I/O converter).
I run ZFS to mitigate half of what you say. I trust ZFS more than backups to keep my data correct - checksums ensure against bitrot. I also have snapshots, so unless I don't notice the file is gone for years it is there to restore easily.
What you say about protection from fire/theft is still real though. I'm looking for a good (which includes easy) solution for that.
What you say about protection from fire/theft is still real though. I'm looking for a good (which includes easy) solution for that.
I think the complexity of RAID might even itself be the reason of data loss in quite a few cases. Of course in a corporate environment with whole racks full of disks it makes sense, but at home I just rather synchronise single disks. Make/buy a NAS with two disks for media files, use one as master and copy the changes from one to the other at midnight or whatever frequency makes most sense. It gives you a chance to undo stupid command line mistakes, and you don't need to have similar models of disk. Then keep an external eSATA/USB disk to backup the main disk once a week/month.
When I was younger, all these experiments with firewalls, routers, vlans, RAID and other more professional technologies were a good learning experience, but after all those years, I rather keep work at work and keep it simple at home rather than to simulate a corporate environment on a shoestring budget.
When I was younger, all these experiments with firewalls, routers, vlans, RAID and other more professional technologies were a good learning experience, but after all those years, I rather keep work at work and keep it simple at home rather than to simulate a corporate environment on a shoestring budget.
We were running some services on a stack of older Dell servers with mostly identical hardware. The RAID controller and mainboard on one of them let the magic smoke out. We (a sysadmin team of three) promptly decided to move the RAID array to another unused server in the stack in order to avoid a lengthy reinstall and manual restoring of daily backups – this happened before the almost-one-click provisioning of devops and virtualization tools was the norm.
The other server lacked the RAID license dongle that was on a pin header on the mainboard, so we just tried to cannibalize it from the server corpse that barely had stopped smoking. The new mainboard promptly rejected the license because the service tag differed, and the only way to change it was to install Windows 95 or 98 and run the tag changer tool. In addition, we had to downgrade the controller firmware to match that of the old one. One nightlong unattended rebuild later, the server was up and running the next morning.
When we upgraded the server hardware a bit later, we migrated it over to mdraid and haven't had any problems with hardware failures since.
Moral of the story: If you are depending on certain hardware features, you need to have a cold spare available with identical hardware and firmware. If you cannot afford one, it's better to either use a software implementation, have a really good service contract with the vendor or just let a VM provider take care of your infrastructure.