SanDisk’s silence deafens as high-profile users say Extreme SSDs still broken(arstechnica.com)
arstechnica.com
SanDisk’s silence deafens as high-profile users say Extreme SSDs still broken
https://arstechnica.com/gadgets/2023/08/sandisk-extreme-ssds-are-still-wiping-data-after-firmware-fix-users-say/
34 comments
> Having seen quite a few stories like this about WD and sub brands, I'm tempted to just return it for a refund and go elsewhere...
The thing is, where? We don't really have a choice, it's either WD(and sub) or Seagate(and sub) with horror stories on both end. I had a WD red fail 3 weeks after the warranty period, while also having an older WD red that outlived it, chugging along for 6 years now. Now I run a mixes pool of WD and Seagate because ultimately, I don't trust either of them.
The thing is, where? We don't really have a choice, it's either WD(and sub) or Seagate(and sub) with horror stories on both end. I had a WD red fail 3 weeks after the warranty period, while also having an older WD red that outlived it, chugging along for 6 years now. Now I run a mixes pool of WD and Seagate because ultimately, I don't trust either of them.
Toshiba is actually pretty good in my book. I've yet to have a major failure in my NAS from Toshiba drives I procured over a few years ago.
The problem with Toshiba is that their enterprise drives have no warranty for non-enterprise customers in the EMEA region.
HDDs are very durable when off, I’d be fine running it but I have backups.
Otherwise you may want to purchase them in person so you can see the box.
I’ve received drives from Newegg that were just tossed in a box with some of those large bubbles and they’re still going fine.
Otherwise you may want to purchase them in person so you can see the box.
I’ve received drives from Newegg that were just tossed in a box with some of those large bubbles and they’re still going fine.
What are you talking about? Newer drives support shocks of around 200G, while older ones supported 250G even 300G.
Step 1: Never use Sandisk products. There cheaply made and the company has a history of not really caring about its products.
Step 2: never store data you don't want to lose on a SSD. Use SSD's for system operations and caching, but never long term storage.
Always stay abreast of high quality products by comparing prices, performance and endurance. By looking and real world benchmark/reviews. In my experience, I go with the products that are more expensive.
IMHO, generally speaking, Samsung SSD products are the best on the market.
Step 2: never store data you don't want to lose on a SSD. Use SSD's for system operations and caching, but never long term storage.
Always stay abreast of high quality products by comparing prices, performance and endurance. By looking and real world benchmark/reviews. In my experience, I go with the products that are more expensive.
IMHO, generally speaking, Samsung SSD products are the best on the market.
Is point 2) still valid nowadays? I know many years ago ssd durability (when unpowered) was low, but hasn’t that significantly improved to the point of not mattering?
And what’s the alternative you’re proposing, spinning rust? Is that what you yourself are using, is all your non-system data on spinning drives? Imo better would be “store all your data on quality ssds if you can afford it, and do 3:2:1 backup for whatever you can’t afford to lose”.
Just one piece of anecdata - I have a nas at home with 5x16tb seagate spinning drives, and a consumer grade 8tb sata qlc Samsung ssd in the sixth slot. One of the spinny drives completely died after few months and needed replacement. And the consumer-grade samsung ssd still going fine after 1.5 years of power on hours with a small but continuous write workload.
And what’s the alternative you’re proposing, spinning rust? Is that what you yourself are using, is all your non-system data on spinning drives? Imo better would be “store all your data on quality ssds if you can afford it, and do 3:2:1 backup for whatever you can’t afford to lose”.
Just one piece of anecdata - I have a nas at home with 5x16tb seagate spinning drives, and a consumer grade 8tb sata qlc Samsung ssd in the sixth slot. One of the spinny drives completely died after few months and needed replacement. And the consumer-grade samsung ssd still going fine after 1.5 years of power on hours with a small but continuous write workload.
Backblaze has larger scale statistics which show that this theory that ssds are inherently unreliable is just bs. In fact they turn out to be slightly more reliable than hdds:
https://www.backblaze.com/blog/ssd-drive-stats-mid-2022-revi...
Just make sure you have backups because hdd or ssd, disks fail.
https://www.backblaze.com/blog/ssd-drive-stats-mid-2022-revi...
Just make sure you have backups because hdd or ssd, disks fail.
Everything about NAND flash only gets worse with newer smaller processes, all in the name of lower $/GB.
SSD's are rated for 3-10 years or some number of terabytes written to disk.
My system's SSD's are both 3 years old, and most of my friends' are over 5 years old. So I would say it is a problem.
My system's SSD's are both 3 years old, and most of my friends' are over 5 years old. So I would say it is a problem.
Don't misunderstand. Bit rot occurs on unpowered solid state in as little as one month, guaranteed by one year. If you're thinking, "but what about my $brandModel?", that's the point of recognizing it's not long term storage media.
Bit rot doesn't mean you're going to lose all your data, just that somewhere on the chip, there will be a cell with a charge that has potentially silently altered. If that's not okay, there are long term storage practices you can follow to ensure that in however many years, your data will be identical to its originally written state.
Bit rot doesn't mean you're going to lose all your data, just that somewhere on the chip, there will be a cell with a charge that has potentially silently altered. If that's not okay, there are long term storage practices you can follow to ensure that in however many years, your data will be identical to its originally written state.
> there are long term storage practices you can follow
Can you please point me towards some resources?
Can you please point me towards some resources?
Systems that use error correcting and robustness codes, checksums, like used on CDs, plus duplication/redundancy.
There are resilient file systems that will spread ECC data chunks across multiple devices/media where bit-flips and partial media loss can be detected and repaired, sometimes automatically. There's offline backup software that'll do similar things.
There are resilient file systems that will spread ECC data chunks across multiple devices/media where bit-flips and partial media loss can be detected and repaired, sometimes automatically. There's offline backup software that'll do similar things.
Ssd's fail suddenly and catastrophically one day with the controller failing most of the time. Hard disks will show sector errors and warnings before that event. Most of the time the data on the drive can be copied out even in case of corruption.
My experience, I use usb flash drives as the system drive in my nas, is that sandisk makes the best flash, samsung is next best and kingston is garbage.
It has been a few years since one failed(thus the ringing endorsement of sandisk) but I started out with kingston datatravelers and burnt through 3 of them in 6 months, I am still a bit sore about this because the data traveler was the perfect form factor, I then bought a mix of sandisk cruiser fits and samsung bars, in the 5 years following, one of the samsung units has failed and I am still on that first sandisk fit.
The kingston units might be perfectly fine under normal use, but exhibited all sorts of strange problems when used as a system drive.
If curious it is a truenas box with the system installed on a two unit usb flash mirror. The way truenas works it is sort of a waste to install it on anything large, so while less than ideal(buy lots of spares) usb flash fits better.
It has been a few years since one failed(thus the ringing endorsement of sandisk) but I started out with kingston datatravelers and burnt through 3 of them in 6 months, I am still a bit sore about this because the data traveler was the perfect form factor, I then bought a mix of sandisk cruiser fits and samsung bars, in the 5 years following, one of the samsung units has failed and I am still on that first sandisk fit.
The kingston units might be perfectly fine under normal use, but exhibited all sorts of strange problems when used as a system drive.
If curious it is a truenas box with the system installed on a two unit usb flash mirror. The way truenas works it is sort of a waste to install it on anything large, so while less than ideal(buy lots of spares) usb flash fits better.
Speaking of TrueNAS, in theory ZFS is a lot better for long-term storage. The ARC (RAM caching) and L2 ARC (SSD caching) means that there's a lot less wear on the drives.
My TrueNAS Scale box has 16GB of RAM and a 1TB SSD as a cache. If the SSD fails, it doesn't really matter: the data is always checked against the checksum (as I understand it). If the drive fails, it just uses the backing storage or the ARC.
My TrueNAS Scale box has 16GB of RAM and a 1TB SSD as a cache. If the SSD fails, it doesn't really matter: the data is always checked against the checksum (as I understand it). If the drive fails, it just uses the backing storage or the ARC.
I thought that USB flash drives were not made to withstand the duty cycle required of system drives, and that's why SSDs cost more - because they have the hardware to transparently detect errors and remap them to a reserve set of locations.
This is mostly true, things like truenas and routers(many ubiquiti edgerouters use usb flash drives as the system drive) get away with it by reducing writes. truenas only writes to the system drive when you make a configuration change. all logging is done on the data drives.
Honestly I am not sure why usb flash drives are so terrible. no market for good drives?
Honestly I am not sure why usb flash drives are so terrible. no market for good drives?
Truenas recommends SSDs since half a decade ago or so due to the high failure rate. The forums felt pretty flooded with posts about failed USB-drives for a while there, and even before that I switched to installing on a SSD since I got tired of having a triple-mirror USB-setup just to keep the system alive long enough to be able to replace them (system was located across town back then).
USB drives are fine, but they’re just made for a different use. You can spend money and get good ones, but super cheap ones exist so it’s safer to say “avoid”.
Personally, I swore off Samsung SSDs after enjoying them for a while when I suddenly had two of them fail in a month after varying lengths of use.
It wasn't the failure that made me swear them off - failures happen, seven different layers of backups and all that fun, it was that I tried to RMA them, spent an entire day trying to do it, and literally could not find a way to do it.
Their web site was broken in every browser I tried with entirely vanilla no extensions anything, and I found reports of that being true going back years, and told me to try calling them.
Twice their phone tree eventually hung up on me, the third time I got a human who seemed incapable of understanding I could be calling about anything but a phone.
At the point where your warranty coverage is useless, the value proposition you offer changes.
It wasn't the failure that made me swear them off - failures happen, seven different layers of backups and all that fun, it was that I tried to RMA them, spent an entire day trying to do it, and literally could not find a way to do it.
Their web site was broken in every browser I tried with entirely vanilla no extensions anything, and I found reports of that being true going back years, and told me to try calling them.
Twice their phone tree eventually hung up on me, the third time I got a human who seemed incapable of understanding I could be calling about anything but a phone.
At the point where your warranty coverage is useless, the value proposition you offer changes.
Make sure you have a backup of the data you care about.
If you don't care about backup, you don't care about data.
No storage lasts forever. SDD, HDD, DVD, TAPE, etc will fail, regardless of brand.
Backup is cheap. Losing data can be priceless...
No storage lasts forever. SDD, HDD, DVD, TAPE, etc will fail, regardless of brand.
Backup is cheap. Losing data can be priceless...
On one hand I have a 4GB sandisk cruzer drive for sneakernet transfers still working well since 2008. On the other, 3 of my sandisk extreme ssds died this year. Luckily I had mirrored backups elsewhere.
> IMHO, generally speaking, Samsung SSD products are the best on the market.
There seems to be 870 EVO reliability issue going on for years now [1]. Shit happens, what matters is how you handle it. Samsung has yet to issue any public statement on the matter.
[1] https://www.techpowerup.com/forums/threads/samsung-870-evo-b...
There seems to be 870 EVO reliability issue going on for years now [1]. Shit happens, what matters is how you handle it. Samsung has yet to issue any public statement on the matter.
[1] https://www.techpowerup.com/forums/threads/samsung-870-evo-b...
I realize you might not be talking about external storage but I love my T5s. They’re the realization of SSD as a storage medium for me - durable, light, small, fast and very affordable
As an added redundancy, I use Arq backup for Mac to backup to s3 using my own encryption key.
As an added redundancy, I use Arq backup for Mac to backup to s3 using my own encryption key.
Storing data I would rather not transport again currently on 30 TiB Kioxias but if you didn't want to you can use overlayfs to make a single view over a slower disk.
HDDs are older technology and logically are an order of magnitude simpler than SSDs.
A really bad HDD will most likely fail within the first few hours of use. If not, the AFR (annualized failure rate) of the worst HDDs are under 3%.
A really bad SSD often provides no clue until it's too late --- as this case shows.
If safety and security and integrity is the main concern, you should be using HDD in my opinion.
A really bad HDD will most likely fail within the first few hours of use. If not, the AFR (annualized failure rate) of the worst HDDs are under 3%.
A really bad SSD often provides no clue until it's too late --- as this case shows.
If safety and security and integrity is the main concern, you should be using HDD in my opinion.
The difference is that a good SSD has a far lower AFR than a good hard drive, at least if Puget systems and backblaze are to be believed.
You mean "good" like in maybe a Samsung SSD?
https://www.pugetsystems.com/blog/2023/02/02/update-on-samsu...
You gotta love the manufacturer's response --- good news, we have update for that --- now that your data is gone.
https://www.pugetsystems.com/blog/2023/02/02/update-on-samsu...
You gotta love the manufacturer's response --- good news, we have update for that --- now that your data is gone.
Yeah if you never updated your firmware for years and years something bad could happen. For context the problem was patched in 2021 before being discovered in 2023. In the mean time with modern firmware they had a 0.2% failure rate according to Puget’s blog.
Sorry, does not compute.
If the problem was patched in 2021, why does their new product line launched in late 2022 still have problems?
If safety and security and integrity is the main concern, you should be using HDD in my opinion.
If the problem was patched in 2021, why does their new product line launched in late 2022 still have problems?
If safety and security and integrity is the main concern, you should be using HDD in my opinion.
There were separate unrelated issues with the 980 pro and 990 pro early firmware both of which were patched.
All these unrelated issues are just extra failure modes that don't exist with HDD.
But because it's software that caused the loss of all your data and it can be patched, it doesn't really count as a "failure" in hardware terms.
But because it's software that caused the loss of all your data and it can be patched, it doesn't really count as a "failure" in hardware terms.
I sent the customer support team an email asking them to confirm that the level of box damage seen wouldn't affect the drive, or alternatively to provide a return label and replacement drive. Their response was "try it out, and if it fails, we'll replace it".
Having seen quite a few stories like this about WD and sub brands, I'm tempted to just return it for a refund and go elsewhere...