Magnetic Tape Storage Technology: usage, history, and future outlook(dl.acm.org)
dl.acm.org
Magnetic Tape Storage Technology: usage, history, and future outlook
https://dl.acm.org/doi/10.1145/3708997
32 comments
I'm surprised nobody has mentioned Microsoft's research towards "replacing" tape storage in the future. Have a look at Project Silica.
https://www.microsoft.com/en-us/research/project/project-sil...
https://www.microsoft.com/en-us/research/project/project-sil...
Research like this has existed for decades. Cerabyte is a new company trying it again. The problem is that you need essentially a large and expensive optical lab to be able to read and write. This means you want to multiplex data carriers even more, and this it eats into bandwidth even more.
My company moved EiBs of data off of tape a few years ago. It was reliable and durable, but the problem was read speed.
It took so long to move tapes around and read the sequentially (no random access!), and as the data corpus grew it got harder to have a practical backup, even though the data was still theoretically extant.
It took so long to move tapes around and read the sequentially (no random access!), and as the data corpus grew it got harder to have a practical backup, even though the data was still theoretically extant.
Most of the people who parrot the "tape is dead" stuff haven't used tape.
When it comes to the reliability of putting something on a shelf, then pulling it off twenty years later, tape still is better than everything else.
When it comes to the reliability of putting something on a shelf, then pulling it off twenty years later, tape still is better than everything else.
Yes, definitely true. And maybe they are even more relevant now because of all the data needed for deep learning.
However, the drives are expensive. This industry is in dire need of disruption.
However, the drives are expensive. This industry is in dire need of disruption.
Yep since IBM has the monopoly they have been squeezing everyone to the maximum degree. Too bad HP stopped their tape drive development. They kept IBM under control.
$5k for a LTO9 drive isn’t terribly bad; once you have any significant number of tapes.
Expensive for home use, but they can buy older technology off-lease.
Expensive for home use, but they can buy older technology off-lease.
> Expensive for home use, but they can buy older technology
I can't imagine home users being interested in buying mostly used SCSI or SAS tape drives while navigating a world of format compatibility challenges and problems with improper storage. Environmental requirements for archival are narrow and most homes don't tick that box over many years or when moving.
This medium is expensive, inconvenient to use and store, and in the world of home use those are killers. You don't need to take my word for it, look around at tape home use.
Home users are better served by cloud storage or an external hard drive, maybe a home NAS, especially for the relatively low data volumes home usage usually involves.
I can't imagine home users being interested in buying mostly used SCSI or SAS tape drives while navigating a world of format compatibility challenges and problems with improper storage. Environmental requirements for archival are narrow and most homes don't tick that box over many years or when moving.
This medium is expensive, inconvenient to use and store, and in the world of home use those are killers. You don't need to take my word for it, look around at tape home use.
Home users are better served by cloud storage or an external hard drive, maybe a home NAS, especially for the relatively low data volumes home usage usually involves.
The choice between magnetic tapes and disks depends mainly on the total amount of archived data.
Some years ago, after I bought a LTO-7 drive at around $3000 as a home user, I have recovered its costs after about a couple hundred terabyte of stored data.
Unfortunately, nowadays the drives for LTO-9 have increased in price, so the cutoff threshold has probably increased to several hundred terabytes.
Even when the amount of stored data does not provide significant savings in the cost of storage media, it may still be worthwhile to use magnetic tapes, for improved peace of mind and for avoiding the hassle of copying the data to newer HDDs every few years.
I am old enough to have seen enough data loss disasters, so I would never trust cloud storage, where the access to my own data would be dependent on my ability of making continuous payments to an external entity, which is really hard to predict for any distant future. Moreover, even with a fast Internet link the access speed to cloud storage is an order of magnitude slower than to a local tape drive or HDD.
Some years ago, after I bought a LTO-7 drive at around $3000 as a home user, I have recovered its costs after about a couple hundred terabyte of stored data.
Unfortunately, nowadays the drives for LTO-9 have increased in price, so the cutoff threshold has probably increased to several hundred terabytes.
Even when the amount of stored data does not provide significant savings in the cost of storage media, it may still be worthwhile to use magnetic tapes, for improved peace of mind and for avoiding the hassle of copying the data to newer HDDs every few years.
I am old enough to have seen enough data loss disasters, so I would never trust cloud storage, where the access to my own data would be dependent on my ability of making continuous payments to an external entity, which is really hard to predict for any distant future. Moreover, even with a fast Internet link the access speed to cloud storage is an order of magnitude slower than to a local tape drive or HDD.
Look, I get it, you're a power user with very special needs. But the rest of the home user world is on a different page. It's funny if you think home users in general have these kinds of wants or needs just because you're a user, at home.
The data volumes, the cost even before we look at the TCO, the performance characteristics, the time/expertise requirements, the need (hassle) for proper storage and retrieval really kill the attraction of tape for home use.
For the backup (and storage as a bonus) needs of most home users cloud or external drives are unbeatable, especially in combination.
The data volumes, the cost even before we look at the TCO, the performance characteristics, the time/expertise requirements, the need (hassle) for proper storage and retrieval really kill the attraction of tape for home use.
For the backup (and storage as a bonus) needs of most home users cloud or external drives are unbeatable, especially in combination.
> For the backup (and storage as a bonus) needs of most home users cloud or external drives are unbeatable, especially in combination.
Cloud is not a backup. Cloud is someone else's storage.
Cloud is not a backup. Cloud is someone else's storage.
[deleted]
I should have been more precise - by “home user” I meant people who run DataCenter equipment at home; datahoarders and homelabs.
What I have said is that for this kind of home users external HDDs are obviously the right solution as long as the total amount of stored data is not much more than 100 TB.
For some threshold in the hundreds of TB range, magnetic tapes become cheaper, despite the huge price of a tape drive, while offering additional advantages, e.g. higher sequential reading and writing speeds and higher reliability.
Moreover, when computing the size of the stored data, one should take into account that the useful data size, after data compression, should be multiplied with 1.05 or 1.10, because you should add redundancy with a code able to reconstruct the data when only a small part of it is corrupted, then you should multiply by 2 or by 3, because any long-term archives must be stored as duplicated or even triplicated on different HDDs or tape cartridges, which are preferably kept at different geographical locations.
Only with such precautions you can be pretty certain that no data loss will occur after many years of data storage, reaching a reliability comparable with that of printed paper.
For some threshold in the hundreds of TB range, magnetic tapes become cheaper, despite the huge price of a tape drive, while offering additional advantages, e.g. higher sequential reading and writing speeds and higher reliability.
Moreover, when computing the size of the stored data, one should take into account that the useful data size, after data compression, should be multiplied with 1.05 or 1.10, because you should add redundancy with a code able to reconstruct the data when only a small part of it is corrupted, then you should multiply by 2 or by 3, because any long-term archives must be stored as duplicated or even triplicated on different HDDs or tape cartridges, which are preferably kept at different geographical locations.
Only with such precautions you can be pretty certain that no data loss will occur after many years of data storage, reaching a reliability comparable with that of printed paper.
The only part of this that is semi accurate is that SAS interface devices are a bit of a chore for the home user. I have an ATTO HBA in my Mac Pro 5,1 and it works like a charm with LTO-7 external drive. Otherwise, there are no format compatibility challenges; no stringent environmental issues. The 6 TB LTO-7 tapes are cheap and with LTFS, pretty easy to use - like drag and drop easy. I’m not sure what other issues you have encountered but if you have enough data to justify the investment, using tape in 2025 is quite straightforward.
One thing that worries me about home tape use is dust and cat hair.
I've had a DDS4 tape way back, which ended up dying, and that could well be the cause. My house is not going to be as clean as a server room.
I've seen tape drives taken apart and there seems to be a worrying lack of concern with any kind of air filtration on the ones I've seen at least. And I don't think it should be all that hard to deal with it. Maybe something like sucking air in through a replaceable filter and exhausting it out of the tape door.
I've had a DDS4 tape way back, which ended up dying, and that could well be the cause. My house is not going to be as clean as a server room.
I've seen tape drives taken apart and there seems to be a worrying lack of concern with any kind of air filtration on the ones I've seen at least. And I don't think it should be all that hard to deal with it. Maybe something like sucking air in through a replaceable filter and exhausting it out of the tape door.
Wouldn't a home air filter sitting adjacent to the tape drive remove almost all of that problem?
Maybe, but it seems to me that the one dust sensitive and very expensive piece of hardware would be not be designed to at least try to minimize the problem.
I was extremely interested in tape when I started my homelab several years ago.
The only options with more data capacity than a hard drive are all high end datacenter equipment. I would have had to buy fiber channel adapters and media, find a drive, a housing to put it in, and tapes. All separately, and each for several times what my entire homelab is worth.
It really is not an option for home-gamers.
The only options with more data capacity than a hard drive are all high end datacenter equipment. I would have had to buy fiber channel adapters and media, find a drive, a housing to put it in, and tapes. All separately, and each for several times what my entire homelab is worth.
It really is not an option for home-gamers.
> $5k for a LTO9 drive isn’t terribly bad; once you have any significant number of tapes.
It's the library/robot is what mostly matters for volume (and being able to automate). Not sure what the price is on those (along with continued support).
It's the library/robot is what mostly matters for volume (and being able to automate). Not sure what the price is on those (along with continued support).
How about small startups?
The break even point between tape and hard drives is somewhere around 400TB. A lot for a personal data collection, but not that much in absolute terms.
Blueray or even (possibly combined with) a 3rd party service probably makes way more sense for small businesses and individuals. That's what I currently use for archival.
Where is cerabyte? Fucking tapes in 2025. Tell me this isn't an artificially controlled market.
> tape
Only once (or very few times) overwritten?
> on a shelf
At home or in a specialized room with controlled climate?
At least I cannot find quickly the requirement in someone (or a robot) pulling each tape once a year(?) and doing a rewind? Which is formally needed for HDD.
Btw, shouldn't HDDs be much more resistant to magnetic fields than magnetic tapes?
Edit:
26TB HDD Non-operating / storage:
Temperature -40 to 70°C (Storage 0 to 70°C)
Relative humidity 5 to 95% non-condensing
Maximum wet bulb temperature: 35°C non-condensing
Maximum temperature gradient: 30°C/Hour
LTO 9+:
Recommended Storage Environment: 15 to 25℃ / 20 to 50%RH / Max Wet Bulb Temperature: 26℃.
Stray magnetic field at any point on tape not to exceed 50 oersteds (4000 ampere/meter).
[1] https://documents.westerndigital.com/content/dam/doc-library...
[2] https://www.fujifilm.com/uk/en/business/data-management/data...
[3] https://www.ibm.com/docs/en/storage-deep-archive?topic=media...
Only once (or very few times) overwritten?
> on a shelf
At home or in a specialized room with controlled climate?
At least I cannot find quickly the requirement in someone (or a robot) pulling each tape once a year(?) and doing a rewind? Which is formally needed for HDD.
Btw, shouldn't HDDs be much more resistant to magnetic fields than magnetic tapes?
Edit:
26TB HDD Non-operating / storage:
Temperature -40 to 70°C (Storage 0 to 70°C)
Relative humidity 5 to 95% non-condensing
Maximum wet bulb temperature: 35°C non-condensing
Maximum temperature gradient: 30°C/Hour
LTO 9+:
Recommended Storage Environment: 15 to 25℃ / 20 to 50%RH / Max Wet Bulb Temperature: 26℃.
Stray magnetic field at any point on tape not to exceed 50 oersteds (4000 ampere/meter).
[1] https://documents.westerndigital.com/content/dam/doc-library...
[2] https://www.fujifilm.com/uk/en/business/data-management/data...
[3] https://www.ibm.com/docs/en/storage-deep-archive?topic=media...
Tape can be used to safely store all typical HD files and etc for that long without a risk of degrading?
Honestly curious about your experience.
Honestly curious about your experience.
[deleted]
Given the gains in tape and hard disk densities in recent years, why can't we revisit floppy disks as a potential long-term archival media for the future? Are they particularly volatile? Would the design of the physical disk case be able to solve for such issues?
Everytime I see a Zip Drive in an old movie I get a serious pang of nostalgia. I wanted one of these so bad, yet, by the time I could afford it, they were no longer practical.
https://en.wikipedia.org/wiki/Zip_drive
https://en.wikipedia.org/wiki/Zip_drive
Floppies are, in fact, volatile. They suffer from bit rot like CDs, and (AIUI) the magnetic flux is weaker and more prone to disappearing over time.
Plus, all floppy drives are now multiple decades old.
A new, modern removable magnetic disc format will certainly perform orders of magnitude better, but not anywhere close to to any other modern format in speed, density, or short-to-medium term stability.
We could make fantastic floppy drives today, but there is simply no economic reason to spend a billion dollars in R&D on it.
Plus, all floppy drives are now multiple decades old.
A new, modern removable magnetic disc format will certainly perform orders of magnitude better, but not anywhere close to to any other modern format in speed, density, or short-to-medium term stability.
We could make fantastic floppy drives today, but there is simply no economic reason to spend a billion dollars in R&D on it.
I won't consider my business a success until we own and operate a tape library.
Turns out that modern drives can stretch the tape to make tracks line up right. It makes sense that as density grows, the real world effects of things like temperature and humidity require more and more work to compensate for.