Ask HN: What is the best media for long term archival storage?
52 comments
For photos or documents like wills I think the best long term storage solution is still physical prints in a shoebox. Yes, they'll fade slowly over time, but will still be viewable for decades, or even centuries if printed using pigment inks on archival papers.
Of course there's still a chance that physical prints can be lost, stolen or destroyed in a house fire. But I think overall that's statistically much less likely to happen than digital media becoming obsolete or unreadable, or data stored online being deleted because the original uploader stopped paying the storage bill or didn't arrange for transfer of their accounts after their death.
Of course there's still a chance that physical prints can be lost, stolen or destroyed in a house fire. But I think overall that's statistically much less likely to happen than digital media becoming obsolete or unreadable, or data stored online being deleted because the original uploader stopped paying the storage bill or didn't arrange for transfer of their accounts after their death.
100% agreed.
The chances of losing digital and physical copies at the same time are pretty low.
I have gotten into habit of printing photos and ordering photobooks for not just us but also for relatives. The idea is sort of distributed backup of physical photos. Also they love it.
The chances of losing digital and physical copies at the same time are pretty low.
I have gotten into habit of printing photos and ordering photobooks for not just us but also for relatives. The idea is sort of distributed backup of physical photos. Also they love it.
Stone tablets are the only media I know of that lasts 1000s of years.
In all seriousness, this isn't a popular opinion around here, but cloud storage. I back my photos up to both iCloud and OneDrive. This way there is triple redundancy if you include the local copy. I trust Apple and Microsoft a lot more than myself to protect my data from spilled beverages, floods, fires, other acts of god, or just my own stupidity in general.
In all seriousness, this isn't a popular opinion around here, but cloud storage. I back my photos up to both iCloud and OneDrive. This way there is triple redundancy if you include the local copy. I trust Apple and Microsoft a lot more than myself to protect my data from spilled beverages, floods, fires, other acts of god, or just my own stupidity in general.
Any pictures from my phone get synced to:
- iCloud Drive
- OneDrive
- Google Photos - I still haven’t gone over the 15GB free space since Google instituted the limit. But I’ll pay for extra storage when needed.
- Amazon Drive - free with Prime
My videos get synced to everything but Amazon Drive.
- iCloud Drive
- OneDrive
- Google Photos - I still haven’t gone over the 15GB free space since Google instituted the limit. But I’ll pay for extra storage when needed.
- Amazon Drive - free with Prime
My videos get synced to everything but Amazon Drive.
I think you might be right. Redundant cloud storage seems to be the way to go. I wish there was something a little more passive.
Bad idea because even if they don't lose the data you may lose your access to the account due to things like random account suspensions.
Which is why I recommend redundant cloud storage. There's a lot of options out there, pick 2. I like iCloud for the integration and convenience and OneDrive because it comes bundled with Office 365.
How do you backup photos to onedrive ? Did you copy from Photos app on Mac to another drive?
The onedrive app on Android has a "camera backup" functionality.
It saves images and videos from any folder to the onedrive cloud.
I've configured it, and then I sync the onedrive to a Synology Nas at home.
It saves images and videos from any folder to the onedrive cloud.
I've configured it, and then I sync the onedrive to a Synology Nas at home.
I’ve been considering this for over 5 years.
So far, cutting edge commercial is https://www.piql.com/, the Norwegian company that did the GitHub Arctic code vault. Their tapes have instructions on how to decode the data inscribed on the tape itself.
Truly cutting edge (non-commercial) is Microsoft’s Project Silica optical storage as well as DNA storage.
I’ve been thinking about building something in this space since I started my career, but the business seems incredibly hard to figure out a revenue model for. Trying to have someone pay up front for eternity(?), always seemed like a steep price. Would love to hear others thoughts if they’ve talked to people about this.
So far, cutting edge commercial is https://www.piql.com/, the Norwegian company that did the GitHub Arctic code vault. Their tapes have instructions on how to decode the data inscribed on the tape itself.
Truly cutting edge (non-commercial) is Microsoft’s Project Silica optical storage as well as DNA storage.
I’ve been thinking about building something in this space since I started my career, but the business seems incredibly hard to figure out a revenue model for. Trying to have someone pay up front for eternity(?), always seemed like a steep price. Would love to hear others thoughts if they’ve talked to people about this.
Piql and Silica are interesting at a civilization level but they aren't useful for personal archiving. You won't have a reader and you probably won't be able to build or buy one.
For the economics of storage see https://economicmodel.dshr.org/ and DSHR's other writing about archiving.
For the economics of storage see https://economicmodel.dshr.org/ and DSHR's other writing about archiving.
One thing I would pay for is a way to engrave my SSH keys into thin steel plates as a QR code.
I have a safe with my critical keys in it but that is only fire proof for 3 hours. If things got really hot those are gone. In theory it shouldn't be an issue as I have them backed up other places but it would give me peace of mind to have them on a fire proof media.
I have a safe with my critical keys in it but that is only fire proof for 3 hours. If things got really hot those are gone. In theory it shouldn't be an issue as I have them backed up other places but it would give me peace of mind to have them on a fire proof media.
Credit where credit is due, the crypto space has some innovations in this area. Fully analog secret storage has a few options I have come across. One is where you punch/stamp your secret into a piece of steel. The blank is pre-scored with a template or letters so you can do this with just a hammer. The other is where you have little rings pre-stamped with letters/numbers which you then assemble onto a tube.
A random site[0] I found offering some of the solutions. Given the industry, I assume this is just the most highly astro-turfed platform, but there you go.
[0] https://cryptosteel.com/
A random site[0] I found offering some of the solutions. Given the industry, I assume this is just the most highly astro-turfed platform, but there you go.
[0] https://cryptosteel.com/
That's very cool. A lot of work to encode a long ssh key.
The way that works in crypto is that, you generate a private key (such as an SSH or GPG key) from a mnemonic passphrase and punch/engrave those words on a metal plate.
Signing keys like ssh keys aren't so important, you should be able to replace those out of band. Decryption keys are more important to back up.
If I lose my ssh keys the business I work for will no longer have access to their production sites.
They can rebuild them from backups somewhere else but that is about it. They're printed on paper in a safe because of this.
They can rebuild them from backups somewhere else but that is about it. They're printed on paper in a safe because of this.
I would place my bets on:
CD-R because it is still around after more than two decades and the media is still in production, there are many drives and they are repairable.
My second choice would be DVD-R for similar reasons, but less history.
Third choice would be FAT formatted spinning disks used as write once. But they are much more susceptible to environmental flux.
Anything that is expensive or hard to come by or new, I would avoid…they’re almost certainly going to be a Zip drive equivalent in 20 years because there is no consumer demand for physical storage and less and less commercial demand because of the cloud.
But that’s me so YMMV. Good luck.
CD-R because it is still around after more than two decades and the media is still in production, there are many drives and they are repairable.
My second choice would be DVD-R for similar reasons, but less history.
Third choice would be FAT formatted spinning disks used as write once. But they are much more susceptible to environmental flux.
Anything that is expensive or hard to come by or new, I would avoid…they’re almost certainly going to be a Zip drive equivalent in 20 years because there is no consumer demand for physical storage and less and less commercial demand because of the cloud.
But that’s me so YMMV. Good luck.
The issue with CD/DVDs is that they degrade quickly over time. The CDs I burnt in early 2000s had tons of read errors when I tried restoring in 2022.
Personally, I think storing data on an external Harddrive is the most stable option. You can keep upgrading the setup as required and the bit rot is minimal.
I backup my data on multiple drives, along with redundant copies stored in multiple locations.
I have heard good things about Tapes as well but have no personal experience with them.
Personally, I think storing data on an external Harddrive is the most stable option. You can keep upgrading the setup as required and the bit rot is minimal.
I backup my data on multiple drives, along with redundant copies stored in multiple locations.
I have heard good things about Tapes as well but have no personal experience with them.
I have fewer problems reading my C64 tapes and 5,25" floppies from 1984-1987 than reading my 1994-2001 CDRs and DVDRs.
I have heard that writable Blu-Rays are thought to be much more durable than all but the best DVD-R.
I wouldn’t doubt blue ray media being at least as reliable as DVD.
I don’t think the hardware was/is as widespread and I don’t expect blue ray was/is as commonly specified in government contracts, integrated into medical devices, etc.
My primary consideration with all digital media is what will the ecosystem be like in twenty years. How easy will it be to start with only a disk?
I don’t think the hardware was/is as widespread and I don’t expect blue ray was/is as commonly specified in government contracts, integrated into medical devices, etc.
My primary consideration with all digital media is what will the ecosystem be like in twenty years. How easy will it be to start with only a disk?
Blu-ray drives are in Playstations and Xboxen; if those drives can be cannibalized and used in a computer there will be millions available in the used market. Or you could jailbreak a console and install Linux just to use the drive.
Right now, today, I can just buy a Blu-ray burner at my nearby big box.
In 1997, I could say the same about Zip because Zip drives were everywhere until they weren’t because consumer behavior changed and CD-RW replaced the market.
There are still working Zip drives, but not a terrible many. Running one is only a matter of conjuring up a parallel port or scsi connection. Nevertheless most people who bet on Zip archives have regretted it sooner or later…and probably sooner.
Drives that read CD-R are almost anything that reads DVD or Blu-ray, plus all but the earliest CD drives. There’s not going to be a need to compete with gamers or retro computing enthusiasts.
That’s what I want in an archive strategy based on my experience over almost forty years. For me, archiving is enough of a project without involving jailbreaking a game console.
But that’s me and my bet. Other people have other priorities and I respect that.
In 1997, I could say the same about Zip because Zip drives were everywhere until they weren’t because consumer behavior changed and CD-RW replaced the market.
There are still working Zip drives, but not a terrible many. Running one is only a matter of conjuring up a parallel port or scsi connection. Nevertheless most people who bet on Zip archives have regretted it sooner or later…and probably sooner.
Drives that read CD-R are almost anything that reads DVD or Blu-ray, plus all but the earliest CD drives. There’s not going to be a need to compete with gamers or retro computing enthusiasts.
That’s what I want in an archive strategy based on my experience over almost forty years. For me, archiving is enough of a project without involving jailbreaking a game console.
But that’s me and my bet. Other people have other priorities and I respect that.
Bitrot will claim the first two.
Go cloud, ZFS or tape.
Go cloud, ZFS or tape.
Archival grade optical disks have an expected life of 100 years in archival conditions.
The cloud has a very low bus factor. Miss a payment and the data is gone. It provides high availability but low permanence and requires active management. It is inherently a poor archival medium.
“Tape” is a cluster of incompatible specifications and implementations. I know you don’t mean Quic 40, but that’s tape and about the last system that sold to regular computer users in big box retailers.
The cloud has a very low bus factor. Miss a payment and the data is gone. It provides high availability but low permanence and requires active management. It is inherently a poor archival medium.
“Tape” is a cluster of incompatible specifications and implementations. I know you don’t mean Quic 40, but that’s tape and about the last system that sold to regular computer users in big box retailers.
CDs and DVDs have error correction. As do hard drives, but it isn't baked in the spec. It's whatever the manufacturer thinks is enough.
I've been through 8mm Exabyte tape, 4mm DAT tape, Sony AIT tape, and a couple of generations of LTO. I kept a pair of drives for each format, all using SCSI, and had suitable SCSI adaptors to drive them, and formated the recorded data using standard Open Source software with no compression or encryption. Every year, I'd read each tape, just to re-tension it, and try to avoid print-through, etc.
It was a massive effort. So, I no longer do that.
I now just keep everything on my NAS. The volumes are mirrored. I have a removable HDD onto which I snapshot the entire NAS volume every night. I swap that drive out every month and send it off-site (and replace it with the one previously off-site). So I have live, local snapshot (up to 24 hours out-of-date), and off-site snapshot (up to 1 month out of date).
Everything I care about is rsync-ed daily to the NAS: home directories, photos, music, Time Machine laptop backups, machine configs, IMAP sync, CalDAV sync, CardDAV, Git repo clones, etc.
Every few years, I double the size of the NAS volume.
There's no monthly cost, and there's no concern about degradation of the media. The storage format is always current, as are the OS and tools required to read it. The only effort is the monthly off-site drive swap.
It won't outlive me, unless my heirs decide to continue to maintain it, but ... at that point I no longer care.
It was a massive effort. So, I no longer do that.
I now just keep everything on my NAS. The volumes are mirrored. I have a removable HDD onto which I snapshot the entire NAS volume every night. I swap that drive out every month and send it off-site (and replace it with the one previously off-site). So I have live, local snapshot (up to 24 hours out-of-date), and off-site snapshot (up to 1 month out of date).
Everything I care about is rsync-ed daily to the NAS: home directories, photos, music, Time Machine laptop backups, machine configs, IMAP sync, CalDAV sync, CardDAV, Git repo clones, etc.
Every few years, I double the size of the NAS volume.
There's no monthly cost, and there's no concern about degradation of the media. The storage format is always current, as are the OS and tools required to read it. The only effort is the monthly off-site drive swap.
It won't outlive me, unless my heirs decide to continue to maintain it, but ... at that point I no longer care.
MDisc DVDs and Bluray discs are supposed to last for a thousand years.
https://www.mdisc.com/
I’ve been thinking of this question too, and think a combination of cloud storage (GDrive + iCloud), HDs, and these MDiscs is what I’ll do. Just haven’t bought the MDrive writer yet.
https://www.mdisc.com/
I’ve been thinking of this question too, and think a combination of cloud storage (GDrive + iCloud), HDs, and these MDiscs is what I’ll do. Just haven’t bought the MDrive writer yet.
One thing about DVDs and Bluray is they have a long life expectancy because of the Lindy Effect. DVDs have been around 25 years, and the latest console generation can still read them. Expect another 25 years.
And more specifically "pressed" discs, not those you can burn at home.
Those you can burn at home relies on a chemical reaction which will degrade after several years, faster if exposed to sunlight.
Those you can burn at home relies on a chemical reaction which will degrade after several years, faster if exposed to sunlight.
M-Disc you can burn at home but do not rely on chemical reactions.
From the website: Verbatim M DISC™ optical media is the new standard for digital archival storage. Unlike traditional optical media, which utilize dyes that can break down over time, data stored on an M DISC is engraved on a patented inorganic write layer – it will not fade or deteriorate. This unique engraving process renders these archival grade discs practically impervious to environmental exposure, including light, temperature and humidity.
ISO/IEC 16963 standard longevity tests have proven the durability of M DISC technology, and it withstood rigorous testing by the US Department of Defense. Based on ISO/IEC 16963 testing, M DISC media has a projected lifetime of several hundred years. https://www.verbatim.com/subcat/optical-media/m-disc/#:~:tex....
From the website: Verbatim M DISC™ optical media is the new standard for digital archival storage. Unlike traditional optical media, which utilize dyes that can break down over time, data stored on an M DISC is engraved on a patented inorganic write layer – it will not fade or deteriorate. This unique engraving process renders these archival grade discs practically impervious to environmental exposure, including light, temperature and humidity.
ISO/IEC 16963 standard longevity tests have proven the durability of M DISC technology, and it withstood rigorous testing by the US Department of Defense. Based on ISO/IEC 16963 testing, M DISC media has a projected lifetime of several hundred years. https://www.verbatim.com/subcat/optical-media/m-disc/#:~:tex....
It's not necessarily as bad as you think: https://blog.dshr.org/2022/08/optical-media-durability-updat...
I use AWS Glacier Deep Archive for exactly this, a few TB of photos, documents, etc.
RClone takes care of uploading to it. Works out to barely over $1/TB/month and AWS takes care of the media lifetime and all that.
I didn't bother with burned disks because then the backups are too local to me, nevermind spanking TBs across multiple disks.
RClone takes care of uploading to it. Works out to barely over $1/TB/month and AWS takes care of the media lifetime and all that.
I didn't bother with burned disks because then the backups are too local to me, nevermind spanking TBs across multiple disks.
Stone Tablets and large fonts have withstood the test of time. Low memory density however.
It really depends on your requirements.
It really depends on your requirements.
In Film acá TV things get archived to LTO. The tapes last a long time and aren't that expensive. The writer/reader can be though (start around a few thousand bucks IIRC)
There are various exotic things but you probably won't find working readers for them in the future. I would go with M-disc or a NAS that's continuously maintained.
I've heard good old magnetic tapes (LTO nowadays) are the medium of choice for long-term storage within large organizations.
What's the oldest tape a drive still in production can read?
In extremis, you need to archive the means of reading the data as well.
This is more true the less standard your media is: 1600bps tape using Unix tar format is much easier to read today than some 1990's commercial backup program on some proprietary, hardware compression implementation using some oddball tape-in-cartridge media.
You still have issues with, eg. melted rubber on tape rollers, leaking capacitors, etc, etc, but that's a more tractable problem than finding the right weirdo media reader 50 (or 100, or whatever) years after it was obsolete.
This is more true the less standard your media is: 1600bps tape using Unix tar format is much easier to read today than some 1990's commercial backup program on some proprietary, hardware compression implementation using some oddball tape-in-cartridge media.
You still have issues with, eg. melted rubber on tape rollers, leaking capacitors, etc, etc, but that's a more tractable problem than finding the right weirdo media reader 50 (or 100, or whatever) years after it was obsolete.
It's not good. For example, LTO-9 drives can only read back to LTO-7. If you want to read a 20-year-old tape you'd need a ~15-year-old drive.
The drives are expensive and the tapes are pretty much only useable for offline storage and not for keeping the data accessible.
I only backup my synced icloud photos library on Mac to rsync.net using their borg services.
LTO tape
Is there something better for long term storage of family photos and other important data?