I have backups on a backup hard drive and also synced to B2, but I am thinking about backing up to some format to put in the cupboard.
The issue I see is that if I don’t have a catastrophic failure and instead just accidentally delete some files one day while organising and don’t realise, at some point the oldest backup state is removed and the files are gone.
The other thing is if I get hit by a bus and no one can work out how to decrypt a backup or whatever.
So I’m thinking of a plain old unencrypted copy of photos etc that anyone could find and use. Bonus points if I can just do a new CD or whatever each year with additions.
I have about 700GB of photos and videos which is the main content I’m concerned about. Do people use DVDs for this or is there something bigger? I am adding 60GB or more each year, would be nice to do one annual addition or something like that.
You must log in or register to comment.
Tape. Amazon glacier if you’re okay with that.
And regular test restores. An untested backup is not a backup.
But when considering what I need to back up, I usually overestimate how much I or other people will care if it’s lost. Family photos are great, but what are the odds of someone saying “damn I wish we still had two dozen photos of that one barbecue?”
I use tape but haven’t been happy with my drive for a while, where do you get your drives? (Also OP I wouldn’t recommend tape until you cross the 10TB mark personally)
Yeah after looking at the price of a drive, I agree it doesn’t seem necessary at the level of data I have.
I don’t personally use tape, but I get most of my stuff from eBay. Tape drives are surprising expensive, even LTO-6 is going to run you a few hundred. But you still can’t beat the density and longevity.
I have cloud with B2, I’m looking for cupboard storage that a random family member can pull out and browse through after I get put in a resthome (only half joking).
Is home tape storage feasible (and good for this use-case)?
In terms of what to backup, I’m running on the assumption that technology will be able to autofilter the good stuff at some point, no need to put much effort in now haha.
I have not used them myself, but M-DISC sounds like what you’re looking for. There are a few other alternatives listed on that Wikipedia article, too.
Thanks, I think the risk here is that there may not be hardware to read it.
From the suggestions here I’m thinking a hard drive with USB connection would be best. It won’t last 50 years but instead I’d replace it every 5 years or so. I’d use an error resistant file system and plug it in each year to add the new files.
This way I also get the chance to move it to newer technology in future instead of a new hard drive. It would then only need to survive for some period of time after I last replaced it, so there’s a good chance of it remaining readable for most of my life.
Thanks, I think the risk here is that there may not be hardware to read it.
M-Disc DVDs are readable by ordinary DVD drives. So you could simply put a USB DVD drive alongside those backup M-Discs on the shelf.
Yeah that’s an idea. It does seem like I’d need a lot of disks though. And I don’t actually have a disk reader or writer at all at the moment.
Wrong post, ignore this.
Like this? https://www.pbtech.co.nz/product/DVWVER4618789/Verbatim-43888-External-Slim-Bluray-Writer-Ultra-H
That’s NZD by the way, conversion rates are terrible at the moment so about halve it for USD, seems in the price range you said.
Ah, a Kiwi. Say hello to your sheep’s from me. (Sorry,former WA resident here, couldn’t resist)
But yeah, we are using exactly that model - and it’s currently only 20 NZD less from what I pay wholesale in Europe for it. So it seems like a pretty decent price.
The drive itself is solid. We currently have around 10, maybe 15, at our clients and it works without any hassle.
I personally recommend to store the disk’s offsite(I store them in a locked box in a bank vault) and some of my clients choose to store another drive there to be extra safe,but I personally don’t see the point.
🐑
Sweet, thanks, I think that’s a good plan. I am thinking duplicate disks, one on site one off site. I do have a cloud backup, but if I die in a house fire then having the offsite disks is a much better solution than the random B2 bucket.
Thanks for the help 🙂
USB hard drive? If we’re talking about a cold backup that’s easy to access a USB drive is reliable and easy.
Yeah based on the suggestions so far this seems like the best option. Just need to make sure I have a way to verify no files are corrupted (and if they are, which ones), and remember to swap it out for a new one every 5 years or so or each time I need a new drive.
Thinking about this, the only ancient information we are still able to access is painted or edged on stone or clay. How about some sort huge wall with thousands of QRcode like engravings?
Reminds me of project Silica. Media historically was more durable (stone/ ink and cloth paper, etc) but had a low data density. As density increased, so did fragility
Haha like
spoilers for the three body problem series
at the the end of the third three body problem book where they need to write something that will last for millions of years so they carve a message in huge letters into the rock
But I’m a millennial so if you think I own a huge amount of land you would be wrong 😛
In your scenario, I’d be looking at ZFS or BTRFS for your live data, especially when taking photos into account. They’ll self-repair files that may run into decay issues, which I’ve seen a lot of with photos in all formats. Since you already keep off-site backups, I’d then just keep an extra drive around that you snapshot to from time to time.
So my offsites are an incremental backup, but at some point the oldest version is gone. I am keen for a completely separate, long term snapshot of what I had that could be thrown in a cupboard, and any random family member clearing my house out as I get moved into a rest home at 108 can go through the photos and find a good one to put on my headstone.
I am also keen for protection against doing something dumb and losing everything (like losing my hard drive and finding out for some reason I can’t access my backups because I lost the encryption key because I put it in bitwarden and they shut down years ago and I never moved the key over because I forgot it was stored there).
ZFS and BTRFS both provide that functionality. Have a look into the features.
So the drive doesn’t need to be hot, I can just plug in once a year and it auto-repairs?
No, the “live” filesystems will repair themselves when they detect problems. They keep revisions of your data, and run checksums constantly. When they find a file has inadvertently changed without access, it will restore said files. Think of it like Mac “Time Machine”, but it’s just the filesystem . You can restore stuff from points in time when needed.
Just read up on it.
Yeah I will read up on it, thanks for the tip!
For photos? Archival prints. As a bonus, you also get a cool album to reminisce later in life.
I don’t want to sort through the 50k photos, and can’t print videos. I’m hoping in 10 or 20 years I’ll be able to feed it into AI to spit out all the best ones, then I’ll consider it.
We do have photos printed, but only a very small percentage of the total.
Completely understandable. I say this as someone with way more photos in digital only media than I should.
I’ve decided I should have a small number of physical prints, as extra redundancy. I’m thinking I’ll print 100 each year to store with the hard drive backup.
This was a recent point of discussion on the 2.5 Admins podcast (https://2.5admins.com/2-5-admins-228/). Some good discussion on there.
My own thought is the best way to handle your family-member-finding-your-old-photos problem is the analog way: make some prints. It’s absolutely idiot proof, the methodology of keeping paper goods is well understood, and the technology is platform independent.
Yeah I really want a digital copy, but for redundancy physical photos sounds like a good idea. Maybe I should do the hard drive rotation thing mentioned in other comments, but each year when I’m updating I could print 100 photos as well. Have a suitcase or container where I keep the hard drives and the physical photos. Then I’d have a decent chance of all photos making it in digital form, plus a bunch of photos just in case the hard drives fail. The hardest part will be going back through the previous years finding an printing 100 photos from each year, but after that I don’t have to get through too many each year.
A hybrid is probably a good way forward. I had a career as a photographer for a while and I learned from that: going through 1000 photos takes very little time, but going through 10,000 takes an eternity. If you can star or mark your obviously important photos as you go along, it’ll take very little to print them at the end of the year.
Having actual prints has always been the consensus among
activistsarchivists. No digital media lasts as long. The media may persist but the technology to read them is long gone.Edit: autocorrupt.
I’m thinking of using a HDD and keeping it at work, which is climate controlled. I’d bring it back every few months to sync the latest.
Since it’s constantly being used, I’m pretty confident it’ll be usable as a backup if my NAS fails, so it only needs to be “shelf stable” for a few months at a time. If you’re retired or something, a safe deposit box at your local bank should do the trick.
If it’s powered off, you’ll have no idea when it dies. And they do die just sitting there.
I’ve actually had more failures of drives sitting around than ones running constantly.
But I will because it won’t work the next time I take it home to sync. The chance that it’ll fail during the few months between a sync and an emergency is incredibly low.
I wouldn’t leave it on a shelf for years, just a few months at a time (approximately quarterly).
I have a cloud backup, so this isn’t about a critical loss of data. It’s about an accessible copy that isn’t encrypted and a layman could get the data off.
If my house burns down and I lose the copy, I can restore the data from the cloud backup (so long as I’m not in the house when it burns).
For your amount, just an external hard drive attached to a NAS or something is fine, or a 2-bay synology would be more than enough. Drives are coming in 20-24TB models now, that’d keep you going for a long time.
That’s effectively what I have now. However I seem to kill a drive every couple of years, so I am keen for something that can be stored for many years (preferably decades).
Just a hdd in usb caddy? IMHO good enough for 4 tier backup.
Yes this seems to be the general theme. Main issue is sorting out a file system. I can use a self-repairing one, to recover from long term storage issues, but then it likely won’t work in Windows which it may need to if I want a layman to be able to access it. So still some refinement of the plan but it’s coming together.
I’ve also decided to print some physical photos, aiming for 100 per year, and will put everything in a container together. The physical photos are for in case the container is lost for decades and the drives die, then there will at least be something.
If you need something which can withstand some bitrot on single drive, just use par2. As long is filesystem is readable, you can recover files even if bit of data get corrupted
After reading the previous discussion I think that you should get more than single drive to store cold backups. That way you can at least spread out the risk of single drive failing. 2TB spinning drives are pretty cheap today and if you have, for example, 4 of them, you can buy one now, write your backups to it and in 6 months buy another, write data on that and so on.
This way you’ll have drives with year or two difference on purchase date, so it’s pretty unlikely all of them fail at once and a single drive gets powered on and checked every other year or so. My personal experience is that spinning drives are pretty stable on the shelf, but I wouldn’t rely on them for decades. And of course even with multiple drives you’ll still want to replace them every 3-5 years each. Plus with multiple drives, if I were to build setup like that, I’d set up some sort of scripts or other solution where I can just plug the thing in and doubleclick an icon on desktop to refresh the data and maybe get a notification automatically that the drive you’re using should be replaced.
And for actual, long term storage, printouts are the way to go. At least in here you can get books made out of photo paper with your pictures. That’s one media which is actually stable over long period and using them doesn’t require a lot of technical knowledge nor hardware. But I’d still keep digital copies around, as the printouts aren’t resistant to things like house fire or water damage.
Yeah I am thinking of getting a couple of drives and cloning across both. Update both at the same time. I didn’t think of getting two drives at different times but that makes sense, thanks for the suggestion!
I am thinking printouts is a good idea too. I might get a big container, and keep a couple of mirrored drives as well as say 100 photos from each year. Every year I update the drives with additions and then print 100 photos from the previous year to add to the collection.
I personally prefer printed out books of our photos. We are missing quite a few years due to life getting in the way, but the end goal is to have actual books of photos with titles like ‘Our family in 2018’ and ‘Sports of our first born at 2022’. In europe we have a company called ‘ifolor’ where you can design and order printouts of your photos. They’re not really cheap, but the quality is pretty damn good. And their offerings go to pretty decent sized photo albums, up to A3 size and 180 pages (which is over 200€). So, not cheap, but at least so far their quality has been worth the money.
And they have cheaper options too, but personally I think it’s worth the money to get the best quality you can for printouts. And even the smallest and cheapest option is far superior over not having anything at all due to hardware failure or whatever.
I’ve done some photo books in the past. A lot of work though, I prefer the idea of printed photos since it’s less work. Plus if they are the only surviving photos, then it’s nice to have them in an easily scannable format.
I came across this container (that’s NZD - about $10 USD), where each of the smaller containers holds 100 photos. Seems like it could be a good option, one container per year with 100 photos in each. 12 years per box. Get say 6 of them to cover the rest of my life (one could probably cover my life to date, a lot less photos before I had kids), put in a larger container that also holds a couple of (mirrored) hard drives. One big time capsule.
Don’t use them, but these seem right for you: https://lemm.ee/post/55012343
Thanks, I missed that post! Looks like the comment section would have answered a lot of my question.
In the end I have pulled the trigger and bought an M-Disc capable burner and a stack of M-Discs, so I’m gonna give that a go and see how it works out.
Sounds good. They do make 100GB MDisc but they are a shorter life span: https://a.co/d/dTOLvW8
What makes you say they are shorter life span? The 25GB and 100GB both have the same “several hundred years” claim.
Oops, just me misreading.
The other thing is if I get hit by a bus and no one can work out how to decrypt a backup or whatever.
Documentation, documentation, documentation. No matter what system you have, make sure your loved ones have a detailed, image-heavy, easy to follow guide on how restorations work - at the file level, at the VM level, at whatever level you are using.
That being said, DVDs actually have quite a short shelf life, all things considered. I’d be more inclined to use a pair of archival strength USB NVME drive, updated and tested routinely(quarterly, yearly, whatever makes sense). Or even an LTO tape, if you want to purchase the drive and some tapes.
You can put your backups in something like VeraCrypt. Set an insanely long password, encoded in a QR code, printed on paper. Store it in the same secured location you store your USB drives (or elsewhere, if you have a security posture).
You may also consider, if money is not a concern, a cloud VPS or other online file storage, similarly encrypted. This can provide an easy URL to access for the less tech-savvy, along with secured credentials for recovery efforts. Depending on what your successors might need to access, this could be a very straightforward way to log into a website and download what they need in an emergency.
archival strength USB NVME drive,
Does such a thing exist? Ordinary flash storage is pretty bad at keeping its content when powered off for a long time, due to how flash memory works. I’d be curious about such drives.
No, they don’t, I pulled it out of my butt. I rewrote my original draft and that slipped in. NVME wouldn’t make sense unless you were powering them up every few months for updates.
As much as I’m worried about family not being able to do it, I’m just as worried that I will do something dumb and lose the encryption key, losing everything. I am keen on the digital equivalent of a suitcase full of photos that could be stumbled upon.
I also already have borg backup set up to a backup drive and synced to the cloud (Backblaze B2).
For tape drives, is many thousands of dollars a normal price? Not sure I’m that keen.
If you buy your LTO drive new, then yes they rip you a new one, for sure! Buy it used…but it still will cost you a few hundred. Like I said, if money is not a concern. If losing the encryption key is a concern, then USB is still your best bet. Make two, keep them simple and unencrypted, stick em in two different safes, update them regularly. And print the documentation with pictures!
I have had terrible experience with USBs failing, including losing a bunch of photos beyond recovery (some 15 years ago, but it still hurts). Plus it’s quite aot of data.
I’m thinking a hard drive + USB SATA cable might be my best option. Add the new content each year. Work out some way of verifying it’s not corrupted. Replace drive every 5 years or something, it can be swapped in when I get a failure and the new one can be the new backup.
For local backups it depends on what you want to have:
- The cheapest option is a usb or thumb drive. But you have to regularly plug it in and copy your backup on it.
- The lazy option is to buy a NAS and configure a backup job that regularly creates a backup. Versioned, incremental, differentials and full backups are possible as is WORM to add a bit of extra security. You can configure a NAS to only turn on specified times, do a backup and then turn off again. This will increase protection against encrypting malware. WORM also helps in this case.
Or just let it run 24/7, create backups every hour and install extra services on it like AI powered image analysis to identify people and objects and let it automatically tag your photos. Cool stuff! Check out QNAP and Synology or build a NAS yourself.
A NAS can also be configured to present its content in a LAN by itself. Any computer will automatically connect to it if the access isn’t secured by user/password or certificate.
I recommend buying a NAS.
So, I have a server that has a backup drive, automated backups, and replication to laptops as well as cloud storage in Backblaze B2. What I’m looking for is something completely separated from the automation that is a backup for if I screw up the automation, as well as a backup that a layman can access (i.e. no encryption, media that is usable by anyone). I have had some very bad experiences with flash drives but I am thinking a HDD with SATA->USB cable attached (I already have the cable).
From the other conversations in this thread mentioning many options, the hard drive option seems the best for my use case, but I’ve also been convinced of the benefit of printing out some physical photos as well, so my current plan is to get a big container, put a couple of mirrored hard drives in there (to validate against each other as protection against bit-rot), and print 100 photos each year to add to the container to have an extra layer of redundancy.
Printing the photos won’t help much. After 20 or so years they are all discolored. You can’t prevent that.
I think SSDs might be the best storage medium for you. Consumer-grade ssds have a 1 year data retention when powered off. That means at least once per year you have to turn it on and copy the data around one time to refresh the cells. This way it’ll probably last several 100 years.
You can’t exactly make it fool-proof. Outside people will never know what you did to create your backup and what to do to access it. Who knows if the drives file system or file types are still readable after 20 years? Who knows if SATA and USB connectors are still around after that time?
For example it is very likely that SATA will disappear within the next 10-15 years as hdds are becoming more and more an enterprise thing and consumers are switching to M.2 ssds.The printed photos are only there as an extra layer of redundancy in case everything else fails. It’s ok if they get discoloured a bit, it never put me off going through my grandparents’ suitcases of photos. Ideally the digital files survive, if not then at least there is something rather than nothing.
Is SSD really necessary? Everything I search up says SSDs have worse retention than HDD in cold storage. A couple TB of HDD is pretty cheap these days, and seems like a better cold storage option.
You can’t exactly make it fool-proof. Outside people will never know what you did to create your backup and what to do to access it. Who knows if the drives file system or file types are still readable after 20 years? Who knows if SATA and USB connectors are still around after that time?
Yes, so now I’m thinking a rotation cycle. About every 5 years replace the drives with new ones, copy over all data. If newer technology exists then I can move to that newer technology. This way I’m keeping it up to date as long as I can.
For example it is very likely that SATA will disappear within the next 10-15 years as hdds are becoming more and more an enterprise thing and consumers are switching to M.2 ssds.
Does this matter if I have a SATA->USB cable stored with it? Other than if USB A standards change or get abandoned for USB C, but that should be covered by the review every 5 years.
Is SSD really necessary? Everything I search up says SSDs have worse retention than HDD in cold storage. A couple TB of HDD is pretty cheap these days, and seems like a better cold storage option.
SSDs are by design less susceptible and more robust. No moving parts and able to work in much harsher conditions than hdds will ever be able to. The standard set by JEDEC requires every consumer ssd to have a 1 year data retention while powered off at 30 °C (I think). That’s the minimum it has to archieve but usually they are better than that. Do not buy the cheapest thumb drives because they contain the all the crap that wasn’t good enough to make ssds from it.
Btw you need to fire hdds up regularly too or the motor gets stuck. I think every 3-6 months was the recommendation.Yes, so now I’m thinking a rotation cycle. About every 5 years replace the drives with new ones, copy over all data.
Don’t make it flat every 5 years. Let a software monitor the SMART values of the drives and send notifications if the values indicate an increased change of a dying disc/ssd.
Does this matter if I have a SATA->USB cable stored with it?
Those are the first that fail, followed by the usb controller chip in the tray. Keep it as simple as possible. Removable trays are probably the best way but I’m not sure how much wear they can take.
Do not buy 2.5" drives. This class will die out soon™. There were no new hdds introduced in years and ssds are often replaced by M.2 ones because of the faster connection.
You are the first person who has recommended SSD for cold storage. Everyone else (including what I’ve googled) says HDD for cold storage, just spin up every year or two and they will be fine. Can you point me at further reading?
Don’t worry, I’ll SMART check the drives each year as I update as required.
As for types of drives dying out soon, I can reassess the situation every 5 years when I do drive replacement. I would be confident 2.5" drives will still be readable in 5 years.
Do not use an SSD for cold storage - it will fail. SSDs need to be plugged in every once to refresh the charge in their NAND, otherwise they’ll lose the data.
This is not a theoretical thing - I’ve had a good Samsung 850 Pro drive fail while being off for 2 years.
I usually use a dehydrator for ~3 days on my drives to make them shelf stable. So far I haven’t had any issues.
This will do nothing at all. Drives don’t die by rust. They usually die because the motor somehow can’t get the discs to spin. Very often dry lube is the reason. That can occur if you leave the drive off too long.
