I have a ZFS RAIDZ2 array made of 6x 2TB disks with power on hours between 40,000 and 70,000. This is used just for data storage of photos and videos, not OS drives. Part of me is a bit concerned at those hours considering they’re a right old mix of desktop drives and old WD reds. I keep them on 24/7 so they’re not too stressed in terms of power cycles bit they have in the past been through a few RAID5 rebuilds.
Considering swapping to 2x ‘refurbed’ 12TB enterprise drives and running ZFS RAIDZ1. So even though they’d have a decent amount of hours on them, they’d be better quality drives and fewer disks means less change of any one failing (I have good backups).
The next time I have one of my current drives die I’m not feeling like staying with my current setup is worth it, so may as well change over now before it happens?
Also the 6x disks I have at the moment are really crammed in to my case in a hideous way, so from an aesthetic POV (not that I can actually seeing the solid case in a rack in the garage),it’ll be nicer.
You must log in or register to comment.
$ for i in /dev/disk/by-id/ata-WD*; do sudo smartctl --all $i | grep Power_On_Hours; done 9 Power_On_Hours 0x0032 030 030 000 Old_age Always - 51534 9 Power_On_Hours 0x0032 033 033 000 Old_age Always - 49499Once a year or so, I re-learn how to interpret Smart values, which I find frustratingly obtuse. Then I promptly forget .
So one’s almost 6 y/o and the other is about 5½?
Seagate “raw read error rate” is a terrifyingly big number if everything is hunky dory.
One has a total powered-on time of 51534 hours, and the other 49499 hours.
As for their actual age (manufacturing date), the only way to know is to look at the sticker on the drive, or find the invoice, can’t tell you right now.
6 years old and running perfectly fine.
I have 5 WD RED disks in a RAIDZ1 config. In the first year I was experimenting with the sleep or spindown options. Then I have read that drives live longer if they run constantly. Now they are spinning 24/7.
The additional SSD has broken and been replaced 2x during these years.
Yeah flat out spinning is definitely better for reliability.
Mine are only 25k hours or so, around 3 years. My prior set of disks had a single failure at 6 years but I replaced them all and went to bigger capacity. There is also the power saving aspect of going down to 2 drives as well, it definitely saves some power not spinning 4 extra drives all the time.
I just got done swapping all my drives out. I had 6x8tb drives in raidz2. About 8 months ago I had some sort of read errors on one drive with about 33k hours on it. I started swapping my drives out with 20tb drives one at a time, and just finished last week. So now I have 6x20tb drives with between 200 and 6k hours on them. The most hours on any of my older drives was about 40k, but other than a couple minor errors on the one drive, I’d had no issues with any of them. I’ve held onto all of the old drives, and was planning on setting up a second nas with 4x8tb drives in raidz1 to use as a backup server.
This was my second time replacing all my drives. My NAS is a bit like the ship of Theseus at this point, as it’s gone through many upgrades over the years. Started out with 6x3tb drives, and after about 4 years swapped the drives with 8tb units. About 5 years later (where we are now) it’s now 20tb drives. I’ve also swapped the chassis, mobo, CPU, and everything else out multiple times, etc.
My original setup was a mixture of desktop and Nas drives, but I’ve since been running all Nas/Enterprise drives. Based on my personal experience, it seems like I’ll replace drives every 4-5 years, regardless of actual failures… Both times I started the drive swaps there were read/write errors or sector failures on a drive in the pool. However, at around the same time I needed more space, so it was a convenient enough excuse to upgrade drive size.
As far as your concern about cramming drives into the chassis, always worth considering swapping chassis’s, but that’s up to you. I think 6 drives in Z2 is pretty happy compromise for number of drives and reliability. Thankfully your storage capacity is low enough you can pretty easily transfer everything off of that Nas to some interim storage location while you make whatever changes you want to.
Part of the reason I want to repurpose my old drives into another server is so I can have enough backup storage for critical files, etc should I need to start over with my main Nas.
3:2:1 - Cattle not pets - If your data is backed up in multiple sites, the death of one site shouldn’t overwhelm you, and give you time to recover.
If your primary site drives are getting above their designed lifetime, rotate them out, sure - but they could be used as part of the backup architecture else where (like a live offsite sync location with enough tolerance for 2 disk failures to account for the age).
3 copies of your data; 2 types of media; 1 copy offsite.
I mean if it’s homelab, it’s ok to be pets. Not everything has to be commoditized for the whims of industry.
“Cattle not pets” in this instance means you have a specific plan for the random death of a HDD (which RAIDZ2 basically already handles), and because of that you can work your HDDs until they are completely dead. If your NAS is a “pet” then your strategy is more along the lines of taking extra-good care of your system (e.g. rotating HDDs out when you think they’re getting too old, not putting too much stress on them) and praying that nothing unexpected happens. I’d argue it’s not really “okay” to have pets just because you’re in a homelab, as you don’t really have to put too much effort into changing your setup to be more cynical instead of optimistic, and it can even save you money since you don’t need to worry about keeping things fresh and new.
“In the old way of doing things, we treat our servers like pets, for example Bob the mail server. If Bob goes down, it’s all hands on deck. The CEO can’t get his email and it’s the end of the world. In the new way, servers are numbered, like cattle in a herd. For example, www001 to www100. When one server goes down, it’s taken out back, shot, and replaced on the line.”
~from https://cloudscaling.com/blog/cloud-computing/the-history-of-pets-vs-cattle/
I get that. But I think the quote refers to corporate infrastructure. In the case of a mail server, you would have automated backup servers that kick-in and you would simply pull the rack of the failed mail server.
Replacing drives based on SMART messages (pets) means you can do the replacement on your time and make sure you can do resilvering or whatever on your schedule. I think that is less burdensome than having a drive fail when you’re quite busy and being stressed about having the system is running in a degraded state until you have time to replace the drive.
I don’t think ‘cattle not pets’ is all that corporate, especially w/r/t death of the author. For me, it’s more about making sure that failure modes have (rehearsed) plans of action, and being cognizant of any manual/unreplicable “hand-feeding” that you’re doing. Random and unexpected hardware death should be part of your system’s lifecycle, and not something to spend time worrying about. This is also basically how ZFS was designed from a core level, with its immense distrust for hardware allowing you to connect whatever junky parts you want and letting ZFS catch drives that are lying/dying. In the original example, uptime seems to be an emphasized tenet, but I don’t think it’s the most important part.
RE replacements on scheduled time, that might be true for RAIDZ1, but IMO a big selling point of RAIDZ2 is that you’re not in a huge rush to get resilvering done. I keep a cold drive around anyway.
Ah, thank you for explaining. I understand where you’re coming from. Nevertheless, from the point of a view a small NAS, RAIDZ1 is much more space and cost efficient so I think there is room for “pets” in the small homelab or NAS.
Yep, numbering’s the key.
When you create NAS01, you know there’s going to be a NAS02 one day
When one server goes down, it’s taken out back, shot, and replaced on the line.
And then Skynet remembers…
A bit rookie, but going strong with my two 4TO Seagate, for around 24K hours each :)
Used HGST and Seagate all running about 50k hours
4x8tb they had 8.5k hours on them when I got them four years ago, they work non-stop since.
According to my Synology:
- WD40EFRX-68WT0N0 - 86,272 hours
- WD40EFRX-68WT0N0 - 86,207 hours
- WD40EFRX-68N32N0 - 34,417 hours
- ST4000VN006-3CW104 - 10,054 hours
According to my Synology:
Where are you finding this data? It’s not Info Center -> Storage…
Look into the S.M.A.R.T. reports of each drive.
On Synology DSM 7.x: Storage Manager › HDD/SSD › Health Info › S.M.A.R.T. › S.M.A.R.T. Attribute › Details › Power_On_Hours
The power-on hours are shown directly on the Health Info page, no need to click through to the SMART attributes.
Ultimately it’s a matter of personal choice and risk tolerance.
The Z1 will be simpler and have larger capacity, but if you have a drive fail you’ll need to quickly get it replaced or risk having to rebuild/restore if the mirror drive follows the first one to the grave.
Your Z2 setup right now can have two drives fail and still be online, and having a wider spread of power-on hours is usually a good thing in terms of failure probability.
I manage a large (14,000±) number of on-site RAID1 arrays in various environments and there is definitely a trend for drives shipped at the same time to fail at roughly the same time. It’s common enough that we often intentionally swap drives out before shipping a new unit to the customer site.
On my homelab, I’m much more tolerant of risk since I have trust in my 3-2-1 backup solution and if my NAS goes down it’s not going to substantially affect anything while I wait for a drive replacement.
My first batch (6x 20TB) is at 8611 hours.
The 2nd batch (3x 20TB) is at 5612 hours
So a year ago you spent over 3k on disks?
Yeah and a new 12 bay NAS to put them all in. I had a 2 bay that I expanded with a bunch of USB drives before, but that was starting to get really messy. Basically took my entire thirteenth salary.
💸 🎬
Second hand so I’m sure ancient.
I tend to buy two at a time. Some are months old, others three years old.
Professionally, I have seen drives over 10 years always on at low utilization without issue. (The data was easily replaceable.)
crammed in to my case in a hideous way
Heat is a killer. Check them regularly.
They’re in a drafty garage. This time of year I keep them spinning to stop them freezing 🤣
- 2x18k - mirrored ZFS pool.
- 1x47k - 2.5" drive from an old laptop used for torrents, temp data, non-critical pod volumes, application logs etc.
- 1x32k - automated backups from ZFS pool. It’s kinda partial mirror of the main pool.
- 1x18k - (NVME) OS drive, cache volumes for pods.
Instead of single pool, I simply split my drives into tiers: cache, storage, and trash due to limited drive counts. Most R/W goes to the cheap trash and cache disks instead of relatively new and expensive NAS drives.
I recently decommissioned my old poweredge T620. Beast of a thing, 5U heavy af. It had 8x10T drives and was the primary media server.
Now that it is replaced I bought 2x Synology RS822+ and filled them with the old disks. Using SHR2. They are mixed brands bought at different times so I’ve made sure each NAS has a mix of disks.
Lowest is 33k hours, highest is 83k.
