My space needs have been growing, especially this year as I spent about a little over a month converting my entire DVD collection to MP4.  Earlier this year I lost a 1.5TB drive.   I’ve lost drives before, but the largest was a 160GB drive, and it was just ISO’s of my optical disks, so I just had to re-rip them.  1.5TB put a lot more things at risk.  I still got lucky.  I lost my Steam directory which can be reloaded, a folder of funny videos and pictures from the internet that is largely covered by YouTube now, and some movies that I could reload from my DVD shelf.  Fortunately, I had just started offloading a lot of data to an external 2TB drive.  I committed myself to having a storage server that can survive a drive failure by 2013.  I just had to figure out what I wanted, but I had no idea the options that are out there.  I want to share this experience for others that may be in the same boat.  I’ll be doing some research on some of the following options:

1. “Hardware” RAID (AMD Chipset)
2. Software RAID 5 (Windows Home Server 2011)
3. Software RAID 5 (Ubuntu)
4. Windows 8 Storage Spaces
5. FreeNAS with ZFS (BSD)
6. disParity
7. SnapRAID
8. DrivePool
9. Drive Bender
10. FlexRAID

Things I’ll be considering:

1. Ease of Use (Including configuration, maintenance, and recovery)
2. Complete survival of at least 1 drive loss
3. Performance
4. Expansion / Upgrade options
5. Efficiency (I’d like to avoid complete duplication a la RAID 10)
6. Cost

Conclusion:

I have been running the “Hardware” RAID with Windows Home Server 2011 for nearly a year now.  I opted for this solution largely due to performance and simplicity.  It does, in some ways, lock me to AMD chipsets for this machine unless I want to do a full data migration.  However, in testing I was comforted to find I could move my drives to any AMD chipset machine and be recognized.  Between the three machines I have with this chipset, two a good friend has, and the common availability of them, I am set at ease against a hardware failure.  I started with 3 drives in the array and a 4th as a “hot spare”.  At some point this summer, I needed a larger array, so the hot spare was loaded into the main array.  I receive an e-mail report once a month on drive status, as well as (in theory) an e-mail any time a problem is detected.  I am really looking forward to seeing what Windows 8 Storage Spaces will be capable of as it matures, but until that time, my preference and recommendation will be the built in RAID, at least on AMD chipsets.

UPDATE: 10/2015

After about 3 years of running RAID5, I have now completely and unregrettably made the transition to DrivePool.  Over the years on RAID, I had drives drop and the array would rebuild.  This happened periodically, and I’d always scan the dropped drive with usually had no problems and then place it back in as a hot spare. This didn’t bother me too much until I had a rebuild fail.  In troubleshooting, the reason I had a rebuild fail was due to a bad sector on a drive.  What bothered me about this was that I had the AMD media scanner that was supposed to scan the drives every month and alert me of problems.  I ended up losing a few files in this recovery, but I was never able to rebuild the array.  Fortunately, it was not important files I lost.

I had also been facing capacity issues.  My 3TB based array was out of room to grow, and I couldn’t add larger drives until I replaced them all.  By switching to DrivePool, I increased the cost of storage redundancy from 25% to 100%, but I was able to use literally any cheap drive I could find on sale.  USB2 and USB3 externals and drives of varying capacity became available.  I could add a 5TB drive and use all of it, even if it took 2x 3TB drives to duplicate its contents.  I have been using DrivePool since April 2015 and can not even imaging looking back.  Combined with DriveScanner, this combination gives me more control, flexibility, and better reporting of problems, at only the cost of a little extra disk space.

UPDATE: 11/2020

Well time for another significant change.  I switched to DrivePool in 2015 as noted above running Windows Home Server.  I don’t recall at what point, but eventually WHS was discontinued.  No worries, I had an extra Windows 8 license which could upgrade to Windows 10, so I just did a clean install of Windows 10. Aside from losing the amazing windows backup features of WHS 2011, everything else was pretty smooth.  I installed DrivePool on my Windows 10 image, plugged in my 12 data disks, and everything was ready to go. DrivePool has served me well, especially paired with Scanner, and I have not lost anything in this time, despite many drive upgrades, removals, and rebalances.  All of my drives were encrypted, which is completely transparent to DrivePool.

I am now switching to unRaid.  I haven’t done a write up on unRaid, but there are many out there.  Why am I making this switch? Ultimately, because Windows is pissing me off. Windows forced automatic reboots are held at bay with RebootBlocker. WHS 2011 was a perfect OS for my home, largely due to the windows full backup solutions. Now though, I find myself nervous to have too much running on my environment, and I don’t even get that benefit. unRaid will allow me to run docker containers, which will offer significant flexibility in the services I run. It also offers a parity and dual parity solution, while retaining the ability to never lose more than the drives that fail as a worst case. Additionally, I was able to mount my Windows 10 drive, with drive pool and pass through all the disks directly to the VM, to have a near seemless switch.  I’m still running Windows 10 with DrivePool as a VM inside of unRaid, but I am slowly moving everything off of Windows!  Dockerization has been great, and I now run far more services on this machine.

If Windows is your OS, DrivePool and DriveScanner are the way to go! But a lot has changed since I initially did this research, and built this machine.  Only 1 of the original drives remains (which I must say is quite impressive after 8 years of being powered on 24/7, though in my environment it gets a significant percentage of sleep time). I’m looking forward to the flexibility that unRaid will offer!