So let’s say a guy had a lot of stuff — like, a lot of stuff — that he wanted to archive? My fellow nerds, I propose this question to you: what is the best way to archive a lot of stuff? Here are the four factors I’ve come up with: time, cost, accessibility, and permanence.
Let’s take the example of backing up 600gb of mp3s.
CD-R
00. Requirements: 925 blank CD-Rs.
01. Time: At 10 minutes each? 154 hours.
02: Money: $20 for 100 = ~$200
03. Accessibility: Accessible with proper filing system.
04. Permanence: ??
DVD-R
00. Requirements: 135 DVD-R Discs
01. Time: 23 hours.
02: Money: $20/100 = ~$40
03. Accessibility: Accessible with proper filing system.
04. Permanence: ??
Blu-Ray
00. Requirements: 24 DVD-R Discs
01. Time: 2x so … ??
02: Money: 24x$10 = $240 + $200 for a burner.
03. Accessibility: Accessible with proper filing system (and a blu-ray drive)
04. Permanence: ??
Hard Drives
00. Requirements: 1 1TB Hard Drive.
01. Time: Hours? (Not hands on time.)
02: Money: $99 (this week; less in the future, no doubt)
03. Accessibility: Accessible with proper filing system.
04. Permanence: ??
So really what it looks like is DVD vs. Hard Drives. Blank DVD’s would run me $40, but take 24 hours worth of burning. A Hard Drive would cost $100, but almost no time at all (I could start the file copy and walk away).
One concern of course is reliability/permanence. If I take either one of these solutions, stick them in the closet, and pull them out in five years, I wonder what the odds of each one retaining 100% of the data is?
The other problem is, of course, I have much more than just 600gb of mp3s to back up. Let’s say I decided to rip every one of my DVDs (well over a thousand) to DivX/XviD format and wanted to back those up as well. That’s another terabyte.
So, what say you, nerds? Is HD > DVD or the other way around? What’s more important to you — time, or money?
Another thing that makes me nervous as I amass more digital “stuff” is backing up all of this data. Do I burn/copy all of these things in duplicate? Am I wasting my time and money by having two copies of all of this crap, or does that make sense?
Part of me is considering building a giant(er) RAID to hold everything, and then backing all of THAT up to external hard drives. I’m doing that now with my mp3s, but not with anything else like movies or anything. I dunno … sometimes I think I may be getting a little weird when it comes to data loss (if that’s possible).
Thoughts? Opinions? Advice? Donations?
I use several hard drives to back everything up. It is easiest and I am lazy. If I have something really important, it will be on more than one hard drive and/or on dvd or cd.
As I get older time becomes increasingly more important to me, I hate wasting it. I couldn’t imagine sitting around and flipping Blank DVD’s (or CD’s) to archive my “stuff” …. god knows I patiently flipped my share of 5.25’s back in the day and I’d like to think that 25 years later…we’ve progressed :) I’m a HD guy. I have a single external TB packed, and I have a RAID Setup in my one server with 3 750TB drives for added redundancy. With the insanely low cost of HD’s these days (and in very respectable sizes I might add) it seems the fastest, unattended archiving solution. Plus locating, filing and indexing is much easier than storing and cataloging a massive disc collection (and there are several web debates on the longevity of CD/DVD media. HD Longevity remains to be seen but is more stable in my opinion. That said, I have 20MB..yes MEG..hard drive in my ORIGINAL 386-SX computer that I bought loooong ago, and its still working perfectly to this day.
Thats my 2 cents.
With hardware as cheap as it is and software cost not a concern for you. I would recommend setting up a second set of raided drives. Cost is a little higher but the odds of losing a critical number of drives in both sets is low. however, I have been told I am paranoid. In addition I would use a sata toaster for the very transient data backups with a couple of discrete hard drives rotated out. This seems like a dramatic overkill for mp3’s until you factor in the time to replace finding and organizing 600 Gig of MP3’s…….Did I mention I might be paranoid?
do not trust CDR or DVD that you record on!!!
I have lost hundreds of albums, pics and movies that way. stay with hard drives as long as you can.!!!
I would suggest hard drive, but I would suggest that you do not use RAID. Your best bet is to stick with single drives. RAID will just have more points of failure, and RAID containers can be difficult to recover if you lost the controller. You are better off just copying the data twice to each of the two hard drives instead of using RAID-1.
I think your most reliable bet is hard drive too. An external one only turned on only to make a backup would probably last for a decade.
I wouldn’t bet on optical discs holding data for 5 years, necessarily. Probably most of the discs would be fine, but most likely one would have problems. I have CD-Rs that I burned 5, 10 years ago that work fine, and others that don’t.
Leaving a hard drive sitting doing nothing for 5 years isn’t highly recommended, but more often than not they survive it. And if it doesn’t spin up right away, heating it up to operating temperature with a hair dryer often gets it going. Frankly, for all the complaining we do about hard drives, in my experience hard drives are still less likely to fail than the various media we use to back them up.
I’d go with a terabyte drive, fire it up every six months or so to make sure it still runs fine, and wait for the price of flash media to come down. Ultimately I think flash is the solution, but it’s nowhere near cost-effective for this purpose yet.
Thoughts? Optical media is the pits (pun intended). DVD-R “appears” to last longer than CD-R but I wouldn’t bet on it either. Then there’s the capacity limit.
Opinions? It’s my experience that it’s the electronics of a HD that usually give up the ghost. Unless you have have a (rare) mechanical failure or drop the drive, the data on the platters remains in place and most of it *can* be retrieved by rebuilding the drive using an identical spare.
Advice? Go with RAID or a simple backup via external drive/s. But do spin them up every few months or so. Flash media *could* be interesting but the capacity/cost factor weighs against them and, I’ve read somewhere, they’re also limited to a life span of some 10 years — other than losing their contents completely out of the blue and no warning beforehand, and then working again fine after a reformat.
Donations? Kindly accepted, thank you.
I have backups of my backups in a fireproof safe on dvds.
not a whole bunch of stuff mainly family photos. not really an answer to your question but whenever i make a backup dvd i do 2 copies and put one in the safe.
Look at the DroboPro!
RAID is a bad idea for home users since its management complexity and failure modes create more problems than it solves. But protection from 1 – or better yet 2 – drive failures is a very GOOD idea.
Data protection for the rest of us
Drobo is a product that protects against drive failure like RAID does without the headaches of traditional RAID arrays.
More here: http://blogs.zdnet.com/storage/?p=454&tag=nl.e040
Just a head’s up Dean … I know of at least two people who had fireproof safes in their home during this last rash of wildfires who lost their homes and were surprised to find their “fireproof” safes were little more than piles of melted goo when they returned. Jeff’s (response number 3) uncle’s house burned and I believe even some of the contents of his gun safe were ruined.
The very most important data I own are my digital pictures, and I have three copies of them. A local copy on my server, a backup copy stored on my RAID, and a copy on an external hard drive which is stored off site. I am serious about those. As much as I would not want to, everything else I own (movies, music, games, etc.) *could* be replaced (it would take years, but it could be done). The digital photos are irreplaceable.
Thanks for all the input and opinions so far. Right now I’m leaning toward one of those SATA docking stations, a stack of 1TB drives, and some sort of carrying case to store them in. Not fire proof, but somewhat resistant.
Also one other thing I should have mentioned originally … my RAID is RAID5. I a considering upgrading it to either RAID6 (two spares) or RAID50 (RAID5, mirrored). I am serious about data loss, probably past the point of reasonability.
I try to keep my overall drive usage trimmed so that a regular backup every six months takes about 2-3 CD-Rs. But I make no attempt to save most MP3s, games or videos because I know that the Internet provides the best redundancy of all, and so unless it’s something I have created, it should be possible to find it again somewhere else online.
That didn’t answer your question though. Um… for that kind of storage needs I would say you should set yourself up with a Network Attached Storage system stuffed with some 1TB quick-removable drives (in case you have to bolt but you want to take everything with you). Share that with your house so your backup system doubles as a fileserver for the house, you’d probably get more use out of it that way, plus it would prevent the drives from sitting still for years on end.
Um… you did not put in the calculations for floppy disk… so I am going to asume that using 754298 3.5in floppies per TB is out of the question for you?
Huh? :-)
Mom
I have about 500 gigs of mp3s, i simply just have one copy of the collection at home, and another at work on an external drive. i dont see what the big deal is.
If that’s the only data you need to protect then you are lucky and I agree, having two copies of 500gb probably isn’t “that big of a deal.”
Let me throw some other numbers at you: add somewhere around 50 DVDs worth of software (~225 gig), two terabytes of movies, and a veritable shit load of other “stuff” (from music videos to home movies to e-books to …) that brings the total closer to 3TB, maybe more.
My data falls into three categories: things that can be easily re-obtained (movies), things that would be difficult to re-acquire (mp3 conversions of old cassette tapes from local bands that broke up ten years ago) and things that would be impossible to replace (things I have created, digital photos, etc). Obviously not all of those things need the same type of redundancy; irreplaceable digital photos are more important to me than divx rips of movies that I’ll never watch.
I can’t tell you how many people have called me (in tears) because their computer imploded for whatever reason and they lost “everything”. I have over a thousand digital pictures of my daughter and probably a dozen physical prints. My digital data is EVERYTHING to me, which might be why I invest “more than the average bear*” into making sure it doesn’t die.
(* That’s a Yogi Bear joke. I am not a “Bear”.)
yes, that’s exactly what i meant! just back up stuff from most important to least. im a photographer, i have over 100 gigs of photos, i have two copies of everything. mp3s are second most important, i have a lot of stuff that would be very hard to find again, or stuff that i ripped from vinyl myself. so that gets a second copy. everything else, i don’t care. movies, software, all that stuff can easily be found again.
Again, I repeat, STAY AWAY from RAID for any long term storage solution. In 5 years when your controller is fried and you need to move your RAID volume to a new controller, you are going to be wishing you had listened to Brian and me.
RAID is a redundancy/high availability solution and is not a good backup solution at all. It also increases the risk of accidentally wiping out both your original copy and the backup because it is a blind copier. If you write bad data to your volume, it automatically gets replicated to the entire volume. So if you go with RAID you are still going to have to have a tertiary drive for backups of the RAID drives. Why bother buying 3 drives to do essentially the same thing you could have done with two drives? Performance on long term storage is not a factor, so the last remaining benefit of RAID is pointless for you as well.
RAID is good if you want to increase read performance of your disks (and don’t care about write performance since that can even go down in RAID setups) or if you want to ensure that a mission critical environment does not go down because of the loss of one or more drives (depending on how many drives you setup, you can have a higher fault tolerance). None of these things seems to be important to you in your environment. It would take you a few minutes to swap out one of your backup drives to be the primary in the event you lost a disk in a non-raid setup. And then you would only be without a backup for whatever amount of time it takes you to buy a new drive. Your downtime would already be minimal and you would be keeping your points of failure as low as possible.
If you are serious about data loss, then ditch the RAID completely and make all the drive stand alone.
The usable capacity of a RAID 6 array is (N-2) * S, where N is the total number of drives in the array and S is the capacity of the smallest drive in the array. So, which do you think would provide you a better level of data security? One volume of 4 disks or 4 seperate disks each with their own individual copy of the data set? I will take the 4 copies every time personally.
Amazon’s Cloud Storage? http://aws.amazon.com/solutions/aws-solutions/#backup
======= it says:
Backup and Storage
Even as storage becomes more plentiful and affordable, you’re still faced with the task of managing your growing storage infrastructure. Amazon Web Services provides a cost-effective solution for storing information in the cloud that eliminates the burden of provisioning and managing hardware. The Amazon Simple Storage Service (Amazon S3™) is storage for the Internet. Amazon S3 provides a simple web services interface that can be used to store and retrieve any amount of data, at any time, from anywhere on the web. With Amazon S3, you get a highly scalable, reliable, fast, and inexpensive data storage infrastructure that enables you to deliver cost-effective and dependable backup solutions. Thousands of customers already use Amazon S3 as their personal backup location, and several more customers deliver compelling end-user backup, storage, and disaster recovery solutions using Amazon Web Services, including:
Backup and Storage Case Studies: 37signals, Altexa, BeInSync, ElephantDrive, Jungle Disk, MediaSilo, MyOwnDB, Sonian, Zmanda
========
But I don’t know if it’s feasible for the amounts you’re getting into. Or if content might violate the TOS. :) And with all that you’d need a really fast upstream connection. In addition to getting over the reservations one might have trusting their data to a ‘cloud’ storage solution.
thanks for the heads up. Im now making a back of my backup for a backup that I will keep at my mother in laws. Today me and the wife chewed over this thread and went to best buy and brought a 1TB drive. it was $100.
I dunno if it was a good deal or not I think it was about the going rate. Just wanted to get things going before the laptop gets stolen, the house burns down or blown away.
$100 for 1TB is pretty good, especially for a brick and mortar store. They have been going for a bit cheaper on Slickdeals every once and awhile, but not that much cheaper.
I used to really like raid – until I had a faulty raid controller fill up several of my Maxtor drives service areas (g-list) with false drive errors a while back. Once that happened, I could no longer access those drives. I still use raid, but I do not rely on raid. Luckily, I had most things backed up.
Perfect example of what Brian and I are talking about here. RAID controllers just add another point of failure to your backup. That is not something that you want to do when you are talking about IMPORTANT backups. Run very fast from RAID for backup.
What about using a software RAID instead?