I fly down the stairs, feet moving so quickly I almost trip over them. Rounding the corner I dodge Morgan, pop into my office, and remove my workstation from the domain. As that machine begins to reboot I jog back to the living room and begin moving downloaded utilities from my laptop to a memory stick. Once the copy begins it’s back upstairs to try and pull more data out of backups. No, this isn’t some new sort of nerd Olympics — unforunately, it was my Monday night.
We’ve become so reliant and comfortable with Microsoft patches that patching and rebooting a machine, even an important server, seems mundane. (I don’t want to hear it from the Linux crowd; my Ubuntu machine downloads as many if not more patches on a day-to-day basis.) When I checked my home server Monday morning and saw that it had downloaded new patches, I didn’t think twice about applying them and rebooting. In fact I thought so little of it that I didn’t even stick around to make sure the server came back online. For the record, it didn’t.
Data recovery is a funny thing that almost never works as well or as smoothly as you think and hope it will. My personal backup scheme goes above and beyond that of most end users and some businesses. Once machines are loaded and configured, snapshot images (using Acronis) are made and stored. In addition, “data” is backed up on a nightly basis as well. By “data” I don’t mean applications, but my own personal files. In other words, I don’t back up Microsoft Word on a nightly basis, but I do backup my Word documents.
On a recent episode of Mythbusters, Adam and Jamie demonstrated how to escape from a sinking car. The technique involved waiting for the car to fill with water, opening door, and swimming to safety. Even with a tried-and-true plan in place and the knowledge that they were filming a television show, Adam still panicked and had to be given emergency oxygen. Even the best plans fall apart in a moment of panic; such is the often case when a hard drive fails. Even though I perform regular backups, it is sometimes difficult to function when all you can think about is the fear of losing almost fifteen years of digital photos and tons of original documents. During these times it is imperitive to follow preset plans and not lose your head.
I worked from 3:30pm Monday until almost 3:30am Tuesday morning reloading, restoring, and recovering data. My backup plan worked good, but not great. In retrospect, I should take more frequent snapshots of my server, or at least my application directory. After working for almost 12 hours straight on getting IIS, MySQL, WordPress and PHP to talk to one another, I threw in the towel and called my good buddy Jeff at Managed Data Solutions to bail me out. Within the hour Jeff had everything up and running, and once again I owe him dinner.
I learned a bit more about backups this time without too sharp of a sting. 24 hours later I’m back up and running, with 0% data loss.
We now return to our regularly scheduled program.