My groggy mind thought back and confirmed that we hadn't seen the backup app inturrupt our workflow in a few weeks, since about the the time we'd last mucked with cron jobs (the most likely cause). We added "manually trigger backups and fix automated scripts" to the end of our "to do" list for the day. It couldn't go at the beginning because I was under the gun to get a presentation polished up and sent out by the end of the day. Everything not directly involved in presentation production just fell to the back of the list.
I went through my day normally, until at about 1:00 in the afternoon my laptop refused to come up out of sleep. Nothing would wake it up, although the "sleep" light ceased throbbing while I'd tried. I tapped the "sleep" button, and my heart sank when I heard the startup chime come on instead of the screen waking up. It had crashed. That's okay, I thought: I'm quite positive I saved my presentation before putting it to sleep. However, my heart soon sank even lower when the machine refused to progress beyond the gray-apple screen. It just went black, shut down again. I repeated. It repeated right back.
The machine would boot up, at all. I brought it back to my desk and ran Disk Warrior on it. DW churned for a good long time, and seemed to "stick" on an "Overlapping File Extents" message. I did a web search for that message and found that it is not uncommon for DW to get "stuck" on that activity for days or even weeks! One of the blogs which had encountered this problem came back a month later (after having let DW churn for over three weeks) and said Data Rescue pulled their data off the failing hard drive in a few hours. This rang a bell; I'm pretty sure I'd heard one of the This Week in Tech regulars extolling the praises of this rather unimaginatively-named product earlier in the summer. Still, I didn't have it, and a license was more money than I could afford.
I did, however, also have Tech Tool Pro in my toolbelt. So, at 3:30 (DW having churned on rebuilding the volume in memory for about two hours) I canceled DW and started up Tech Tool Pro. TTP calls "Overlapping File Extents" "Allocation Overlap Errors" or somesuch, but in the end gave the same analysis: it can't fix the volume because the same cluster is allocated to more than one file on disk. TTP has a data restoration procedure, so I kicked that off.
TTP churned for three hours then crashed. I rebooted the machine, started TTP again, and "canceled" the process after about an hour. The results it had been able to "scavange" from the drive were depressing, placing folders like "Applications" inside application packages and generally all of what little it could find backwards and/or inside-out. This seemed to be a dead end.
It was now past 8:00 at night. I emailed my boss and let him know I'd try rebuilding the presentation from scratch, but that it was going to be late. I kicked off Disk Warrior again, and braced myself for weeks of waiting for DW to give me back a semi-working hard drive. I spent several more hours trying to put together the presentation I'd been working on, but knew I was missing some vital points. Without my notes and emails to consult, the "reconstruction" task was next to impossible.
So, mostly on a whim, I downloaded and installed Data Rescue II at about 1:00 AM. I stopped DW, it having not progressed in five hours any more than it had previously progressed in two. I started Data Rescue up, got to where it was performing a "quick scan", then went back to working on pounding my head against the wall and hoping that the things I was missing in the presentation would fall out. Needless to say, they did not. No more than fifteen minutes later I glanced over at the machine running Data Rescue and saw that it seemed to have stopped working. Typical, I thought. That's pretty much the nail in the coffin, then.
However, upon closer examination, Data Rescue had not, in fact, failed. Instead, it had completed in less than a quarter hour what Disk Warrior and Tech Tool Pro had failed at over several hours! All my data was listed in tree form, perfect.
The "demo" version only allowed me to pull one file. So, still a little skeptical that it would be able to get the data as easily as it had apparently found the directory structure, I navigated about six folders deep, to my presentation, and told it to recover it.
A few minutes later, the presentation was on my desktop, bit-for-bit perfect!
Now, here is the one bad point about Data Rescue. It has a shareware hook apparently devised by satan himself: you can see your whole hard drive, and you can pick one (just one) of your little children to rescue from annihilation. But, to get the rest, you need to fork over money. It is pure evil. Of course, no more evil really than just losing everything with no hope of recovery at all, but still. Such conflicts are not healthy.
I thought of all my email messages. I thought of my project plans. I thought of the months I'd spend trying to reconstruct - and poorly - even just a fraction of all this. Then I sighed and paid the "ransom". Truly, though: even that one document I'd gotten "free" was enough to justify the $99 cost.
The next morning, the entire contents of the PowerBook's drive were sitting, intact, on my large desktop drive. The presentation was off my desk, and my nerves were soothed enough that I actually might be able to get through the preso without bursting into tears over my lost documents.
Lessons Learned:
Of course, the most important lesson is: when the little voice in the back of your head says something about backups not working, that needs to go to the front of your "to-do" list, not the back. Secondarily, though, is that while each tool mentioned above does indeed work in certain situations, when push comes to shove and all seems lost, Data Rescue does what it says: rescues your data. After all, that is what is most important here.
Minor Gripes:
- "dot" files all lost their dots in the process. For instance, the ".MacOSX" folder in my home folder (holds environment settings) came out as "MacOSX".
- Some files lost ownership. I didn't have identical user names on the two machines (the one used to recover and the original machine), which I believe was the root issue. When recovered, many files and folders ended up owned by "System" and inaccessible to me (a "chown -R *" fixed that, although I'm not sure if anything below my home folder was *intentionally* owned by other than the current user so I don't know if this will cause other issues).