Why I haven't read your email... Thunderbird ate it!

A reader of this blog — yes, there is one… I’m as shocked as you are — sent me two emails over the weekend that I haven’t been able to read. I saw the headers, but Thunderbird, my former email client, chose to shrink my email box down to size 0, deleting all my emails in that box!

This is the second time Thunderbird has destroyed my email Inbox! The first time was last summer. And I’ve had enough.

When this happened last year, I decided to avoid ever again losing months worth of email by: 1) telling my email server to save my emails for a while, even after I’ve downloaded them; and, 2) making regular data backups.

Stupidly, however, laziness overwhelmed my sense of loss. My hard drive is so large (500GB) that full backups take about 13 hours, so I went months between backups. Also, since most of my code and databases are also saved on my servers, I wasn’t sufficiently concerned about losing data.

This time, I’ve already told my server not to delete my emails for 90 days. I’ve also completely backed up my hard drive.

But, most importantly, I’ve decided that Thunderbird is fatally flawed by its use of the “Mbox” email storage format. Mbox is very popular and works well for people with low email usage. But for people like me who store gigabytes of email, mbox is a disaster waiting to happen. Why? Because mbox stores ALL your emails (at least, all the emails for each email account) in a single file. When your email client is constantly adding to and deleting from a multi-gigabyte file, corruption will eventually occur. In my case, Thunderbird apparently choked on the size of the file. Instead of warning me (e.g., “Your email file is very large. Please save a copy and then I will attempt to shrink it down”), Thunderbird somehow decided to shrink it to size 0 and start all over again without warning… for the SECOND time in twelve months!

So I’m moving on. I’m going to use a MailDir-based or MH-based email client (possibly Claws) that stores each message in a separate file.

P.S. When a multi-gigabyte file vanishes from your Linux hard drive, shut down your computer immediately, boot up again from a LiveCD (I used Ubuntu Rescue Remix), clone the hard drive (with GNU’s ddrescue, packaged as “gddrescue”… don’t use package “ddrescue”), and then use a program such as PhotoRec to search through deleted files. I knew this was the proper procedure, but my email disaster struck just as our family was heading out the door. I panicked and installed a program I hoped might fix my problem quickly. I may well have overwritten part of my email file. (I’ll find out tonight when I try to recover the email file from my hard drive clone.)

Posted by James on Monday, June 07, 2010