Email archiving woes

Slashdot has two articles up about email archiving woes. One which is more of a question posed by an individual who wants to keep all of their email over the years (over multiple platforms and IM logs as well). The other raises the opposite side of the issue in that it appears Microsoft is claiming to have lost email from a period of time which would be crucial in a current lawsuit against them (from Burst.com over alleged patent infringement within Windows Media Player). They have the data on either of side of the time period, but not the 35 days in question.

The first one is more of an amusement to me since I personally am not sure I would *want* that much back-history of data since it wouldn't be particularly useful and just a hassle. But it is very possible, and probably likely, that this person has far more interesting and/or important data than I have had so far.
My recommendation to him and those like him, would be to use several media types since a failure of one type would not necessarily mean failure to all of them. USB flash drives are large enough now that you can hold a huge amount of data - 1GB is very likely enough to hold all of one person's email over the years and IM logs (text is small) - that is assuming you aren't also keeping large attachments. (one of the executives where I work has nearly a 2GB mailbox and it is because he read that Bill Gates never deletes anything - just for the record, that is a bad reason to not delete anything from your inbox)
Between CDs, DVDs, USB flash drives, and portable hard drives, you could back-up most any home email archive with a bit of overkill. You could put a tape drive in there too if you were really paranoid of media failure - but if you are that paranoid, then you should probably also distribute the media in different secure physical locations and perhaps also upload it to a networked server too.
(a lot of that is tongue in cheek since it is extreme overkill for pretty much any home use - but actually probably not enough for some corporate environments, so it isn't quite as ridiculous as it sounds)

Microsoft's problem on the other hand is entirely different. Now there are plenty of small companies that have lost backups and archived data. But in this case, Microsoft has the data on either side of the loss and the window of loss just happens to be during the period which the lawsuit wants.
There are a few ways to look at this, each equally valid depending on which side you are viewing it from. One side sees it that Microsoft is clearly covering their butts in this case and know that they have the money and legal team to tie this up in the courts until the other side gives up or runs out of money. Another side is that Microsoft is claiming that they didn't find the emails interesting enough to merit keeping and deleted them, but did find everything else interesting and so kept it. Finding something "interesting" is a subjective thing, so it is hard to claim legally that they are lying - even though they very likely are.
I have seen a few lawsuits where they pull out the email archives and use that to determine who screwed-up and where. The firms which have dealt with this before know to avoid paper/digital trails and they will handle questionable conversations in person - plausible deniability. If they say it didn't happen, there is nothing to prove otherwise. In these types of lawsuits, I have even seen companies claim that they lost their backups or had faulty equipment. Since that actually does happen, unless the law for their industry holds them accountable for such things, they aren't in the wrong (there have been changes in the fund world so that they have to keep massively detailed backups, but not every industry has specific laws for that).

Personally, I'm fairly laid back about my personal backups and I put data I must have onto CD, removable hard drives, and USB flash drives. At work we have backups to a server, redundant alternating removable hard drives, multiple USB flash drives (not large backups, just certain small but vital databases and programs), Ultrium tapes (with a relatively short rotation of a single week, since backup is more of our interest over that of archival purposes - but we are moving that up to a month with a year's worth of archival points), and a remote location backup site as well (still in progress). We rotate the removable media offsite on a weekly basis.
Those strategies, both at home and at work, are more for the purpose of getting things up and working again, less on going back to see the status of something on day X of year Y - because the nature of our business doesn't require it.
Your mileage may vary.

Feel free to post up your backup and/or archiving process and why it is at the level of scrutiny it is.

Posted by Eric at March 12, 2005 10:00 PM | TrackBack