A hack to implement SPF on Exchange (2000 and 2003) via SpamAssassin 2.70

There has been a recent (Feb 12-20th 2004) article posted by Microsoft that discusses their "Caller ID for E-Mail" which is basically their way of the usual branding of other people's ideas as their own.
In this particular case, the idea that they are branding as their own is the SPF (Sender Policy Framework) concept.

What these technologies both do are essentially try to fix the flaw in SMTP which allows anyone to send mail so that it looks as if it is from anyone. I could send mail out from the spamblogging.com domain but make it look in the from field that it is from whitehouse.gov if I was so inclined to be a jerk.
These two ideas both do a lookup to see if the domain that the mail was sent from is part of the same group as the from field claims to be from. If not, then action can be taken, based on how the mailserver is configured.

Microsoft Exchange, like many of their products, claims to make various maintenance processes easier - and perhaps in many cases it succeeds. But in cases like this, unless MS specifically writes in a tool to allow you to do something, you can't do it at all. So if you want to implement their Caller-ID system, you can't yet since they haven't written anything.
And at first glance you can't use the SPF either.

That said, these two pages help a little in explaining how to enable SPF on your server and setting up the DNS as well. The DNS bit is relatively straightfoward - quoting from this page:

You can create the reverse MX record by hand or through a wizard provided by the creators of the SPF RFC http://spf.pobox.com/wizard.html. The format of a typical reverse MX record is simple enough:

domain-name. IN TXT "v=spf1 a mx ptr -all"

In this example, is the domain name to publish a reverse MX record for. Note the trailing dot after the domain name; if you published the domain name foo.com, it would look like this:

foo.com. IN TXT "v=spr1 a mx ptr -all"

To add the appropriate reverse MX record to a server running Microsoft's DNS, follow these steps:

1. Open the DNS snap-in via Microsoft Management Console
2. Right-click the domain that you want to add an SPF record for.
3. Choose Other New Records.
4. Choose a record type of TXT.
5. Enter the SPF record to specify valid mail senders.

For other DNS servers, such as BIND or the free DNS server BIND-PE, you need to add the record manually.

To implement the SPF checking into Exchange is harder though. The same page lists out a way that you can use an Exchange event sink and some third party tools to do it.

So now this brings me to the idea that I had which spawned this blog post...

It occurred to me that I have already written a hack for Exchange that allows SpamAssassin to run on Exchange 2000/2003.
This hack also runs on an event sink that gets triggered every time a message is received by Exchange. The general process is:
1) A message comes in and triggers the event sink.
2) The event sink script holds the message in memory as an object.
3) The script writes the message out to a temp directory on the server as a file.
4) The script calls the SpamAssassin program to be executed on that file.
5) SpamAssassin does its thing and outputs a file with the headers in it that essentially say that it is either spam or not spam based on what the settings are.
6) The script scans that file that it outputs to for the X-Spam header and checks to see if it is "YES" or not.
7a) If it is spam, it then marks the subject of the message object in memory with and additional "*****SPAM****" (or whatever you have it configured to write).
7b) If it is not spam, then it does nothing at all and does so with a big smile on its face.
8) The message in memory is then left to continue on through the event triggers and if it passes all of those successfully and isn't kicked to badmail or killed, then it gets delivered to the end user.
9) The end user then has mail that was triggered marked as "******SPAM*****" (or whatever you chose to mark it as in the script settings) and can thereby sort it however they want with their own local filters.

This is all relevant because the new (as of the time of this writing) 2.70 (still in testing/cvs, but available to use) version of SpamAssassin will handle SPF lookups - so this is an "easy" way to get both spam filtering and SPF lookups into Exchange which would otherwise put up a fight.

A small zip file with the Exchange 2000/2003 event sink code can be found here.

A description of how to install SpamAssassin on your Win32 system can be found here. (that talks about how to install SA 2.60, but should still be applicable for 2.70 - although to setup the SPF part might take slightly more familiarity with the configuration settings - I haven't looked too hard yet)
Also note that this guy has written a Win32 version of the spamc/spamd as well - that is likely mentioned on his site.

You can find the latest SpamAssassin 2.70 releases at the bottom of this page. If you look in the "devel" directory you can find a 3.00 version - but I haven't look at anything at all regarding this - so I personally wouldn't use it just yet (at least not in the Exchange setup since I only know it works with the 2.x systems for sure).

Notes about the hack of mine:
1) This solution is serial - while it is working on one message, no other event sinks can fire during that time. So if you get a lot of messages in a day, then this might not be the best solution for your system. I have a network of about 25 users and we get maybe 1000 messages a day (largely spam too) and it runs fine for us (both on a PIII 1Ghz with 512MB RAM and on our newer P4 HT 3Ghz with 2G RAM).
2) This only gets triggered on incoming mail, it does nothing to outgoing mail - that said, you should already configure your system to not allow relays anyway.
3) I get about 1 e-mail a day from around the world asking me for the same feature to be added - everyone wants the headers that SpamAssassin creates to be passed on to the message that will be delivered to the end user. This is just a quick and dirty hack that I made - in an ideal world that would happen, it would behave just like SA - but as far as I know (someone please correct me if I'm wrong), the scripting API does give access to editing the subject of the message object before it is delivered to the end user, but it doesn't allow you to edit the headers.
It is either that or the way Exchange formats the headers is "wrong" compared the way SA assumes that they are formatted (Unix style).
I honestly don't recall since it has been nearly a year since I banged this hack out.
4) The code is no way guaranteed not do horrible things to your machine, your family, or the price of tea in Southeast Asia. That said, it has only ever done something weird on our server once - and that was that it just randomly stopped running. I reinstalled the sink and it was fine again - no clue what caused it (likely a server issue and not an issue with the code).
We briefly turned it off when we upgraded from Exchange 2000 to 2003 and planned on getting a commercial solution, but then when the higher-ups decided that my budget being $0 was too high as it is, we would be just fine without any commercial products.
5) This isn't a total no-brainer - it probably helps a bit if you know how to use SpamAssassin on other systems so that you can configure your user-prefs file.
6) The code currently will write out the spam/non-spam to folders that you specify - I need to add in the ability to turn that off. You can do that yourself if you so desire simply by commenting out code or cutting it out, but if that worries you, eventually I will get around to updating this code.
7) If you change something in the SA user-prefs file and expect it to show up in the message delivered to the end user... don't. The user-prefs file will change what the temporary file has done to it, but the message delivered to the end user with this hack is what the event sink controls - not SA directly. If you want to change the subject addition text to "*-*BEANS*-*" instead of "*****SPAM*****", then do so via the event sink script and not via your user-prefs config file.

If that didn't scare you off from it, then good luck with it and feel free to write me and I'll likely write back and whine about having too much other stuff to do right now to make modifications on it.

Also, a special thanks to Nugget who mentioned SPF/Caller-ID in his blog (I am "stenz" with the brain dead responses in that post).
I had been so busy with work lately that I had somehow not noticed the SPF/Caller-ID news.

Posted by Eric at March 11, 2004 11:29 AM | TrackBack