processed pork

Like most internet users, I get spam. Unlike most, I get ridiculous, overwhelming amounts of it: on average days, upwards of a thousand pieces. Consider it an occupational hazard. To find the screenwriters, directors, actors, producers and slew of other collaborators on which Cyan’s (and now Long Tail’s) projects depend, I need my email address flung far and wide through the cybersphere. But, in that flinging, it inevitably ends up on junk mail lists everywhere.

For the past year or so, I’ve been getting around the problem using KnowSpam.net, a server-side challenge-response system. Which basically means that, every time someone sent me an email, if I hadn’t previously received an email from them, KnowSpam would ask them to demonstrate they were human by answering a question on their website. And, on the plus side, it worked exceedingly well in cutting my spam down to zero. On the minus, it also started to increasingly piss off the humans whose humanity was being verified.

So, yesterday evening, I downloaded SpamSieve, a Bayesian filter for the Mac. Bayesian filters (near and dear to my heart from the neuroscience and computer science days back at Yale) essentially figure out the fuzzy overall similarity between two things – in this case, the similarity between an incoming piece of email, and the entire body of previously received email already sorted into spam and ham. Bayesian filtering, in theory, works remarkably well. But, like communism, it rarely seems to pan out quite as nicely in practice.

Which is why I was more than pleasantly surprised by SpamSieve; I was joyously shocked. After training the filters on a stack of old emails, it caught all but one of the 313 pieces of spam I received since midnight. Now, perversely, I’m taking pleasure not in the real email I receive, but in the flow of penis enlargement ads and mortgage refinancing offers, as they pile up, message by unmissed message, in my junk folder.

Finding happiness in watching good technology at work. Further proof that, hide out in the world of film as I may, I’m still pretty much 100% dork.