Spam is a tricky problem. Or as Matt Haughey
says "spam bloggers sure are resourceful little bastards."
For a while now, the Blogger team has been contending with spam on Blog*Spot through mechanisms like
Flag as Objectionable and
comment/
blog creation CAPTCHAs. The spam classifier that
Pal described has also dramatically reduced the amount of spam that folks experience when browsing NextBlog.
However, spam is still being created and, as was
widely noted, Blogger was especially targeted this weekend.
One group of folks who are particularly affected by blog spam are those who use blog search services and those who subscribe to feeds of results from those services. When spam goes up, it directly affects the quality of those results. I'm exceedingly sympathetic with these folks because, well, we run
one of those services ourselves.
So given that the problems is hard, what more are we doing? One thing we can do is improve the quality of the Recently Updated information we publish.
Recently Updated lists like the one Blogger publishes are used by search services to determine what to crawl and index. A big goal in deploying the filtered NextBlog and Flag as Objectionable was to improve our spam classifiers. As we improve these algorithms, we plan to pass the filtered information along automatically. Just as a first step, we're publishing a list of
deleted subdomains that were created this weekend during the spamalanche.
Greg from Blogdigger (one of the folks who consumes blog data)
points out that "ultimately the responsibility for providing a quality service rests on the shoulders of the individual services themselves, not Google and/or Blogger." However, we think by sharing what we've learned about spam on Blogger we can hopefully improve the situation for everyone.
We can also make it more difficult for suspected spammers to create content. This includes placing challenges in front of would-be spammers to deter automation.
Of course, false positives are an unavoidable risk with automatic classifiers. And it's important to remember that the majority of content being posted on Blog*Spot is not spam (we know this from the ongoing manual reviews used to train the spam classifier).
Some have suggested that we go a step farther and place CAPTCHA challenges in front of all users before posting. I don't believe this is an acceptable solution.
First off, CAPTCHAs represent a burden for all users (the majority of whom are legit), an impossible barrier for some, and are incompatible with API access to Blogger.
But, most importantly, wrong-doers are already breaking CAPTCHAs on a daily basis. And not through clever algorithmic means but via the old-fashioned human-powered way. We've actually been able to observe when human-powered CAPTCHA solvers come on-line by analyzing our logs. You can even use the timestamps to determine from whence this CAPTCHA-solving originates.
One thing we've learned from Blog Search, is that even if spam were completely solved on Blog*Spot, there would still be a problem. As others have concluded, we've realized that this is going to be an on-going challenge for Blogger, Google and all of us who are interested in making it easier for people to create and share content online.
Ever want to know what bloggers are saying about a web page but been too lazy to do anything more than watch slick animations? You clearly have a fever for which the only cure is Blogger Web Comments, our new extension for Firefox 1.5. Once it’s installed, you can see instant results from Blog Search showing you posts about the page you’re looking at. Plus, there’s a Blogger posting form built in, so you can immediately make a post on your own blog. Glen Murphy has a little story about making the extension in his announcement post on the Google Blog.
Can you tell it’s a bit of a slow Friday at Blogger? Anyway, here’s Prashant and me being happy and excited in our new Blogger hoodies, which just showed up in a big box of warm fuzziness.
Reporters Without Borders just released a "Handbook for bloggers and cyber-dissidents:"