Live Search turns off search crawler, starts SEM blog series

bot Search Engine Roundtable has been following a thread at WebMaster World concerning some disturbing behavior from MSNbot, the crawler Live Search uses to find and index information on the web.  As of last night (April 28), the offending “feature” has been turned off, according to Brett Yount from Microsoft (MSNdude on WebMaster World):

I just spoke with the team. They said they are working on a fix for this, but the feature causing the problem is turned off currently, so you shouldn’t be seeing any fake referrals from us.

Apparently the behavior was part of an attempt to detect cloaking, which Wikipedia defines as “..a black hat search engine optimization (SEO) technique in which the content presented to the search engine spider is different to that presented to the user’s browser.”  The problems first occurred in December of 2007, and prompted the Live Search team to post an explanation on the issue and what they were doing about it.  However in the months since, continued problems with inflated site statistics caused by MSNbot keep resurfacing.

And while it isn’t quite Steve Ballmer speaking at SEM conferences, like Danny Sullivan wants, a new series of blog posts on Search Engine Marketing 101 begins on the Live Search Webmaster blog with a post by Rick DeJarnette of Live Search.  DeJarnette explains why the series is warranted:

When people first look into SEM, they are usually overwhelmed by new concepts, new jargon, new technologies, and the fact that there is usually an assumed level of technical knowledge that they don’t have and lack the time to learn (they’ll often tell you they’re too busy trying to run their business to devote time to the website!). As a result, they either hire a consultant to “make it work,” or if they are DYIers, they spit out some basic HTML code and call it a day. But unless the person doing the work knows the right strategies and tactics, these efforts are often only marginally successful. Let’s change that.

We’re looking forward to the series, as we know far too little about SEM ourselves.