Geeklog Forums

As I expected, after some days of testing the latest Bad Behaviour2 I found that BB2 is blocking several search engine bots on our website.

Some, I can understand, they are Chinese, some Japanese, and a few regular lesser known search bots. The latest I found was http://www.exalead.com (193.47.80.51 2008-10-11 04:39:39 Mozilla/5.0 (compatible; Exabot/3.0; +http://www.exabot.com/go/robot) Required header 'Accept' missing)

Also, be careful using BB2 if you did some extensive work to get your website registered in web directories which check regularly if the URL is still valid, or if you still have a link back...

But then again, do you really want to be indexed by such a clueless company? What exactly is the benefit of being indexed by a small search engine that nobody every heard of (or uses)? If they can't even get their bot to speak HTTP properly (as indicated by BB's reason for the blocking) - what does that tell us about their competence?

bye, Dirk

P.S. The above is my personal opinion only, of course.

I personally belief that BB2 is a good solution if you have a website which depends on mainstream visitors, but if you are running a website in a niche market, say like diving or Asian culture, BB2 becomes less a good idea.

exalead.com indexes about 8,029,402,560 web pages, with enough visitors to generate a serious visitor rate. The hit rate is higher then Altavista... or Ask....

We still testing the Bad Behavior2 plug-in and found that most pings we do to blog directories are being blocked by BB2. I have to say that not all is bad, I can see usefulness in the plug-in I only and to see if there is a possibility to have a white list?

For example, http://www.feedsky.com indexes our website regular, and it is now blocked by BB2.
(Okay, Feedsky is Chinese, but still in China alone are more active Internet users then in whole Western Europe..., we run a few travel websites, and more and more Chinese travelers us our websites)

Bad Behavior does have a whitelist in the file public_html/bad-behavior/whitelist.inc.php

bye, Dirk