Welcome to Geeklog, Anonymous Saturday, September 07 2024 @ 08:45 pm EDT
Geeklog Forums
Bad Behavior2 blocking Search Bots
As I expected, after some days of testing the latest Bad Behaviour2 I found that BB2 is blocking several search engine bots on our website.
Some, I can understand, they are Chinese, some Japanese, and a few regular lesser known search bots. The latest I found was http://www.exalead.com (193.47.80.51 2008-10-11 04:39:39 Mozilla/5.0 (compatible; Exabot/3.0; +http://www.exabot.com/go/robot) Required header 'Accept' missing)
Also, be careful using BB2 if you did some extensive work to get your website registered in web directories which check regularly if the URL is still valid, or if you still have a link back...
Some, I can understand, they are Chinese, some Japanese, and a few regular lesser known search bots. The latest I found was http://www.exalead.com (193.47.80.51 2008-10-11 04:39:39 Mozilla/5.0 (compatible; Exabot/3.0; +http://www.exabot.com/go/robot) Required header 'Accept' missing)
Also, be careful using BB2 if you did some extensive work to get your website registered in web directories which check regularly if the URL is still valid, or if you still have a link back...
7
7
Quote
Status: offline
Dirk
Site Admin
Admin
Registered: 01/12/02
Posts: 13073
Location:Stuttgart, Germany
But then again, do you really want to be indexed by such a clueless company? What exactly is the benefit of being indexed by a small search engine that nobody every heard of (or uses)? If they can't even get their bot to speak HTTP properly (as indicated by BB's reason for the blocking) - what does that tell us about their competence?
bye, Dirk
P.S. The above is my personal opinion only, of course.
bye, Dirk
P.S. The above is my personal opinion only, of course.
5
5
Quote
I personally belief that BB2 is a good solution if you have a website which depends on mainstream visitors, but if you are running a website in a niche market, say like diving or Asian culture, BB2 becomes less a good idea.
exalead.com indexes about 8,029,402,560 web pages, with enough visitors to generate a serious visitor rate. The hit rate is higher then Altavista... or Ask....
exalead.com indexes about 8,029,402,560 web pages, with enough visitors to generate a serious visitor rate. The hit rate is higher then Altavista... or Ask....
7
5
Quote
Status: offline
richard.bkk
Forum User
Junior
Registered: 09/27/08
Posts: 21
We still testing the Bad Behavior2 plug-in and found that most pings we do to blog directories are being blocked by BB2. I have to say that not all is bad, I can see usefulness in the plug-in I only and to see if there is a possibility to have a white list?
For example, http://www.feedsky.com indexes our website regular, and it is now blocked by BB2.
(Okay, Feedsky is Chinese, but still in China alone are more active Internet users then in whole Western Europe..., we run a few travel websites, and more and more Chinese travelers us our websites)
For example, http://www.feedsky.com indexes our website regular, and it is now blocked by BB2.
(Okay, Feedsky is Chinese, but still in China alone are more active Internet users then in whole Western Europe..., we run a few travel websites, and more and more Chinese travelers us our websites)
10
8
Quote
All times are EDT. The time is now 08:45 pm.
- Normal Topic
- Sticky Topic
- Locked Topic
- New Post
- Sticky Topic W/ New Post
- Locked Topic W/ New Post
- View Anonymous Posts
- Able to post
- Filtered HTML Allowed
- Censored Content