Bad Bot blocking Revisited

Question

I've read a lot about bad bot blocking, php scripts, .htaccess techniques, etc...

Is this a valid method?

Since .htacces can rewrite and send a bad bot a 403 deny or forward to something like spam poison, is it possible to Disallow a folder, then through .htaccess in that specific folder redirect to spampoison? Since Apache reads each .htaccess independently and follows specific instructions, then a bad bot not following robots.txt would just be redirected. Or anyone trying to access, /badbot/ or whatever I choose to call my trap folder.

You're basically describing a form of honeypot. The bot trap John references in the answer he linked to below is one particular implementation, but this is a more general term for the technique. — Su', Jul 01 '12 at 03:49

score 2 · Answer 1 · edited Apr 13 '17 at 12:33

2

I've recommended similar tactics before.

edited Apr 13 '17 at 12:33

Community

1

answered Jul 01 '12 at 01:02

John Conde

86,255
27
146
241

score 2 · Answer 2 · edited Apr 13 '17 at 12:33

What you suggest (use a bot trap to trap the bad bod, and then block it using .htaccess or your favourite scripting language) is indeed a much-used technique for dealing with robots from the dark side.

You may want to look at another thread here about this particular technique: Tactics for dealing with misbehaving robots

If you're looking for sample code that tells you how to implement it, just Google for "bot trap". Here is just one such page: http://www.kloth.net/internet/bottrap.php that I found very instructive.

score 1 · Answer 3 · answered Aug 30 '12 at 05:54

1

If the bots aren't changing IP's often why let them hit your server and take up usage? I would ban the IPs's first in hosts.deny / ipchains then use .htaccess and robots.txt as back up

answered Aug 30 '12 at 05:54

Anagio

11,195
1
26
49

Bad Bot blocking Revisited

3 Answers3