Massive Spider Attack


Subject: Massive Spider Attack
Hi

Got a small problem with one of my users pages.

Results that he´s having massive spider attacks from rumanien sites, but all using an american spider,

called 80.legs

turns out to use up to 80 conections or more at the time, driving the server cpu use, nearly into red area.

spider

the funny thing is, it uses ip´s from rumania etc

but we can´t off course block IP by IP, by the time you finnish with your list, he´s back with new IP´s. And look´s like theres no way to block them by spider id ( tried it, but it´s still there).


Any idea how to get rid of this ???

Edit:

Searching 80.legs website, i found This

Any idea how to include this spider into the robot block list??

By htaccess or by robot text

Subject: Re: Massive Spider Attack
.htaccess is the best way!

Subject: Re: Massive Spider Attack
Deny the access from all the range of the network (if you don't care of Romanians)

for example: I found today on the error log someone that was trying to find files that doesn't exist my place (usually they search for the "links" mod or the "knowledge base" mod (I've uninstalled it)).

So in order to prevent the IP change I banned the whole network, you can find the range easily without calculation using whois.domaintools.com

for example if the ip is

58.22.0.0

the range is from 58.22.0.0 to 58.22.1.255

and the line in the .htaccess is

deny from 58.22.0.0/15

(you can find the range "58.22.0.0/15" in the site's page, but there are other site's that can make the calculation for you, I forgot the links, sorry :oops: )

Subject: Re: Massive Spider Attack
well for first thing i did put the spider id in the tracker *008*.

took about 10 minutes , and all 80 legs conections were gone.

now i probably start to block ukrainiens "ru" range.

turns out that they´re trying to access referers list


Page 1 of 1


  
You cannot post new topics
You cannot reply to topics
You cannot edit your posts
You cannot delete your posts
You cannot vote in polls
You cannot attach files
You can download files
You cannot post calendar events

   

This is a "Lo-Fi" version of our main content. To view the full version with more information, formatting and images, please click here.

Powered by Icy Phoenix based on phpBB
Generation Time: 0.0897s (PHP: 20% SQL: 80%)
SQL queries: 10 - Debug Off - GZIP Enabled