https://www.icyphoenix.com/viewtopic.php?f=35&t=8850&p=57788#p57788
-----------------------------------
mort
Thu 13 Dec, 2012 13:00

Re: A Vicious Spider Bot (AhrefsBot)
-----------------------------------
[quote user="Informpro" post="57785"][quote]Thanks for reporting, it may happen from time to time that a new crawler opens many pages... I'll try to limit its access if it continues.[/quote](from MG)[/quote]

It can be controlled no doubt - [i][b]limit Crawl-Delay: 360000[/b][/i], but as it has no respect for how it goes about its business, I'd class it as a web-reaper and ban it completely.

;)

[quote user="Joshua203" post="57786"]Ooopz, I thought thought you were one of those special cases, I'll turn it arround and add the link to here in the other thread (it's about reporting ahrefs and 80legs behaving badly on icy sites (whois page).[/quote]

Hehehe! Nope!  :mryellow: 

This thing appeared to have had 16 streams running when I killed it.  Probably Out-DDos'd itself.  :lol:


