Icy Phoenix

Old Support Topics - Google Cannot Access The Site Correctly Any More

kesha [ Sat 29 Nov, 2008 14:00 ]
Post subject: Google Cannot Access The Site Correctly Any More

Hi everybody,

this is the second time that I am facing a problem with Google. The first time, a year ago, Google bot started to get some strange errors on all pages of my site, and this phenomenon occurred only to Google.
I posted here the problem and was suggested to clean up the cache and recompile all modules. That solved the problem.
This time Google is again getting an error, but I cannnot see what kind of error. I tried to clean up the cache and recompile all modules, but after waiting for Google to respider the pages, I saw that the problem is still there.
You can see by yourself if you see the copy in the cache of Google.
With this query on google: "site:www.mandolino-balalajka.it" it's possible to see all the pages of the site indexed by Google (they used to be more than 2000 and there are now really very few remaining).
Following the link of the first result will take you to the home page of my site, without any errors, but if you click on the "Copia cache" link for that result you will see what Google gets when accesses the same page.
Any of you has an explanation for this and, most important, a solution?
Thank you so much!

kesha [ Mon 01 Dec, 2008 15:09 ]
Post subject: Re: Google Cannot Access The Site Correctly Any More

Well, I've been doing some investigations on my own, and found out that the error message "Critical Information" is displayed only to those users that have been banned, so I checked out the "banlist" table and found out that there were 14 IPs in the table.
I suppose that one of those must be the address of Google bot.
I removed all the entries in the banlist table, and now I am waiting for Google to respider the site.
But now my question for you is this.
Since I am the only administrator, and I never added any IPs in the banlist, I wonder whether there is some other way that could lead to IPs to be automatically added to the banlist table, or it could be the result of hacking.
What do you think?

Mighty Gorgon [ Tue 02 Dec, 2008 10:52 ]
Post subject: Re: Google Cannot Access The Site Correctly Any More

I don't know why those IP are in your banlist, can you please list them?

I can say for "sure" that it hasn't been hacked... why an hacker should hack your site by only banning Google BOT?

kesha [ Tue 02 Dec, 2008 13:41 ]
Post subject: Re: Google Cannot Access The Site Correctly Any More

These are the IPs that were in my banlist:

77.41.104.29
74.6.20.40
82.99.30.34
82.99.30.46
82.99.30.68
82.99.30.11
82.99.30.57
38.99.13.125
58.65.239.18
222.237.78.50
61.234.105.61
82.115.76.37
85.179.153.249

This is quite odd. I agree with you that it makes little sense for an hacker to ban only Google BOT, but on the other hand I know for sure that I didn't ban those IPs.
I checked on Whois and none of those IPs resulted to belong clearly to Google, and this is why I am now waiting impatiently that Google respiders the site to see if this was the problem.
But, as I already said, the error that Google gets is "Critical Information", without any other information. This error, if I understand correctly, comes out only to those users or IPs that have been added to the banlist.
Or there might be some other reason?

Mighty Gorgon [ Tue 02 Dec, 2008 17:48 ]
Post subject: Re: Footer Not Displayed Correctly

Those IP addresses could have been banned by CrackerTracker... can you check how it is configured?

Try to monitor the Ban Panel constantly, and let me know if it happens again.

I would discard the "hacking" option!

kesha [ Wed 03 Dec, 2008 11:28 ]
Post subject: Re: Google Cannot Access The Site Correctly Any More

In the settings, IP Blocking was active (I have now deactivated it), and this is the list present in the IP & Agent Filter Section:

*anonym*
*BlowFish*
*BruteForce*
*Bullseye*
*CheeseBot*
*CherryPicker*
*EmailCollector*
*EmailSiphon*
*EmailWolf*
*ExtractorPro*
*Harvest*
*ia_archiver*
*InfoNaviRobot*
*LinkextractorPro*
*LinkScan*
*LinkWalker*
*moget*
*NetMechanic*
*Proxy*
*SiteSnagger*
*TightTwatBot*
*WebBandit*
*WebCopier*
*WebDownloader*
*WebEnhancer*
*WebSauger*
*Website Quester*
*WebStripper*
*WebZip*
*WWW-Collector*
lwp
lwp-*

None of these should interfere with Google Bot, however...

Mighty Gorgon [ Wed 03 Dec, 2008 12:31 ]
Post subject: Re: Google Cannot Access The Site Correctly Any More

I agree.

Let me know if we have definitely solved this.

kesha [ Tue 09 Dec, 2008 17:54 ]
Post subject: Re: Google Cannot Access The Site Correctly Any More

I have just verified that Google is not getting the "Critical Information" error anymore.
We might say that this case is solved except that.... we couldn't figure out an explanation for what happened.
Indeed I succeded in fixing this problem removing the IPs listed above from the banlist, but none of those IPs belongs to Google.
Nice, isn't it?