https://www.icyphoenix.com/viewtopic.php?f=35&t=2263
-----------------------------------
borbo
Sun 10 Jun, 2007 05:34

Get Rif Of Spider Bots
-----------------------------------
Hello...

I've got a page in a subfolder outside from IP root and i'd like to keep it private...

How can i drop the bots even googlebot to avoid spider?

I want this page never to be visible in google search!!!




Thankssssss


-----------------------------------
Vortex
Sun 10 Jun, 2007 10:27

Re: Get Rif Of Spider Bots
-----------------------------------
[quote user="borbo" post="16316"]Hello...

I've got a page in a subfolder outside from IP root and i'd like to keep it private...

How can i drop the bots even googlebot to avoid spider?

I want this page never to be visible in google search!!!




Thankssssss[/quote]


This is not an Icy related question... anyway, take a look to this ;)


http://www.robotstxt.org/wc/exclusion-admin.html


-----------------------------------
borbo
Sun 10 Jun, 2007 16:09

Re: Get Rif Of Spider Bots
-----------------------------------
I'm verry sorry for the offtopic...It was 06:34 in greece so you ca understand how destroyed i was ! :shock:


-----------------------------------
borbo
Sun 10 Jun, 2007 16:20

Re: Get Rif Of Spider Bots
-----------------------------------
mmm...nowan icy phoeinix question....

My IP is in a folder ex. /httpdoacs/phpBB2/ 

Do i have to move robots.txt one folder up and edit it changing the paths?


-----------------------------------
Vortex
Sun 10 Jun, 2007 19:51

Re: Get Rif Of Spider Bots
-----------------------------------
[quote user="borbo" post="16343"]mmm...nowan icy phoeinix question....

My IP is in a folder ex. /httpdoacs/phpBB2/ 

Do i have to move robots.txt one folder up and edit it changing the paths?[/quote]



It is explained in that page ;)


I suggest you to put robots.txt in the root and specify in it which folder allow or not to the bot spidering.


