[squeak-dev] [Q] How deal with unwanted looking to private stuff?
siguctua at gmail.com
Fri Jan 29 10:17:30 UTC 2010
I think, you should define rules for bots in robots.txt file, and put
there anything which you want robots to index, and what's not.
On 29 January 2010 11:14, Edgar J. De Cleene <edgardec2001 at yahoo.com.ar> wrote:
> This days I working on HVNaughtieWiki , send info for feedback to squeakros
> and to aida list.
> Besides some squeakers looking this work in progress, this morning I have
> the attention of Google
> This is the print string of request.
> HttpRequest (URL=/file/squeak.sts; protocol=HTTP/1.1; header=a
> Dictionary('accept'->'*/*' 'accept-encoding'->'gzip,deflate'
> 'connection'->'Keep-alive' 'from'->'googlebot(at)googlebot.com'
> 'host'->'220.127.116.11:8085' 'user-agent'->'Mozilla/5.0 (compatible;
> Googlebot/2.1; +http://www.google.com/bot.html)' ); getFields=a
> Dictionary(); postFields=a Dictionary())
> All is Comanche + HV2 and some on top.
> How deal with this?
> Refuse the connect ?
> Send a error page?
> Any hints is welcomed
Igor Stasenko AKA sig.
More information about the Squeak-dev