[Seaside] Static sites and spider handling

[Avi Bryant]

On Tue, 26 Aug 2003, Colin Putney wrote:

> One thing I learned while working at Whistler.com is that search engine
> robots are incredibly paranoid and cynical. It would seem that the
> primary design goal of a search engine is to detect and defeat
> spamdexing in order to present an accurate view of the web to its
> users. As a result, the bots absolutely *hate* two-faced web sites, and
> will penalize them in the rankings accordingly.

I'm curious - how do they *know*?  Do they occasionally use fake UA
strings to keep you honest?