[Seaside] Static sites and spider handling

[Avi Bryant]

On Tue, 26 Aug 2003, Cees de Groot wrote:

> 3. every time the user agent has a value known in the UA database when a
> session needs to be created, the same session id is used. This ensures
> that URL's appear static to the spider;

The problem is that although the session ID will be the same, all of the
links (the continuation IDs) will be changing for every visit.  The spider
will never see the same full URL twice.  I'm really not sure what the best
way of dealing with this is - Seaside's URLs are designed to reflect a
unique page view, not a unique "page" or "resource", and it's going to be
hard to fight that.

> I want to make robots.txt accessible maybe with a simple comanche module
> (or is there a way with Seaside to return data without a redirect to a
> session first?), so that all hits on the file can be handled from the
> application server image.

You could override #enterSessionWithRequest: in your WASession subclass to
look something like this:

enterSessionWithRequest: aRequest
  ^ aRequest url = 'robots.txt'
      ifTrue: [self responseForRobotsFile]
      ifFalse: [super enterSessionWithRequest: aRequest]

responseForRobotsFile
  ^ WAGenericResponse new ...

There would be earlier places you could hook in, but this is probably the
easiest. You're catching the request just before the redirect and session
key registration happens.