spiders and docs (was: [Seaside] REST and Seaside)

[Brian Brown]

On Apr 11, 2005, at 4:48 PM, Avi Bryant wrote:

> On Apr 11, 2005 11:36 PM, Michal <miso.list at auf.net> wrote:
>>
>> Just a me-too here to confirm that this can be a catch when using
>> seaside, at least v2.3. I've recently seen the same two problems:
>>
>> i. spiders hitting seaside every few seconds, apparently due to the
>> dynamically generated unique urls, and bringing the session count to
>> thousands (in my case the spider was jeeves/teoma)
>
> Ok, so clearly this is a problem we need to deal with (I've never seen
> it because all the apps I've deployed have login pages :).  Does
> anyone have any suggestions?  Cees already has code in Janus for
> detecting spiders; what do we do with them once we have them?
>

Heh, we can hardcode a handler for /robots.txt and tell them to go 
away.... or point to some static pages ;)


>> ii. WADocumentHandler never expiring (isActive always returns true)
>> and thus never getting collected, again inflating the count of seaside
>> entities sticking around.
>
> Yeah, that's a bit of a problem, but I'm not really sure what to do
> about it.  You should have a finite number of static documents being
> used anyway, shouldn't you?
>
> Avi
> _______________________________________________
> Seaside mailing list
> Seaside at lists.squeakfoundation.org
> http://lists.squeakfoundation.org/listinfo/seaside