[squeak-dev] Server timeouts and 504 return codes

Tobias Pape Das.Linux at gmx.de
Mon Jan 28 07:48:45 UTC 2019

> On 28.01.2019, at 01:39, Chris Muller <ma.chris.m at gmail.com> wrote:
> Hi,
>>>>> Yes, the SqueakMap server image is one part of the dynamic, but I
>>>>> think another is a bug in the trunk image.  I think the reason Tim is
>>>>> not seeing 45 seconds before error is because the timeout setting of
>>>>> the high-up client is not being passed all the way down to the
>>>>> lowest-level layers -- e.g., from HTTPSocket --> WebClient -->
>>>>> SocketStream --> Socket.  By the time it gets down to Socket which
>>>>> does the actual work, it's operating on its own 30 second timeout.
>>>> I would expect subsecond reponse times. 30 seconds is just unacceptably
>>>> long.
>>> Well, it depends on if, for example, you're in the middle of
>>> Antarctica with a slow internet connection in an office with a fast
>>> connection.  A 30 second timeout is just the maximum amount of time
>>> the client will wait for the entire process before presenting a
>>> debugger, that's all it can do.
>> We can be sure that Tim should get subsecond response times instead of
>> timeouts after 30 seconds.
> Right, but timeout settings are a necessary tool sometimes, my point
> was that we should fix client code in trunk to make timeouts work
> properly.
> Incidentally, 99% of SqueakMap requests ARE subsecond -- just go to
> map.squeak.org and click around and see.  For the remaining 1% that
> aren't, the issue is known and we're working on a new server to fix
> that.
>>>>> It is a fixed amount of time, I *think* still between 30 and 45
>>>>> seconds, that it takes the SqueakMap server to save its model after an
> and so if in the meantime it can simply be made to wait 45s instead of
> 30s, then current SqueakMap will only be that occasional delay at
> worst, instead of the annoying debugger we currently get.
>>>> You would save seconds, not milliseconds by not downloading files again.
>>> IIUC, you're saying we would save one hope in the "download" --
>>> instead of client <--> alan <--> andreas, it would just be client <-->
>>> alan.  Is that right?
>> No. If the client doesn't have the mcz in the package cache but nginx has
>> it in its cache, then we save the transfer of data between alan and
>> andreas.
> Are alan and andreas co-located?

They're VMs on rackspace. The slowest bandwidth Rackspace has is 200 MBit/s, the fastest 2 GBit/s, i forgot which we have.
The network is not the limiting factor here, Squeak is.

>> The file doesn't have to be read from the disk either.
> I assume you mean "read from disk" on alan?  What about after it's
> cached so many mcz's in RAM that its paging out to swap file?  To me,
> wasing precious RAM (of any server) to cache old MCZ file contents
> that no one will ever download (because they become old very quickly)
> feels wasteful.  Dragster cars are wasteful too, but yes, they are
> "faster"... on a dragstrip.  :)  I guess there'd have to be some kind
> of application-specific smart management of the cache...
> Levente, what about the trunk directory listing, can it cache that?
> That is the _#1 thing_ source.squeak.org is accessing and sending back
> over, and over, and over again -- every time that MC progress box that
> says, "Updating [repository name]".
>> If the client does have the mcz, then we save the complete file transfer.
>>> I don't know what the speed between alan <---> andreas is, but I doubt
>>> it's much slower than client <---> alan in most cases, so the savings
>>> would seem to be minimal..?
>> The image wouldn't have to open a file, read its content from the disk and
>> send that through a socket.
> By "the image" I assume you mean the SqueakSource server image.  But
> opening the file takes very little time.  Original web-sites were
> .html files, remember how fast those were?  Plus, filesystems "cache"
> file contents into their own internal caches anyway...
> Yes, it still has to return back through alan but I assume alan does
> not wait for a "full download" received from andreas before its
> already pipeing back to the Squeak client.  If true, then it seems
> like it only amounts to saving one hop, which would hardly be
> noticeable over what we have now.
>> Nginx does that thing magnitudes faster than
>> Squeak.
> The UX would not be magnitudes faster though, right?
>>>>>> That would also let us save bandwidth by not downloading files already
>>>>>> sitting in the client's package cache.
>>>>> How so?  Isn't the package-cache checked before hitting the server at
>>>>> all?  It certainly should be.
>>>> No, it's not. Currently that's not possible, because different files can
>>>> have the same name. And currently we have no way to tell them apart.
>>> No.  No two MCZ's may have the same name, certainly not withiin the
>>> same repository, because MCRepository cannot support that.  So maybe
>> Not at the same time, but it's possible, and it just happened recently
>> with Chronology-ul.21.
>> It is perfectly possible that a client has a version in its package cache
>> with the same name as a different version on the server.
> But we don't want to restrict what's possible in our software design
> because of that.  That situation is already a headache anyway.  Same
> name theoretically can come only from the same person (if we ensure
> unique initials) and so this is avoidable / fixable by resaving one of
> them as a different name...
>>> we need project subdirectories under package-cache to properly
>>> simulate each cached Repository.  I had no idea we were neutering 90%
>>> of the benefits of our package-cache because of this too, and just
>>> sitting here, I can't help wonder whether this is why MCProxy doesn't
>>> work properly either!
>>> The primary purpose of a cache is to *check it first* to speed up
>>> access to something, right?  What you say about package-cache sounds
>> I don't know. It wasn't me who designed it. :)
> I meant ANY "cache".
>  https://en.wikipedia.org/wiki/Cache_(computing)
> For Monticello, package-cache's other use-case is when an
> authentication issue occurs when trying to save to a HTTP repository.
> At that point the Version object with the new ancestry was already
> constructed in memory, so rather than worry about trying to "undo" all
> that, it was simpler and better to save it to a package-cache, persist
> it safely so the client can simply move forward from there (get access
> to the HTTP and copy it or whatever).
> - Chris
>>> really bad we should fix that, not surrender to it.
>> Yes, that should be fixed, but it needs changes on the server side.
>> What I always had in mind was to extend the repository listing with
>> hashes/uuids so that the client could figure out if it needs to download a
>> specific version. But care must be taken not to break the code for
>> non-ss repositories (e.g. simple directory listings).
>> Levente
>>> - Chris

More information about the Squeak-dev mailing list