[Vm-dev] Socket's readSemaphore is losing signals with Cog on Linux

[Igor Stasenko]

On 16 August 2011 18:33, Henrik Johansen <henrik.s.johansen at veloxit.no> wrote:
>
>
> On Aug 16, 2011, at 4:00 49PM, Igor Stasenko wrote:
>
> On 16 August 2011 16:30, Henrik Sperre Johansen
> <henrik.s.johansen at veloxit.no> wrote:
>
> On 15.08.2011 23:14, Henrik Sperre Johansen wrote:
>
> http://code.google.com/p/pharo/issues/detail?id=4655
>
> Is it already integrated in image(s)?
>
> As I wrote it a couple of hours ago, I doubt it.
> One half is integrated in 1.4 (raising errors if you try to allocate enough external objects that you'd have to adjust the size), other half in neither 1.3 nor 1.4. (try a GC to free slots before growing beyond current max size).
>
> Could of course be totally unrelated to the lost signals Levente and others
>
> are seeing on Unix, but it'd be interesting to hear if that still happened
>
> with changes in place equivalent to the ones described in issue.
>
> Cheers,
>
> Henry
>
>
> Is there another hard limit? Like  size of VM table for socket/file handles?
>
> Huh?
> File handles don't use external objects at all that I'm aware of.
> Didn't even know the VM had a special table for them.

Seems its not for files/sockets.
I remember i saw there is some plugin, who keeps a separate table with
low-level data structure(s),
while exposing them as a 'handles' (which is simply an index to that
table) to image side.

> Sockets use 3 semaphores registered in the externalobjects table each, not cleaned up until Sockets are either explicity destroyed or finalized.
> Xtreams tests used 3 Sockets per test, with no explicit destruction in tearDown, so with 87 tests the default 512 externalObjects table filled up rather quickly with no finalization happening.
> There is no hard limit in Cog per se, except one really shouldn't be adjusting the maxExternalObjects size after startup, as that CAN lead to lost signals. (thoroughly documented in the code)
> max-max size of the table is 64k in current image format (stored in image header), as indicated in the comment in maxExternalObjects: method.
>
> PS. At least on Windows, independently of the above, if I manually set
>
> maxExternalObjects to > 4095 (Ie its real size is 8192), I inevitably run
>
> into
>
> "CreateThread() failed (8) - Not enough storage is available to process this
>
> command" errors in the output console if I run the
>
> XTSocketReadingWritingTest...
>
> Tried replicating this on OSX, and couldn't do it, so I guess it is Windows-specific?

On windows, a plugin creates a separate OS thread for every new socket.
(not sure about Cog, but in Squeak it is like that)

So, if you open too many sockets, you can quickly hit the hard limit
of max number of OS threads :) Which if i remember correctly is around
1024.

I think tests should be rewritten to release resources as soon as they
are no longer needed.
Because there's actually little what can be done, since it is more an
OS limitation (and partly, a limitation of socket plugin
implementation).

> Cheers,
> Henry
>



-- 
Best regards,
Igor Stasenko AKA sig.