[squeak-dev] Re: [Vm-dev] [OSProcess] forking and file descriptors

Eliot Miranda eliot.miranda at gmail.com
Thu Jan 8 18:01:34 UTC 2015

On Thu, Jan 8, 2015 at 7:56 AM, Henrik Johansen <
henrik.s.johansen at veloxit.no> wrote:

> On 08 Jan 2015, at 11:37 , Max Leske <maxleske at gmail.com> wrote:
> Hi
> We currently use ImageSegment to create snapshots of our object graphs. To
> ensure consistency (and for performance reasons) we create a fork of the
> image and then run the segment creation in the fork. We’ve always had minor
> issues with TCP sockets but they are pretty rare and have never corrupted
> any data (we close the TCP connections in the child).
> Recently however, we created a new application which also makes heavy use
> of a database and now it seems that forking creates a real problem. In
> anticipation of possible problems I opted to destroy all sockets (with
> Socket>>destroy) in the fork, thinking that, since all file descriptors are
> copies of the ones in the parent process, the sockets in the parent process
> should be unaffected [1], [2].
> With that mechanism in place however, we are seeing very weird things,
> such as multiples sockets in the parent (!) having the same file handle
> (which leads to the wrong data being read from the database and, in turn,
> corrupt objects).
> AFAICT, the OSProcess plugin doesn’t offer any way of dealing with such
> problems so I was wondering if anybody has had any experience with these
> kinds of issues and whether there is some kind of best practice.
> I am aware that the most simple option is to close the sockets in the
> parent before forking, but that will mean that we would have to wait for
> all database connections to finish executing, then blocking them to prevent
> new connections to the database. Depending on the time a query takes (which
> may well be a couple of seconds in our case) clients would need to wait for
> quite a long time before their request can be answered (and this scenario
> of course assumes that we only close the database sockets and leave the TCP
> sockets open…).
> So under the condition that I need to fork that image, what is the best
> way to deal with open file descriptors?
> Thanks for your time.
> Max
> [1] http://man7.org/linux/man-pages/man2/fork.2.html
> [2] http://man7.org/linux/man-pages/man2/clone.2.html
> Well...
> If I understand the source correctly (at least on Unix,
> https://github.com/pharo-project/pharo-vm/blob/master/platforms/unix/plugins/SocketPlugin/sqUnixSocket.c
> )
> The socketHandle in a Socket instance is a pointer to a private
> (platform-specific) struct.
> That struct again has a handle to the native socket, which I assume is
> what gets copied when you fork a process?
> Socket >> primDestroySocket frees the memory pointed to by socketHandle.
> So, are you using clone or fork to create a fork of the image?
> If their memory is shared (clone) instead of copied (fork), you might be
> kicking the feet out from under the parent image as well, so to speak...

Hmmm, that reminds me that in Spur we can perhaps move some of these
structs up into the image if we keep them in pinned objects.  I just wanted
to note that for my own memory.

> Henry

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20150108/d9bfe44e/attachment.htm

More information about the Vm-dev mailing list