[Vm-dev] Fwd: Re: [Pharo-dev] OSProcess in 3.0

[phil at highoctane.be]

How to include aioplugin in pharovm?

See messages below for the reason why.

Phil
---------- Message transféré ----------
De : "David T. Lewis" <lewis at mail.msen.com>
Date : 16 oct. 2014 01:30
Objet : Re: [Pharo-dev] OSProcess in 3.0
À : "Pharo Development List" <pharo-dev at lists.pharo.org>
Cc :

On Tue, Oct 14, 2014 at 09:35:44PM +0200, Thierry Goubier wrote:
>
>
> Le 14/10/2014 14:15, David T. Lewis a ?crit :
> >On Tue, Oct 14, 2014 at 07:19:10AM +0200, Thierry Goubier wrote:
> >>Hi Dave,
> >>
> >>took me a while to do some testing. It seems there is an issue, but I
> >>couldn't find out where. First, the data:
> >>
> >>Loading a package from a local gitfiletree repository (get from the
> >>command a zipped git archive containing the package) :
> >>
> >>4.5.11, 4.6.8: 2132 msec.
> >>4.5.12, 4.6.9: 50847 msec.
> >>
> >>I tried to rewrite the main command, remove the closePipes, etc... No
> >>change in the runtime, so your right.
> >>
> >>Profiling doesn't help much: execution time is mostly idle time (but
> >>with a significant increase on PipeableOSProcess commands: goes from
> >>invisible to 2700 ms. A user break in the middle shows it delay waiting
> >>on output of the external process).
> >>
> >>Only hint I could see is that it seems to be linked with the size of the
> >>output of the command.
> >
> >Thierry,
> >
> >Aha! I am quite sure that you have located the problem. There must be
> >something preventing the OS process from running to completion with
> >a larger amount of output data. It definitely sounds like a bug in my
> >latest update to Pipeable OSProcess.
>
> The strange thing is that it runs to completion (i.e. I got no errors;
> if any data was missing or corrupted on output, my load would fail); but
> a lot slower than it should be.
>
> >Thank you very much for finding this. I will get it fixed as quickly
> >as I can.
>
> Thanks. I'm unable to help you much, I'm afraid :(

Thierry,

I do not yet have a solution to this, but I have been able to confirm
that the performance difference that you see is primarily associated with
the AioPlugin not being present in your VM.

When I changed PipeableOSProcess to handle pipe handle closing properly,
I used a buffered output stream that is driven by aio events. When data
is available to be read from the external OS process, an aio notification
pulls the available data into the buffered stream, and when no more data
is available, the pipe (file) handles are closed.

If the plugin is not present in the VM, I provided a polling mechanism
to read data from the external OS process. Apparently this is very
inefficient, and I am sure it could be greatly improved. That is the
cause of the extremely slow performance that you are seeing.

So in summary: The enhancement to PipeableOSProcess relies on an aio
event driven reader, and if the AIO plugin is missing, the fallback code
relies on polling, and is horribly inefficient.

More to follow, and thank you again for identifying the problem.

If you can convice your VM vendor to include AioPlugin in your VM, that
would be helpful also ;-)

Dave
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20141016/b471ff49/attachment.htm