[squeak-dev] Printing standard output from PipeableOSProcess on the Transcript

David O'Toole deeteeoh1138 at gmail.com
Sat Jan 15 18:07:15 UTC 2022


I tried reading character by character using #next, and this worked okay
albeit producing output slowly in the Transcript---and it still stalls at
the same point in the output.

On Sat, Jan 15, 2022 at 12:48 PM David O'Toole <deeteeoh1138 at gmail.com>
wrote:

> Thank you. This did not change the behavior. I did discover that output is
> always read in exact 1000 byte multiples, first it reads 1000 then in a
> later invocation of #step, it reads 4000 and then output stalls.
>
> On Sat, Jan 15, 2022 at 12:34 PM David T. Lewis <lewis at mail.msen.com>
> wrote:
>
>> Hi David,
>>
>> Hmmm, I'm not sure where the missing output is going.
>>
>> I do have another suggestion though. I think you can simplify your
>> initialization by using just the #prepareOutputForInternalReader method.
>> This will set things up properly, with non-blocking applied only to
>> the Scheme output reader pipe (the writer to Scheme should remain in
>> blocking mode).
>>
>> So the code can be reduced to this:
>>
>>
>> startScheme
>>
>>         sourcePath ifNil: [self error: 'You must set sourcePath to the
>> directory where Mosaic''s Scheme source files are stored.'].
>>         self schemeSourceFileExists ifFalse: [self error: 'Cannot find
>> Scheme source in sourcePath.'].
>>         schemeProcess ifNil: [
>>                 schemeProcess := PipeableOSProcess
>>                         new: schemeExecutable
>>                         arguments: nil
>>                         environment: nil
>>                         descriptors: nil
>>                         workingDir: nil
>>                         errorPipelineStream: nil].
>>         schemeProcess prepareOutputForInternalReader.
>>         "actually start the process now"
>>         schemeProcess value.
>>         self sendInitialize.
>>         self sendSchemeExtensions.
>>         self sendScheme: '(provide ''squeak-mosaic)'
>>
>>
>>
>> On Sat, Jan 15, 2022 at 11:40:33AM -0500, David O'Toole wrote:
>> > Hi David, thanks for your help. I indeed have the following in my About
>> > Squeak dialog: AioPlugin VMConstruction-Plugins-AioPlugin-eem.22 (i)
>> >
>> > The Delay suggestion did not work. I notice that I can start a new
>> Scheme
>> > job, and the standard output begins flowing again, without ever
>> producing
>> > the remaining output from the previous run, even though the run
>> completed
>> > successfully. So the output is getting lost somewhere.
>> >
>> > Should I be doing [schemeOutput atEnd] whileFalse: [ | output | output
>> :=
>> > schemeOutput upToEnd ] ? That seemed to prevent output entirely.
>> >
>> > Thanks for the references to those other places I could look. I will
>> > endeavor to debug this.
>> >
>> > On Sat, Jan 15, 2022 at 10:53 AM David T. Lewis <lewis at mail.msen.com>
>> wrote:
>> >
>> > > Hi David,
>> > >
>> > > On Fri, Jan 14, 2022 at 10:03:46PM -0500, David O'Toole wrote:
>> > > > I forgot to include the output method. Adding "flush" seems to have
>> > > helped,
>> > > > but output still stops coming well before it is complete.
>> > > >
>> > > > On Fri, Jan 14, 2022 at 9:49 PM David O'Toole <
>> deeteeoh1138 at gmail.com>
>> > > > wrote:
>> > > >
>> > > > > I have a question relating to PipeableOSProcess. I'm trying to
>> read the
>> > > > > standard output from the process and (optionally) print it to the
>> > > > > Transcript. Several large chunks consisting of dozens of lines are
>> > > > > produced, but then output halts always after a certain number of
>> > > characters
>> > > > > (stopping in the middle of a line) even though I can verify that
>> the
>> > > Scheme
>> > > > > process completed its entire job successfully and should have
>> produced
>> > > more
>> > > > > output. Am I setting up / accessing the PipeableOSProcess
>> incorrectly
>> > > in my
>> > > > > snippet here? I would greatly appreciate any tips you might be
>> able to
>> > > give
>> > > > > on my investigating this further. I am running Squeak 5.3 on
>> Linux.
>> > > > >
>> > >
>> > > A likely source of the problem relates to reading available data from
>> > > an OS pipe, particularly if a large amount of data is involved. When
>> > > you send #upToEnd to the reader, you will get all of the data that is
>> > > currently available in the pipe. Meanwhile, pipe itself may be waiting
>> > > for someone to read from it before it allows the process that is
>> writing
>> > > to the pipe (the Scheme interpreter in this case) to complete the
>> write.
>> > >
>> > > The end result is that when you read upToEnd from a pipe, you may not
>> > > receive the entire chunk of data that is being written by Scheme, and
>> > > you may need to wait for a few milliseconds and do another read to
>> > > get the rest.
>> > >
>> > > For purposes of debugging, try putting a short delay before the
>> #upToEnd
>> > > (Delay forMilliseconds: 200). If that makes it work, it will confirm
>> > > my guess above.
>> > >
>> > > A couple of related notes for your information:
>> > >
>> > > Take a look at the #upToEndOfFile method, as compared to #upToEnd.
>> > > You will not be able to use it for your Scheme interpreter because
>> > > it would wait for the Scheme interpreter to close its end of the
>> > > pipe, which is not what you want for in interactive connection. But
>> > > it may clarify the issues of reading from a pipe.
>> > >
>> > > Also look at class BufferedAsyncFileReadStream, which deals with
>> > > the issue by populating its buffer based on AIO events (data available
>> > > on the pipe) to minimize the need for polling loops to get data
>> > > from a pipe. This is how you are reading the pipe data, and depending
>> > > on whether you have the AIOPlugin in your VM, it may be falling back
>> > > on its polling loop instead of being event driven. This should still
>> > > work fine but it affects timing of the data available from the pipe.
>> > >
>> > > Regarding the #flush, I'm not sure I understand what you are doing
>> > > in the #step method, but when you write to the external Scheme
>> > > process on pipeToInput you do want to flush it. That ensures that
>> > > your data gets pushed out to the OS pipe, and makes it available
>> > > for Scheme to read from its end of the pipe.
>> > >
>> > > HTH,
>> > > Dave
>> > >
>> > >
>> > >
>>
>> >
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20220115/23217a9f/attachment.html>


More information about the Squeak-dev mailing list