Hi David,

Hmmm, I'm not sure where the missing output is going.

I do have another suggestion though. I think you can simplify your
initialization by using just the #prepareOutputForInternalReader method.
This will set things up properly, with non-blocking applied only to
the Scheme output reader pipe (the writer to Scheme should remain in
blocking mode).

So the code can be reduced to this:


startScheme

	sourcePath ifNil: [self error: 'You must set sourcePath to the directory where Mosaic''s Scheme source files are stored.'].
	self schemeSourceFileExists ifFalse: [self error: 'Cannot find Scheme source in sourcePath.'].
	schemeProcess ifNil: [
		schemeProcess := PipeableOSProcess
			new: schemeExecutable
			arguments: nil
			environment: nil
			descriptors: nil
			workingDir: nil
			errorPipelineStream: nil].
	schemeProcess prepareOutputForInternalReader.
	"actually start the process now"
	schemeProcess value.
	self sendInitialize.
	self sendSchemeExtensions.
	self sendScheme: '(provide ''squeak-mosaic)'



On Sat, Jan 15, 2022 at 11:40:33AM -0500, David O'Toole wrote:
> Hi David, thanks for your help. I indeed have the following in my About
> Squeak dialog: AioPlugin VMConstruction-Plugins-AioPlugin-eem.22 (i)
> 
> The Delay suggestion did not work. I notice that I can start a new Scheme
> job, and the standard output begins flowing again, without ever producing
> the remaining output from the previous run, even though the run completed
> successfully. So the output is getting lost somewhere.
> 
> Should I be doing [schemeOutput atEnd] whileFalse: [ | output | output :=
> schemeOutput upToEnd ] ? That seemed to prevent output entirely.
> 
> Thanks for the references to those other places I could look. I will
> endeavor to debug this.
> 
> On Sat, Jan 15, 2022 at 10:53 AM David T. Lewis <lewis at mail.msen.com> wrote:
> 
> > Hi David,
> >
> > On Fri, Jan 14, 2022 at 10:03:46PM -0500, David O'Toole wrote:
> > > I forgot to include the output method. Adding "flush" seems to have
> > helped,
> > > but output still stops coming well before it is complete.
> > >
> > > On Fri, Jan 14, 2022 at 9:49 PM David O'Toole <deeteeoh1138 at gmail.com>
> > > wrote:
> > >
> > > > I have a question relating to PipeableOSProcess. I'm trying to read the
> > > > standard output from the process and (optionally) print it to the
> > > > Transcript. Several large chunks consisting of dozens of lines are
> > > > produced, but then output halts always after a certain number of
> > characters
> > > > (stopping in the middle of a line) even though I can verify that the
> > Scheme
> > > > process completed its entire job successfully and should have produced
> > more
> > > > output. Am I setting up / accessing the PipeableOSProcess incorrectly
> > in my
> > > > snippet here? I would greatly appreciate any tips you might be able to
> > give
> > > > on my investigating this further. I am running Squeak 5.3 on Linux.
> > > >
> >
> > A likely source of the problem relates to reading available data from
> > an OS pipe, particularly if a large amount of data is involved. When
> > you send #upToEnd to the reader, you will get all of the data that is
> > currently available in the pipe. Meanwhile, pipe itself may be waiting
> > for someone to read from it before it allows the process that is writing
> > to the pipe (the Scheme interpreter in this case) to complete the write.
> >
> > The end result is that when you read upToEnd from a pipe, you may not
> > receive the entire chunk of data that is being written by Scheme, and
> > you may need to wait for a few milliseconds and do another read to
> > get the rest.
> >
> > For purposes of debugging, try putting a short delay before the #upToEnd
> > (Delay forMilliseconds: 200). If that makes it work, it will confirm
> > my guess above.
> >
> > A couple of related notes for your information:
> >
> > Take a look at the #upToEndOfFile method, as compared to #upToEnd.
> > You will not be able to use it for your Scheme interpreter because
> > it would wait for the Scheme interpreter to close its end of the
> > pipe, which is not what you want for in interactive connection. But
> > it may clarify the issues of reading from a pipe.
> >
> > Also look at class BufferedAsyncFileReadStream, which deals with
> > the issue by populating its buffer based on AIO events (data available
> > on the pipe) to minimize the need for polling loops to get data
> > from a pipe. This is how you are reading the pipe data, and depending
> > on whether you have the AIOPlugin in your VM, it may be falling back
> > on its polling loop instead of being event driven. This should still
> > work fine but it affects timing of the data available from the pipe.
> >
> > Regarding the #flush, I'm not sure I understand what you are doing
> > in the #step method, but when you write to the external Scheme
> > process on pipeToInput you do want to flush it. That ensures that
> > your data gets pushed out to the OS pipe, and makes it available
> > for Scheme to read from its end of the pipe.
> >
> > HTH,
> > Dave
> >
> >
> >

>