I got a little further. #updateErrorPipelineStream helped to get the error messages as they came.

p := PipeableOSProcess command: '/usr/local/bin/HandBrakeCLI -e x264 -q 20 -m --start-at duration:1 --stop-at duration:5 -i /Volumes/DISK_01 -o "/path/to/DISK_01.mp4"'. [p isComplete] whileFalse: [ p updateErrorPipelineStream. Transcript show: p errorUpToEnd ].

-- View this message in context: http://forum.world.st/PipeableOSProcess-incremental-output-tp3946435p3946565... Sent from the Squeak - Beginners mailing list archive at Nabble.com.

On Thu, Oct 27, 2011 at 04:58:34PM -0700, Sean P. DeNigris wrote:

How do I get the output and error of a PipeableOSProcess command line-by-line as it comes, like CommandShell does?

In PipeableOSProcess, the output stream is #pipeFromOutput and the error stream is #errorPipelineStream. For example:

proc := PipeableOSProcess command: 'ls -l * NOSUCHFILE'. out := proc pipeFromOutput. err := proc errorPipelineStream.

From any of these streams you can read character by character

with #next, or you can read the entire contents of the stream with #upToEnd or #upToEndOfFile. On a unix system, lines of output in a pipe will usually be separated by line feed characters, so you can read character by character looking for a Character lf.

But actually you do not even need to access those streams directly, because PipeableOSProcess itself behaves like a stream. You can read output directly from a PipeableOSProcess with #next, #next:, #upToEnd, #upToEndOfFile, #errorUpToEnd, and #errorUpToEndOfFile.

To illustrate the idea, one way to get the output of a command line by line as follows:

(PipeableOSProcess command: 'ls -l') upToEndOfFile findTokens: String lf

I found #setNonBlockingOutput for the output, but "pipeFromError setNonBlocking" doesn't seem to do the same for the errors.

In PipeableOSProcess this is all handled automatically, so you can just use the output and error streams as described above.

Dave

On Fri, Oct 28, 2011 at 09:28:25AM -0700, Sean P. DeNigris wrote:

Thanks David. Is there a way to query whether output/errors are available? I saw some mention of an event driven mechanism, but wasn't clear how to use it.

The event-driven stuff is "under the covers". It basically watches for available data, for example when data is available on the stderr stream of the external process it is automatically pushed into the errorPipelineStream.

The easiest way to see if data is available is to read from the stream and see what you get. For example, if you do #upToEnd and get an empty string, then there is no data available. Note that #upToEnd gives you all of the currently available data (there may be more on the way, for example if the external process is something that runs continuously and puts out data once in a while). If you are dealing with an external program that you expect to run once to completion, then you can use #upToEndOfFile to read everything until the process is complete.

The output and error streams are just streams of data containing whatever happens to come out of the external process. If you are interested in error conditions, you can check exit status of the external process after it runs to completion. Note that the external process may not exit until after you have read its output data (it will block on writing to the pipe if it needs to write a lot of data and you have not read any of it yet), so in general you will want to read the output data, then check exit status afterwards (and maybe read the errorOutputStream also).

Dave

On Sat, Nov 26, 2011 at 07:40:42AM -0800, Sean P. DeNigris wrote:

Sean P. DeNigris wrote

...

I got a little further. #updateErrorPipelineStream helped to get the error messages as they came.

Dave, based on your answers, this seems like it should not be necessary, yet without it #errorUpToEnd returned an empty string even though there was output available. Is there an internal flush that's not getting done?

PipeableOSProcess is designed to be used in a pipeline of connected process proxies (or other types of PipeJunction). In that context, The standard output of each PipeJunction is connected to the standard input of the next, and the error output (if any) from each PipeJunction needs to go somewhere. The errorPipelineStream is shared by multiple process proxies in the pipeline, and provides a place to accumulate the error output from any of the process proxies. The updates to this accumulator stream are triggered by change notifications that happen as data becomes available on the error output streams of each process proxy.

HTH, Dave

On Mon, Nov 28, 2011 at 06:03:18AM -0800, Sean P. DeNigris wrote:

David T. Lewis wrote

...

PipeableOSProcess is designed...

I learned a lot from your answer (I always forget I can chain them together), but I'm not clear whether it answered my question :)

Let me rephrase... In my original example: PipeableOSProcess command: '/usr/local/bin/HandBrakeCLI -e x264 -q 20 -m --start-at duration:1 --stop-at duration:5 -i /Volumes/DISK_01 -o "/path/to/DISK_01.mp4"'

As a naive user, I would assume that "p errorUpToEnd" would show me the error output. I found that it did not unless I first sent "p updateErrorPipelineStream". From your answer, it sounds like the error stream should be automatically notified and updated when new data is available, but this doesn't seem to be happening.

Try using #errorUpToEndOfFile rather than #errorUpToEnd. This will wait for all available data, as opposed to just the data that is currently available.

The reason that you are getting an empty string is that you are reading errorUpToEnd before that error output has been completely updated. The data will not be available until the external process has written it to its stderr, flushed the steam (usually by exiting), and your Squeak VM has detected data available, your OSProcess has read the data, and all of those buffered streams have been updated. This all takes place asynchronously with semaphores being signaled and background processes waking up to update the data.

You can also illustrate this simply by inserting a delay between starting the command and reading its error output. If you wait for a few hundred milliseconds, the data will magically appear and the errorUpToEnd will now give you the results you expect.

HTH, Dave