Hi,
On etoys4#2229+OSProcessV4-3-7, the waitForCommand: hangs after executing: ReleaseBuilderSqueakland new prepareReleaseImageForSqueakland.
Doing: Cursor wait showWhile: [ OSProcess waitForCommand: 'sleep 2']
works fine just before this command but if run after this send, the method hangs waiting for runState to turn from #running to #complete even after the child process is done.
I was able to narrow down the difference to ReleaseBuilderForSqueakland>>finalStripping ... >>discardFFI ..
recreateSpecialObjectsArray
Commands issued (in a separate workspace) just before this call work fine. The one issued after this call hangs. Has anyone else bumped into this problem? Any hints on tracing race conditions or side effects?
Thanks in advance .. Subbu
On Monday 15 June 2009 09:20:27 pm K. K. Subramaniam wrote:
Hi,
On etoys4#2229+OSProcessV4-3-7, the waitForCommand: hangs after executing: ReleaseBuilderSqueakland new prepareReleaseImageForSqueakland.
I forgot to add squeak version: Squeak3.10beta of 22 July 2007 [latest update: #7159] on Kubuntu 9.04 with latest updates.
TIA .. Subbu
On Mon, Jun 15, 2009 at 09:20:27PM +0530, K. K. Subramaniam wrote:
Hi,
On etoys4#2229+OSProcessV4-3-7, the waitForCommand: hangs after executing: ReleaseBuilderSqueakland new prepareReleaseImageForSqueakland.
Doing: Cursor wait showWhile: [ OSProcess waitForCommand: 'sleep 2']
works fine just before this command but if run after this send, the method hangs waiting for runState to turn from #running to #complete even after the child process is done.
I was able to narrow down the difference to ReleaseBuilderForSqueakland>>finalStripping ... >>discardFFI ..
recreateSpecialObjectsArray
Commands issued (in a separate workspace) just before this call work fine. The one issued after this call hangs. Has anyone else bumped into this problem? Any hints on tracing race conditions or side effects?
Hi Subbu,
I have not tried this specific Etoys scenario but in general if an external process proxy does not go from #running to #complete, it is usually because that external process is still blocked while trying to write to its standard output or standard error stream, and thus is not yet able to exit.
If this is happening, you may get better results if you load the CommandShell package in addition to OSProcess. Then you can use the PipeableOSProcess and ProxyPipeline classes, which provide more complete support for handling the output streams from external processes (see the class side #command: methods in these two classes).
Sorry I do not know the specifics of what #prepareReleaseImageForSqueakland is doing, so maybe this is not helpful. But feel free to ask me any OSProcess questions off list and I'll try to help if I can.
Dave
On Tuesday 16 June 2009 07:30:26 am David T. Lewis wrote:
I have not tried this specific Etoys scenario but in general if an external process proxy does not go from #running to #complete, it is usually because that external process is still blocked while trying to write to its standard output or standard error stream, and thus is not yet able to exit.
In this case, no i/o was involved. The command (sleep 2) had indeed ended but the runState was not changed. I was able to get the code going by manually changing the state to #completed.
I suspect that the bug is related to specialObjectArray or VM changes but it just happened to affect waitForCommand:
Subbu
On Monday 15 June 2009 09:20:27 pm K. K. Subramaniam wrote:
On etoys4#2229+OSProcessV4-3-7, the waitForCommand: hangs after executing: ReleaseBuilderSqueakland new prepareReleaseImageForSqueakland.
Doing: Cursor wait showWhile: [ OSProcess waitForCommand: 'sleep 2']
works fine just before this command but if run after this send, the method hangs waiting for runState to turn from #running to #complete even after the child process is done.
I was able to narrow down the difference to ReleaseBuilderForSqueakland>>finalStripping ... >>discardFFI ..
recreateSpecialObjectsArray
Filed bug http://tracker.squeakland.org/browse/SQ-262. Attached is a simple script that shows the behavior (even without LPF.st). Run it on a *copy* of a etoys dev image (the script overwrites the image).
The bug is holding up Etoys 4 rollout to a few thousand students in local public schools. Any help in tracking it down is greatly appreciated.
Subbu
On 16.06.2009, at 07:49, K. K. Subramaniam wrote:
On Monday 15 June 2009 09:20:27 pm K. K. Subramaniam wrote:
On etoys4#2229+OSProcessV4-3-7, the waitForCommand: hangs after executing: ReleaseBuilderSqueakland new prepareReleaseImageForSqueakland.
Doing: Cursor wait showWhile: [ OSProcess waitForCommand: 'sleep 2']
works fine just before this command but if run after this send, the method hangs waiting for runState to turn from #running to #complete even after the child process is done.
I was able to narrow down the difference to ReleaseBuilderForSqueakland>>finalStripping ... >>discardFFI ..
recreateSpecialObjectsArray
Filed bug http://tracker.squeakland.org/browse/SQ-262. Attached is a simple script that shows the behavior (even without LPF.st). Run it on a *copy* of a etoys dev image (the script overwrites the image).
The bug is holding up Etoys 4 rollout to a few thousand students in local public schools. Any help in tracking it down is greatly appreciated.
You intend to ship OSProcess to children?!
- Bert -
On Tuesday 16 June 2009 01:49:41 pm Bert Freudenberg wrote:
You intend to ship OSProcess to children?!
Yes, indirectly. It is needed for LatexMorph. The easiest way to produce good looking Kannada/Devanagari/Math text is by using LaTeX typesetter. Last year, I extended Simon's LatexMorph to create an Etoy that 9-year olds can use in their projects.
Subbu
On 16.06.2009, at 10:39, K. K. Subramaniam wrote:
On Tuesday 16 June 2009 01:49:41 pm Bert Freudenberg wrote:
You intend to ship OSProcess to children?!
Yes, indirectly. It is needed for LatexMorph. The easiest way to produce good looking Kannada/Devanagari/Math text is by using LaTeX typesetter. Last year, I extended Simon's LatexMorph to create an Etoy that 9-year olds can use in their projects.
This is either really dangerous or you cannot exchange projects anymore. Which of the two evils did you choose?
- Bert -
On Tuesday 16 June 2009 02:49:39 pm Bert Freudenberg wrote:
This is either really dangerous or you cannot exchange projects anymore. Which of the two evils did you choose?
The latter. But it is not really evil ;-). It serves our purpose admirably. LatexMorph Etoy is a tool that uses OSProcess to invoke latex on the underlying host to typeset LaTeX commands but the resulting graphic is cached in the Etoy. On hosts without LaTeX, the graphic can be seen but not edited.
Of course, a project with this Etoy will not load into an image without LatexMorph. The Etoy itself can be trashed after the graphic is extracted into an ImageMorph if its use outside of our schools is intended.
The code is available in http://www.squeaksource.com/LatexMorph. It needs OSProcess (in the image) and latex and dvipng on the host. See LatexMorph class for examples of usage.
How is OSProcess more dangerous than console shell access?
Subbu
On 16.06.2009, at 12:08, K. K. Subramaniam wrote:
On Tuesday 16 June 2009 02:49:39 pm Bert Freudenberg wrote:
This is either really dangerous or you cannot exchange projects anymore. Which of the two evils did you choose?
The latter. But it is not really evil ;-). It serves our purpose admirably. LatexMorph Etoy is a tool that uses OSProcess to invoke latex on the underlying host to typeset LaTeX commands but the resulting graphic is cached in the Etoy. On hosts without LaTeX, the graphic can be seen but not edited.
Of course, a project with this Etoy will not load into an image without LatexMorph. The Etoy itself can be trashed after the graphic is extracted into an ImageMorph if its use outside of our schools is intended.
The code is available in http://www.squeaksource.com/LatexMorph. It needs OSProcess (in the image) and latex and dvipng on the host. See LatexMorph class for examples of usage.
How is OSProcess more dangerous than console shell access?
It can be invoked from an Etoys script. Someone could make a project that deletes your user directory as soon as you load it on your machine. To prevent this, the file sandbox gets enabled as soon as you load a project authored by someone else.
That's why Dave put in a feature to disable OSProcess if the file sandbox is enabled. But that means you cannot invoke OSProcess functions anymore, hence you cannot edit a LatexMorph someone else sent you even if you have Latex installed.
That's what I meant with having to choose between security and project exchange (even if all users are using the same image).
Or are you running in Sugar on the XO under Rainbow? Then of course it would be fine, because Rainbow provides a sandbox that covers native code, too (and hence we do not enable the Etoys sandbox under Rainbow). Too bad there is so little interest in Rainbow elsewhere :(
- Bert -
On Tuesday 16 June 2009 04:04:32 pm Bert Freudenberg wrote:
How is OSProcess more dangerous than console shell access?
It can be invoked from an Etoys script. Someone could make a project that deletes your user directory as soon as you load it on your machine. To prevent this, the file sandbox gets enabled as soon as you load a project authored by someone else.
I understand. OSProcess+LatexMorph is no different from giving access to a command shell. Before LatexMorph, kids generate Kannada text using external programs like Kile or Kate, take a snapshot, trim it and import it into Etoys. For something as simple as labeling buttons or diagram parts, this was an overkill.
I had to come up with something simpler that would work in these remote schools that doesn't require any IT support. Our notebook computers are standalone and have very little state. All kids share a single autologin account. Each one carries a personal flash chip containing VMs, image, projects. They pick any available computer for working on their projects. The machine itself is unimportant but personal data is precious and gets carried across academic years.
Subbu
On Tuesday 16 June 2009 04:04:32 pm Bert Freudenberg wrote:
hence you cannot edit a LatexMorph someone else sent you even if you have Latex installed.
LatexMorph serves only as a tool to convert textual LaTeX code into a graphic. One can always ship this code as TextMorph across projects. I can do: myLatexMorph contents: importedLatexMorph contents. myLatexMorph openInHand.
Once Etoys gets a good typesetter plugin, OSProcess may no longer be required. Any takers?
Or are you running in Sugar on the XO under Rainbow?
No. Etoys is run off a USB flash chip on off-the-shelf machines with Linux or Windows (machines borrowed from other local community projects).
Subbu
On 16.06.2009, at 16:11, K. K. Subramaniam wrote:
On Tuesday 16 June 2009 04:04:32 pm Bert Freudenberg wrote:
hence you cannot edit a LatexMorph someone else sent you even if you have Latex installed.
LatexMorph serves only as a tool to convert textual LaTeX code into a graphic. One can always ship this code as TextMorph across projects. I can do: myLatexMorph contents: importedLatexMorph contents. myLatexMorph openInHand.
Once Etoys gets a good typesetter plugin, OSProcess may no longer be required. Any takers?
Pango is supposed to be pretty good for regular text paragraphs. Doesn't layout formulas of course.
Or are you running in Sugar on the XO under Rainbow?
No. Etoys is run off a USB flash chip on off-the-shelf machines with Linux or Windows (machines borrowed from other local community projects).
The proper way would have been to make a very simple custom plugin that has just a single primitive invoking a shell script doing the LaTeX processing. That way it could not be misused for anything else - OSProcess is just too mighty a tool for this specific purpose. And it would be easily customizable, too, by editing the shell script.
- Bert -
On Tuesday 16 June 2009 08:11:34 pm Bert Freudenberg wrote:
Once Etoys gets a good typesetter plugin, OSProcess may no longer be required. Any takers?
Pango is supposed to be pretty good for regular text paragraphs. Doesn't layout formulas of course.
Or Kannada text when we started two years back. In fact, the quality of Kannada class test papers typeset by LaTeX was one of the motivators using Squeak in our project. LaTeX was tempting but the learning curve was steep for teachers and kids who barely knew English. With Squeak, they could play with LaTeX easily in a virtual computer.
The proper way would have been to make a very simple custom plugin that has just a single primitive invoking a shell script doing the LaTeX processing.
True. In fact, LatexMorph uses a Latex class that was to become a plugin. OSProcess allowed me to prototype many ideas rapidly in Squeak itself. It turned to be fast enough to typeset in near real-time (2+ per second), so I never got around to writing that plugin :-).
Subbu
On Tue, Jun 16, 2009 at 11:19:00AM +0530, K. K. Subramaniam wrote:
On Monday 15 June 2009 09:20:27 pm K. K. Subramaniam wrote:
On etoys4#2229+OSProcessV4-3-7, the waitForCommand: hangs after executing: ReleaseBuilderSqueakland new prepareReleaseImageForSqueakland.
Doing: Cursor wait showWhile: [ OSProcess waitForCommand: 'sleep 2']
works fine just before this command but if run after this send, the method hangs waiting for runState to turn from #running to #complete even after the child process is done.
I was able to narrow down the difference to ReleaseBuilderForSqueakland>>finalStripping ... >>discardFFI ..
recreateSpecialObjectsArray
Filed bug http://tracker.squeakland.org/browse/SQ-262. Attached is a simple script that shows the behavior (even without LPF.st). Run it on a *copy* of a etoys dev image (the script overwrites the image).
OSProcess relies on a semaphore to notify it when a child process exits. If you recreate the special objects array, it may be causing OSProcess to lose this connection, hence external processes would appear to never reach the #complete state.
Try doing this immediately after recreating the special objects array:
OSProcess accessor initialize
This will restart the process that waits on the semaphore, which will hopefully now be waiting on the correct semaphore.
Dave
On Tuesday 16 June 2009 04:52:29 pm David T. Lewis wrote:
OSProcess relies on a semaphore to notify it when a child process exits. If you recreate the special objects array, it may be causing OSProcess to lose this connection, hence external processes would appear to never reach the #complete state.
Try doing this immediately after recreating the special objects array:
OSProcess accessor initialize
This will restart the process that waits on the semaphore, which will hopefully now be waiting on the correct semaphore.
I tried initialization in recreateSpecialObjects method just after the array got reset. The child semaphore became nil. I also tried it just after the Rebuild send. Ditto. log is attached.
BTW, should'nt recreate raise an event to other classes to get them to reinitialize their semaphores? or can OSProcess do a timed poll loop instead of a semaphore? Can we discuss this on #squeak?
Subbu
On Tue, Jun 16, 2009 at 06:08:58PM +0530, K. K. Subramaniam wrote:
On Tuesday 16 June 2009 04:52:29 pm David T. Lewis wrote:
OSProcess relies on a semaphore to notify it when a child process exits. If you recreate the special objects array, it may be causing OSProcess to lose this connection, hence external processes would appear to never reach the #complete state.
Try doing this immediately after recreating the special objects array:
OSProcess accessor initialize
This will restart the process that waits on the semaphore, which will hopefully now be waiting on the correct semaphore.
I tried initialization in recreateSpecialObjects method just after the array got reset. The child semaphore became nil. I also tried it just after the Rebuild send. Ditto. log is attached.
Hi, I'm back. Try this instead:
Smalltalk recreateSpecialObjectsArray. OSProcess accessor primForwardSignal: OSProcess accessor primSigChldNumber toSemaphore: nil. OSProcess accessor initialize.
Explanation: The signal forwarding primitive in OSProcess requires the signal handler be unregistered, then re-registered, if you change the special objects array. There is an explanation in the comment of #primitiveForwardSignalToSemaphore.
BTW, should'nt recreate raise an event to other classes to get them to reinitialize their semaphores?
Well maybe, but really I think this is too much complication for an infrequent situation. It is very unusual to to call #recreateSpecialObjects, and normally OSProcess would recover from it anyway just by restarting the image.
or can OSProcess do a timed poll loop instead of a semaphore?
Actually, it does do exactly this when the primitives are not available.
Can we discuss this on #squeak?
Sorry, I was away doing other things. But I think the code above will take care of the problem. Please let me know if it works.
Dave
On Wednesday 17 June 2009 05:47:03 am David T. Lewis wrote:
Smalltalk recreateSpecialObjectsArray. OSProcess accessor primForwardSignal: OSProcess accessor primSigChldNumber toSemaphore: nil. OSProcess accessor initialize.
I swapped the first and second sends and could build my image successfully. Thanks a ton for your responses.
Subbu
etoys-dev@lists.squeakfoundation.org