[Vm-dev] Help/advice tracking down a squeak-vm regression

Thu May 10 02:43:21 UTC 2012

Well, thanks to our trusty Mantis bug tracker, it looks like we have
a record of the Scratch related issues. The VM compatibility issues
are documented here:

  Mantis 0007454: Removal of obsolete prim support for closures is a problem for Scratch images
  http://bugs.squeak.org/view.php?id=7454

And the Scratch plugins and licensing are here:

  Mantis 0007654: Add Scratch plugins CameraPlugin and ScratchPlugin to standard VMs
  http://bugs.squeak.org/view.php?id=7654

It seems that the VM compatibility issues were introduced quite intentionally
as part of the move to support proper block closures in Squeak. As of 2011,
the thought was that the proper way forward was to do the corresponding
updates in the Scratch image to incorporate full block closures, which
would be an improvement for Scratch and would also remove the VM compatibility
concerns.

This would still be the right thing to do, although I gather from
the Scratch web site that further development of the Scratch image
may no longer be a priority.

Amos, can you please clarify if further development of the Scratch
image is planned? If yes, then updating the Scratch image may be
the right thing to do. If no, then we can take a harder look at
coming up with a way to produce a backward compatible VM. If neither
of these is feasible, then it will probably be necessary to arrange
a separate source distribution for a Scratch VM.

Thanks,
Dave

On Wed, May 09, 2012 at 09:09:26PM -0400, David T. Lewis wrote:
>  
> Well, I for one would not put "John Maloney on the team" in the same
> sentence with "unfortunately" ;-)
> 
> In any case, to bracket the problem a bit I tried running a Scratch
> image on the 3.7-7 Unix VM from http://squeakvm.org/unix/ and it seems
> to work fine. This VM was built from source in March 2005. Source code
> for that VM is at http://squeakvm.org/unix/release/Squeak-3.7-7.src.tar.gz
> (which may not include the the Scratch plugins, but is otherwise a
> workable source code base).
> 
> Current Squeak VMs do not support a Scratch image, so the differences are
> due to changes in the VM / image interface that took place since 2005.
> 
> Time permitting I will try next week to identify the major changes that
> that have taken place since 2005 that would affect Scratch support. In
> principle it should be possible to produce a VM that will run Scratch
> along with current Squeak/Pharo/etc although I'm afraid that this will
> be a difficult task in practice, so a separate source distribution for
> a VM to run Scratch may be a more pragmatic approach.
> 
> With respect to patching the image, this is an extremely easy thing
> to do, but a difficult thing to manage. Unless there is active development
> of the Scratch image itself, it is probable better to focus effort on
> providing a VM that runs the Scratch image as currently distributed.
> But that is an idea that you might want to run by John for a second
> opinion if you can.
> 
> Dave
> 
> 
> On Wed, May 09, 2012 at 01:48:19PM -0400, Amos Blanton wrote:
> > Thanks to everyone who is looking into this issue.
> > 
> > Unfortunately, the only current member of the Scratch Team with Squeak
> > experience is John Maloney, and he is tied up working on Scratch 2.0 at the
> > moment.  I'm told it will not be possible to divert his time towards
> > looking deeply into these issues - although he probably can respond to
> > general questions.
> > 
> > So - if bundling the VM seems like the best solution, that would be fine.
> > We do bundle the VM in the Mac / Windows version of Scratch. The latest
> > official Debian package also bundles an older VM (not included in the
> > current GPL'd Scratch source tarball, but still available here:
> > http://www.assembla.com/spaces/scratchonlinux/trac_subversion_tool ).
> > However, we were once told that this may run afoul of packaging guidelines,
> > and that we should try to avoid it if possible. Perhaps those on thread
> > with more knowledge of packaging guidelines for Debian can say more?
> > 
> > In addition - if someone would like to look into making changes to the
> > Scratch.image to improve compatibility with later VMs, that would be great.
> > I'm not sure how patching works with Squeak images, and if it would be
> > possible for packagers to patch the current source. But if that's
> > difficult, we could update the source package based on recommendations /
> > changes from the OSS / Squeak community.
> > 
> > 
> > On Tue, May 8, 2012 at 10:52 AM, David T. Lewis <lewis at mail.msen.com> wrote:
> > 
> > > On Tue, May 08, 2012 at 02:39:45PM +0200, Bert Freudenberg wrote:
> > > >
> > > >
> > > > On 08.05.2012, at 14:21, David T. Lewis wrote:
> > > >
> > > > >
> > > > > On Tue, May 08, 2012 at 10:44:26AM +0100, Alex Bradbury wrote:
> > > > >>
> > > > >> David and Igor: thanks for your responses. Comments below.
> > > > >>
> > > > >> On 7 May 2012 00:28, David T. Lewis <lewis at mail.msen.com> wrote:
> > > > >>> If I remember correctly, Scratch derives from a Squeak 3.6 image and
> > > uses
> > > > >>> a VM of similar vintage. Since the days of Squeak 3.6, there have
> > > been
> > > > >>> a number of fundamental changes in the Squeak VM. I recall some
> > > discussion
> > > > >>> of this perhaps a couple of years ago, and I think that the
> > > prevailing
> > > > >>> view was that it made more sense to bring the Scratch image up to
> > > date
> > > > >>> with respect to the VM changes, as opposed to trying to make a single
> > > > >>> VM that could be backward compatible all the way to Squeak 3.6.
> > > > >>
> > > > >> Though the current squeak image did work flawlessly in squeak-vm 4.0?
> > > > >> So the upstream view is that this is not officially supported and the
> > > > >> correct solution if it no longer works on 4.4 is to just use an older
> > > > >> vm?
> > > > >>
> > > > >>> From a license point of view, is there any reason that the Scratch VM
> > > > >>> and source code could not be distributed on Debian? If this is
> > > acceptable,
> > > > >>> then I expect that the fastest way to achieve a working distribution
> > > would
> > > > >>> involve arranging for Scratch to run using a VM at e.g.
> > > /usr/bin/scratch,
> > > > >>> and Squeak to run with /usr/bin/squeak or /usr/bin/cog.
> > > > >>
> > > > >> It does seem that the Ubuntu packages bundles its own
> > > > >> scratch_squeak_vm. Either that, or supporting the installation of
> > > > >> squeak-vm 4.0 alongside 4.4 might be good options. For my purposes
> > > > >> with the Raspberry Pi, I'm probably just going to install and pin the
> > > > >> squeeze squeak-vm (4.0) until a better solution comes along.
> > > > >>
> > > > >> Alex
> > > > >
> > > > > On 8 May 2012 10:44, Alex Bradbury <asb at asbradbury.org> wrote:
> > > > >>> Though the current squeak image did work flawlessly in squeak-vm 4.0?
> > > > >>> So the upstream view is that this is not officially supported and the
> > > > >>> correct solution if it no longer works on 4.4 is to just use an older
> > > > >>> vm?
> > > > >>
> > > > >> squeak image -> scratch image
> > > > >>
> > > > >> Alex
> > > > >
> > > > > As far as I know, Scratch images have not worked on standard Squeak VMs
> > > > > for a number of years, and Scratch has always relied on shipping its
> > > > > own VM. In addition to fundamental changes in the VM itself that have
> > > > > taken place in recent years, I believe that the Scratch version of the
> > > > > VM also includes a couple of plugins (add-in functionality) that are
> > > > > required for Scratch but not included in Squeak.
> > > > >
> > > > > I think I see one source of confusion here. I just read the link at
> > > > > http://info.scratch.mit.edu/Source_Code and find the following
> > > statement:
> > > > >
> > > > > "Since Scratch simply uses a stock Squeak virtual machine, we are
> > > > >  not distributing the Squeak virtual machine source code. Both the
> > > > >  source code and pre-compiled binaries for the Squeak virtual machine
> > > > >  are available at www.squeakvm.org."
> > > > >
> > > > > Unfortunately, I don't think that this is true, and hopefully someone
> > > > > can make a correction to that web page.
> > > > >
> > > > > The squeakvm.org code repository along with VMMaker at
> > > source.squeak.org
> > > > > does contain fully versioned copies of the Squeak VM, presumably
> > > including
> > > > > all or most of the code in the current Scratch VM (except for the
> > > Scratch
> > > > > specific plugins). So it would probably be possible to build a working
> > > > > Scratch VM using code from the standard Squeak VM repositories. But
> > > > > I fear this would take some work on the part of someone who knows
> > > > > Scratch well enough to find and build the right versions and put the
> > > > > pieces together.
> > > > >
> > > > > Dave
> > > >
> > > > Well, actually, Scratch (based on Squeak 2.8 IIRC) worked fine up to the
> > > 3.10 VM. E.g. OLPC's Scratch package does not bundle a VM but uses the
> > > stock Fedora Squeak VM (3.10).
> > > >
> > > > For a very long time we managed to keep VMs being able to run all Squeak
> > > images back to 1.x. The promise was new VM + old image = fine, but new
> > > images may require new VMs. Upgrading the VM should not unnecessarily break
> > > old images.
> > > >
> > > > So what did we change in 4.x VMs that it does not work anymore?
> > >
> > > I am not certain, but we will need to figure it if it is causing problems
> > > now for Scratch and Debian. I probably will not be able to look into
> > > this myself until next week though.
> > >
> > > I know that I can run a Squeak 3.6 image on the most up-to-date
> > > SqueakVM, but going back further in time gets problematic, and it
> > > definitely is not possible to run a 1.x image on current VMs.
> > >
> > > My recollection (not necessarily correct) is that the mechanism for
> > > delivering events from the VM to the image was changed in a fundamental
> > > way. There would probably also be some issues related to the special
> > > objects array and primitive number assignments.
> > >
> > > I think there was some discussion of this on the vm-dev list a
> > > couple of years ago, and some related emails with John Maloney at
> > > the time. I don't have a link right now, but I'll see if I can
> > > find it.
> > >
> > > Dave
> > >
> > > >
> > > > If these changes are really unavoidable and incompatible, then we should
> > > indeed advise distros to package both 3.x and 4.x VMs.
> > > >
> > > > - Bert -
> > > >
> > >
> > >