Squeakers,

Stephan Rudlof wrote:

> Here I'd like to hear something from Ian...

Here goes...

Dan Ingalls wrote:

> As long as he is willing, I would like to continue with Ian as the
> official purveyor of unix VMs.

He hasn't been idle (and is still more than willing), just very quiet
(and very "otherwise occupied" ;-).

> Recently there has been much of interest and activity surrounding
> the installation of unix VM sources on SourceForge.  [...]  It also
> has the potentially disastrous property that, if not undertaken in
> partnership with the official purveyor, it can lead to utter
> confusion over what VM has what features, bugs, etc.

I don't know about anybody else, but I was confused.

> it's in the spirit of open source development that anyone can do
> anything, and it's OK.

Fine by me, too. ;-)  (But see implied critique below.)

Andrew C. Greenberg wrote:

> > the installation of unix VM sources on SourceForge.  [...]  It also
> > has the potentially disastrous property that, if not undertaken in
> > partnership with the official purveyor, it can lead to utter
> > confusion over what VM has what features, bugs, etc.
> 
> While I am not sure Dan is entirely right, I will note that while I
> had no problem building one of Ian's VMs on a FreeBSD system, I have
> yet to get a successful build of the SourceForge version.

Took me about 10 hours to make a working and consistent VM from the
SourceForge tree (in the end I simply gave up and pulled the patches
into my sources [I say "patches" but I mean I spent hours staring at
an "ediff" buffer ;-]).  Then another 5 to fix the directory structure
and make configure work in it.  Then another 15 or so to fix various
portability problems and several bits of code that were just plain
wrong.  Then another (ok, I'll stop there, but: I haven't done much
else [including sleeping] since last wednesday).

Tim Rowledge wrote:

> The only argument I can remember hearing against making the VM totally
> bare and all the plugins external is one of convenience for people
> fetching/installing the VM (and indirectly for the folks supporting
> them).

This is the least of the worries, I assure you.

> For some platforms that is a cogent argument (Andreas shudders at
> the though of Windows users being expected to fetch several files
> and install them for example)

That's just a matter of education.  The nicest thing you can do for
someone who doesn't understand is to teach them.

> whereas for others (Acorn for example) there is no reason not to
> make everything external since the packaging mechanism just handles
> everything for you.

Well, this kind of ignores an important source of headaches.  Squeak
is trying to use .so files in cruel and unusual ways that many Unix
designers never dreamed of (even in their worst nightmares).  On some
platforms (including at least one of the supported ones, I might add)
it is *impossible* for the loader to figure out the dynamic
dependencies for the VM unless the plugins are (i) installed in a
standard place that ld.so knows about; (ii) prefixed with "lib", and
(iii) suffixed by ".so.<maj>.<min>".  This makes the build kind of
tricky, since a partial install is absolutely required at "half-time".

A significant portion of the <insert huge number> of hours spent on
this since last week were devoted to finding a way to make plugins
work as uniformly as possible across the various different platform
constraints.  (I wasn't really very successful [yet], although they do
seem to work on the six Unix platforms that I've tried so far.)

(Besides all that, I've quite a few comments about the way the
pluginised VM stuff works in general, but I can't be bothered right
now. ;-)

Somebody else [sorry -- my email archive, when I can wrest control of
it from procmail, is as disorganised as my apartment] wrote:

> I support the use of SourceForge and cvs for the C sources.

Which make no sense whatsoever without the generated sources (and
vice-versa).  The current CVS setup is quite simply unmanageable in
the light of this.

> - Collecting many ports at the same place:

Try inserting the Windows sources into the current CVS tree, and
you'll see what I'm getting at.

>   - will improve possibilities of unification. That is needed. Handling line
>     endings, keyboards, mouse buttons, command line arguments, etc. This kind
>     of thing can be made uniform and more of it can be platform independent.

I don't see any logical reason as to why this should be so.

>     The present organization make such things awkward to handle.

Agreed.  But the current CVS organisation makes things worse.

>     Having a stupid thing such as the cr/lf/crlf-problem still
>     around after several years proves this.

I'm not sure that this still the case.  When I generate files from
2.{8,9a}, the line endings are perfect.

>   - Having all (or many) ports available at the same site unifies and
>     simplifies downloading.

Agreed.

>   - helps porters: look around for a while, decide what to use, make the
>     port

Agreed, although anyone brave enough to contemplate porting Squeak
would already have a pretty good idea which other platforms are fairly
close.

> There are alternatives to cvs

"rsync" springs effortlessly to my mind.  It works, its fast, and it
doesn't force anyone anywhere to learn any arbitrary SCCS commands
whatsoever.

Bert Freudenberg wrote:

[Hi Bert -- and wow, that plugin stuff is neat.  But we need to talk
about security.  Positive vetting of pathnames etc. isn't sufficient.]

> The CVS did a good job at coordinating the individual changes. No longer
> asking where to get the latest sources. But now it seems an ordering hand
> is needed, someone telling what direction to go, someone blessing or
> quashing stuff.

Agreed, but I think this might be difficult to organise with five
people (at the last count) having global write permission in the tree.

It's not just the technical implications that worry me, but the
aesthetic ones too.  I've tried hard to keep the sources crystal clear
and easy (even enjoyable, for people who like that kind of thing ;-)
to read...

> Just look at the mess at SourceForge ;^)

Quite. ;-)

If you have a rational solution to this "uncontrolled proliferation of
good intent" them I'm all ears.

> I did some little cleanup to make Squeak (more exactly, the recent
> additions like serial support) compile on Solaris and Irix. It's
> gmake and gcc for now, not even tried with cc yet, but it works :)

I've pulled much all this stuff into my sources (which are still ANSI
C and use only the intersection of GNU, BSD and SysV make ;-p).
Thanks! to everyone who's contributing stuff.

(Bert: if you know which strange flags I have to pass to the Irix
compiler/linker for the various different cases in the new Makefile.in
then then I'll buy your beer for an evening if you're at OOPSLA this
year.)

And now, let's get serious...

I'm sorry for the lack of 2.8 (or 2.9a) -- I was waiting for the
"official" release (as usual) before acting.  But since there seems to
be significant interest in this, in the directory:

  ftp://ftp.inria.fr/INRIA/Projects/SOR/users/piumarta/squeak/

you will find the following:

  Squeak2.8-src.tar.bz2
    - my current sources (even less guaranteed than usual)
	
  Squeak2.8-ppc-unknown-linux-gnu.tar.bz2
    - monolithic VM (squeak) and modules (Squeak3D.so SoundCodecPrims.so)
      for GNU/Linux on PowerPC

  Squeak2.8p-ppc-unknown-linux-gnu.tar.bz2
    - "pluginised" VM (psqueak) and plugins (AsynchFilePlugin.so
      B2DPlugin.so BitBltPlugin.so FFTPlugin.so FilePlugin.so
      FloatArrayPlugin.so JoystickTabletPlugin.so MIDIPlugin.so
      Matrix2x3Plugin.so SerialPlugin.so SocketPlugin.so
      SoundPlugin.so SurfacePlugin.so ZipPlugin.so) for GNU/Linux on
      PowerPC.

plus similar archives for:

  sparc-unknown-linux-gnu
  sparc-sun-solaris2.5.1  (yes, I know, I should upgrade to 2.7.  Next week...)
  alpha-dec-osf4.0e
  i686-pc-linux-gnu

So what's new in the above?

 - A new directory structure that makes a strict bi-axial split in the
   sources:

	- platform specific vs. platform independent, and
	- hand written vs. automatically generated

   The intent is that generated and hand-written code never occupy the
   same directory, that generated code all live in one place (makes
   maintainers' lives *sooo* much easier), that code specific to
   a given platform be in an isolated subtree, and that adding code for
   other platforms be merely a question of making one dir and copying
   the sources there.

 - Simplified (really, I promise, even if it looks way more complex)
   configure and Makefiles.  Configure now figures out _all_ the
   dependencies between generated and hand-written code for itself,
   and sets up the Makefile to (for example) group files belonging to
   the same plugin into the same .so file.  The pluginised VM does
   not, for example, contain any of the Unix file, sound, serial,
   network, etc. primitives, nor the generated sqFilePrims or
   sqSoundPrims, and so on, and so forth...  (Configure figures this
   out implicitly from the file names, and *not* explicit from the
   directory structure.)

   Taken together, the above two features mean that generating a new
   VM (even in the presence of new and previously unheard-of modules
   and/or plugins) involves (at most):

   	squeak MyLatest.image
		Interpreter translate: 'interp.c' doInlining: true.
		InterpreterSupportCode writeSourceFiles."
	rm src/generated/*
	mv *.[ch] src/generated
	make clean
	make

   One minor plea: can we rename LargeIntegers.c to LargeIntegerPlugin.c
   (or something similar)?  Please??!

 - Modules and plugins now (finally!) work properly on Digital Unix and
   SunOS.  (Although Digital Unix still gives a SIGFPE in the 3D stuff,
   and it's a mystery why -- especially since the plugins work like a
   dream.  If gdb worked properly on DU with ".so"s I might have a
   fighting chance...)

 - Network and serial improvements, thanks to Lex and Ned.  (Ned: the
   serial stuff now compiles perfectly on *lots* of Unices.)

 - Support for Netscape plugin, thanks to Bert.  However, I'm not yet
   distributing the npsqueak binary.  (I'll integrate it properly into
   the rest of the artefact before for the final 2.8, I promise.)

 - Lots of little bugs and irritations fixed (e.g. amongst others, if
   there are *still* SoundPlayerProcess-related problems on
   sound-disabled systems then it's _serious_: tell me about it!).

 - Updated BUILD instructions.  (You probably DON'T want to risk typing
   "make install" for the moment: some last-minute shared object
   headaches have probably broken this on some platforms.)

 - For Andreas's amusement: the Unix VM now understands how to look for
   and read an (optionally gzipped) internal image file.  (He asked me
   last week: "Do you think it would be possible to make a Unix VM
   that ...", and I took it as a challenge. ;-)  In the above directory
   are also (and FOR ENTERTAINMENT PURPOSES ONLY):

	Squeak2.8i-{ppc,sparc,i686}-unknown-linux-gnu.tar.bz2
	  - monolothic VM (isqueak) that contains a built-in copy of the
            2.8 image file.

	Squeak2.8z-{ppc,sparc,i686}-unknown-linux-gnu.tar.bz2
	  - monolothic VM (isqueak) that contains a built-in copy of the
            2.8 image file, gzipped -9 (about 60% of the image size smaller
	    than isqueak).  Inflation is pretty fast too.  (I could easily
	    support other compressed formats that use the deflate method
	    too, just ask and it shall be done.)

   Andreas: look for targets "isqueak" and "isqueaz" in the configured
   Makefile, and the sections conditional on USE_INTERNAL_IMAGE in
   sqXWin.c, to see how this is done.  (It would be neat to reuse the
   ZipPlugin for inflating instead of zlib + zipio, but the interface 
   looks too "Squeaky" at first glance.)

 - One more note: the crux of the above kaboodle was generated out of a
   2.8-2348 image.  I'll grab 2.8gamma and fix any obvious problems
   within the next couple of days.

2.9a is also kind of ready, but before putting it up for FTP I'm
waiting until the new three-primitive Socket and GUI event semantics
are finalised so that preliminary versions of the relevant code can go
into the relevant places.  When Andreas gives the green light I'll be
onto it.  (Besides, getting 2.8 ready for prime time has to take
priority -- and there's a way to go yet.)

I've not tested much of the obscure stuff in all this.  OSS Sound
still appears to work on GNU/Linux-ix86 (I've tried to figure out why
it's broken on PPC to no avail -- none of the ioctl()s appear to
misbehave and all the right stuff is configured in my kernel... ho
hum...), and a half-hearted test of Sockets reveals that peers are
responding to all the starfleet regulation hailing frequencies.  I
haven't gone over some of the new stuff with a fine-toothed comb yet
(and there _are_ problems that I am aware of, for example in the new
socket code someone has modified something someplace that puts the
fds[] array out of sync with a private socket struct causing "bad
match" errors -- I'll fix this while adding the three-prim socket
stuff for 2.9a, then retrofit the working code to 2.8).  Par
conséquence I don't really want to vouch for the quality of this
"2.8 hackers release" for the moment.  Don't upgrade your granny's
pacemaker with it.

(For modules and plugins: AFAIAC if the regular squeak can make bunny
hop around the PWM7 window then modules are working perfectly, and if
psqueak doesn't dump little core crottes all over the place before it
even tries to start up then the plugins are ok. ;-)

As usual if you spot any suspicious behaviour (in Unix Squeak, that
is) then let me know; there are bound to be problems in the mass of
(sometimes rather desperate) hacking that I've done in the last week.
Email to me direct is the best way -- no need to swamp the list with
bug reports for the moment.  (I think my .procmailrc knows since this
afternoon how to separate out VM-related Squeak traffic [anything
including one or more of UNIX, Unix, unix, VM, Vm, vm, LINUX, Linux,
or linux should now reach me pretty fast], and should no longer be
"silently reading on my behalf" all the list messages that were "cc"ed
to me [I should have RTFM a little more carefully a long time ago],
but one never knows...  Maybe I'll never receive any more mail from
any source whatsoever.  Hmm... paradise! ;-)

One final thing, just in case anyone is interested (and/or still
reading), for some reason I felt like putting the following tarball in
the same FTP directory: j3-2.6.0-powerpc-linux-gnu.tar.bz2 (Marcus:
are you out there somewhere??? ;-)  Please don't redistribute it!!!
It's there for "educational purposes" only.

That's it.  Time for bed.  Over and out.  (And above all, enjoy!)

Ian