[Squeakfoundation]re: Flow integration

[Andreas.Raab@gmx.de]

Hi Craig,

> > Secondly, I am scared to death about the issue of backward
> > compatibility.
> 
> 	I think the best thing I can say to assuage such fear is that I've
> already done "complete" installations of Flow into Squeak, in which I
> removed *all* the old stream and socket (and file!) support, in three
> days (24 hours' work), without incident.

Unfortunately this doesn't reassure me in the least. Here is why:

> > With the current version of flow you will instantly break any code
> > that - for example - uses "FileStream readOnlyFileNamed: 'foo.bar'"
> > or "Socket newTCP" (and believe me, there are plenty of those in
> > Croquet...
> 
> 	That's only true, possibly, for post-3.2final code; in the current
> release of Flow I've fixed all appropriate class references as of 3.2
> final.

That's exactly the problem. You fixed all references ... you knew about. The
thing that scares the hell out of me is that we're (again) replacing stuff
that has a long-standing and well-defined notion and that the community has
been used for years. There _is_ plenty of code out there that has been written
over the years and which you have not fixed - simply because it's beyound
visibility.

I should say (and I think I wasn't clear in this) that when I said "leave
the current stuff basically as is" then I meant _literally_. Including the
names of the classes and their semantics. The problem is that there's no easy way
of migrating when you start to replace classes by something semantically
different. And that was conceptually no different with modules. Oh, Henrik fixed
everything he knew about in the image but all the stuff that was outside and
being written and no longer compatible was what blew it ("you cannot add new
modules at the top level" anyone?!). Because the existing semantics were
known for many years and people just seem to expect that if you want to change
the status quo you better provide the means for them keeping it.

Note that Henrik was actually describing in very detailed ways what you have
to do to port things to 3.3alpha - but did it help? Nope. Because people
just weren't ready yet to buy into these changes big time and there wasn't a
path for them to make the change step by step. And you cannot force people to
adopt your point of view unless you got absolute control (like Microsoft).
People just "know" that they create files by using #readOnlyFileNamed: or use
this or that pattern and it is _hard_ to learn the new stuff and often very
inconvenient (don't forget that many Squeakers are really in for the fun of it).

And, honestly, I am in a similar situation right now with Croquet. I cannot
imagine to port all this stuff to flow right away. So if you don't provide a
way of "buying in lightly" by showing how I can use my stuff without lots of
modifications then - at this point - I won't buy in. It's as simple as that.
And I would expect that other people who are doing anything serious with
files and sockets may feel similar. I don't want to insult you but flow has not
been widely tested yet. Chances are that there are problems which could be
much worse than the problems we have with anything right now - but these are
mostly well known due to thousands of users and years of testing. So it's a risk
you are requiring me to take here and I don't know if I want to take all of
it at once. Perhaps I'd like a little toying around with it first. But if
that means it's going to break all the file and network related code that I've
ever written and that you knew nothing about it's just unlikely that I'm going
to do it (unless I'd have a really good reason for doing so - and "better
factoring" just does not count).

But the important part about what I am saying is: There is no reason why we
would *have* to break anything knowingly. E.g., if there are only a few
conflicts in names, and I think we're really talking about five class names or so,
most of which - except Socket and FileStream - are rarely (if ever) used,
then why not just keep the two hierarchies just completely disjunct?!

So my idea is basically: Leave the names "FileStream" and "Socket" to the
current classes (there's really no need to use them in flow since file streams
are created by sending Filename>>stream and Sockets by instantiating a
concrete subclass such as IncomingTCPStream) and simply avoid any intersection of
the two hierarchies. In this case you can keep any existing
not-in-the-image-code running and it will spare you all of those endless renamings you had to
do. Then, once the image has been converted to flow, make up a compatibility
package with those classes that are no longer present and allow people to load
this package if needed.

What I am trying to avoid here is a similar situation to what we had
already. A long leap forwards which not everyone may be willing to take without
something that bridges the gap. Like I was saying in the context of the modules
discussion: Part of what is so useful about the work being done right now with
DVS, PackageInfo, SqueakMap is that it leaves people with a choice and is
non-intrusive. I think it's pretty simple to make flow similarly non-intrusive
by just changing a few class names, and I think it's worth it. Because then,
it plays in the same league.

> > I'd be interested in finding out how feasable it is to implement the
> > flow socket primitives in a non-threaded way. On PS/2 for example, or
> > any *nix with broken pthreads. The code looks like it is relying
> > heavily on a threaded implementation.
> 
> 	Yes, it is, and I think that's a very good thing, from simplicity and
> performance standpoints (see http://netjam.org/flow/advocacy.html). As
> I've said before, I don't think it's worth compromising on either of
> these points for the sake of supporting broken host platforms.

Well if flow can't support those platforms at all I think this would in fact
be a very good point in compromising a bit of efficiency (and just for the
records - I am still waiting for benchmarks which actually indicate that flow
makes any noticable difference in speed ;-)

> It seems quite feasible to me, by using Smalltalk
> threads that simulate some of what the host threads do in the current
> Flow implementation (not entirely unlike what happens in the mainstream
> Squeak system currently).

This isn't quite what I meant. The question was more like: Is it possible to
write an implementation of the socket primitives that adheres exactly to the
specification of the primitives which may not use a threaded
implementation?! Typically, it's best to stay compatible at the primitive level so that if
there's a certain behavior expected by the ST code it should be able to rely
on it. At the very least I wouldn't expect people running on HP-UX (which
IIRC, has horribly broken pthreads) to expect that they have to "reprogram flow"
in order to get something to run which works perfectly well on Linux.

> 	However, I do consider supporting the thread-capable platforms (e.g.,
> win32, *BSD, Linux, Macintosh), which are overwhelmingly more widely
> used than the others (e.g., PS/2, Acorn), a much higher priority.

Certainly. Yet, there are lots of platforms that Squeak can run on and some
of them already have and more will have networking capabilities. And I don't
want to arbitrarily limit those - it's bad enough that we are requiring a
flat 32bit memory model. Throwing out all of those that have "non-standard
threading behavior" might be more than you think.

> Flow
> works on those popular platforms right now. And I think the work of
> supporting the thread-impaired platforms now may be rendered moot in the
> near future. I'm not familiar with the details of PS/2, but I know that
> there are, for example, people working on pthreads for the Acorn.

On PS/2 it has been a deliberate decision to implement non-preemptive
threads. That's to give the game designer full control over what code is run when
(this is different for PS2/Linux btw).
Cheers,
  - Andreas

-- 
+++ GMX - Mail, Messaging & more  http://www.gmx.net +++
NEU: Mit GMX ins Internet. Rund um die Uhr für 1 ct/ Min. surfen!