[Squeakfoundation]re: Flow integration

Fri, 22 Nov 2002 13:37:29 +0300

Let me summarize the points so far -

1. Missing socket options + "security stuff in file access" - Craig
plans to implement within this version, before we start integrating.
2. Categorization imperfect - side effect of 3.3 transitions, will be
corrected.
3. Initialization magie prevents loading Flow twice - not really a
problem, so long as it's only the magic. We'll require other testing to
make sure Flow does everything it's required to.
4. Backwards compatibility - 
 - Flow changes a few names from X to OldX, otherwise everything is
intact.
 - There are many applications refering to the old names, in the image
and outside it.
 - This means that Flow does break everything not explicitly fixed, with
fixes being just name changes (correct me if I got this wrong).
 - An alternative to the current situation is to make flow disjunct from
all the existing code and then let people port as they wish. This
requires some changes to names.
 - The old stuff probably should remain loadable as a package even after
any transition is "over".
5. Proposals to take the primitive interfaces but not the stream
refactoring
 - I can't summarize this because there were no specific
proposal/problems raised (Michael? Goran?)
 - Inserting an idea here - *if* we agree the stream refactoring leads
to a better state than streams are in right now, let's integrate it
independently of the "better network primitives" issues.
6. Threads dependency
 - Flow as is requires OS threads.
 - Two options are to wait for platforms to aquire them or to work
around missing threads in the Smalltalk code.
 - Some platform don't/shouldn't have preemptive threads (PS/2)
 - There are reasons to do the thread emulation inside the primitive and
outside it.

1 + 2 are relevant to whether the code is actually better, and Craig
agrees to fix them.
3 seems to be a non issue, since it's just initialization magic that
prevents reloading (not lacking functionality).
4 is about the transition, which I agree we might have to hone a bit in
order to prevent breakage, and make porting come after acceptance, and
not vice versa. 
OTOH, these are only relevant if we clarify all the "should we do it at
all issues". Which are: 

5 - Michael, could you please clarify you intent here? I'm not sure
whether your comments here are about transition or the desireability of
the refactoring itself.

6 seems to hinge on whether Flow can have reasonable support in the
image for emulating threading on platform that lack it, and on whether
there's a reason that's not good enough. I think we agree that our goal
is Squeak becomes more portable by this change. On the face of it, one
implementation of platform agnostic thread emulation in the image is
better than many different schemes in the platform specific primitives
(our current situation). This is just my impression, and still doesn't
prove that such in image emulation is reasonably doable.

Daniel

Craig Latta <craig.latta@netjam.org> wrote:
> 
> Hi Andreas--
> 	
> > From what I can see looking over the primitives, there are two things
> > missing. One is support for socket options and one support for
> > security stuff in file access. Both are not hard to add but can be
> > painful if missing.
> 
> 	Fair enough; I plan to implement those things for version 2.
> 
> > ...it'd be nice to have a version that doesn't put 82 classes into
> > "reorganize me" - makes it really hard to grok this stuff for
> > someone who has no idea how to reorganize flow ;-)
> 
> 	Of course. This is an artifact of the way class categorization got, er,
> attacked in Squeak 3.3 (in which I was working for quite a while),
> before there was a stable 3.2. I think it was reasonable to leave things
> this way for an alpha release (the current Flow release is 2 alpha). I
> had planned proper categorization for 2 alpha 2. It'll be easier than I
> thought, now that categorization support has emerged unchanged. :)
> 
> > Secondly, I am scared to death about the issue of backward
> > compatibility.
> 
> 	I think the best thing I can say to assuage such fear is that I've
> already done "complete" installations of Flow into Squeak, in which I
> removed *all* the old stream and socket (and file!) support, in three
> days (24 hours' work), without incident.
> 
> 	Nevertheless...
> 
> > I urge you to consider leaving the current streams and sockets
> > basically intact and rather have flow be an _alternative_ to instead
> > of a _replacement_ of the current classes.
> 
> 	Yeah, that's why I currently distribute Flow that way. After installing
> Flow, all Flow features and all old features work concurrently.
> 
> > From what I can see there are only very few places affected where
> > global names collide and if that's the entire problem someone ought
> > to be able to come up with a few alternative names.
> 
> 	Actually, the current Flow release does this. Anywhere there's an
> existing class with a name I want to use, I rename it with "Old"
> prepended (e.g., "Stream" becomes "OldStream"). I also fixed all
> references to those classes (as of 3.2 final).
> 
> 	So...
> 
> > With the current version of flow you will instantly break any code
> > that - for example - uses "FileStream readOnlyFileNamed: 'foo.bar'"
> > or "Socket newTCP" (and believe me, there are plenty of those in
> > Croquet...
> 
> 	That's only true, possibly, for post-3.2final code; in the current
> release of Flow I've fixed all appropriate class references as of 3.2
> final. E.g., (FileStream readOnlyFileNamed: 'foo.bar') became
> (OldFileStream readOnlyFileNamed: 'foo.bar'), and (Socket newTCP) became
> (OldSocket newTCP).
> 
> 	The new internal stream classes support the protocol of the old ones
> completely. The other "name-stealing" ones (e.g., FileStream and Socket)
> do this to a lesser degree. I decided it was easy enough to just fix all
> the class references.
> 
> > ...if you need another instructive example just try to install flow
> > two times ;-)
> 
> 	That's not a fair example. :)  There's some very appropriate but
> one-time magic in the Flow installer. Do you really need this
> capability? It can be done.
> 
> > The entire situation here strikes me of being amazingly similar to
> > the backward compatibility issues with modules - in both cases
> > fundamental notions are being replaced instead of augmented. Let's
> > try hard not to repeat this debacle.
> 
> 	I think this situation is very different, since the new and old stuff
> runs at the same time. We can be as gradual as we like in adapting
> current applications to the new framework. Also note that, as has been
> the case since Flow 1.2 (August 2000) no new VM is required (the new
> primitives are in a plugin). For me, this is all in marked contrast to
> the 3.3 modules experience; for example, class categories and pool
> dictionaries just broke. :)  (This is not an invitation to reopen a
> discussion on modules. ;)
> 
> > I'd be interested in finding out how feasable it is to implement the
> > flow socket primitives in a non-threaded way. On PS/2 for example, or
> > any *nix with broken pthreads. The code looks like it is relying
> > heavily on a threaded implementation.
> 
> 	Yes, it is, and I think that's a very good thing, from simplicity and
> performance standpoints (see http://netjam.org/flow/advocacy.html). As
> I've said before, I don't think it's worth compromising on either of
> these points for the sake of supporting broken host platforms. I think
> it'd be better to support those platforms by augmenting Flow's current
> implementation. It seems quite feasible to me, by using Smalltalk
> threads that simulate some of what the host threads do in the current
> Flow implementation (not entirely unlike what happens in the mainstream
> Squeak system currently).
> 
> 	However, I do consider supporting the thread-capable platforms (e.g.,
> win32, *BSD, Linux, Macintosh), which are overwhelmingly more widely
> used than the others (e.g., PS/2, Acorn), a much higher priority. Flow
> works on those popular platforms right now. And I think the work of
> supporting the thread-impaired platforms now may be rendered moot in the
> near future. I'm not familiar with the details of PS/2, but I know that
> there are, for example, people working on pthreads for the Acorn.
> 
> 
> 	thanks,
> 
> -C
> 
> --
> Craig Latta
> improvisational musical informaticist
> craig@netjam.org
> www.netjam.org/resume
> Smalltalkers do: [:it | All with: Class, (And love: it)]
> _______________________________________________
> Squeakfoundation mailing list
> Squeakfoundation@lists.squeakfoundation.org
> http://lists.squeakfoundation.org/listinfo/squeakfoundation