(This is a draft I think we should post to the squeak-dev to generate the main discussion there. If you guys have comments/additional issues to raise...)
As I noted before, a few people have requested that the Flow work by Craig be integrated into the base image.
This work: - Replaces the primitive interface to sockets and other External Resources - Refactors the stream hierarchy
I've had a few concerns that the changes might create too much change all at once, considering that many things use the stream hierarchy. After speaking to Craig about these issue, it seems that (Craig, please correct me if I'm wrong here)- 1. The new streams implement all the old protocols, so that the main compatibility problem should be direct references to the old class names. This is easy to fix incrementally for packages, and fairly quickly for the image itself. 2. The package as it is now can coexist with the existing libraries, making testing quite feasible at low risk.
So I guess this move should be pretty easy. Open questions - 1. Should we do it? 2. How do we test to minimize risk? 3. When?
1. Do or not? If people know of any general objections/problems with this code, it's best to put them in the open now.
I'm not qualified to answer about the benefits of the new primitive interfaces, I'm quite ignorant in these things. John M has (IIUC) spoken in favor of it. Could we get some more input from the experts?
About the stream refactorings, I think several people have used Flow to develop their own applications. Luciano has mentioned that he's used it for several and liked it. More evaluations would be useful here.
2. Testing As to format, Craig has agreed to post these changes on SM in the precise form he'd prefer to issue the proposed updates. As to how to make sure it doesn't cause problems for this or that package or application - that's up to you guys out there that use your sockets for more than reading mail... ideas?
3. When? Probably not in 3.4 (just to keep the release finite and near), but possibly in 3.5. This depends on Craig posting on SM in the proposed format, and mostly on when we get enough testing input.
Daniel
On Thu, 21 Nov 2002, Daniel Vainsencher wrote:
This work:
- Replaces the primitive interface to sockets and other External Resources
I'm not qualified to answer about the benefits of the new primitive interfaces, I'm quite ignorant in these things. John M has (IIUC) spoken in favor of it. Could we get some more input from the experts?
Are these new primitives documented (rigorously ;) somewhere?
Ian
Are [the Flow] primitives documented (rigorously ;) somewhere?
I think the source (at both the Smalltalk and virtual machine levels) is very well commented. Is there some other form you'd like the documentation to take? I'm willing to recast the information (make tables, use hypertext, etc.).
thanks,
-C
-- Craig Latta improvisational musical informaticist craig@netjam.org www.netjam.org/resume Smalltalkers do: [:it | All with: Class, (And love: it)]
Daniel Vainsencher wrote:
This work:
- Replaces the primitive interface to sockets and other External
Resources
- Refactors the stream hierarchy
I would like to propose to actually separate the two. It should be feasible to replace the current external stuff (sockets etc) with Craig's implementation + the network rewrite. I also think we can actually have the new network layer isolate the lower layer enough so that we can switch without most people noticing.
In the Stream refactoring I see two parts: a) streams for the external resources b) refactoring the hierarchy
I definitely want to see a) in one form or another, but I'm not entirely convinced yet that I/we want b) in its current form. Just MHO.
Michael
Forgot one thing:
I'm not qualified to answer about the benefits of the new primitive interfaces, I'm quite ignorant in these things. John M has (IIUC) spoken in favor of it. Could we get some more input from the experts?
I'd be interested in finding out how feasable it is to implement the flow socket primitives in a non-threaded way. On PS/2 for example, or any *nix with broken pthreads. The code looks like it is relying heavily on a threaded implementation.
Cheers, - Andreas
Andreas.Raab@gmx.de is claimed by the authorities to have written:
Forgot one thing:
I'm not qualified to answer about the benefits of the new primitive interfaces, I'm quite ignorant in these things. John M has (IIUC) spoken in favor of it. Could we get some more input from the experts?
I'd be interested in finding out how feasable it is to implement the flow socket primitives in a non-threaded way. On PS/2 for example, or any *nix with broken pthreads. The code looks like it is relying heavily on a threaded implementation.
Me too, even though Craig is convinced we can actually do threads on Acorn. I'm not so sanguine about it....
tim
Hi Andreas--
From what I can see looking over the primitives, there are two things missing. One is support for socket options and one support for security stuff in file access. Both are not hard to add but can be painful if missing.
Fair enough; I plan to implement those things for version 2.
...it'd be nice to have a version that doesn't put 82 classes into "reorganize me" - makes it really hard to grok this stuff for someone who has no idea how to reorganize flow ;-)
Of course. This is an artifact of the way class categorization got, er, attacked in Squeak 3.3 (in which I was working for quite a while), before there was a stable 3.2. I think it was reasonable to leave things this way for an alpha release (the current Flow release is 2 alpha). I had planned proper categorization for 2 alpha 2. It'll be easier than I thought, now that categorization support has emerged unchanged. :)
Secondly, I am scared to death about the issue of backward compatibility.
I think the best thing I can say to assuage such fear is that I've already done "complete" installations of Flow into Squeak, in which I removed *all* the old stream and socket (and file!) support, in three days (24 hours' work), without incident.
Nevertheless...
I urge you to consider leaving the current streams and sockets basically intact and rather have flow be an _alternative_ to instead of a _replacement_ of the current classes.
Yeah, that's why I currently distribute Flow that way. After installing Flow, all Flow features and all old features work concurrently.
From what I can see there are only very few places affected where global names collide and if that's the entire problem someone ought to be able to come up with a few alternative names.
Actually, the current Flow release does this. Anywhere there's an existing class with a name I want to use, I rename it with "Old" prepended (e.g., "Stream" becomes "OldStream"). I also fixed all references to those classes (as of 3.2 final).
So...
With the current version of flow you will instantly break any code that - for example - uses "FileStream readOnlyFileNamed: 'foo.bar'" or "Socket newTCP" (and believe me, there are plenty of those in Croquet...
That's only true, possibly, for post-3.2final code; in the current release of Flow I've fixed all appropriate class references as of 3.2 final. E.g., (FileStream readOnlyFileNamed: 'foo.bar') became (OldFileStream readOnlyFileNamed: 'foo.bar'), and (Socket newTCP) became (OldSocket newTCP).
The new internal stream classes support the protocol of the old ones completely. The other "name-stealing" ones (e.g., FileStream and Socket) do this to a lesser degree. I decided it was easy enough to just fix all the class references.
...if you need another instructive example just try to install flow two times ;-)
That's not a fair example. :) There's some very appropriate but one-time magic in the Flow installer. Do you really need this capability? It can be done.
The entire situation here strikes me of being amazingly similar to the backward compatibility issues with modules - in both cases fundamental notions are being replaced instead of augmented. Let's try hard not to repeat this debacle.
I think this situation is very different, since the new and old stuff runs at the same time. We can be as gradual as we like in adapting current applications to the new framework. Also note that, as has been the case since Flow 1.2 (August 2000) no new VM is required (the new primitives are in a plugin). For me, this is all in marked contrast to the 3.3 modules experience; for example, class categories and pool dictionaries just broke. :) (This is not an invitation to reopen a discussion on modules. ;)
I'd be interested in finding out how feasable it is to implement the flow socket primitives in a non-threaded way. On PS/2 for example, or any *nix with broken pthreads. The code looks like it is relying heavily on a threaded implementation.
Yes, it is, and I think that's a very good thing, from simplicity and performance standpoints (see http://netjam.org/flow/advocacy.html). As I've said before, I don't think it's worth compromising on either of these points for the sake of supporting broken host platforms. I think it'd be better to support those platforms by augmenting Flow's current implementation. It seems quite feasible to me, by using Smalltalk threads that simulate some of what the host threads do in the current Flow implementation (not entirely unlike what happens in the mainstream Squeak system currently).
However, I do consider supporting the thread-capable platforms (e.g., win32, *BSD, Linux, Macintosh), which are overwhelmingly more widely used than the others (e.g., PS/2, Acorn), a much higher priority. Flow works on those popular platforms right now. And I think the work of supporting the thread-impaired platforms now may be rendered moot in the near future. I'm not familiar with the details of PS/2, but I know that there are, for example, people working on pthreads for the Acorn.
thanks,
-C
-- Craig Latta improvisational musical informaticist craig@netjam.org www.netjam.org/resume Smalltalkers do: [:it | All with: Class, (And love: it)]
Hi Craig,
Secondly, I am scared to death about the issue of backward compatibility.
I think the best thing I can say to assuage such fear is that I've already done "complete" installations of Flow into Squeak, in which I removed *all* the old stream and socket (and file!) support, in three days (24 hours' work), without incident.
Unfortunately this doesn't reassure me in the least. Here is why:
With the current version of flow you will instantly break any code that - for example - uses "FileStream readOnlyFileNamed: 'foo.bar'" or "Socket newTCP" (and believe me, there are plenty of those in Croquet...
That's only true, possibly, for post-3.2final code; in the current release of Flow I've fixed all appropriate class references as of 3.2 final.
That's exactly the problem. You fixed all references ... you knew about. The thing that scares the hell out of me is that we're (again) replacing stuff that has a long-standing and well-defined notion and that the community has been used for years. There _is_ plenty of code out there that has been written over the years and which you have not fixed - simply because it's beyound visibility.
I should say (and I think I wasn't clear in this) that when I said "leave the current stuff basically as is" then I meant _literally_. Including the names of the classes and their semantics. The problem is that there's no easy way of migrating when you start to replace classes by something semantically different. And that was conceptually no different with modules. Oh, Henrik fixed everything he knew about in the image but all the stuff that was outside and being written and no longer compatible was what blew it ("you cannot add new modules at the top level" anyone?!). Because the existing semantics were known for many years and people just seem to expect that if you want to change the status quo you better provide the means for them keeping it.
Note that Henrik was actually describing in very detailed ways what you have to do to port things to 3.3alpha - but did it help? Nope. Because people just weren't ready yet to buy into these changes big time and there wasn't a path for them to make the change step by step. And you cannot force people to adopt your point of view unless you got absolute control (like Microsoft). People just "know" that they create files by using #readOnlyFileNamed: or use this or that pattern and it is _hard_ to learn the new stuff and often very inconvenient (don't forget that many Squeakers are really in for the fun of it).
And, honestly, I am in a similar situation right now with Croquet. I cannot imagine to port all this stuff to flow right away. So if you don't provide a way of "buying in lightly" by showing how I can use my stuff without lots of modifications then - at this point - I won't buy in. It's as simple as that. And I would expect that other people who are doing anything serious with files and sockets may feel similar. I don't want to insult you but flow has not been widely tested yet. Chances are that there are problems which could be much worse than the problems we have with anything right now - but these are mostly well known due to thousands of users and years of testing. So it's a risk you are requiring me to take here and I don't know if I want to take all of it at once. Perhaps I'd like a little toying around with it first. But if that means it's going to break all the file and network related code that I've ever written and that you knew nothing about it's just unlikely that I'm going to do it (unless I'd have a really good reason for doing so - and "better factoring" just does not count).
But the important part about what I am saying is: There is no reason why we would *have* to break anything knowingly. E.g., if there are only a few conflicts in names, and I think we're really talking about five class names or so, most of which - except Socket and FileStream - are rarely (if ever) used, then why not just keep the two hierarchies just completely disjunct?!
So my idea is basically: Leave the names "FileStream" and "Socket" to the current classes (there's really no need to use them in flow since file streams are created by sending Filename>>stream and Sockets by instantiating a concrete subclass such as IncomingTCPStream) and simply avoid any intersection of the two hierarchies. In this case you can keep any existing not-in-the-image-code running and it will spare you all of those endless renamings you had to do. Then, once the image has been converted to flow, make up a compatibility package with those classes that are no longer present and allow people to load this package if needed.
What I am trying to avoid here is a similar situation to what we had already. A long leap forwards which not everyone may be willing to take without something that bridges the gap. Like I was saying in the context of the modules discussion: Part of what is so useful about the work being done right now with DVS, PackageInfo, SqueakMap is that it leaves people with a choice and is non-intrusive. I think it's pretty simple to make flow similarly non-intrusive by just changing a few class names, and I think it's worth it. Because then, it plays in the same league.
I'd be interested in finding out how feasable it is to implement the flow socket primitives in a non-threaded way. On PS/2 for example, or any *nix with broken pthreads. The code looks like it is relying heavily on a threaded implementation.
Yes, it is, and I think that's a very good thing, from simplicity and performance standpoints (see http://netjam.org/flow/advocacy.html). As I've said before, I don't think it's worth compromising on either of these points for the sake of supporting broken host platforms.
Well if flow can't support those platforms at all I think this would in fact be a very good point in compromising a bit of efficiency (and just for the records - I am still waiting for benchmarks which actually indicate that flow makes any noticable difference in speed ;-)
It seems quite feasible to me, by using Smalltalk threads that simulate some of what the host threads do in the current Flow implementation (not entirely unlike what happens in the mainstream Squeak system currently).
This isn't quite what I meant. The question was more like: Is it possible to write an implementation of the socket primitives that adheres exactly to the specification of the primitives which may not use a threaded implementation?! Typically, it's best to stay compatible at the primitive level so that if there's a certain behavior expected by the ST code it should be able to rely on it. At the very least I wouldn't expect people running on HP-UX (which IIRC, has horribly broken pthreads) to expect that they have to "reprogram flow" in order to get something to run which works perfectly well on Linux.
However, I do consider supporting the thread-capable platforms (e.g., win32, *BSD, Linux, Macintosh), which are overwhelmingly more widely used than the others (e.g., PS/2, Acorn), a much higher priority.
Certainly. Yet, there are lots of platforms that Squeak can run on and some of them already have and more will have networking capabilities. And I don't want to arbitrarily limit those - it's bad enough that we are requiring a flat 32bit memory model. Throwing out all of those that have "non-standard threading behavior" might be more than you think.
Flow works on those popular platforms right now. And I think the work of supporting the thread-impaired platforms now may be rendered moot in the near future. I'm not familiar with the details of PS/2, but I know that there are, for example, people working on pthreads for the Acorn.
On PS/2 it has been a deliberate decision to implement non-preemptive threads. That's to give the game designer full control over what code is run when (this is different for PS2/Linux btw). Cheers, - Andreas
Hi Andreas--
You fixed all references... you knew about.
That's not the end of the process. :) I think you're missing my intent that those changes be temporary. The full integration process would go like this:
1. Change some of the original class names ("Stream" becomes "OldStream", etc.) 2. Fix references to those classes. Everything keeps working during the rest of the integration process. 3. Install new classes (several of which use the original names, and provide backward-compatible protocol). 4. Incrementally change the above references to use the original class names again (references to "OldStream" become references to "Stream", which is the new but backward-compatible implementation, etc.) 5. The users of internal streams will see no cumulative change, since the Flow support for internal streams provides complete backward compatibility. The users of other support (e.g., sockets) can decide whether to use the new or old framework. 6. As we wish, we can incrementally remove the backward compatibility and old framework support (perhaps over a very long period of time).
The Flow 2a release implements steps 1 to 3, I'm proposing to finish the process. Given this process, I don't think your analogies to 3.3 modules or Croquet (or Microsoft :) are valid.
...why not just keep the two hierarchies just completely disjunct?
Because, e.g., I think what I call "Socket" should be called "Socket". :) As I see it, there is a genuine ontological conflict between the current mainstream implementation and mine. And after renaming the mainstream classes (which, for the reasons above, I think is no big deal), they *are* disjunct. :)
More seriously, I think it's very important to have the right names for things. Sure, one normally makes a file stream by evaluating an expression which doesn't refer to the actual instantiating class, but I think the system is easier to use if people run into appropriately evocative names for things in the system browsers, and this ease of use is critical.
Well if flow can't support those [thread-less] platforms at all...
Um, I just said I think it can...
...just for the record - I am still waiting for benchmarks which actually indicate that flow makes any noticable difference in speed ;-)
The reality of spare-time programming being what it is, if you want *me* to provide these benchmarks, you'll have to be prepared to wait until there are real applications I care about. :)
Please remember that my primary motivation in designing Flow the way I did was programming simplicity, not performance. It so happens that I also think that it will also be faster. But I don't start profiling things until they perform meaningful work. I haven't made time to write applications yet, mostly because the ones I want rely on yet more infrastructure that has little to do with streaming (how to store mail, how to represent system changes, etc.). (I don't mean this as a dig against, e.g., Celeste or Scamper, although I must admit I don't use them.)
But while you're waiting, please feel free to send tests that *you* would consider meaningful. :)
It seems quite feasible to me, by using Smalltalk threads that simulate some of what the host threads do in the current Flow implementation (not entirely unlike what happens in the mainstream Squeak system currently).
This isn't quite what I meant.
I know, but I stand by my answer. :)
The question was more like: Is it possible to write an implementation of the socket primitives that adheres exactly to the specification of the primitives which may not use a threaded implementation?
Sure, but I'd rather keep that simulation at the more malleable object-memory level.
Typically, it's best to stay compatible at the primitive level so that if there's a certain behavior expected by the ST code it should be able to rely on it.
Sure again, but Flow's design allows that behavior to be provided either by "auxilliary primitives" (which aren't called directly by Smalltalk code) or by yet more Smalltalk code. I'd rather go the Smalltalk route.
At the very least I wouldn't expect people running on HP-UX (which IIRC, has horribly broken pthreads) to expect that they have to "reprogram flow" in order to get something to run which works perfectly well on Linux.
Right, they wouldn't. Squeak would know what platform it's on, and act accordingly.
By the way, HP-UX has had perfectly fine pthreads since version 11.0 (two years ago). (Before that, it had DCE threads, which were indeed problematic.)
...there are lots of platforms that Squeak can run on and some of them already have and more will have networking capabilities [but not properly functioning threads]. And I don't want to arbitrarily limit those...
As I said, I think we can accomodate those platforms too.
thanks,
-C
-- Craig Latta improvisational musical informaticist craig@netjam.org www.netjam.org/resume Smalltalkers do: [:it | All with: Class, (And love: it)]
Hi- As a completely naive user -- could you not make the 'old; stuff become a loadable package? David
Craig Latta wrote:
Hi Andreas--
You fixed all references... you knew about.
That's not the end of the process. :) I think you're missing my intentthat those changes be temporary. The full integration process would go like this:
Change some of the original class names ("Stream" becomes"OldStream", etc.) 2. Fix references to those classes. Everything keeps working during the rest of the integration process. 3. Install new classes (several of which use the original names, and provide backward-compatible protocol). 4. Incrementally change the above references to use the original class names again (references to "OldStream" become references to "Stream", which is the new but backward-compatible implementation, etc.) 5. The users of internal streams will see no cumulative change, since the Flow support for internal streams provides complete backward compatibility. The users of other support (e.g., sockets) can decide whether to use the new or old framework. 6. As we wish, we can incrementally remove the backward compatibility and old framework support (perhaps over a very long period of time).
The Flow 2a release implements steps 1 to 3, I'm proposing to finishthe process. Given this process, I don't think your analogies to 3.3 modules or Croquet (or Microsoft :) are valid.
...why not just keep the two hierarchies just completely disjunct?
Because, e.g., I think what I call "Socket" should be called "Socket".:) As I see it, there is a genuine ontological conflict between the current mainstream implementation and mine. And after renaming the mainstream classes (which, for the reasons above, I think is no big deal), they *are* disjunct. :)
More seriously, I think it's very important to have the right names forthings. Sure, one normally makes a file stream by evaluating an expression which doesn't refer to the actual instantiating class, but I think the system is easier to use if people run into appropriately evocative names for things in the system browsers, and this ease of use is critical.
Well if flow can't support those [thread-less] platforms at all...
Um, I just said I think it can......just for the record - I am still waiting for benchmarks which actually indicate that flow makes any noticable difference in speed ;-)
The reality of spare-time programming being what it is, if you want*me* to provide these benchmarks, you'll have to be prepared to wait until there are real applications I care about. :)
Please remember that my primary motivation in designing Flow the way Idid was programming simplicity, not performance. It so happens that I also think that it will also be faster. But I don't start profiling things until they perform meaningful work. I haven't made time to write applications yet, mostly because the ones I want rely on yet more infrastructure that has little to do with streaming (how to store mail, how to represent system changes, etc.). (I don't mean this as a dig against, e.g., Celeste or Scamper, although I must admit I don't use them.)
But while you're waiting, please feel free to send tests that *you*would consider meaningful. :)
It seems quite feasible to me, by using Smalltalk threads that simulate some of what the host threads do in the current Flow implementation (not entirely unlike what happens in the mainstream Squeak system currently).
This isn't quite what I meant.
I know, but I stand by my answer. :)The question was more like: Is it possible to write an implementation of the socket primitives that adheres exactly to the specification of the primitives which may not use a threaded implementation?
Sure, but I'd rather keep that simulation at the more malleableobject-memory level.
Typically, it's best to stay compatible at the primitive level so that if there's a certain behavior expected by the ST code it should be able to rely on it.
Sure again, but Flow's design allows that behavior to be providedeither by "auxilliary primitives" (which aren't called directly by Smalltalk code) or by yet more Smalltalk code. I'd rather go the Smalltalk route.
At the very least I wouldn't expect people running on HP-UX (which IIRC, has horribly broken pthreads) to expect that they have to "reprogram flow" in order to get something to run which works perfectly well on Linux.
Right, they wouldn't. Squeak would know what platform it's on, and actaccordingly.
By the way, HP-UX has had perfectly fine pthreads since version 11.0(two years ago). (Before that, it had DCE threads, which were indeed problematic.)
...there are lots of platforms that Squeak can run on and some of them already have and more will have networking capabilities [but not properly functioning threads]. And I don't want to arbitrarily limit those...
As I said, I think we can accomodate those platforms too. thanks,-C
-- Craig Latta improvisational musical informaticist craig@netjam.org www.netjam.org/resume Smalltalkers do: [:it | All with: Class, (And love: it)] _______________________________________________ Squeakfoundation mailing list Squeakfoundation@lists.squeakfoundation.org http://lists.squeakfoundation.org/listinfo/squeakfoundation
-- David Harris OmniPort Home Page: http://www3.telus.net/OmniPort1/ Discussion egroup: http://groups.yahoo.com/group/OmniPort Swiki: http://omniport.swiki.net/1
Hi David--
As a completely naive user [and top-poster ;] -- could you not make the 'old; stuff become a loadable package?
Sure, whatever old stuff gets removed can always be available somewhere thereafter.
-C
-- Craig Latta improvisational musical informaticist craig@netjam.org www.netjam.org/resume Smalltalkers do: [:it | All with: Class, (And love: it)]
On Thursday, November 21, 2002, at 06:56 PM, Craig Latta wrote:
- Change some of the original class names ("Stream" becomes
"OldStream", etc.) 2. Fix references to those classes. Everything keeps working during the rest of the integration process. 3. Install new classes (several of which use the original names, and provide backward-compatible protocol). 4. Incrementally change the above references to use the original class names again (references to "OldStream" become references to "Stream", which is the new but backward-compatible implementation, etc.) 5. The users of internal streams will see no cumulative change, since the Flow support for internal streams provides complete backward compatibility. The users of other support (e.g., sockets) can decide whether to use the new or old framework. 6. As we wish, we can incrementally remove the backward compatibility and old framework support (perhaps over a very long period of time).
This process would be appropriate for a monolithic image, but it doesn't fly very well in a modular universe. Flow can't co-exist with any packages which use the old networking classes, because they might be loaded after flow, or flow might not correctly patch them when it changes the names of the old classes. No matter how thorough the reference-fixing strategy is, there will always be some goofy package out there that got missed.
I'd like to see a migration process that went more like this:
1) Rename some classes in flow and make it into a DVS package that only modifies the image by adding to it.
2) Carve out the existing networking code into a similar package.
3) Have a transition period where networking applications get registered on squeak map as depending on either old-sockets or flow. Make it known that flow is the "blessed" networking library and encourage package maintainers to port to flow. During this period, development of flow would be carried out in such as way as to maintain peaceful co-existence with old-sockets.
4) Once dependence on the old-sockets package has waned, remove the compatibility requirement and allow flow to evolve in whatever way seems ideal.
This process fits in better with the image factoring that the Mousekeepers have been discussing, and would make use of the tools that will be required for that anyway - package dependencies, versioning, conflict detection etc.
Notice that I haven't mentioned anything about what should be included "in the image." I think the factoring / packaging efforts will produce a multitude of image distributions, which can include either, neither or both networking libraries as appropriate.
My two bits.
Colin
Hi Colin--
This process would be appropriate for a monolithic image, but it doesn't fly very well in a modular universe.
Normally I'd agree, but since virtually all the applications we're talking about recently *were* in a monolithic image (we're still busy taking them all out :), I think it's much less work right now to proceed the "monolithic" way. For other applications which were never part of the release image, we can adapt just by making sure they're loaded, installing Flow, and fixing what breaks. I effectively did that for everything in 3.2final, and it was easy.
thanks,
-C
-- Craig Latta improvisational musical informaticist craig@netjam.org www.netjam.org/resume Smalltalkers do: [:it | All with: Class, (And love: it)]
Craig Latta craig.latta@netjam.org wrote:
Hi Colin--
This process would be appropriate for a monolithic image, but it doesn't fly very well in a modular universe.
Normally I'd agree, but since virtually all the applications we're talking about recently *were* in a monolithic image (we're still busy taking them all out :), I think it's much less work right now to proceed
No they were not. VH (a project of Bluefish) is not in the image and use the Sockets heavily. My point being that I agree fully with Colin, Andreas and Cees on this one.
the "monolithic" way. For other applications which were never part of the release image, we can adapt just by making sure they're loaded, installing Flow, and fixing what breaks. I effectively did that for everything in 3.2final, and it was easy.
First of all this would require we round up all apps out there that do depend on this - and note that there may be non open source apps out there too... Doesn't sound like a plan to me.
Secondly this would mean adopting Flow before we actually have tried it out heavily. That also seems foolish.
Instead I agree with the process Colin outlined with the little exception that I would like to make clear that the original networking package would be the one "blessed" from step 2 and forwards. First when Flow attracts enough users and has proven itself "in battle" we can consider moving the blessing over to Flow - if we decide that Flow is a better base package than the original.
Note also that I also agree with Andreas/Colin that instead of renaming the old classes (thus breaking any code outside the image) why not simply having new names in Flow? It seems very obvious to me.
regards, Göran
goran.hultgren@bluefish.se wrote:
No they were not. VH (a project of Bluefish) is not in the image and use the Sockets heavily. My point being that I agree fully with Colin,
It would be interesting to see how much of your application gets broken by the network rewrite. That would give a real life estimate on how much pain refactorings may cause.
On the other hand we should learn from the Mac OS experience where Apple decided to avoid pain for too long by staying backwards compatible to broken APIs. It's like saving money by not changing the oil in your car ;-)
Michael
Michael Rueger m.rueger@acm.org wrote:
goran.hultgren@bluefish.se wrote:
No they were not. VH (a project of Bluefish) is not in the image and use the Sockets heavily. My point being that I agree fully with Colin,
It would be interesting to see how much of your application gets broken by the network rewrite. That would give a real life estimate on how much pain refactorings may cause.
Yes, but I don't think I will take the time to do that. VH is shelved for the moment.
On the other hand we should learn from the Mac OS experience where Apple decided to avoid pain for too long by staying backwards compatible to broken APIs. It's like saving money by not changing the oil in your car ;-)
Just for the record: I generally do *not* think that backwardscompatibility is that important - I want Squeak to evolve as "fast" as possible! But I also think that a package like Flow could get much more acceptance if I could install it without having it break my old stuff. And I also think that people need to understand that the image isn't the world of Squeak code.
But please let us stop talking about choosing one before the other - let's aim at having them *both* as packages so people can pick and choose. That is after all what we want, right?
regards, Göran
On Sat, 23 Nov 2002 goran.hultgren@bluefish.se wrote:
Just for the record: I generally do *not* think that backwardscompatibility is that important - I want Squeak to evolve as "fast" as possible!
Ignoring backwards compatibility might allow you to evolve a little faster than if you took it into account. OTOH, backwards compatibility gives you a much better chance of early adoption of your evolved code by a far larger number of users -- which means your evolved code will be stress tested and debugged a lot sooner. The additional time spent making it backwards compatible might actually _reduce_ the overall time needed to get your evolved code to production quality.
Mike's comparison with Apple's APIs is a little misleading here since Apple were dealing with a huge code base and large, unweildy APIs. In the case of Sockets the additional work and complexity involved with backwards compatibility is significantly smaller.
But please let us stop talking about choosing one before the other - let's aim at having them *both* as packages so people can pick and choose. That is after all what we want, right?
Precisely. (At least, that's what I would want. ;)
BTW, Craig: can we currently connect an AsyncFile to a flow `endpoint' and use streaming protocols to talk to it?
Ian
goran.hultgren@bluefish.se wrote:
But please let us stop talking about choosing one before the other - let's aim at having them *both* as packages so people can pick and choose. That is after all what we want, right?
Is that possible? I can image two different loadable packages for networking, but I think the base image will include some kind of file system support, and then you'll have to choose between Flow or the current FileStream hierarchy. The same happens with Streams. I don't believe the Streams are ever going to become a loadable package, so we'll have to choose between the current Streams or Flow's,
The Guides will take the final decision, but if it was my call I would forget backward compativility and move Squeak forward. I have old code, and I know I might not have the time to port everything to Flow, but that's why we have a repository with old images. My old code runs perfectly in old images, and I don't expect it to run in newer images. I have used Flow in two projects: - A TCP/IP stack, where the sockets are resources (polymorphic with Flow's ExternalResources); - A framework for distributed agents (my "Army of Mobile Agents"). And in this project I implemented a hierarchy of ProxyResources: files, sockets and pipes than run thorugh a syscalls proxy... some form of RPC for system calls. All system calls are marshaled and executed in a very tiny server (~ 100 bytes) running on a different host.
This is my experience with Flow, and I believe Flow is simpler, cleaner and more useful than the Streams currently in Squeak.
IMHO, etc. Really :) Luciano.-
On Thursday, November 21, 2002, at 11:01 PM, Craig Latta wrote:
Hi Colin--
This process would be appropriate for a monolithic image, but it doesn't fly very well in a modular universe.
Normally I'd agree, but since virtually all the applications we're talking about recently *were* in a monolithic image (we're still busy taking them all out :), I think it's much less work right now to proceed the "monolithic" way.
These are the networking packages that I've used for my day-to-day work:
Comanche Mysql drivers RPC-XML rST VNCServer
None of these have ever been in the image. I'm sure there are quite a few others out there as well.
For other applications which were never part of the release image, we can adapt just by making sure they're loaded, installing Flow, and fixing what breaks. I effectively did that for everything in 3.2final, and it was easy.
Sure, but once I've done that, then what? Then I've got to maintain two code bases. I won't be able to convert all these packages to use flow in an afternoon. Most of them were written by somebody else, (as was flow) so debugging them will take time as I figure out what is supposed to happen as well as what actually is happening.
The other way would be to convert all these packages to use OldSocket and friends and make sure I have always have flow installed in all my images. Then I can port the packages one at a time, and still be able to use the as-yet-unconverted packages at the same time. But if there are any show-stoppers, I'm going to have a fair amount of work to move back to the old socket code. And in the meantime I'm dependent on flow, which complicates my image building process, even though I'm not using it yet.
Don't get me wrong, I'm a big fan of ontological purity. But come on, you're asking me to do a lot of work and take a big risk to preserve the esthetics of your class names. How about a compromise? Just stick FL at the beginning of each flow class to create a poor man's name space. Then flow can happily along side the old network classes and we can use the packaging system to sort out these problems.
Cheers,
Colin
Craig Latta craig.latta@netjam.org said:
- Change some of the original class names ("Stream" becomes
"OldStream", etc.)
Without my giving any qualification whatsoever on the qualities of Flow, I think I have to agree with Andreas. Why not leave Stream and introduce a FlowStream (or FlwStream, whatever)? People can then have both in the image and the market can decide whether Flow is really better. In a year or so, when Flow has proven itself, you can then remove the old code from the image; and when Squeak finally has namespacing, you can remove these ugly prefixes.
Talking to the outside world is one of the most important things; with all the great Squeak software that I see lying around but that only works under 2.x, I think that going through great lengths to provide compatibility for old code is extremely important. If that means that the new stuff has to revert to not-so-nice class names (b/c that is basically the only real issue here, if I understood it right), I think that's the way to go.
Hi Cees--
Without my giving any qualification whatsoever on the qualities of Flow, I think I have to agree with Andreas.
Hmm. I'm not sure what "agreeing with Andreas" actually means at the moment, since, from his responses so far, I think he missed some of what I was saying.
Why not leave Stream and introduce a FlowStream (or FlwStream, whatever)?
Because, in my experience, people tend to actually test things more when the *old* stuff gets the name changes.
In a year or so, when Flow has proven itself...
Heh. Another year...
I think that going through great lengths to provide compatibility for old code is extremely important.
I agree, and I think what I propose does this.
thanks,
-C
-- Craig Latta improvisational musical informaticist craig@netjam.org www.netjam.org/resume Smalltalkers do: [:it | All with: Class, (And love: it)]
Why not leave Stream and introduce a FlowStream (or FlwStream, whatever)?
Because, in my experience, people tend to actually test things more when the *old* stuff gets the name changes.
It sounds like you're saying "because if I change the names, it forces people to make sure their stuff works with Flow".
That's a really good way of making sure Flow never gets adopted. Personally, I'd love to use it, but it's not gonna happen (or not anytime soon) if it means porting all my existing code to it. On the other hand, if Flow can live happily side by side with the current streams, and I can, *without* any porting effort, have new code that uses Flow in the same image as old code that doesn't, I'm likely to actually use Flow - and it's only by getting used that Flow will get tested at all.
Hi Avi--
It sounds like you're saying "because if I change the names, it forces people to make sure their stuff works with Flow".
That's not what I'm saying, nor what I think the reality is.
Personally, I'd love to use it, but it's not gonna happen (or not anytime soon) if it means porting all my existing code to it.
It doesn't. As I've said, it means temporarily using a few different class names.
On the other hand, if Flow can live happily side by side with the current streams, and I can, *without* any porting effort, have new code that uses Flow in the same image as old code that doesn't...
That is the case. :)
...I'm likely to actually use Flow - and it's only by getting used that Flow will get tested at all.
Yup.
thanks,
-C
-- Craig Latta improvisational musical informaticist craig@netjam.org www.netjam.org/resume Smalltalkers do: [:it | All with: Class, (And love: it)]
On Fri, 22 Nov 2002, Craig Latta wrote:
Personally, I'd love to use it, but it's not gonna happen (or not anytime soon) if it means porting all my existing code to it.
It doesn't. As I've said, it means temporarily using a few different class names.
Sigh. So I'm working on code that uses the old Streams, and I need some other code that uses Flow. Ok, I change all those class names, bring in Flow, and keep going. Now I've finished my changes, and need to commit. Wait - this code needs to work on a stock 3.2 image. Need to change all those names back before I commit. Change them, commit and need to continue working. But Flow's loaded, so the code won't work until I change all the names again...
I realize that you're saying that eventually Flow will be 100% backwards compatible with the current streams. Great. When that's true, I'll be glad to start testing it. Until then, I can't picture the process.
squeakfoundation@lists.squeakfoundation.org
-
Andreas.Raab＠gmx.de -
Avi Bryant -
cg＠cdegroot.com -
Colin Putney -
Craig Latta -
Daniel Vainsencher -
David Harris -
goran.hultgren＠bluefish.se -
Ian Piumarta -
Luciano Notarfrancesco -
Michael Rueger -
Tim Rowledge