Partitioning the image (was Re: Shrinking sucks!)

Doug Way dway at mailcan.com
Tue Feb 8 00:39:51 UTC 2005


On Mon, 7 Feb 2005 11:37:48 +0200 , goran.krampe at bluefish.se said:
> (some stuff snipped here and there)
> 
> > Although it would be nice to be able to unload packages that are there.  
> >   But you can unload Monticello packages (and any other package type  
> > based on PackageInfo), which I guess you already know.
> 
> Yup, I know. :) And hey, I will add it to the SqueakMap Package Loader.
> I just haven't since it felt like a "teaser" that will only awaken the
> deamons. ;)

Yeah, it would be nice to have in the Package Loader.

> > Yeah, let's get TFNR kick-started again.  I will hop on board this  
> > time. :)
> 
> Great! TFNR is now my prio uno too, apart from replying to all the SqF
> hoopla, spitting out the bi-weekly and making the new SM. :)

Someone else should really take on the bi-weekly report, you already
have enough on your plate. ;)

> But really, we JUST HAVE TO DO IT this time. Damn shame we are in
> different timezones. Can you pop over to IRC whenever you can and we can
> see if we overlap? I am there during my work hours, hardly anything on
> weekends these days - family takes prio.

I will try to get on IRC, but it may be tough to do from work, I have to
figure out proxy/firewall issues.  (Is there a way to do IRC through a
web browser? ;) )  And then evening-time here (EST) is 1am-5am Sweden
time, ah well...

> Sorry, I meant PackageOrganizer. And note it has a class instvar called
> #default.
> And btw, I now installed version 18 of PI into 3.9a-6550 and that silly
> halt bugs me. Can we get rid of that somehow?
> 
> > package on SqueakMap.  Is this just the list of packages which shows up  
> > when you open the "Package List" window?
> 
> No, that list is... well, is is quite slimmed down. :)

For what it's worth, that list seems to correspond to "PackageOrganizer
default" whenever I happen to look at it...

> Personally I think it is a bit confusing - we have 3 things currently
> making users confused:
> 
> 1. Package browser. The old browser that packagifies class categories.
> IMHO it should be called something else.

Yes, we need to rename it ASAP.  Just call it something like
CategoryBrowser and be done with it.  It may soon become somewhat
irrelevant if Packages become the principal organization of classes,
rather than Class Categories.

> 2. The above mentioned Package list. It feels a bit... well, can't we
> integrate this stuff in the SM Loader?
> 3. SqueakMap Package Loader which perhaps should be renamed to "Package
> Manager". That way it implies more than just "loading". I mean, it does
> also show what packages are installed etc.
> 
> So either we make something more out of the "package list" or we zap it
> and improve the SqueakMap Package Loader to become the One And True
> Place For Packages. Which is my suggestion. :) It should for example be
> easier to see what packages are installed - currently you need to apply
> a filter - and newbies might not even find the damn menu. ;) Perhaps a
> button row somewhere in this tool would make it more newbie-friendly.

Well, this issue needs some more thought.  The Package Loader currently
also treats changesets, .sar files etc as "packages".  But they're not
"real" PackageInfo packages.  I'd like a list or browser tool somewhere
which only shows the "real" packages, which can be browsed, unloaded,
and otherwise manipulated.

I suppose one solution is to have the Package Loader show the installed
items, including real packages and other SMpackages (changesets, etc). 
But only have the Unload/Browse/etc capability enabled for the real
packages, which you suggested below.

I guess I mainly just want to see the list of all real packages when
browsing source code in the image.  So perhaps all we need is a real
Package Browser (sort of a different System Browser) which lists Package
names in the upper left corner, and then the next four panes across the
top are the same as the MC Snapshot Browser, including Extensions etc,
and then a code pane at the bottom.  You use this tool to browse/edit
all the code in the image (but it could also do a few extra things like
unload packages, too).  Then we wouldn't need the Package List.  And
this is somewhat separate from the Package Loader/Manager's
functionality.

Actually, you still need to keep the System Browser, though, or
something like it.  One browser needs to have the package-centric view,
with extensions grouped with the package.  And the other browser needs
to have the class-centric view, with extensions grouped with the class. 
(This is exactly what ENVY had.)

Anyway, this Package Browser could be worked on in parallel with the
initial partitioning work.

> [SNIP]
> > >> Undefining messages is not a problem until two
> > >> packages will modify the same concurrently.
> > >
> > > Eh, well IMHO the problem with uninstalling packages is not exactly  
> > > what you describe.
> > > If all packages were forced to be constrained to be Monticello packages
> > > (without dirty code running in class initializers) - then we would more
> > > or less already have that. But they aren't.
> > 
> > The solution to this is obvious, I think. :)
> 
> Just note that this part of my previous post was quite theoretical just
> to try to explain the intricacies with "uninstall" in Squeak country.
> People tend to think that "packages" in Squeak are just source code. But
> in fact - they aren't, because of the simple fact that arbitrary code is
> allowed to execute on install. Which of course theoretically can have
> irreversible side effects.

Right.  I think we have to live with the fact that class initialization
will happen on load, and it may not be reversible, but we are trusting
the packages not to do a lot of bad/unreversible stuff during class
initialization.

> > First, make it easy to unload packages that can be unloaded.  For  
> > example, I don't understand why the "Package List" window (available  
> > even without MC) doesn't offer an "unload package" menu item.  They  
> > should be unloadable.  Of course, you can't guarantee unloadability if  
> > your image is dependent on the code being unloaded, but the package  
> > should already be "untangled" before it is made available.  (And there  
> > can be issues with unloading if you have a bunch of instances of  
> > unloaded classes in your image doing important things, but I don't  
> > think that's our biggest problem right now.)
> 
> I vote for - according to my rambling above - adding a button row in
> SqueakMap Package Loader (and renaming it to Package Manager) - which
> includes an "Uninstall" button with ballon help explaining why it is
> almost always disabled. :)

Yep.  It would eventually become disabled less often. ;)

> > Basically, the Package concept needs to be more available in the UI.   
> 
> Yes. And again, I am not saying this because it is a tool that *I* am
> maintaining - but I really think we should push the SM Package Loader so
> that it is more available. This includes adding it in the right hand
> flap.
> 
> Sidenote: I intend to take a look at SUnit and copy how all that is done
> and release a new loader. Should we also rename it then? Whadda ya say?
> "Package Manager"? Or we stick with it as today but then we really need
> to have yet another tool (better than PackageList) and that feels
> suboptimal.

"Package Manager" might be alright.  Maybe think about it for a bit
first, and how it might fit in with a Package Browser described above.

> > There should be a package-centric browser.  The MC Snapshot Browser is  
> > pretty much the right UI for this, but it might be nice to have it  
> > available outside of MC, just operating on code in the image.  (But if  
> > Basic already includes MC, well, I guess the Snapshot Browser could be  
> > used.  But make it available from the Package List via a "browse  
> > package" menu item and in other places.)
> 
> Mmmm. Again, I wonder if "Package List" really is worth having around.
> :) What are the compelling reasons?

Okay, maybe get rid of it if we have both a Package Manager/Loader and
Package Browser.

> [SNIP]
> > Forget about changesets, .st files and .sar files, they have their  
> > purpose, but they don't need to be unloadable.  All real "packages"  
> > (i.e. applications or subsytems) don't need to be in those formats,  
> > they can be MC/PI-based.
> 
> .sar files could be unloadable if they only contain .mcz files and no
> "dirty" postscripts.

True.
 
> > .sar files are good for installing things like fonts or arbitrary data  
> > into an image, but there's not a pressing need to have those be  
> > unloadable either.
> 
> Right, on the other hand we might need them for packaging resources.
> So perhaps we could "tag" them somehow? As in "this .sar file is
> uninstallable, because it only has .mcz files and a proper postscript".
> Eh... would it also need a uninstall-postscript then perhaps? Hmm. :)
> 
> Anyway, let's dump that issue on Ned and Avi. :) I will mail them.

OK, I'm sure something could be worked out there.

> [SNIP]
> > > 1. TFNR. Yes. Let's finally partition the image and put people in  
> > > charge
> > > of each part. Yes, the parts will still be totally intertangled - but
> > > each line of code will have a maintainer. And each logical "tool" or
> > > "mechanism" will have a caring Squeaker. We should do this NOW.
> > 
> > Sounds good.  I am on board!
> 
> Great. Let's do it. We simply just add a cs that makes PIs more or less
> based on the current class categories IMHO. Stuff can be moved around
> later. Agree?

Ok, although it may be worth pausing *briefly* to consider Stephane's
comments about whether we should use a real object instead of
PackageInfo with its string-based naming hack, so that you can e.g.
easily rename a package without worrying about the class category names.

Maybe there is a simple solution using a real object which has a similar
API to PackageInfo and organizes packages in exactly the same way, which
is also reasonably backward-compatible with existing browsers.  But I
don't want to wait long either, we need to look at it now and make a
decision.

I think we are agreed that we do not want to tackle namespaces or
anything like that right now, at least.

> 
> > > 4. Start moving tools over to Tweak, so that we can get rid of Morphic.
> > > And move to newer tools where appropriate - like Omnibrowser instead of
> > > the old browser etc. In essence we need Omnibrowser, a workspace,
> > > exporer/inspectors and a debugger. At least. :)
> 
> Btw, the above was based on the assumption that Tweak was definitely
> scheduled to be the next "official" UI framework in official Squeak.
> AFAIK this is definitely not "definite" at this point. In short - this
> is a topic of its own, will not discuss it in this posting.

OK.

> > > (tweak and spoon work)
> 
> Yeah, the above was much further down the road. At *this* point (I
> change my mind quickly you know) I actually say:
> 
> - We shouldn't include Spoon in this plan just yet. If we want to go
> small, we can try with any small image that at least has a head (Morphic
> or MVC).
> - We shouldn't automatically think Tweak is the given successor to
> Morphic. The issue is more complex. And Morphic will have to be around
> for a long time to come.
> 
> In fact - I think we should make sure Squeak can work with both
> frameworks. I know Andreas has mentioned resurrecting ideas/code in
> StableSqueak (using an abstract UI framework layer) but on the other
> hand, given recent postings about Tweak (the Islands post for example) I
> am not so sure that Tweak is going to be so easy to just "put in
> Squeak". Again, a different topic altogether.

Sounds good.

> > Overall, your plan sounds pretty compatible with what I was rambling  
> > about in this post:
> > 
> > http://lists.squeakfoundation.org/pipermail/squeak-dev/2005-January/087639.html
> 
> Yes ( I reread it) that post is DEAD ON what I am after. And yes, that
> is exactly what TFNR was/is all about.

Cool. :)

> > With #1, it might be good to have a basic plan of attack figured out  
> > here on squeak-dev first, before forming a separate mailing list of  
> > volunteers.  If there is a basic plan in place, it might actually help  
> > with getting more volunteers, because they'll know something will  
> > "really happen this time". ;)
> 
> Yeah, but too much planning also... well, I am in "do it" mode for the
> last few weeks. :)
> But sure, I am with ya.

Right, well, I'm just talking about the very basics, I think we already
have most of what I would consider a basic plan:

- We will use the update stream to partition and then detangle the
image.
- We'll assign maintainers to each package after partitioning, before
detangling.
- We will use XXX (PackageInfo or something very similar) as the basis
for packaging. (This one isn't decided yet.)
- etc.

> (snip)
> I am all with that.
> 
> > Basically, I don't think we can really consider getting rid of the  
> > update stream until #6 (full dependencies) is done.  And even then we  
> > may keep it.  Or not, who knows? ;)
> 
> Sure. Let's stick to it.

Great.

> > My other thought is that we should make the initial partitioning as  
> > coarse as possible.  "Graphics" is one package, "Morphic" another,  
> > "Kernel", "Tools", etc.  This will make it easier to detangle, if  
> > you're only worrying about large pieces... don't worry about breaking  
> > the large pieces into smaller ones until later.  At this point, no one  
> 
> Exactly my view. It doesn't matter if they "overlap" in wrong ways in
> the beginning, just as long as every line of code belongs to *someone*.
> Then the people in charge can work it out between themselves.

Great.  Getting back to finding maintainers, it might be best to assign
four or five maintainers to a big package like Morphic (with one lead
maintainer), rather than assigning individual people to smaller pieces
like Morphic-Windows, etc.  We will probably have more success getting
complete coverage that way.

> > (Then there's the whole issue that PackageInfo is based on the strings  
> > of the class categories and method categories to define the code that  
> > it contains, which is kind of a hack, but it works and is compatible  
> > with existing tools.  It wouldn't be hard to convert PackageInfo-based  
> > packages to a more "real" modules system later, after the detangling is  
> > done... it's the detangling that's the hard part.  I'm not sure we want  
> > to wait around for a more proper modules system before we begin  
> > partitioning & detangling.  I guess one problem with this is that you  
> 
> No, we should definitely not wait anymore. I am not even sure what we
> would be waiting for.

Well, let's hear what the other alternative is, anyway.  (For a *Brief*
moment. :) )  But it is true that we could convert PackageInfo packages
to something more robust later, since the eventual package
partitioning/detangling should be roughly the same whichever one we use.

> > may end up changing the class categories of a lot of classes to make  
> > this work.  For example, "Collections-*" might have to be renamed to  
> > "Kernel-Collections-*" if we want the Collections-* classes to be in  
> > the Kernel package, which is where most of them would need to be.  Can  
> > the package names be unrelated to the class category names?)
> 
> Well, I would favour having separate packages. Why can't collections be
> a separate package?
> 
> I mean, sure - it will never be uninstallable, but it seems you would be
> able to upgrade it to newer releases at least. Hey, that is a thought
> worth noting...

My initial thought was that the single Kernel "package"/image could run
on its own.  But I don't have a strong opinion on that.  We can let the
people in charge of those areas decide.

> > (And I know the above is only 80% thought through, but announcing a  
> > plan and starting on it is a sure way to get feedback!)
> > 
> > - Doug
> 
> Let's just roll with it - the plan is fine. I say we just MOVE FORWARD
> and people can yell if there is anything. :)

Alrighty, we're getting ready to roll. :)

- Doug



More information about the Squeak-dev mailing list