Packages

packages@lists.squeakfoundation.org

1 participants
90575 discussions

Re: Feedback
by Lex Spoon 09 Mar '05

09 Mar '05

"Lex Spoon" <lex(a)cc.gatech.edu> wrote: > For the record, Universes is under maintenance and in use, and (no > surprise) it achieves all of the properties I mentioned in my opening > email. Crud, I hit send too fast. There are other properties that would be nice, to be sure, and we'll get there. -Lex

1 0

partitioning a moving target
by Avi Bryant 09 Mar '05

09 Mar '05

Should we be doing our partioning work based on Squeak 3.8 or 3.9a? Either way we have to deal with moving target issues; the question is whether we want to have a stable base while we do the bulk of the work, and then have to update to match 3.9, or whether we want to have the ground constantly pulled out from beneath us by the new updates in the 3.9a stream. Or is there some third option? Avi

3 2

proposal for starting work on partitioning
by Avi Bryant 08 Mar '05

08 Mar '05

Cees and I just chatted on IRC about tools for supporting partitioning, and coordinating between the package splitting we do and the work done by the morphic splitters. Here's the simple proposal I have to let us get started as quickly as possible, hopefully with minimal conflict and minimal need for coordination: - I'll set up a page on the swiki with a list of packages in the image (if people have string preferences for how granular to make this list, or what should be on it, please say so). To begin with they'll all be up for grabs except the morphic ones. - If you want to do some work on splitting out a package you put a note beside it on the wiki saying that you've started working on it and what the current date is. - When you've done some work on it and are stopping, post a changeset with what you've done to the list, and take your name off of the page. - I'll post the changeset to a special update stream as soon as possible so that people can easily stay in sync. Always try to update from the stream before starting to work on your own splitting. - If someone's had a package "checked out" for a while without posting, use your judgement as to whether or not to override their lock and work on it yourself - maybe email them to see if they'll release the lock to you, or post to the list about it. - In terms of how we attack the splitting itself, it seems to me that it's easiest if people can really just focus on one package at a time - that is, I want to look through all the methods of a package and just decide "is it in this package or isn't it". So I suggest that we work in two passes: first, we try to kick everything out of packages that shouldn't be there, by going through all of the classes in the package and moving any of their methods that shouldn't be in the package to a special "Orphaned" package. Second, or in parallel, we go back through the Orphaned methods and adopt the relevant ones into our package. It would be nice if we had some tools to go through lists of methods in a package, or in Orphaned, with big "Mine" and "Not Mine" buttons, but we can probably do without that for now. If this approach seems sensible to people, I'll set up the small bits of infrastructure required right away. Avi

2 4

Re: Splitting tools
by Dan Ingalls 08 Mar '05

08 Mar '05

Daniel V. wrote... >Just a note here: >The tool Dan describes here, AFAIU, allows one to determine what can be removed after a named set of categories is removed. IIUC, this is useful after the named categories are themselves removable (not depended on by other stuff). > >MudPie can be used before that, to see what depends on what, and therefore what is preventing your package from being removable. > >So they seem to me to be complements. Yes, this was my sense as well. When I (wrote and) used these methods, I already had a sense that, for instance, Celeste, Scamper and MailMessage were the "roots" of most of the web classes [at that time]. Nowadays, you would want to include several other root apps, probably, to get the nice results I got. I believe that Daniel's MudPie would help a lot, both in identifying these natural roots, and in breaking any circularities. - Dan

1 0

Re: splitting tools
by Daniel Vainsencher 08 Mar '05

08 Mar '05

Just a note here: The tool Dan describes here, AFAIU, allows one to determine what can be removed after a named set of categories is removed. IIUC, this is useful after the named categories are themselves removable (not depended on by other stuff). MudPie can be used before that, to see what depends on what, and therefore what is preventing your package from being removable. So they seem to me to be complements. I've updated the SqueakMap version of MudPie to work on Squeak 3.9+Connectors 2.2. Daniel Dan Ingalls wrote: > [I accidentally sent this to Modules, and am cross-posting this to the Packages list, since it seems that similar work is going on there as well. My apologies if you get multiple copies -- Dan] > > >> On Tue, 1 Mar 2005 19:28:09 -0800, Dan Ingalls <Dan(a)SqueakLand.org> wrote: >> >> >>> Does such a tool exist right now? If not, I think it would be a great aid to the project of partitioning the image. >> >> >> It certainly would. When I worked with VW last year, we had such a tool which greatly helped in determining prerequisites. First it would tell you what packages where needed, but it had a detail view that would tell you why these were needed. >> >> With that, we could in theory point at ToolBuilder and some 'favorite morphs' and start hacking away. > > > > > Folks - > > I went back and tracked down my old code and messages to the Squeak list. It is from February 2001 (!!) and I attach the summary below. It turns out that most of the code is still in the image. I haven't tested it, but I'd be happy to help anyone to fix it or use it. > > The principle was extremely simple, and very few lines of code. > Here's an example of its use: > Start with a set of "package roots", such as {Celeste. Scamper. MailMessage}. > First establish the list of classes and methods that appear to be unused > in the full system > Then compute the same result but in the absence of the package roots. > The *newly* inaccessible classes and methods presumably are only > used by package roots, so they should be added to the package. > Iterate the previous two steps until the package does not grow any more. > > The comments with update #3735 below give the excellent results achieved in deriving a package for the three network applications of the time. > > This code all worked four years ago, and it was the basis for the preliminary partitioning in the Environments project. I mainly wanted to point it out, in case it might be of use to people in partitioning the current image. It may be entirely subsumed by Daniel V's MudPie but, if not, then it might save people some time. > > Hope this is of some use. > > - Dan > ----------------------------------------- > > > 3735BetterShrink-di -- Dan Ingalls -- 27 February 2001 > Introduces a new image partitioning tool that is essentially a local version of majorShrink. The idea is to pick some application like Scamper, or some cluster like all the 3D classes, and remove it, and also remove everything that is solely referred to by it. The way it works is: > Record all unsent messages and unused classes at the outset > Mark the application or cluster as removed > Note all *newly* unreferenced methods and unused classes > and iteratively remove them as well > For example, the expression > Smalltalk reportClassAndMethodRemovalsFor: #(Celeste Scamper MailMessage) > reports 63 classes and 155 other messages that can be removed. > Also, introduces a new method, fileOutAndRemove:... that will take such results, build a changeSet from them, fileOut everything that is about to be removed, and then remove all of the classes and related messages as well. For the above example, this method produces a 290k fileOut, and saves about 104k from the system. More importantly, if you fileIn the changeSet afterward, the system returns to approximately its former size, and you can run Scamper again. It's a way to fileOut a package that never existed. Tools for Pools yet to come. >

1 0

re: Splitting tools
by Dan Ingalls 07 Mar '05

07 Mar '05

[I accidentally sent this to Modules, and am cross-posting this to the Packages list, since it seems that similar work is going on there as well. My apologies if you get multiple copies -- Dan] >On Tue, 1 Mar 2005 19:28:09 -0800, Dan Ingalls <Dan(a)SqueakLand.org> wrote: > >>Does such a tool exist right now? If not, I think it would be a great aid to the project of partitioning the image. > >It certainly would. When I worked with VW last year, we had such a tool which greatly helped in determining prerequisites. First it would tell you what packages where needed, but it had a detail view that would tell you why these were needed. > >With that, we could in theory point at ToolBuilder and some 'favorite morphs' and start hacking away. Folks - I went back and tracked down my old code and messages to the Squeak list. It is from February 2001 (!!) and I attach the summary below. It turns out that most of the code is still in the image. I haven't tested it, but I'd be happy to help anyone to fix it or use it. The principle was extremely simple, and very few lines of code. Here's an example of its use: Start with a set of "package roots", such as {Celeste. Scamper. MailMessage}. First establish the list of classes and methods that appear to be unused in the full system Then compute the same result but in the absence of the package roots. The *newly* inaccessible classes and methods presumably are only used by package roots, so they should be added to the package. Iterate the previous two steps until the package does not grow any more. The comments with update #3735 below give the excellent results achieved in deriving a package for the three network applications of the time. This code all worked four years ago, and it was the basis for the preliminary partitioning in the Environments project. I mainly wanted to point it out, in case it might be of use to people in partitioning the current image. It may be entirely subsumed by Daniel V's MudPie but, if not, then it might save people some time. Hope this is of some use. - Dan ----------------------------------------- 3735BetterShrink-di -- Dan Ingalls -- 27 February 2001 Introduces a new image partitioning tool that is essentially a local version of majorShrink. The idea is to pick some application like Scamper, or some cluster like all the 3D classes, and remove it, and also remove everything that is solely referred to by it. The way it works is: Record all unsent messages and unused classes at the outset Mark the application or cluster as removed Note all *newly* unreferenced methods and unused classes and iteratively remove them as well For example, the expression Smalltalk reportClassAndMethodRemovalsFor: #(Celeste Scamper MailMessage) reports 63 classes and 155 other messages that can be removed. Also, introduces a new method, fileOutAndRemove:... that will take such results, build a changeSet from them, fileOut everything that is about to be removed, and then remove all of the classes and related messages as well. For the above example, this method produces a 290k fileOut, and saves about 104k from the system. More importantly, if you fileIn the changeSet afterward, the system returns to approximately its former size, and you can run Scamper again. It's a way to fileOut a package that never existed. Tools for Pools yet to come.

1 0

Get this party started...
by Avi Bryant 06 Mar '05

06 Mar '05

Hi all, I thought I'd kick off this mailing list with my (hopefully brief) thoughts on what our goals and priorities might be; hopefully y'all will post similarly, and we can start to reach a consensus on what it is we're trying to do. On the Teams page at http://anakin.bluefish.se/castaways/6, the Packages team is described as "Partitioning, PI, packages, TFNR, SM, Universes etc. Evolving the tools and assigning parts of the 3.9 image, and much more." So that's our scope. Personally, I read this mainly as two interlinked goals: 1. To separate the code currently in the image into discrete sections with minimal and well understood dependencies on each other. 2. To structure our community so that it's feasible and robust to manage what Lex talks about as the "commons" of Squeak as many small pieces of code rather than as one large one. Note that these are both primarily organizational or social goals rather than technical ones: the point is to know where the boundaries between packages are, and to know (as a user) which packages to load at any given point, not the particular mechanisms that enable this. In particular, I'd like to leave discussions about how to represent individual packages either in the image or on disk to other groups (like the Modules) group - we have ways to do this already (PackageInfo, SAR, .mcz), and although they may not be the best ways, my guess is that they will work as well as any to meet our goals. Even if someone does have a reason why a different technical approach will significantly change the way in which we can organize code, or make us much more productive in doing so (it's conceivable that MC2 would fit into this category, for example), I suggest that we plan to continue using the status-quo packaging systems at first: the main work of defining boundaries should translate easily to any new technology that comes along, and so we shouldn't sit around waiting for it. Where I do think we'll need to develop or advocate new technology is to better manage packages as groups: the more our world is split into packages, the more we will need to be able to specify the dependencies and compatibilities between sets of packages or package versions, and we don't yet have anything widely used that does this. With Göran increasingly busy with other stuff, I don't know how much we should expect him to do on SqueakMap to address this, but I suspect that one role of the Packages team will be to slowly take over the maintainance of that part of Squeak's infrastructure. We can't ignore the social component here too: the major thing that SqueakMap has going for it is how accessible and thus how widely used it is, and we have to make sure not to jeopardize that. The other major technological and organizational decision I expect we'll have to make is how to reconcile the update stream with a packaged image: do we lose the update stream altogether? Do we make some updates optional (only applied if package X is loaded)? How do we convert from snapshots of package versions to changesets appropriate for the stream? And so on. I don't have any immediate answers, but there are definitely a lot of questions there. So a very rough draft of a plan might include: 1. using PackageInfo (and maybe tools like Spaghetti Tracer) to split the image into minimal dependency chunks, which would allow: 2. for as many packages as possible, unloading them and maintaining them purely on SqueakMap, which would require: 3. enhancing SqueakMap to model sets of compatible packages, and dependencies between them, which leaves: 4. for the remaining packages, building technology that lets us convert between a view of the image as a set of versioned packages (for developers), and a view of the image as a monolithic application with incremental updates (for endusers). Those are the things that come to mind, but these are early thoughts and I hope and expect to change them once others have posted theirs. Cheers, Avi

2 4

Re: user types
by Cees de Groot 06 Mar '05

06 Mar '05

On Sat, 5 Mar 2005 16:59:46 -0400, Lex Spoon <lex(a)cc.gatech.edu> wrote: > There are several familiar user types: > We call them 'personas', and we often attach a full bio and even a (random ;)) mugshot to them. So in your project you can just say "Jim" wouldn't like this, and everyone would know that this would apply to "Jim Frensby, 28, CS PhD, wrote his thesis on event horizon user interfaces and is currently working on further research in this area. [etcetera...]". By developing personas you get 'live' customers to think about instead of some abstract group. Might be useful for our community as well... It is very important to keep driving the point home that this is not just for Joe Hacker :)

1 0

user types
by Lex Spoon 05 Mar '05

05 Mar '05

Avi points out that a packaging commons needs to support social mechanisms -- and perhaps even engineer new ones. This observation has a lot of ramifications. It means there is not an obvious Right Solution. It means that version 1 may well not work the way it was expected, and thus that we need to be prepared to watch how the tools actually get used and adapt to what people want. Some features, painfully, might not turn out to be useful at all. We should be careful not to get so attached to our theories, that we refuse to recognize what the users are actually doing. But we have to start somewhere! Let me try and sketch some user types that seem nice to support. This is by no means a final list, but we have to have *some* working model. Please add anyone I left out, or adjust any types you think are off the mark. There are several familiar user types: 1. J. Hacker goes to www.squeak.org, gets hooked, fools around, and is excited enough to develop some code that they'd like to share with the other Squeak users. It's important to support J. Hacker's, because they are the main source of all this great code that we'd like to share around. 2. Fred Exec likes golfing but hates computer innards. If someone gives him a CD with SqGolf 3000 on it, then Fred should be able to use it and to apply updates as Avi described in the kickoff email message. Let's keep in mind some less familiar types, as well: 3. The Scratch project is a local work group within a research lab, and they'd like to build something cool in Squeak. They would like to use stable packages that have been posted to the 'net, but they do not need the bleeding edge. They plan to eventually share their project on the web, but they would like in the meantime to keep their ugly looking code to themselves. Nevertheless, they are multiple people, and they can greatly benefit from having the repository tools still work for them. 4. Joe Porter might want to port BottomFeeder from VisualWorks to Squeak. Our infrastructure should allow redistributing stuff which is primarily maintained somewhere else. 5. HP Labs has some cool Squeak code that only makes sense for use within the organization. It would be nice if HP lab rats can use the package repository infrastructure to share this code amongst themselves. They should be able to do this regardless of which other Squeak projects they are involved in; e.g., they should be able to collaborate with the Scratch guys using an image that has HP Lab packages loaded. Some initial implications on the tools, based on these user types: There should be some very easy publishing mechanism available, so that J. Hacker can get their code seen by people who are interested. There is a big difference between posting something on a web site, and posting it to something like SqueakMap. For mysterious reasons, packages in the former category seem likely to wither compared to the latter. There should be a way to have local repositories, so that the Scratch group can make progress. One implication of this is that identifiers of all kinds are going to have conflicts between the local repositories and the well-known global repositories. It looks like mixin repositories would be useful, in order to support HP Lab's internal packages. Along these lines, it would be ideal if it is possible to manage a package manually, without reference to a repository. It should be possible for one Squeaker to send a newly created package file to the Squeaker sitting beside them on another computer, without invoking lots of infrastructure. -Lex

1 0

user types
by Lex Spoon 05 Mar '05

05 Mar '05

1 0

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Packages