Hi guys!
Evidently it is dangerous not being online for a weekend. :) Anyway, I have read all posts to this list and would like to make some swift comments first:
1. If I read you right Avi, you suggest us sticking with PI as it stands today. I tend to agree, we will learn a lot through this project and we can always migrate to something like Alexandre's recent work afterwards. PI is being depended upon quite a lot right now so sticking to it, disregarding its warts, is the way to go at this time - we all know how it works and the tools work with it. Also, Ned has recently posted a "power pack" for PI including some split-changesets-based-on-PIs that I and he wrote independently. I strongly urge us to perhaps make that stuff an official package in Basic and get it in. :) Or just merge it into whatever places it affects (ChangeSorters etc, the PackageInfo package). Just as long as we "gear up" properly for our quest. :)
2. Your initial post didn't clearly express the idea that we can indeed "stake out" the image without having to actually "break out" parts of it. I think that piece of the puzzle is important. The TFNR idea was precisely that, meaning that it is better to have a tangled mess that is staked out, than to have a tangled mess. :)
So I would like for you to clarify your view on that. In other words - I would like for us to make a 100% stake out *first* (it can be made quite quick if we don't get too granular - let the PIs divide themselves later if their Stewards feel the need), and then *after* that start breaking out stuff. And sure, we may end up with a few "largish" PIs and perhaps even a "what is left in the bucket"-PI but so be it, then we at least have a *fully covered* image AND using the existing field in SM we can map each such PI to an SM package and thus we know *who* stewards it and what emails they have etc. And just by accident we can then (given Ned's power pack) use the changeset splitting mechanism to easily feed multi-package-covering-changesets to the maintainers.
Ok, I reread your post and perhaps you *do* indeed mean this - sorry if I misunderstood. :)
3. I intend to focus on SM now, it seems that we have managed to get ourselves organized pretty good and we do have a plan for the next few months in place. So *my* main Team is *this* team and my main job in this team is SM. :)
And regarding the future of SM:
1. Lex mentions a few needs currently not being addressed by SM. One is the ability to have "private" repos - or in other words to decentralize SM into multiple connected servers in some way. This has been in my plan for a long time but I have postponed it because I didn't want to limit the model because of it. I am all ears for making this happen, but not before dependencies or a few other good things. Then, when the model looks somewhat "complete" we can again try to fogure out how we distribute it. :) And also, Avi has a good point in that SqueakSource etc fills a "developer space" that SM doesn't - so the need is less today than it was before SqueakSource/Monticello I guess.
2. The ability to be able to create a "loose package" that can be sent to someone etc, is part of the above. I just want to say that the model in SM is quite interconnected, so these things aren't trivial to do if we want to keep some of the current features of SM. But again, I am interested in solving this somehow too.
3. Dependencies. I have said it before and don't want to sound like a broken record :), but I do have a working implementation in my dev image. I need to fix a few things and put a few UIs on it. I also should describe the implementation so that you can give feedback.
4. Universes. I intend to look into Lex's code again more carefully and try to figure out how to either: a) Make it work "with" SM. But this is only interesting if Lex or someone else wants to continue maintaining Universes and nor merging it with SM. b) Merge the concepts and mechanisms with SM so that the sum is greater than the parts.
Phew. Ok, now I read the final post from Avi again and... well, I still wonder a bit about the "stake out first" contra "break out".
But anyway, a great start guys!
regards, Göran
On Mon, 7 Mar 2005 08:58:56 +0100, goran.krampe@bluefish.se goran.krampe@bluefish.se wrote:
Evidently it is dangerous not being online for a weekend.
Ah, I wondered why squeak-dev was so quiet... ;)
- If I read you right Avi, you suggest us sticking with PI as it stands
today.
For now, yes.
Also, Ned has recently posted a "power pack" for PI including some split-changesets-based-on-PIs that I and he wrote independently.
http://map1.squeakfoundation.org/sm/package/fd191d06-6efe-4fde-ab0e-f1225059... Yes, that will be very useful for the partitioning work I'm sure.
- Your initial post didn't clearly express the idea that we can indeed
"stake out" the image without having to actually "break out" parts of it. I think that piece of the puzzle is important.
Agreed. I would put it this way: in most cases, it is more important to be able to save and update parts of the image as packages, than it is to be able to unload and load those same parts.
- Lex mentions a few needs currently not being addressed by SM. One is
the ability to have "private" repos - or in other words to decentralize SM into multiple connected servers in some way.
A small clarification, here: what I believe Lex and I were talking about was not "multiple connected servers", but multiple independent servers. The client should be able to integrate the information from these various servers, but I don't believe that we need or want the servers themselves to be aware of each other.
- Universes. I intend to look into Lex's code again more carefully and
try to figure out how to either: a) Make it work "with" SM. But this is only interesting if Lex or someone else wants to continue maintaining Universes and nor merging it with SM. b) Merge the concepts and mechanisms with SM so that the sum is greater than the parts.
Yes. One thing that I'm hoping will come out of this list is a better understanding of how to have one solution which has the advantages of both.
Avi
Hiya!
Avi Bryant avi.bryant@gmail.com wrote:
On Mon, 7 Mar 2005 08:58:56 +0100, goran.krampe@bluefish.se goran.krampe@bluefish.se wrote:
Evidently it is dangerous not being online for a weekend.
Ah, I wondered why squeak-dev was so quiet... ;)
:)
- If I read you right Avi, you suggest us sticking with PI as it stands
today.
For now, yes.
Good, because I agree. :)
Also, Ned has recently posted a "power pack" for PI including some split-changesets-based-on-PIs that I and he wrote independently.
http://map1.squeakfoundation.org/sm/package/fd191d06-6efe-4fde-ab0e-f1225059... Yes, that will be very useful for the partitioning work I'm sure.
Ah, right - he put it on SM, perfect.
- Your initial post didn't clearly express the idea that we can indeed
"stake out" the image without having to actually "break out" parts of it. I think that piece of the puzzle is important.
Agreed. I would put it this way: in most cases, it is more important to be able to save and update parts of the image as packages, than it is to be able to unload and load those same parts.
Exactly, and it is even good if we can just stake out parts and still maintain them using the update stream. Sure, if we stake them out (PI) then of course we have the ability to do both, depending on how we wish to do it. Anyway, my main point is that we shouldn't let the fact that a part is tightly intertwined stop us from staking out the image. Sure, some of the boundaries will be a bit loose - but we will still be able to map each method to a Steward. :)
- Lex mentions a few needs currently not being addressed by SM. One is
the ability to have "private" repos - or in other words to decentralize SM into multiple connected servers in some way.
A small clarification, here: what I believe Lex and I were talking about was not "multiple connected servers", but multiple independent servers. The client should be able to integrate the information from these various servers, but I don't believe that we need or want the servers themselves to be aware of each other.
Sure, but the problems are mostly the same - how to manage a complex object model in a distributed manner. :)
- Universes. I intend to look into Lex's code again more carefully and
try to figure out how to either: a) Make it work "with" SM. But this is only interesting if Lex or someone else wants to continue maintaining Universes and nor merging it with SM. b) Merge the concepts and mechanisms with SM so that the sum is greater than the parts.
Yes. One thing that I'm hoping will come out of this list is a better understanding of how to have one solution which has the advantages of both.
I am intent on making sure SM evolves in the best plausible way.
Avi
regards, Göran
goran.krampe@bluefish.se wrote:
Date: Mon, 7 Mar 2005 13:31:48 +0100 From: goran.krampe@bluefish.se Subject: Re: Feedback To: packages@discuss.squeakfoundation.org mailing-list: contact packages-help@discuss.squeakfoundation.org; run by ezmlm
Hiya!
Avi Bryant avi.bryant@gmail.com wrote:
On Mon, 7 Mar 2005 08:58:56 +0100, goran.krampe@bluefish.se goran.krampe@bluefish.se wrote:
Evidently it is dangerous not being online for a weekend.
Ah, I wondered why squeak-dev was so quiet... ;)
:)
- If I read you right Avi, you suggest us sticking with PI as it stands
today.
For now, yes.
Good, because I agree. :)
Also, Ned has recently posted a "power pack" for PI including some split-changesets-based-on-PIs that I and he wrote independently.
http://map1.squeakfoundation.org/sm/package/fd191d06-6efe-4fde-ab0e-f1225059... Yes, that will be very useful for the partitioning work I'm sure.
Ah, right - he put it on SM, perfect.
- Your initial post didn't clearly express the idea that we can indeed
"stake out" the image without having to actually "break out" parts of it. I think that piece of the puzzle is important.
Agreed. I would put it this way: in most cases, it is more important to be able to save and update parts of the image as packages, than it is to be able to unload and load those same parts.
Exactly, and it is even good if we can just stake out parts and still maintain them using the update stream. Sure, if we stake them out (PI) then of course we have the ability to do both, depending on how we wish to do it. Anyway, my main point is that we shouldn't let the fact that a part is tightly intertwined stop us from staking out the image. Sure, some of the boundaries will be a bit loose - but we will still be able to map each method to a Steward. :)
- Lex mentions a few needs currently not being addressed by SM. One is
the ability to have "private" repos - or in other words to decentralize SM into multiple connected servers in some way.
A small clarification, here: what I believe Lex and I were talking about was not "multiple connected servers", but multiple independent servers. The client should be able to integrate the information from these various servers, but I don't believe that we need or want the servers themselves to be aware of each other.
Sure, but the problems are mostly the same - how to manage a complex object model in a distributed manner. :)
Yes, I was thinking that you probably have to have the servers be indepedent, at least in one direction. You probably can't have the local server being *required* to make round trips to the global server. Only probably, though. I have not thought about it enough to close the door on that design possibility entirely. Mostly, it just seems right that the local repository should be entirely under its own control, and that we should be very careful about what requirements we place on it.
Being disconnected does change the design problems in one way: unique identifiers can no longer be guaranteed. One general strategy for approaching this, is that if the local guy defines an identifier identical to what the global server has, then the local guy is *overriding* the global server -- just like with class inheritance.
Lex
"Lex Spoon" lex@cc.gatech.edu wrote: [SNIP]
A small clarification, here: what I believe Lex and I were talking about was not "multiple connected servers", but multiple independent servers. The client should be able to integrate the information from these various servers, but I don't believe that we need or want the servers themselves to be aware of each other.
Sure, but the problems are mostly the same - how to manage a complex object model in a distributed manner. :)
Yes, I was thinking that you probably have to have the servers be indepedent, at least in one direction. You probably can't have the local server being *required* to make round trips to the global server. Only probably, though. I have not thought about it enough to close the door on that design possibility entirely. Mostly, it just seems right that the local repository should be entirely under its own control, and that we should be very careful about what requirements we place on it.
I agree, one design I have somewhere in the back of my head goes like this:
- Add code to SMAccount so that it can disconnect itself from the rest of the model (replacing the objects with proxies with UUIDs in them) and then be able to reconnect and resolve the UUIDs. This means a disconnected SMAccount can be serialized and sent to a server or downloaded from it.
- Skip the current ImageSegment code (well, we could keep it as a good mechanism to get a full map of course) and instead save each account on its own on disk.
- A server can optionally know a "parent server". If it doesn't then it is on its own totally. If it does, it can regularly connect and download any modified accounts since last time.
- When you "commit" your changes you simply send your full SMAccount to a selected server. This means you make your modifications (creating releases, whatever) in your own image and then using a simple HTTP post or whatever, you smack your full account up on the server.
The above is pretty simple to do - we could start with a "hard" requirement that the SMAccount should be able to resolve all proxies or else it will abort. One downside with the above is that you would typically need one account for each server that you like to publish on. A likely scenario is 3-4 levels (global, department/company, local server, localhost) which would mean having separate accounts for these. I have been toying with going fine granular letting each SMObject know which server it "belongs" to, and thus letting you get away with having a single SMAccount - but when you commit it - it will filter itself appropriately and only send a partial copy without any SMObjects that don't belong for the given server.
Anyway, something like the above seems pretty doable.
Being disconnected does change the design problems in one way: unique identifiers can no longer be guaranteed. One general strategy for approaching this, is that if the local guy defines an identifier identical to what the global server has, then the local guy is *overriding* the global server -- just like with class inheritance.
Well, I think the model don't need that - all SMObjects have UUIDs, but sure, there are a few ways to "clash" today, for example - package name is verified to be unique. I was thinking that any such conflicts should be noticed at "commit time" and if there is a conflict we simply abort and tell the user.
Lex
regards, Göran
In general, Goran, this sounds like a plan that can work, if you want to do all that work! Let's just keep our eyes on the list of desiderata (local development, dependencies, and whatever else), but if they are all met that is wonderful.
Have you thought about mixin universes? Like, someone backports internationalization to 3.7 and sets up a local SqueakMap to hold the packages in it, while someone else creates a cryptography SqueakMap that holds code that can't be used or redistributed by Americans. Some images might want to have 3.7 + m18n + cryptography, and others might want 3.7 + cryptography, etc. For various reasons, these mixin repositories do seem to be useful in Debian, and so presumably they will be useful for us too if we support it. With the design strategy you describe, maybe this could be supported by having multiple parent pointers?
I agree, one design I have somewhere in the back of my head goes like this:
Sounds good to me, not that I know SqueakMap internals all that well....
- When you "commit" your changes you simply send your full SMAccount to
a selected server. This means you make your modifications (creating releases, whatever) in your own image and then using a simple HTTP post or whatever, you smack your full account up on the server.
So an account is the unit of interchange between servers. Well, I have no idea if that will work. If it does, terrific! Does this include support for the same account posting packages to a private server? You don't want to upload the local packages to the public server, but you may want to reuse the account. Alternatively, it may be reasonable to make people have different accounts on different servers.
Incidentally, I organized Universes so that HTTP is used for retrieving copies of the universe, but a maintained TCP connection is used for editing. This way, it opens the door to negotiations, as Craig would say, between the editing image and the server.
But if HTTP POST's are enough, you don't need that. I prefered to avoid having to constrain the protocol that way.
Anyway, something like the above seems pretty doable.
Cool!
Being disconnected does change the design problems in one way: unique identifiers can no longer be guaranteed. One general strategy for approaching this, is that if the local guy defines an identifier identical to what the global server has, then the local guy is *overriding* the global server -- just like with class inheritance.
Well, I think the model don't need that - all SMObjects have UUIDs, but sure, there are a few ways to "clash" today, for example - package name is verified to be unique. I was thinking that any such conflicts should be noticed at "commit time" and if there is a conflict we simply abort and tell the user.
Clashes seem like a fine strategies as well. Notice that they also seem to happen whenever you do an "update" from a more public server; changes that have been committed locally, may turn out to clash with something that happened upstream. Going the clashes route, it seems like the model now needs to have support for "stuff killed off by an update from upstream", plus some way for the local users to fix up the situation.
I guess another general approach, is to try and rename the UUID's locally, in order to undo the conflict. I don't know if this is doable or not.
And (heck) a fourth, is that you change from UUID's to server-unique ID's. So when you refer to an ID, you have to know which server it pertains to.
-Lex
Hi Lex!
"Lex Spoon" lex@cc.gatech.edu wrote:
In general, Goran, this sounds like a plan that can work, if you want to do all that work! Let's just keep our eyes on the list of desiderata
Well, I have intended something like this for a long time, I just have postponed it. :)
(local development, dependencies, and whatever else), but if they are all met that is wonderful.
Have you thought about mixin universes? Like, someone backports internationalization to 3.7 and sets up a local SqueakMap to hold the packages in it, while someone else creates a cryptography SqueakMap that holds code that can't be used or redistributed by Americans. Some images might want to have 3.7 + m18n + cryptography, and others might want 3.7 + cryptography, etc. For various reasons, these mixin repositories do seem to be useful in Debian, and so presumably they will be useful for us too if we support it. With the design strategy you describe, maybe this could be supported by having multiple parent pointers?
Yeah, I have thought about them - and yes, possibly. I do think that would make things more complex though - but at the moment I haven't had my morning coffee so I will simply ignore it. :)
Perhaps we could postpone mixins until we have the rest working? With the intention of trying to support them, but waiting until we have the rest operating I mean.
I agree, one design I have somewhere in the back of my head goes like this:
Sounds good to me, not that I know SqueakMap internals all that well....
The internals in SM aren't that magical. :)
- When you "commit" your changes you simply send your full SMAccount to
a selected server. This means you make your modifications (creating releases, whatever) in your own image and then using a simple HTTP post or whatever, you smack your full account up on the server.
So an account is the unit of interchange between servers. Well, I have
Yes, I know it has its problems - it may be tempting to go more fine granular - but on the other hand that is the granularity of ownership, so it is natural. You own your account and all in there - it is like a document kinda. I know it has issues as you point out, see below...
no idea if that will work. If it does, terrific! Does this include support for the same account posting packages to a private server? You don't want to upload the local packages to the public server, but you may want to reuse the account. Alternatively, it may be reasonable to make people have different accounts on different servers.
I know - first I thought "ouch" and that hey, they will just have to use separate accounts - but that hurts when you start thinking about it. An alternative is this:
In essence an SMAccount has a couple of fields and then a collection of SMObjects. That is more or less all to it. Then SMObject of course have subclasses like SMPackage etc. One idea is to mark each SMObject with a "home server" tag. And when you commit your account you need to specify to which servers (one or more). Then it:
1. Create a copy of the account disconnected from the map (replacing all forreign refs with UUID proxies). 2. For each server to commit to: 2.1 Deepcopy the copy. 2.2 Filter the list of SMObjects based on the server to commit to. 2.3 Serialize the account using ReferenceStream and friends, gzip and POST.
...this way you would only have one account no matter how many servers you participate on.
Incidentally, I organized Universes so that HTTP is used for retrieving copies of the universe, but a maintained TCP connection is used for editing. This way, it opens the door to negotiations, as Craig would say, between the editing image and the server.
Just a note regarding networking, I have always thought SM should continue to use simple HTTP in order to always work behind firewalls etc - but one really nice thing lately is Cees' p2p stuff. I don't know how it can affect us (SM, Universes) but it sure does have tons of interesting mechanisms. :)
But if HTTP POST's are enough, you don't need that. I prefered to avoid having to constrain the protocol that way.
Anyway, something like the above seems pretty doable.
Cool!
Being disconnected does change the design problems in one way: unique identifiers can no longer be guaranteed. One general strategy for approaching this, is that if the local guy defines an identifier identical to what the global server has, then the local guy is *overriding* the global server -- just like with class inheritance.
Well, I think the model don't need that - all SMObjects have UUIDs, but sure, there are a few ways to "clash" today, for example - package name is verified to be unique. I was thinking that any such conflicts should be noticed at "commit time" and if there is a conflict we simply abort and tell the user.
Clashes seem like a fine strategies as well. Notice that they also seem to happen whenever you do an "update" from a more public server; changes that have been committed locally, may turn out to clash with something that happened upstream.
Well, in general systems yes :) but in SM, no, not really. Because locally you will only modify your own account - modifying other parts of the map is a general "no, no". If you do that, then you are on your own IMHO. :)
Going the clashes route, it seems like the model now needs to have support for "stuff killed off by an update from upstream", plus some way for the local users to fix up the situation.
Yes. Typical issues when committing are:
- You created a package (or some other artifact with unique names) with a name that already exists. - Your SMObjects refer to other SMObjects that do not exist anymore (categories could be removed, package releases also etc, etc)
...but without giving it tons of thought I do think the above two things are mostly it.
I guess another general approach, is to try and rename the UUID's locally, in order to undo the conflict. I don't know if this is doable or not.
Hmmm. Not sure I grok that. :)
And (heck) a fourth, is that you change from UUID's to server-unique ID's. So when you refer to an ID, you have to know which server it pertains to.
Well, I might have jumbled up my thoughts - but given the above design - the SMAccount that you are committing to a server will only contain SMObjects that think they "belong" there so all other SMObjects they refer to should be available on that server too.
Which means that, perhaps I didn't write it, but the server marker in each SMObject makes sure it has a "home server" in the server "tree", and it will only be able to refer to other SMObjects available on that server (which includes all servers above it in the tree all the way up to the master server on sqf). That restriction would be quite easy to maintain in the client image when you are modifying your account (since all SMObjects know their home).
So again, the tree of servers with each node down the tree only adding SMObjects makes these things simpler. And yes, mixins would probably make thing more complex :), not sure.
One way of making mixins simpler might be to restrict mixins to be a leaf thing. I mean, the servers are all in a tree like I describe but the leaf (localhost) could possibly use mixins and take the heat of that locally. But again, I would gladly wait with mixins until later.
-Lex
regards, Göran
goran.krampe@bluefish.se wrote:
- Universes. I intend to look into Lex's code again more carefully and
try to figure out how to either: a) Make it work "with" SM. But this is only interesting if Lex or someone else wants to continue maintaining Universes and nor merging it with SM. b) Merge the concepts and mechanisms with SM so that the sum is greater than the parts.
For the record, Universes is under maintenance and in use, and (no surprise) it achieves all of the properties I mentioned in my opening email.
One stable universe is already released (3.7), and, if there is no replacement process by then, I intended to take the lead on making more stable releases, for whoever wants their code to be in such a release. I won't do it for 3.8 though, because my life is insanely busy right now!
-Lex
Hi Lex!
"Lex Spoon" lex@cc.gatech.edu wrote:
goran.krampe@bluefish.se wrote:
- Universes. I intend to look into Lex's code again more carefully and
try to figure out how to either: a) Make it work "with" SM. But this is only interesting if Lex or someone else wants to continue maintaining Universes and nor merging it with SM. b) Merge the concepts and mechanisms with SM so that the sum is greater than the parts.
For the record, Universes is under maintenance and in use, and (no surprise) it achieves all of the properties I mentioned in my opening email.
(btw, it should be "not" and not "nor" above under a))
Ok, so what is your view on this then? Am I to interpret you that you wish me to go for a) ? Meaning integration instead of merge of concepts?
I do have an idea that we could add "Universe" as a new "type" of thing on SM, just as I would like to add "image" as another type, and possibly even "repository" (Monticello).
I want to work for us to end up in the best of worlds - and I need you to work with me here.
regards, Göran
On Wed, 9 Mar 2005 15:51:26 +0100, goran.krampe@bluefish.se goran.krampe@bluefish.se wrote:
I do have an idea that we could add "Universe" as a new "type" of thing on SM, just as I would like to add "image" as another type, and possibly even "repository" (Monticello).
That's a very interesting thought; as I understand it, the goal of SqueakMap was always to "map" the Squeak community in general, rather than to be a list of packages. I think this is a great goal, and one we should try to keep sight of. The question is, what are the most interesting things to map? At first, packages were an obvious choice, but I think Lex makes a very compelling argument that as we move forward we should be thinking in terms of sets of package versions rather than in terms of individual packages. In light of that, what if we migrated SqueakMap towards keeping a list of package Universes rather than a list of packages? It would remain in its familiar and extremely useful status as the gateway to the Squeak world, but would delegate the responsibility for package-level granularity to the individual Universes.
Avi
Hi!
Avi Bryant avi.bryant@gmail.com wrote:
On Wed, 9 Mar 2005 15:51:26 +0100, goran.krampe@bluefish.se goran.krampe@bluefish.se wrote:
I do have an idea that we could add "Universe" as a new "type" of thing on SM, just as I would like to add "image" as another type, and possibly even "repository" (Monticello).
That's a very interesting thought; as I understand it, the goal of SqueakMap was always to "map" the Squeak community in general, rather than to be a list of packages. I think this is a great goal, and one
Exactly.
we should try to keep sight of. The question is, what are the most interesting things to map? At first, packages were an obvious choice, but I think Lex makes a very compelling argument that as we move forward we should be thinking in terms of sets of package versions rather than in terms of individual packages. In light of that, what
I agree that "sets of package versions" or Universes is a nice concept. I still believe that it isn't the *only* way of managing dependencies - but it sure is *one* way of doing it.
if we migrated SqueakMap towards keeping a list of package Universes rather than a list of packages? It would remain in its familiar and extremely useful status as the gateway to the Squeak world, but would delegate the responsibility for package-level granularity to the individual Universes.
Eh, hmm. That would indeed make Universes the *only* way of managing for example dependencies, and I do have another way in mind. What I instead was thinking was this:
In the next SM I am introducing SMResource which is an unversioned "thing" on SM (compared to SMPackage which keeps track of its releases). So an SMResource will also have a URL and various other attributes like name, description etc, but only keep track of the "current" version using typically a version field (much like SM1 did).
A Universe could then be such an SMResource. And if you install a Universe it simply means downloading/installing the current version of it. Since a Universe basically is a list of references to SMPackageReleases (or could be at least) and their direct deps I don't think it would be practical nor suitable to keep track of every "release" of the Universe itself (=every time it is changed) - thus the idea to model it as an SMResource.
I haven't had time to look at Lex's Universe code yet - so I don't know if he uses SM to get to the actual releases, but anyway, the above scheme doesn't really care what a Universe *is* - it will just be something that SM can download (a URL) and then fire up an installer on. An SMResource could use an SHA to be able to know if it has indeed changed since it was last downloaded - or it can be triggered by the version "field" having changed.
I am still unsure if Lex is interested in merging Universes with SM or as I describe above we simply make SM able to map them and get to them. If Lex wants to continue with Universes totally separate from SM (meaning no merging) then I will simply make sure SM handles them as I always want SM to handle stuff in the Squeak community - it would then seem very stupid to add a similar mechanism to SM because then we would have two such competing mechanisms.
Avi
regards, Göran
On Thu, 10 Mar 2005 13:55:48 +0100, goran.krampe@bluefish.se goran.krampe@bluefish.se wrote:
I am still unsure if Lex is interested in merging Universes with SM or as I describe above we simply make SM able to map them and get to them. If Lex wants to continue with Universes totally separate from SM (meaning no merging) then I will simply make sure SM handles them as I always want SM to handle stuff in the Squeak community - it would then seem very stupid to add a similar mechanism to SM because then we would have two such competing mechanisms.
Well, exactly: this is an area where it's unlikely for two competing systems to both survive for long. People don't want to register the same package in two different places, with two different dependency systems, etc. One of three things is going to have to happen:
1. We do a lot of work to merge the two concepts into a new package repository system, that combines the best of both SM and Universes. Frankly, I think this would be a bad idea; the two were designed with different goals and different visions, and I think I've seen enough of squeak-dev dynamics to know that we would spend more time arguing about how to do this merge than it was worth. 2. We do no work at all, and let the two of them fight it out evolutionarily: people will register their packages with one or the other (or, for a time anyway, maybe even both), and eventually one will fade away into disuse. But this would be a shame, because as I mentioned in (1), they really have quite different goals - they happen to overlap in the area of managing a list of Squeak packages, but that's the *only* purpose of Universes, and is (in terms of vision) really just an incidental part of SqueakMap. Which suggests, to me, 3. We split the responsibility: SqueakMap is the "map of the Squeak world", which lists all the various resources like universes, repositories, people, websites, images, whatever. Universes are where you go to actually register a package. Each of them does what they do best, and there's a nice, well defined relationship between them.
Now, it's maybe it's a bad idea for me to use the names of the existing systems when describing (3) - clearly, SqueakMap in that scenario would look very different from how it does now (even if it were fairly similar internally), and Universes would have to change as well. Really, what I'm proposing is that we have *a* central, public system that holds metadata about resources like repositories (both development and release) and user accounts (ideally shared by all the various Squeak systems), and *a* system of distributed repositories in which package versions get registered. SqueakMap and Universes seem, respectively, like great starting points for these, but if (for example) you really don't like the Universes dependency model and want to propose that this fictional package repository system use another one instead, that would certainly fit within the scope of what I'm suggesting.
Avi
Hi!
Avi Bryant avi.bryant@gmail.com wrote:
On Thu, 10 Mar 2005 13:55:48 +0100, goran.krampe@bluefish.se goran.krampe@bluefish.se wrote:
I am still unsure if Lex is interested in merging Universes with SM or as I describe above we simply make SM able to map them and get to them. If Lex wants to continue with Universes totally separate from SM (meaning no merging) then I will simply make sure SM handles them as I always want SM to handle stuff in the Squeak community - it would then seem very stupid to add a similar mechanism to SM because then we would have two such competing mechanisms.
Well, exactly: this is an area where it's unlikely for two competing systems to both survive for long. People don't want to register the same package in two different places, with two different dependency systems, etc.
I agree fully.
One of three things is going to have to happen:
- We do a lot of work to merge the two concepts into a new package
repository system, that combines the best of both SM and Universes. Frankly, I think this would be a bad idea; the two were designed with different goals and different visions, and I think I've seen enough of
Not *that* different.
squeak-dev dynamics to know that we would spend more time arguing about how to do this merge than it was worth.
Well, perhaps. :) It mainly depends on the willingness of me and Lex IMHO. I just looked over the latest Universes source briefly and note that there is a LOT of overlap, IMHO a merge would do both systems some good - but again, this depends on what Lex wants.
- We do no work at all, and let the two of them fight it out
evolutionarily: people will register their packages with one or the other (or, for a time anyway, maybe even both), and eventually one will fade away into disuse. But this would be a shame, because as I mentioned in (1), they really have quite different goals - they happen to overlap in the area of managing a list of Squeak packages, but that's the *only* purpose of Universes, and is (in terms of vision) really just an incidental part of SqueakMap. Which suggests, to me, 3. We split the responsibility: SqueakMap is the "map of the Squeak world", which lists all the various resources like universes, repositories, people, websites, images, whatever. Universes are where you go to actually register a package. Each of them does what they do best, and there's a nice, well defined relationship between them.
Well, I have a variant 4 in mind that is a bit different. :)
Now, it's maybe it's a bad idea for me to use the names of the existing systems when describing (3) - clearly, SqueakMap in that scenario would look very different from how it does now (even if it were fairly similar internally), and Universes would have to change as well. Really, what I'm proposing is that we have *a* central, public system that holds metadata about resources like repositories (both development and release) and user accounts (ideally shared by all the various Squeak systems), and *a* system of distributed repositories in which package versions get registered. SqueakMap and Universes seem, respectively, like great starting points for these, but if (for example) you really don't like the Universes dependency model and want to propose that this fictional package repository system use another one instead, that would certainly fit within the scope of what I'm suggesting.
Well, I like choice - as long as it dosn't harm us. :) So I want people to be able to use Universes easily, but I also want to explore my dependency model etc, I don't want to give that up.
So my proposal is something like this (and of course, it all depends on Lex): - Make SM aware of other things like images, universes, repositories etc. - Still keep SM as the main catalog of packages and releases of them. - Make the Universes code use SM for the packages/releases. Looking at UPackage for example it is very much overlapping SMPackage. There may of course be issues with this approach but I would like to discuss it at least.
This would mean that there is one place to register a package or a release thereof. But then, you would be able to find the universes in SM, and be able to add a package etc to them instead of having to reregister the same data.
Something like that. But now I would like to know more about Lex feelings on the subject before discussing it further.
Avi
regards, Göran
On Thu, 10 Mar 2005 16:08:30 +0100, goran.krampe@bluefish.se goran.krampe@bluefish.se wrote:
So my proposal is something like this (and of course, it all depends on Lex): - Make SM aware of other things like images, universes, repositories etc. - Still keep SM as the main catalog of packages and releases of them. - Make the Universes code use SM for the packages/releases. Looking at UPackage for example it is very much overlapping SMPackage. There may of course be issues with this approach but I would like to discuss it at least.
So, this would mean that a Universe would be an overlay on top of SM that:
- For each package in SM, specified exactly 0 or 1 versions - For each package, specified its dependencies on other packages (no versions needed here, since there's only one version visible anyway) - Starts out open to modification (by who?), but can be locked, after which it doesn't ever change - Doesn't have to reside on the central SqueakMap server
Lex, does that capture the main points? Oh, and I think also:
- Provides a reliable cache of the actual package. Which is, I think, something SM was planning to add anyway, right?
Note that I'm ignoring the issue of private packages (vs. private configurations of packages) here...
Avi
Hi!
Avi Bryant avi.bryant@gmail.com wrote:
On Thu, 10 Mar 2005 16:08:30 +0100, goran.krampe@bluefish.se goran.krampe@bluefish.se wrote:
So my proposal is something like this (and of course, it all depends on Lex): - Make SM aware of other things like images, universes, repositories etc. - Still keep SM as the main catalog of packages and releases of them. - Make the Universes code use SM for the packages/releases. Looking at UPackage for example it is very much overlapping SMPackage. There may of course be issues with this approach but I would like to discuss it at least.
So, this would mean that a Universe would be an overlay on top of SM that:
- For each package in SM, specified exactly 0 or 1 versions
Yes, that is one way of putting it. :)
- For each package, specified its dependencies on other packages (no
versions needed here, since there's only one version visible anyway)
Yes.
- Starts out open to modification (by who?), but can be locked, after
which it doesn't ever change
That part doesn't affect SM - SM just holds a URL to the Universe - how the Universe is modified is up to Universes. So if you recall my thoughts about embedded resources etc (perhaps we haven't discussed that) a Universe would not be such an embedded resource - it would instead be an "external" resource living on its own URL.
- Doesn't have to reside on the central SqueakMap server
No, it can reside wherever it likes.
Lex, does that capture the main points? Oh, and I think also:
- Provides a reliable cache of the actual package. Which is, I think,
something SM was planning to add anyway, right?
I have it working 70% in my dev image, I was at home thinking about the final bits on that yesterday.
Note that I'm ignoring the issue of private packages (vs. private configurations of packages) here...
Me too - but a Universe is all or nothing - isn't it? I mean, if I can access a specific Universe - then I can access all packages in it, right? Otherwise I might have missed something.
Avi
regards, Göran
I have delayed my response due to outstanding levels of work at school. I'll be more responsive in two weeks!
The general strategy of having a set-of-packages as an item on SqueakMap sounds fine. That strategy solves the most important goals, which IMHO are in order:
1. have a way to manage sets of packages and thus produce stable releases
2. have a way to auto-load dependencies
Other achievements of Universes might be harder to map over, but even so, I can't say no to someone volunteering their time to keep this going, even if it loses some of the other nicities. Whenever thoes goals are met in SqueakMap, I'd be happy to help with migrating the existing stable and development universes over to the new mechanisms. While I think the universes model gives an impressive number of properties for its simplicity, I can't say no to someone volunteering to spend their time on keeping this all going.
As one nitpick, maybe 1 version of each package seems fine for stable releases but might not be enough for a development universe. I find that as stable as Debian's "unstable" is, I still download an occasional broken package. It is nice if the model allows more than one package to be available so that people can back up in this case. Even so, Avi's general observation seems to remain: dependencies are simpler, due to there being very few versions of each package to choose from.
-Lex
PS -- a number of people have objected to the name "universe". Feel free to pick a different name, Goran, especially if the thing it names is somewhat different from a classical package universe.
Hi all!
"Lex Spoon" lex@cc.gatech.edu wrote:
I have delayed my response due to outstanding levels of work at school. I'll be more responsive in two weeks!
The general strategy of having a set-of-packages as an item on SqueakMap sounds fine. That strategy solves the most important goals, which IMHO are in order:
- have a way to manage sets of packages and thus produce stable
releases
- have a way to auto-load dependencies
Right.
Other achievements of Universes might be harder to map over, but even so, I can't say no to someone volunteering their time to keep this going, even if it loses some of the other nicities. Whenever thoes
I will study Universes first.
goals are met in SqueakMap, I'd be happy to help with migrating the existing stable and development universes over to the new mechanisms.
Goodie. Will see, perhaps I can automate that when that time comes.
While I think the universes model gives an impressive number of properties for its simplicity, I can't say no to someone volunteering to spend their time on keeping this all going.
Goodie :)
As one nitpick, maybe 1 version of each package seems fine for stable releases but might not be enough for a development universe. I find that as stable as Debian's "unstable" is, I still download an occasional broken package. It is nice if the model allows more than one package to be available so that people can back up in this case. Even so, Avi's general observation seems to remain: dependencies are simpler, due to there being very few versions of each package to choose from.
-Lex
PS -- a number of people have objected to the name "universe". Feel free to pick a different name, Goran, especially if the thing it names is somewhat different from a classical package universe.
Universe is quite fine with me I think.
Anyway, this sounds good - then I consider myself to have a green light to move forward. Great!
regards, Göran
packages@lists.squeakfoundation.org
-
Avi Bryant -
goran.krampe＠bluefish.se -
Lex Spoon