On 6/27/05, Alexandre Bergel bergel@iam.unibe.ch wrote:
But the Project 39a is still empty...
Ah, yes, it is. We do need to commit stuff there. I just didn't understand the purpose of a script that created new packages when we already have a changeset from Andreas that does so.
I'll populate that project shortly.
I have many changes related to the system change notification that are waiting...
Great.
Avi
On Mon, 27 Jun 2005 17:15:16 +0200, "Avi Bryant" avi.bryant@gmail.com said:
On 6/27/05, Alexandre Bergel bergel@iam.unibe.ch wrote:
But the Project 39a is still empty...
Ah, yes, it is. We do need to commit stuff there. I just didn't understand the purpose of a script that created new packages when we already have a changeset from Andreas that does so.
Right, we can just use Andreas' changeset to do this partitioning. (which is very similar to Alexandre's script anyway, I think)
I'll populate that project shortly.
Sounds good. By the way, does a SqueakSource "Developer" have access to do this initial populating (adding new packages), or do you need to be an Admin?
I have many changes related to the system change notification that are waiting...
Great.
Yes, that should give the new process a good workout. Hmm, how many packages do these change-notification changes cut across? :-)
Realistically, it's going to take us a little while to get everything working smoothly with this new process. But we might as well dive in and get started. It's a big change, but I think it's flexible enough that we will be able to adjust the process if necessary, and we won't end up in a 3.3alpha modules situation.
I will set up a new swiki page covering the new process, as a point of reference.
- Doug
On 6/27/05, Doug Way dway@mailcan.com wrote:
I'll populate that project shortly.
Sounds good. By the way, does a SqueakSource "Developer" have access to do this initial populating (adding new packages), or do you need to be an Admin?
You don't have to be an admin (and I should probably add some others as admins anyway, no reason to have me be a bottleneck).
However, I just tried to upload the lot and found that there's still something messed up with the squeaksource installation. Argh. I'll look at this again after dinner.
I will set up a new swiki page covering the new process, as a point of reference.
Great.
Avi
I will set up a new swiki page covering the new process, as a point of reference.
Yes, this is important. Doug, I can help if you want.
I still think that the mailing list email should be used when answering. Who can configure it ? Or better question, where can it be configured...
Alexandre
Am 27.06.2005 um 18:12 schrieb Alexandre Bergel:
I still think that the mailing list email should be used when answering. Who can configure it ? Or better question, where can it be configured...
You can "configure" that in your mail client. I see you use Mutt. Press L to reply to the list.
See http://www.unicom.com/pw/reply-to-harmful.html
- Bert -
On 6/27/05, Avi Bryant avi.bryant@gmail.com wrote:
However, I just tried to upload the lot and found that there's still something messed up with the squeaksource installation. Argh. I'll look at this again after dinner.
Just FYI, I'm currently taking the SqueakSource instance down for a bit while I try (again) to sort this out...
Avi
On 6/27/05, Avi Bryant avi.bryant@gmail.com wrote:
On 6/27/05, Avi Bryant avi.bryant@gmail.com wrote:
However, I just tried to upload the lot and found that there's still something messed up with the squeaksource installation. Argh. I'll look at this again after dinner.
Just FYI, I'm currently taking the SqueakSource instance down for a bit while I try (again) to sort this out...
Ok, it's back up, and I'm slowly feeding it packages. For now you'll have to access it at port 9090 (http://source.squeakfoundation.org:9090/). Cees, when you get a chance, can you change the ProxyPass lines in the apache config to drop the /seaside/ss part?
Avi
So, the first set of packages have been uploaded to the 39a project, and I've placed the image I committed them from to http://beta4.com/~avi/Squeak3.9a-Packages.zip (for now - maybe someone could drop this onto box1?). This should make a good starting point for anyone wanting to start feeding some new changes in.
Doug, we should start doing whatever needs to be done with the 3.9a update stream now...
Avi
On 6/27/05, Avi Bryant avi.bryant@gmail.com wrote:
So, the first set of packages have been uploaded to the 39a project, and I've placed the image I committed them from to http://beta4.com/~avi/Squeak3.9a-Packages.zip (for now - maybe someone could drop this onto box1?). This should make a good starting point for anyone wanting to start feeding some new changes in.
http://source.squeakfoundation.org/static/Squeak3.9a-Packages.zip - would that be alright as well? ;-)
The apache config is now moderately complex, but I think it all works... FLW...
On Mon, 27 Jun 2005 20:55:56 +0200, "Avi Bryant" avi.bryant@gmail.com said:
So, the first set of packages have been uploaded to the 39a project, and I've placed the image I committed them from to http://beta4.com/~avi/Squeak3.9a-Packages.zip (for now - maybe someone could drop this onto box1?). This should make a good starting point for anyone wanting to start feeding some new changes in.
Doug, we should start doing whatever needs to be done with the 3.9a update stream now...
Yes, I'm fiddling around with that now.
So, here are the possible updates that would get us started with the 3.9alpha update stream... (Starting with a fresh Squeak3.8-6665-basic image.)
- Update cs #1: Full Monticello changeset. For now, I'm just using Monticello-ar.245.mcz for this, which is the latest one in the Squeak3.8-Packages iSqueak image, because we want a version that works with MonticelloConfigurations. Or can we just use one of the base Monticello releases such as Monticello-avi.231 ? (Does the Reorg cs depend on a specific Monticello version?)
- Update cs #2: MonticelloConfigurations-ar.17 changeset.
- Update cs #3: Reorganize.3.8-6662.cs (from Andreas)
(Hmmm, what is in the Unnamed cs in the Squeak3.8-Packages image? There's a bunch of changes in there... this is different from the Reorg cs.)
- Update cs #4: The partitioning DoIt cs (from http://source.impara.de -> Projects -> iSqueak -> wiki).
Note that after I do these steps, the list of packages in the Monticello browser is somewhat different than in the Squeak3.8-Packages image, this may need some cleanup? Not sure where Connectors and some of the other packages came from. See attached screenshot.
Part of this is that we just need to set the release repository (to 'http://source.squeakfoundation.org/39a') for the 35 packages. I assume this can be done with a DoIt. (And eventually we should also set master repositories for these packages...)
Hopefully we can do all of this cleanup through updates, so that people can continue to follow the update stream from a 3.8 final image if they want. Otherwise, we'll have to just tell everyone to download a new 3.9alpha partitioned image, which wouldn't be the end of the world, but it'd be nice to avoid that.
- Doug
It makes sense to me. Who set it up ? Let us know once done, I will test.
Alexandre
On Tue, Jun 28, 2005 at 06:21:33PM -0400, Doug Way wrote:
On Mon, 27 Jun 2005 20:55:56 +0200, "Avi Bryant" avi.bryant@gmail.com said:
So, the first set of packages have been uploaded to the 39a project, and I've placed the image I committed them from to http://beta4.com/~avi/Squeak3.9a-Packages.zip (for now - maybe someone could drop this onto box1?). This should make a good starting point for anyone wanting to start feeding some new changes in.
Doug, we should start doing whatever needs to be done with the 3.9a update stream now...
Yes, I'm fiddling around with that now.
So, here are the possible updates that would get us started with the 3.9alpha update stream... (Starting with a fresh Squeak3.8-6665-basic image.)
- Update cs #1: Full Monticello changeset. For now, I'm just using
Monticello-ar.245.mcz for this, which is the latest one in the Squeak3.8-Packages iSqueak image, because we want a version that works with MonticelloConfigurations. Or can we just use one of the base Monticello releases such as Monticello-avi.231 ? (Does the Reorg cs depend on a specific Monticello version?)
Update cs #2: MonticelloConfigurations-ar.17 changeset.
Update cs #3: Reorganize.3.8-6662.cs (from Andreas)
(Hmmm, what is in the Unnamed cs in the Squeak3.8-Packages image? There's a bunch of changes in there... this is different from the Reorg cs.)
- Update cs #4: The partitioning DoIt cs (from http://source.impara.de
-> Projects -> iSqueak -> wiki).
Note that after I do these steps, the list of packages in the Monticello browser is somewhat different than in the Squeak3.8-Packages image, this may need some cleanup? Not sure where Connectors and some of the other packages came from. See attached screenshot.
Part of this is that we just need to set the release repository (to 'http://source.squeakfoundation.org/39a') for the 35 packages. I assume this can be done with a DoIt. (And eventually we should also set master repositories for these packages...)
Hopefully we can do all of this cleanup through updates, so that people can continue to follow the update stream from a 3.8 final image if they want. Otherwise, we'll have to just tell everyone to download a new 3.9alpha partitioned image, which wouldn't be the end of the world, but it'd be nice to avoid that.
- Doug
On 6/29/05, Doug Way dway@mailcan.com wrote:
- Update cs #1: Full Monticello changeset. For now, I'm just using
Monticello-ar.245.mcz for this, which is the latest one in the Squeak3.8-Packages iSqueak image, because we want a version that works with MonticelloConfigurations. Or can we just use one of the base Monticello releases such as Monticello-avi.231 ? (Does the Reorg cs depend on a specific Monticello version?)
We definitely want one of the Impara versions, for the configuration support.
- Update cs #4: The partitioning DoIt cs (from http://source.impara.de
-> Projects -> iSqueak -> wiki).
Note that after I do these steps, the list of packages in the Monticello browser is somewhat different than in the Squeak3.8-Packages image, this may need some cleanup? Not sure where Connectors and some of the other packages came from. See attached screenshot.
I think those come from cs #4, right? Do we need to run that? I'm not sure we want all of those automatically created packages in there.
Hopefully we can do all of this cleanup through updates, so that people can continue to follow the update stream from a 3.8 final image if they want. Otherwise, we'll have to just tell everyone to download a new 3.9alpha partitioned image, which wouldn't be the end of the world, but it'd be nice to avoid that.
The other thing we'll need to generate will be a changeset that provides the ancestry metadata for all of the initial package versions, so that the MC working copies will be in the same state as they would have been if all the packages had been loaded from the repository (which will take a long time and we don't want to put people through if we don't have to).
Avi
On Wednesday, June 29, 2005, at 08:20 AM, Avi Bryant wrote:
On 6/29/05, Doug Way dway@mailcan.com wrote:
- Update cs #1: Full Monticello changeset. For now, I'm just using
Monticello-ar.245.mcz for this, which is the latest one in the Squeak3.8-Packages iSqueak image, because we want a version that works with MonticelloConfigurations. Or can we just use one of the base Monticello releases such as Monticello-avi.231 ? (Does the Reorg cs depend on a specific Monticello version?)
We definitely want one of the Impara versions, for the configuration support.
Ok, I'll go with the versions I mentioned for cs #1 and #2 then.
- Update cs #4: The partitioning DoIt cs (from http://source.impara.de
-> Projects -> iSqueak -> wiki).
Note that after I do these steps, the list of packages in the Monticello browser is somewhat different than in the Squeak3.8-Packages image, this may need some cleanup? Not sure where Connectors and some of the other packages came from. See attached screenshot.
I think those come from cs #4, right? Do we need to run that? I'm not sure we want all of those automatically created packages in there.
Yes, but I thought cs #4 did all of the other PI-creation as well, so I assume it is necessary. I.e. it creates the Kernel, Collections, etc MC packages from the class categories. Installing MC doesn't do this, does it? The Reorg cs does not do it, it only shuffles class categories around.
Anyway, I can go through the steps again with my image tomorrow (which I don't have with me at the moment), and see where these extra packages are coming from... I should be able to figure this out.
Hopefully we can do all of this cleanup through updates, so that people can continue to follow the update stream from a 3.8 final image if they want. Otherwise, we'll have to just tell everyone to download a new 3.9alpha partitioned image, which wouldn't be the end of the world, but it'd be nice to avoid that.
The other thing we'll need to generate will be a changeset that provides the ancestry metadata for all of the initial package versions, so that the MC working copies will be in the same state as they would have been if all the packages had been loaded from the repository (which will take a long time and we don't want to put people through if we don't have to).
Ok, that would be good. Mm, any suggestions on how I should do this? :-) Of course, most of the packages are really the "initial" package versions as you say, so they don't really have any "ancestry", right? E.g. we would have Kernel version 1. But there may be something missing that I should add in any case... as if it were loaded from the repository, I agree.
- Doug
Am 30.06.2005 um 06:02 schrieb Doug Way:
On Wednesday, June 29, 2005, at 08:20 AM, Avi Bryant wrote:
On 6/29/05, Doug Way dway@mailcan.com wrote:
- Update cs #1: Full Monticello changeset. For now, I'm just using
Monticello-ar.245.mcz for this, which is the latest one in the Squeak3.8-Packages iSqueak image, because we want a version that works with MonticelloConfigurations. Or can we just use one of the base Monticello releases such as Monticello-avi.231 ? (Does the Reorg cs depend on a specific Monticello version?)
We definitely want one of the Impara versions, for the configuration support.
Ok, I'll go with the versions I mentioned for cs #1 and #2 then.
I'd rather use the latest from http://source.impara.de/mc.html, we're at #261 / #30 now, which work fine.
- Bert -
Hi -
Some more comments:
Note that after I do these steps, the list of packages in the Monticello browser is somewhat different than in the Squeak3.8-Packages image, this may need some cleanup? Not sure where Connectors and some of the other packages came from. See attached screenshot.
I think those come from cs #4, right? Do we need to run that? I'm not sure we want all of those automatically created packages in there.
Depends on whether you wanted to have the system fully packagized or not. This is a key question - I have no interest whatsover in another "partially packaged" system. We have this already.
If you *are* interested in a fully packagized system then you need to make sure you know what every last line of code belongs to. The doIt does that by touching every method in the system and therefore pointing out (!) places that need to be fixed.
That's what this doIt is good for - it's NOT intended to perform proper packagizing it is intended to show which parts of the system are not included in your current package set and therefore need to be dealt with.
Since I did the work a while ago it is quite possible that later changes have done some modifications that I was unaware about when I did the work. I suggest you just reclassify these classes/methods too so that you have a fully packagized system.
Yes, but I thought cs #4 did all of the other PI-creation as well, so I assume it is necessary. I.e. it creates the Kernel, Collections, etc MC packages from the class categories. Installing MC doesn't do this, does it? The Reorg cs does not do it, it only shuffles class categories around.
The doIt "sort of" does this but *not* properly. To do it properly you want both, a repository that the package resides on and a version of the package. Here is how to do this:
a) Send out the Reorg.cs just to get things in order for people out there (this includes any additional modifications that need to be made) b) Do whatever is necessary to get Monticello/MCConfs to the people out there (e.g., put them into the update stream or so) c) Post versions to the 3.9 repository that include all packages which should then be in a basic image d) Post a configuration that people out there get via update stream
If you do this, here is what will happen for a 3.9 user: * They get the ReOrg.cs * They a configuration map * This will force them to download the versions from the repository * Since the repository version will be equivalent to what is in the system, no changes will be made
In other words, the above is a recipe to get the packages to the users at the expense of having to download all of these packages from the repository - which will take some time but it also means that (for once ;-) you know that people have precisely the versions that are in the repository.
Anyway, I can go through the steps again with my image tomorrow (which I don't have with me at the moment), and see where these extra packages are coming from... I should be able to figure this out.
Almost always they are method extensions which originally belonged to some package but where posted with other fixes.
Hopefully we can do all of this cleanup through updates, so that people can continue to follow the update stream from a 3.8 final image if they want. Otherwise, we'll have to just tell everyone to download a new 3.9alpha partitioned image, which wouldn't be the end of the world, but it'd be nice to avoid that.
See above. The only issue is that everyone will have to download the packages so you guys better get that server in shape ;-)
The other thing we'll need to generate will be a changeset that provides the ancestry metadata for all of the initial package versions, so that the MC working copies will be in the same state as they would have been if all the packages had been loaded from the repository (which will take a long time and we don't want to put people through if we don't have to).
Ok, that would be good. Mm, any suggestions on how I should do this? :-) Of course, most of the packages are really the "initial" package versions as you say, so they don't really have any "ancestry", right? E.g. we would have Kernel version 1. But there may be something missing that I should add in any case... as if it were loaded from the repository, I agree.
No! Please, no! Not "as if" - make it the real thing! What is the problem with just downloading the package and installing it? If people don't want to wait they can always just grab a preloaded image.
Cheers, - Andreas
On 6/30/05, Andreas Raab andreas.raab@gmx.de wrote:
Depends on whether you wanted to have the system fully packagized or not. This is a key question - I have no interest whatsover in another "partially packaged" system. We have this already.
I'm certainly with you on having a fully packagized system; the main question I have is whether we want to fully packagize it eagerly or lazily. You've done a great job of getting the bulk of the packaging done, and that may be enough to start with; as long as we have a fully packagized *process*, the rest can come along at a slower rate. That is, if there's no way to submit a FIX or ENH, or get an update from the stream, for code that's not packagized, then the first time we need to actually change one of those loose methods, it will get dealt with - and probably more thoughtfully than if we try to do them all at once now. Another strategy might be to recategorize them to all be in an *orphaned package rather than creating tons of tiny packages, with the rule that when you modify a method in that package, you need to try to move it out (so it should shrink over time).
But that's just my $0.02 - if others are less inclined to punt, and want to do it properly up front, great. It might not turn out to be that much work.
No! Please, no! Not "as if" - make it the real thing! What is the problem with just downloading the package and installing it? If people don't want to wait they can always just grab a preloaded image.
Fair enough. I suspect most people will just grab the preloaded one anyway.
Avi
Under the assumption that the placing of code under packages is done simply by creating an MC package for every otherwise-uncategorized method, and therefore is no work at all (I think that's what Andreas says that doIt does, if not, that's not hard to write):
I think its better to have everything in packages, so people don't meet technical problems such as "I changed a bunch of methods, uploaded the packages that changed in MC, but one of my methods didn't appear".
The down side is we might have a couple of dumb packages, but we'll be moving code between packages forever anyway (detangling), so who cares?
Daniel
Avi Bryant wrote:
On 6/30/05, Andreas Raab andreas.raab@gmx.de wrote:
Depends on whether you wanted to have the system fully packagized or not. This is a key question - I have no interest whatsover in another "partially packaged" system. We have this already.
I'm certainly with you on having a fully packagized system; the main question I have is whether we want to fully packagize it eagerly or lazily. You've done a great job of getting the bulk of the packaging done, and that may be enough to start with; as long as we have a fully packagized *process*, the rest can come along at a slower rate. That is, if there's no way to submit a FIX or ENH, or get an update from the stream, for code that's not packagized, then the first time we need to actually change one of those loose methods, it will get dealt with - and probably more thoughtfully than if we try to do them all at once now. Another strategy might be to recategorize them to all be in an *orphaned package rather than creating tons of tiny packages, with the rule that when you modify a method in that package, you need to try to move it out (so it should shrink over time).
But that's just my $0.02 - if others are less inclined to punt, and want to do it properly up front, great. It might not turn out to be that much work.
No! Please, no! Not "as if" - make it the real thing! What is the problem with just downloading the package and installing it? If people don't want to wait they can always just grab a preloaded image.
Fair enough. I suspect most people will just grab the preloaded one anyway.
Avi
On 6/30/05, Daniel Vainsencher danielv@techunix.technion.ac.il wrote:
Under the assumption that the placing of code under packages is done simply by creating an MC package for every otherwise-uncategorized method, and therefore is no work at all (I think that's what Andreas says that doIt does, if not, that's not hard to write):
I think its better to have everything in packages, so people don't meet technical problems such as "I changed a bunch of methods, uploaded the packages that changed in MC, but one of my methods didn't appear".
The down side is we might have a couple of dumb packages, but we'll be moving code between packages forever anyway (detangling), so who cares?
Ok. So is it better to have a single dumb package ("Orphaned") or lots of little dumb packages? I lean towards the former.
Avi
Haven't thought this through. I also lean towards the former, because it seems easier to keep track of.
Daniel
Avi Bryant wrote:
On 6/30/05, Daniel Vainsencher danielv@techunix.technion.ac.il wrote:
Under the assumption that the placing of code under packages is done simply by creating an MC package for every otherwise-uncategorized method, and therefore is no work at all (I think that's what Andreas says that doIt does, if not, that's not hard to write):
I think its better to have everything in packages, so people don't meet technical problems such as "I changed a bunch of methods, uploaded the packages that changed in MC, but one of my methods didn't appear".
The down side is we might have a couple of dumb packages, but we'll be moving code between packages forever anyway (detangling), so who cares?
Ok. So is it better to have a single dumb package ("Orphaned") or lots of little dumb packages? I lean towards the former.
Avi
On Thu, 30 Jun 2005 14:06:51 +0200, "Avi Bryant" avi.bryant@gmail.com said:
On 6/30/05, Daniel Vainsencher danielv@techunix.technion.ac.il wrote:
Under the assumption that the placing of code under packages is done simply by creating an MC package for every otherwise-uncategorized method, and therefore is no work at all (I think that's what Andreas says that doIt does, if not, that's not hard to write):
I think its better to have everything in packages, so people don't meet technical problems such as "I changed a bunch of methods, uploaded the packages that changed in MC, but one of my methods didn't appear".
The down side is we might have a couple of dumb packages, but we'll be moving code between packages forever anyway (detangling), so who cares?
Ok. So is it better to have a single dumb package ("Orphaned") or lots of little dumb packages? I lean towards the former.
See my previous email... it's looking like we may not have any orphaned methods at all, so hopefully we won't need this.
(If we have to have this for some reason, I also lean toward having a single big dumb package rather than a bunch of little ones.)
Now, there is still *plenty* of detangling & package boundary work to be done, but at least it looks like all the code will be in real packages. :) (E.g. Moving String>>asUrl from the Collections package to the Network package.)
- Doug
On Wed, 29 Jun 2005 21:41:50 -0700, "Andreas Raab" andreas.raab@gmx.de said:
Hi -
Some more comments:
Note that after I do these steps, the list of packages in the Monticello browser is somewhat different than in the Squeak3.8-Packages image, this may need some cleanup? Not sure where Connectors and some of the other packages came from. See attached screenshot.
I think those come from cs #4, right? Do we need to run that? I'm not sure we want all of those automatically created packages in there.
Depends on whether you wanted to have the system fully packagized or not. This is a key question - I have no interest whatsover in another "partially packaged" system. We have this already.
Agreed 100%... I think we all want a fully packagized system, where every single method in the system belongs to a package. (And preferably, every method belongs to exactly one package, which means the Basic configuration of packages has no overrides, if possible.)
If you *are* interested in a fully packagized system then you need to make sure you know what every last line of code belongs to. The doIt does that by touching every method in the system and therefore pointing out (!) places that need to be fixed.
That's what this doIt is good for - it's NOT intended to perform proper packagizing it is intended to show which parts of the system are not included in your current package set and therefore need to be dealt with.
Actually, we are in better shape than I originally thought. There were a dozen or so extra MC packages that showed up after I executed the DoIt (e.g. "Connectors", "Flexiblevocabularies", "Refactory", etc). However, when browsing each of these packages, they have no code, they just show an empty list of *Extensions and nothing else.
So, my guess is that these were perhaps generated by the DoIt from some extra extension method categories (e.g. "*Connectors") which didn't have any methods in them. Ah, I see a fresh 3.8-6665-basic image has a "*flexiblevocabularies-viewer" method category on Object, which the Reorg cs doesn't do anything with, so that's where they come from... shouldn't be hard to clean that up.
Well, there is one stray package which did have some code in it after I ran the DoIt. A package called "UserObjects" has three classes, CardPlayer55, CardPlayer57, and Player56. Looks like some Etoys prototype classes which don't need to be there. There are no instances of these classes in the 6665-basic image, so I could just add an update cs which deletes them, and deletes the UserObjects class category.
Since I did the work a while ago it is quite possible that later changes have done some modifications that I was unaware about when I did the work. I suggest you just reclassify these classes/methods too so that you have a fully packagized system.
As far as "later changes" go, there are actually only two new updates in 3.8 since your iSqueak Packages image was created, 6663 and 6664. Both of these updates just change a few existing methods, they don't add or reorganize anything. So your Reorg changeset looks like it should be fine as-is, except for the small cleanups I noted above.
Yes, but I thought cs #4 did all of the other PI-creation as well, so I assume it is necessary. I.e. it creates the Kernel, Collections, etc MC packages from the class categories. Installing MC doesn't do this, does it? The Reorg cs does not do it, it only shuffles class categories around.
The doIt "sort of" does this but *not* properly. To do it properly you want both, a repository that the package resides on and a version of the package. Here is how to do this:
a) Send out the Reorg.cs just to get things in order for people out there (this includes any additional modifications that need to be made) b) Do whatever is necessary to get Monticello/MCConfs to the people out there (e.g., put them into the update stream or so) c) Post versions to the 3.9 repository that include all packages which should then be in a basic image d) Post a configuration that people out there get via update stream
If you do this, here is what will happen for a 3.9 user:
- They get the ReOrg.cs
- They a configuration map
- This will force them to download the versions from the repository
- Since the repository version will be equivalent to what is in the
system, no changes will be made
In other words, the above is a recipe to get the packages to the users at the expense of having to download all of these packages from the repository - which will take some time but it also means that (for once ;-) you know that people have precisely the versions that are in the repository.
Yeah, that sounds like the way to go, then.
I can put up a big warning dialog before this update, telling people that they should strongly consider just getting a replacement 3.9alpha image from http://ftp.squeak.org/xxx, noting that proceeding with the big package-installing update WILL take longer than downloading a brand-new image. But if they're a glutton for punishment, they can proceed. :)
...
Hopefully we can do all of this cleanup through updates, so that people can continue to follow the update stream from a 3.8 final image if they want. Otherwise, we'll have to just tell everyone to download a new 3.9alpha partitioned image, which wouldn't be the end of the world, but it'd be nice to avoid that.
See above. The only issue is that everyone will have to download the packages so you guys better get that server in shape ;-)
Hopefully with the big warning message, not too many crazy folks will do the big package install. :)
... Ok, that would be good. Mm, any suggestions on how I should do this? :-) Of course, most of the packages are really the "initial" package versions as you say, so they don't really have any "ancestry", right? E.g. we would have Kernel version 1. But there may be something missing that I should add in any case... as if it were loaded from the repository, I agree.
No! Please, no! Not "as if" - make it the real thing! What is the problem with just downloading the package and installing it? If people don't want to wait they can always just grab a preloaded image.
Yes, sounds good to me, I wasn't looking forward to "faking" this ancestry.
I'll see if I can get these new updates added to the 3.9alpha stream tonight.
- Doug
Am 30.06.2005 um 18:25 schrieb Doug Way:
Well, there is one stray package which did have some code in it after I ran the DoIt. A package called "UserObjects" has three classes, CardPlayer55, CardPlayer57, and Player56. Looks like some Etoys prototype classes which don't need to be there. There are no instances of these classes in the 6665-basic image, so I could just add an update cs which deletes them, and deletes the UserObjects class category.
Classes in the UserObjects category are created whenever you make an etoy script. I guess we have to special-case that to not make it into a versioned package.
- Bert -
On Jun 30, 2005, at 12:25 PM, Doug Way wrote:
I'll see if I can get these new updates added to the 3.9alpha stream tonight.
Right now I'm loading all the packages from the 39a repository into a test image, so I can put together an MCConfiguration. However, the Morphic package causes a problem when I load it... there are a few changes in the code related to the reorg. (Specifically, the reorg moved some essential methods such as Morph>>handlerForMouseDown: from a "*StandardYellow..." category to a plain "standardyellow..." category, so that they're no longer extensions.)
Avi, were the packages you put in the 39a repository created without installing the Reorg changeset first? That would explain the difference. I may have to put some updated versions of the packages in the 39a repository to be exactly consistent with the source in the 3.8+reorg image.
- Doug
On Jul 2, 2005, at 6:22 AM, Doug Way wrote:
Right now I'm loading all the packages from the 39a repository into a test image, so I can put together an MCConfiguration. However, the Morphic package causes a problem when I load it... there are a few changes in the code related to the reorg. (Specifically, the reorg moved some essential methods such as Morph>>handlerForMouseDown: from a "*StandardYellow..." category to a plain "standardyellow..." category, so that they're no longer extensions.)
Avi, were the packages you put in the 39a repository created without installing the Reorg changeset first? That would explain the difference. I may have to put some updated versions of the packages in the 39a repository to be exactly consistent with the source in the 3.8+reorg image.
Yes, I think that's right - so, good, commit them; this will give the system a little more of a workout.
Avi
On Jul 2, 2005, at 5:48 AM, Avi Bryant wrote:
On Jul 2, 2005, at 6:22 AM, Doug Way wrote:
Right now I'm loading all the packages from the 39a repository into a test image, so I can put together an MCConfiguration. However, the Morphic package causes a problem when I load it... there are a few changes in the code related to the reorg. (Specifically, the reorg moved some essential methods such as Morph>>handlerForMouseDown: from a "*StandardYellow..." category to a plain "standardyellow..." category, so that they're no longer extensions.)
Avi, were the packages you put in the 39a repository created without installing the Reorg changeset first? That would explain the difference. I may have to put some updated versions of the packages in the 39a repository to be exactly consistent with the source in the 3.8+reorg image.
Yes, I think that's right - so, good, commit them; this will give the system a little more of a workout.
Alright, I started committing them, I have an image set up where I could commit them all.
Well, I just committed one so far, Balloon-dew.1.mcz. Since it was a new package created from scratch (from my own partitioning DoIt), it starts at version 1. This is good, I think, since it really has no ancestry. All the packages could start at version 1, except for Monticello & SqueakMap, which do have an MC ancestry.
The only problem is that you have a version 4 (Balloon-avi.4.mcz) out there in the 39a repository, and SqueakSource thinks that is the "latest" one. Should I try to create mine as a version 5 instead? (Not sure offhand how to do that.) Or is there a way to delete your packages?
- Doug
p.s. for (mostly my own) reference, here's my list of steps that I'm using for all of this:
1. Generate packages for 39a repository:
- Start with 3.8-6665-basic image, rename - Install Reorg cs - Install my CleanupUserObjects cs (removes UserObjects & its etoy-generated classes) - Install latest MC.261 + MCConfigs.30 from Bert - Run my package-creation DoIt, which creates exactly 35 packages in the MC browser: "Create MC packages for all system category prefixes." (SystemOrganization categories collect: [:ea | (ea findTokens: $-) first]) asSet do: [:prefix | PackageInfo registerPackageName: prefix. MCWorkingCopy forPackage: (MCPackage new name: prefix)]. - Add 39a repository in MC (with my id/pw), add to all packages in MC. - Save all 35 packages to the 39a repository.
2. Post initial updates to the 3.9alpha update stream:
- In updates.list, rename previous #3.9alpha stream tag to #3.9alphaold, to keep it around for reference. Add new #3.9alpha tag at the end to start it over. - Add Reorg cs as first update. (Hmm, may not be entirely necessary, but probably a good idea so that the packages installed later have identical code.) - Add my CleanupUserObjects cs as the 2nd update. - Add two changesets based on Bert's MC & MCConfigurations installs, as 3rd & 4th updates.
3. Create MCConfiguration changeset to be added to the update stream:
- Start with 3.8-6665-basic image, rename - Install Reorg cs - Install MC + MCConfigs from Bert - Load all 35 latest packages from the 39a repository (the ones I saved there previously) via MC, one by one. Before each load, check the "changes" to make sure there are no changes, i.e. the package and the code in the image should be identical. - Open the MCConfigurations browser, add all 35 packages, in the preferred load order. For this first MCConfiguration, I'm not sure the load order matters all that much, since the loaded code should be identical to what's already in the image. I guess I will start with the "root" packages first (Kernel, Collections, etc) and add the outer ones later in the list. - Post the configuration to the 3.9alpha update stream (as the 5th update), see how it goes.
4. Test the 3.9alpha stream by updating from a fresh image!
Hi doug
I just registered to the 3.9 squeaksource. Tell me how I can help. Where can I find a 3.9 packaged image?
Stef
On Jul 3, 2005, at 5:03 AM, stéphane ducasse wrote:
Hi doug
I just registered to the 3.9 squeaksource. Tell me how I can help. Where can I find a 3.9 packaged image?
I'm still working on it, see my other post. I realize I'm the bottleneck here, but I have some time this weekend and I will get it done today or tomorrow.
In the meantime, I added you as a developer on the 3.9a project. (And also Marcus Denker.) The current list of developers is roughly the group which had access to the update stream previously. We probably don't need a lot of developers on the 3.9a project... keep in mind that the bulk of development should be happening in the individual packages in their master repositories, in theory. We will see how it all works out. :)
- Doug
Ok, I've added the updates to the restarted 3.9alpha update stream. (It even puts up a warning if you're trying to update from the old 3.9alpha stream... I remembered that doing a "self notify: 'please close this window'" is a not-too-bad way to bail out of the current UI process.)
However, there's a problem with the MCConfiguration install. It generally works and gets most of the way through it, but it hits an error trying to install the SMBase-dew.64 package. Looking at the debugger, it looks like it's trying to install an .mcd diff file for this. Ah, maybe the base version has to be in the same repository? (which would be SMBase-gk.63)
Anyway, anyone can try to do the updates if you want to see the error. The big MCConfig install of all the packages actually doesn't take as terribly long as I thought, maybe 5-10 minutes. (It probably won't be too hard on the server if just a few people on this list try it.) Here's all you need to do:
- start with a fresh 3.8-6665-basic image (save a copy) - 'SystemVersion setVersion' to 'Squeak3.9alpha' - load updates
I'm going to pack it in for the day, but hopefully this won't take too long to fix tomorrow. Then I can post a working image with the updates already loaded.
- Doug
p.s. these are the updates:
6670New39StreamWarning-dew.cs 6671Reorganize39-6665-ar.cs 6672CleanupUserObjects-dew.cs 6673Monticello-bf.cs 6674MonticelloConfigs-bf.cs 6675InstallPkgsWarning-dew.cs 6676InstallMCPackages-dew.cs
Just a question is the configuration file versioned too because it seems important to be able to load back old configurations?
Stef
Am 04.07.2005 um 08:29 schrieb stéphane ducasse:
Just a question is the configuration file versioned too because it seems important to be able to load back old configurations?
In the current hybrid stream/mc scheme, there is no separate configuration file. All configurations will appear as a changeset in the update stream. You can use the changesorter to see older configurations.
- Bert -
On 5 juil. 05, at 11:26, Bert Freudenberg wrote:
Am 04.07.2005 um 08:29 schrieb stéphane ducasse:
Just a question is the configuration file versioned too because it seems important to be able to load back old configurations?
In the current hybrid stream/mc scheme, there is no separate configuration file. All configurations will appear as a changeset in the update stream. You can use the changesorter to see older configurations.
OK Is there a reason? because publishing the MC configurations also as a package is useful.
Stef
- Bert -
Am 05.07.2005 um 16:15 schrieb stéphane ducasse:
On 5 juil. 05, at 11:26, Bert Freudenberg wrote:
Am 04.07.2005 um 08:29 schrieb stéphane ducasse:
Just a question is the configuration file versioned too because it seems important to be able to load back old configurations?
In the current hybrid stream/mc scheme, there is no separate configuration file. All configurations will appear as a changeset in the update stream. You can use the changesorter to see older configurations.
OK Is there a reason? because publishing the MC configurations also as a package is useful.
It was TSTTCPW. As I said before, yes it would be a good idea to version configs, and yes it would be good to get unify versions and configurations into a general dependency scheme, but the general consensus was to start using what there is right now and improve over time.
- Bert -
- start with a fresh 3.8-6665-basic image (save a copy)
- 'SystemVersion setVersion' to 'Squeak3.9alpha'
- load updates
When I do that, I get a rollback "Error: cannot remove non-empty category". Change 6672CleanupUserObjects-dew yields this. Am I the only one ?
The image I used is a fresh 6665.
Alexandre
I'm going to pack it in for the day, but hopefully this won't take too long to fix tomorrow. Then I can post a working image with the updates already loaded.
- Doug
p.s. these are the updates:
6670New39StreamWarning-dew.cs 6671Reorganize39-6665-ar.cs 6672CleanupUserObjects-dew.cs 6673Monticello-bf.cs 6674MonticelloConfigs-bf.cs 6675InstallPkgsWarning-dew.cs 6676InstallMCPackages-dew.cs
On Jul 4, 2005, at 5:45 AM, Alexandre Bergel wrote:
- start with a fresh 3.8-6665-basic image (save a copy)
- 'SystemVersion setVersion' to 'Squeak3.9alpha'
- load updates
When I do that, I get a rollback "Error: cannot remove non-empty category". Change 6672CleanupUserObjects-dew yields this. Am I the only one ?
The image I used is a fresh 6665.
Hm, you're right, we don't really need to remove the category in the update stream. It mainly needed to be removed when I created the 35 packages, so that it wasn't one of them.
Make sure that you're using a basic image, not a full image. Following this new update stream with a full image will only invite problems, and is probably pointless. (Ok, I just tweaked the 6675 warning update to encourage people to start with a 3.9alpha-basic image to clarify this. Yes, this image is not available yet, but it will be before we announce the new process to the public.) I started with the Squeak3.8-6665-basic.zip on the ftp site.
Anyway, I changed 6672 so that it only removes the three player classes, not the category. Give it another try. (You'll eventually hit the problem with SM-Base mcd file, but at least most of the packages will load, and you can poke around the packages with MC, and also look at the 6676 MCConfig update.)
- Doug
When I do that, I get a rollback "Error: cannot remove non-empty category". Change 6672CleanupUserObjects-dew yields this. Am I the only one ?
The image I used is a fresh 6665.
It works fine.
I have some changes related to the changeset notification. how should I proceed to submit it ?
I loaded my changeset, and the System is modified. Shall I save the package System?
As far as I understood, people willing to submit fixes, have to do it using changeset, and harvester are the only ones who can update the 39a.
If what I understood is correct, I will contribute to the swiki page Doug started once I am back in bern next week.
Cheers, Alexandre
On Tue, 5 Jul 2005 10:20:58 +0200, "Alexandre Bergel" bergel@iam.unibe.ch said:
It works fine.
I have some changes related to the changeset notification. how should I proceed to submit it ?
I loaded my changeset, and the System is modified. Shall I save the package System?
As far as I understood, people willing to submit fixes, have to do it using changeset, and harvester are the only ones who can update the 39a.
Ok, here is where things get interesting. :)
Basically, most of the old rules are now gone. You won't have one group of harvesters controlling everything (so that everyone else has to send them changesets) anymore.
We need to get small groups of 2-6 maintainers for all of the 35 packages making up the base image. Easier said than done, of course, but we need to make a stab at it.
In this case, it sounds like all of your changes are in the System package? (When you make your changes, does only the System package have a "*" by it in the MC Browser?)
If that is the case, since you are doing significant work in the System package, I would suggest that you be one of the co-maintainers of the master System package. Various folks at Berne have made significant changes to System in the last few years (KCP team etc), so it makes sense for you or someone else there to be one of the co-maintainers.
With an important package like System, it's probably good to have at least 3 maintainers, so that the group is self-sustaining if somebody leaves, they can review each other's changes, etc. One of these people would be the "lead" maintainer. More trivial packages such as VersionNumber could probably just have one maintainer. (This is really a topic for a whole other email.)
Anyway, ideally we should have a master System package repository set up somewhere. Could be at source.sqf.org or squeaksource.com or somewhere else. As one of the co-maintainers, you would simply commit your changes to the System package there. Periodically, we (the v3.9 team) would add the latest version of the System package to the 39a repository as well, and then we'd have your changes.
(The cool thing about the master vs release repository separation is that if the maintainers of the System package don't do their job, someone else could create a competing System package in a different repository, and the v3.9 team could simply switch to using that version instead.)
So, the master repository vs release repository aspect is a little bit complicated, but it seems like the only real way to move to a system of truly distributed package development. We do already have master repositories for some packages, such as MC, MCConfigurations, SM-Base, etc. Now we need them for all the other packages.
In the short term, one thing we could do is simply make changes to the System package and other core packages directly in the 39a repository, since we don't have master repositories for these packages yet. I hesitate to suggest this, though, as this is still too centralized a model. But if we have trouble finding package maintainers, we may end up having to do this for some packages in the short term.
It is still true that an outsider submitting a bug fix should submit a changeset via Mantis or whatever current process. The package maintainers would be checking Mantis periodically for fixes from the community, and incorporating them into their package.
If what I understood is correct, I will contribute to the swiki page Doug started once I am back in bern next week.
So, you got that? ;-)
For now, I'm not sure if you should go ahead and make the change in the 39a System package, or if you should wait until we do a real maintainership drive, and create all the remaining master packages. Thoughts from anyone else?
- Doug
On 5 juil. 05, at 22:37, Doug Way wrote:
Anyway, ideally we should have a master System package repository set up somewhere. Could be at source.sqf.org or squeaksource.com or somewhere else. As one of the co-maintainers, you would simply commit your changes to the System package there. Periodically, we (the v3.9 team) would add the latest version of the System package to the 39a repository as well, and then we'd have your changes.
(The cool thing about the master vs release repository separation is that if the maintainers of the System package don't do their job, someone else could create a competing System package in a different repository, and the v3.9 team could simply switch to using that version instead.)
But it seems that for now this can be really difficult for refactorings that are crosscutting several packages since you will have to look in different locations. So this can be a real pain. Or will I publish in another repository so this means that we will create a lot of hidden dependencies because even if I work regularly but another master is on holiday then the v3.9 team can be just out of the loop to get my changes in.
Stef
In the short term, one thing we could do is simply make changes to the System package and other core packages directly in the 39a repository, since we don't have master repositories for these packages yet. I hesitate to suggest this, though, as this is still too centralized a model. But if we have trouble finding package maintainers, we may end up having to do this for some packages in the short term.
My impression is that we should go slowly and do not go crazy and decentralized too fast. For well identified packages such as MC this is really ok but for the others this can be really a mess as I mentioned.
It is still true that an outsider submitting a bug fix should submit a changeset via Mantis or whatever current process. The package maintainers would be checking Mantis periodically for fixes from the community, and incorporating them into their package.
On Tue, 5 Jul 2005 23:05:17 +0200, "stéphane ducasse" ducasse@iam.unibe.ch said:
In the short term, one thing we could do is simply make changes to the System package and other core packages directly in the 39a repository, since we don't have master repositories for these packages yet. I hesitate to suggest this, though, as this is still too centralized a model. But if we have trouble finding package maintainers, we may end up having to do this for some packages in the short term.
My impression is that we should go slowly and do not go crazy and decentralized too fast. For well identified packages such as MC this is really ok but for the others this can be really a mess as I mentioned.
I think I agree. Maybe for some of the well-defined "outer" packages such as Balloon, Movies, Speech, etc., we should try to find maintainers and get someone to set up master repositories soon for those, at least. (Hm, some of those maybe don't even need to be in Basic, they could just be part of Full.)
But some of the core packages haven't really been properly detangled yet, so we may have a lot of changes which cut across several packages. Maybe these core packages (Kernel, Collections, Morphic, etc) should be maintained in the 39a repository for the short term until they are better detangled. Certainly the detangling work itself will definitely cut across multiple packages and would be easier to do directly in the 39a repository. (Although the work of the Morphic Splitters team is obviously relevant here, I'll have to see what progress they made.) Comments from Daniel and any other detangling experts? :)
Actually, both the Morphic Splitters and the Toolbuilder work involve some detangling, and we'd probably want to look at incorporating both of those earlier rather than later. (At a certain point we just have to pick which order these are incorporated... e.g. 1. Morphic Splitters 2. Toolbuilder refactoring 3. previous 3.9alpha changes... whichever goes first has the least amount of merging to do.)
- Doug
Hi Doug, I've posted a couple of relevant posts to the list, did you see them? Some comments below.
Doug Way wrote:
I think I agree. Maybe for some of the well-defined "outer" packages such as Balloon, Movies, Speech, etc., we should try to find maintainers and get someone to set up master repositories soon for those, at least. (Hm, some of those maybe don't even need to be in Basic, they could just be part of Full.)
But some of the core packages haven't really been properly detangled yet, so we may have a lot of changes which cut across several packages. Maybe these core packages (Kernel, Collections, Morphic, etc) should be maintained in the 39a repository for the short term until they are better detangled. Certainly the detangling work itself will definitely cut across multiple packages and would be easier to do directly in the 39a repository. (Although the work of the Morphic Splitters team is obviously relevant here, I'll have to see what progress they made.) Comments from Daniel and any other detangling experts? :)
I say use the Submissions project in the mean time, for anything that does not have clear maintainership, and anything not yet detangled. That way people can cooperate, and moving things to 39a and into peoples alpha is still an explicit action.
Actually, both the Morphic Splitters and the Toolbuilder work involve some detangling, and we'd probably want to look at incorporating both of those earlier rather than later. (At a certain point we just have to pick which order these are incorporated... e.g. 1. Morphic Splitters 2. Toolbuilder refactoring 3. previous 3.9alpha changes... whichever goes first has the least amount of merging to do.)
Remember that we are now managing the code with MC. One benefit is that it is quite easy for someone to just merge, and see exactly what conflicts there are.
Daniel
But some of the core packages haven't really been properly detangled yet, so we may have a lot of changes which cut across several packages.
I agree. Better package separation is probably the most important thing to do right now - simply because otherwise you'll end up with inappropriate changes to (say) Kernel because of some Morphic change. It would be nice if package boundaries could be defined in such a way that we know and can proove the package dependencies and use that as a guide line for how to set up the boundaries.
Maybe these core packages (Kernel, Collections, Morphic, etc) should be maintained in the 39a repository for the short term until they are better detangled. Certainly the detangling work itself will definitely cut across multiple packages and would be easier to do directly in the 39a repository. (Although the work of the Morphic Splitters team is obviously relevant here, I'll have to see what progress they made.) Comments from Daniel and any other detangling experts? :)
Keep in mind that MC is fairly good at merging (and where it isn't it needs fixing ;-) so we ought to be able to take the results of various projects and simply merge them in.
Actually, both the Morphic Splitters and the Toolbuilder work involve some detangling, and we'd probably want to look at incorporating both of those earlier rather than later. (At a certain point we just have to pick which order these are incorporated... e.g. 1. Morphic Splitters 2. Toolbuilder refactoring 3. previous 3.9alpha changes... whichever goes first has the least amount of merging to do.)
As far as I can tell 1. and 2. don't overlap at all (Brian could say more about this). 3. should probably go last since it has the finest granularity and therefore is easiest to merge.
Cheers, - Andreas
On Wed, 06 Jul 2005 08:21:49 +0200, "Andreas Raab" andreas.raab@gmx.de said:
... Keep in mind that MC is fairly good at merging (and where it isn't it needs fixing ;-) so we ought to be able to take the results of various projects and simply merge them in.
Yes, that is one of the cool aspects of the new process, merging should be a lot easier. And the MC tools will be getting a good workout, and maybe even some fixes/enhancements.
Actually, both the Morphic Splitters and the Toolbuilder work involve some detangling, and we'd probably want to look at incorporating both of those earlier rather than later. (At a certain point we just have to pick which order these are incorporated... e.g. 1. Morphic Splitters 2. Toolbuilder refactoring 3. previous 3.9alpha changes... whichever goes first has the least amount of merging to do.)
As far as I can tell 1. and 2. don't overlap at all (Brian could say more about this). 3. should probably go last since it has the finest granularity and therefore is easiest to merge.
Makes sense to me. We could possibly do just a very few 3.9alpha changes (preferably non-Morphic/Toolbuilder related) first, just to try out the new process. But then do the Morphic Splitters/Toolbuilder stuff before much of anything else is done.
- Doug
There's a Submissions repository world-writable, on source.sqf.org.
We can post things there for people to look at, and occasionally merge it into 3.9a. So one way to begin would be to start creating versions on Submissions that include the patches that have accumulated for 3.9a before, merging them into the 39a repository versions as people think is safe.
Might be a good idea to wait with creating a master repository for each package until its clearer who are the maintainers, and the borders between packages stabilize. In the mean time, Submissions could be used for everything.
Daniel
Doug Way wrote:
On Tue, 5 Jul 2005 10:20:58 +0200, "Alexandre Bergel" bergel@iam.unibe.ch said:
It works fine.
I have some changes related to the changeset notification. how should I proceed to submit it ?
I loaded my changeset, and the System is modified. Shall I save the package System?
As far as I understood, people willing to submit fixes, have to do it using changeset, and harvester are the only ones who can update the 39a.
Ok, here is where things get interesting. :)
Basically, most of the old rules are now gone. You won't have one group of harvesters controlling everything (so that everyone else has to send them changesets) anymore.
We need to get small groups of 2-6 maintainers for all of the 35 packages making up the base image. Easier said than done, of course, but we need to make a stab at it.
In this case, it sounds like all of your changes are in the System package? (When you make your changes, does only the System package have a "*" by it in the MC Browser?)
If that is the case, since you are doing significant work in the System package, I would suggest that you be one of the co-maintainers of the master System package. Various folks at Berne have made significant changes to System in the last few years (KCP team etc), so it makes sense for you or someone else there to be one of the co-maintainers.
With an important package like System, it's probably good to have at least 3 maintainers, so that the group is self-sustaining if somebody leaves, they can review each other's changes, etc. One of these people would be the "lead" maintainer. More trivial packages such as VersionNumber could probably just have one maintainer. (This is really a topic for a whole other email.)
Anyway, ideally we should have a master System package repository set up somewhere. Could be at source.sqf.org or squeaksource.com or somewhere else. As one of the co-maintainers, you would simply commit your changes to the System package there. Periodically, we (the v3.9 team) would add the latest version of the System package to the 39a repository as well, and then we'd have your changes.
(The cool thing about the master vs release repository separation is that if the maintainers of the System package don't do their job, someone else could create a competing System package in a different repository, and the v3.9 team could simply switch to using that version instead.)
So, the master repository vs release repository aspect is a little bit complicated, but it seems like the only real way to move to a system of truly distributed package development. We do already have master repositories for some packages, such as MC, MCConfigurations, SM-Base, etc. Now we need them for all the other packages.
In the short term, one thing we could do is simply make changes to the System package and other core packages directly in the 39a repository, since we don't have master repositories for these packages yet. I hesitate to suggest this, though, as this is still too centralized a model. But if we have trouble finding package maintainers, we may end up having to do this for some packages in the short term.
It is still true that an outsider submitting a bug fix should submit a changeset via Mantis or whatever current process. The package maintainers would be checking Mantis periodically for fixes from the community, and incorporating them into their package.
If what I understood is correct, I will contribute to the swiki page Doug started once I am back in bern next week.
So, you got that? ;-)
For now, I'm not sure if you should go ahead and make the change in the 39a System package, or if you should wait until we do a real maintainership drive, and create all the remaining master packages. Thoughts from anyone else?
- Doug
On Wed, 06 Jul 2005 00:16:43 +0300, "Daniel Vainsencher" danielv@techunix.technion.ac.il said:
There's a Submissions repository world-writable, on source.sqf.org.
We can post things there for people to look at, and occasionally merge it into 3.9a. So one way to begin would be to start creating versions on Submissions that include the patches that have accumulated for 3.9a before, merging them into the 39a repository versions as people think is safe.
The Submissions repository seems like it might be an extra step which is not entirely necessary... we already have these changes stored in the old update stream, or on Mantis, etc.
It seems more relevant as a possibly new fix/enh submissions process, which is currently done in Mantis, and is partly/mostly handled by the Janitors group. You might want to bring up the idea on that list, although I guess it's worth pondering here for a bit.
One upside to the Submissions repository is that the UI is available in Squeak via MC, whereas Mantis changesets are not. I guess one possibility is still using Mantis to track bugs & fixes, but posting the code in the Submissions repository instead of to Mantis. One downside is that the repository could easily get cluttered with hundreds of unordered submissions.
Might be a good idea to wait with creating a master repository for each package until its clearer who are the maintainers, and the borders between packages stabilize.
Agreed.
- Doug
Ah, you do see my mails, just a matter of timing.
Doug Way wrote:
On Wed, 06 Jul 2005 00:16:43 +0300, "Daniel Vainsencher" danielv@techunix.technion.ac.il said:
There's a Submissions repository world-writable, on source.sqf.org.
We can post things there for people to look at, and occasionally merge it into 3.9a. So one way to begin would be to start creating versions on Submissions that include the patches that have accumulated for 3.9a before, merging them into the 39a repository versions as people think is safe.
The Submissions repository seems like it might be an extra step which is not entirely necessary... we already have these changes stored in the old update stream, or on Mantis, etc.
[snip]
One upside to the Submissions repository is that the UI is available in Squeak via MC, whereas Mantis changesets are not.
I don't think of it as "show up in Squeak via MC". I think of it as - can be trivially accessed by Squeak code, in a well modeled manner, using MC. On top of that we can build various things, better UI being just one. For example, a test server that runs a test suite on every submitted version, catching regressions before they get into the update stream.
I guess one possibility is still using Mantis to track bugs & fixes, but posting the code in the Submissions repository instead of to Mantis.
Yup. Given a common repository for each package, the mantis entry needs only to name the specific version. And as Bert suggested, we could eventually move bug tracking functionality into Squeak.
One downside is that the repository could easily get cluttered with hundreds of unordered submissions.
I think that's a tools issue. That repositories accumulate dead end versions is natural. If this repository does it faster, and pushes SqueakSource and MC to improve how they deal with this, or specialized UIs to appear, that's a good thing.
It seems more relevant as a possibly new fix/enh submissions process, which is currently done in Mantis, and is partly/mostly handled by the Janitors group. You might want to bring up the idea on that list, although I guess it's worth pondering here for a bit.
Sure. Seems to me that we want the process to be as seamless as possible, while allowing maintainers control.
So I propose for each package, things get merged into 39a from its master repository, which can be Submissions if there's nothing else, or the maintainers repository otherwise. We make sure that in the image, each package includes its master repository, so things get submitted to the right place. If the maintainer sees something interesting for his package that got to Submissions by mistake, he merges it (or not), and again, nobody else needs to do anything special - just merge from the maintainer every so often.
And from everyones point of view (Joe R. Fixit, Maintainer, update stream writer (are those still called harvesters?)), everything is quite uniform. I get my code for a package from one predefined repository, and merge and save what I think is right into another.
Daniel Vainsencher
On Wed, 06 Jul 2005 02:44:20 +0300, "Daniel Vainsencher" danielv@techunix.technion.ac.il said:
Doug Way wrote:
The Submissions repository seems like it might be an extra step which is not entirely necessary... we already have these changes stored in the old update stream, or on Mantis, etc.
[snip]
One upside to the Submissions repository is that the UI is available in Squeak via MC, whereas Mantis changesets are not.
I don't think of it as "show up in Squeak via MC". I think of it as - can be trivially accessed by Squeak code, in a well modeled manner, using MC. On top of that we can build various things, better UI being just one. For example, a test server that runs a test suite on every submitted version, catching regressions before they get into the update stream.
True. Also, I was thinking mainly in terms of the UI being better integrated for the person reviewing/incorporating submissions, but this integrated UI is also better for the submitter. So maybe it's not such a bad idea after all. ;)
I guess one possibility is still using Mantis to track bugs & fixes, but posting the code in the Submissions repository instead of to Mantis.
Yup. Given a common repository for each package, the mantis entry needs only to name the specific version. And as Bert suggested, we could eventually move bug tracking functionality into Squeak.
Ok. For starters, we could give it a try for some submissions, and also discuss with the Janitors team.
I'm guessing most/all submissions would be .mcd files. (I suppose a complete overhaul/replacement of a package could be an .mcz.)
There might be a few cases which are hard to handle, such as a submission which requires a DoIt, which requires a changeset (shouldn't be too common). Those could be left in Mantis for now. Or, submissions which cut across multiple packages couldn't be conveniently bundled into a single .mcd file, I think.
One downside is that the repository could easily get cluttered with hundreds of unordered submissions.
I think that's a tools issue. That repositories accumulate dead end versions is natural. If this repository does it faster, and pushes SqueakSource and MC to improve how they deal with this, or specialized UIs to appear, that's a good thing.
True.
It seems more relevant as a possibly new fix/enh submissions process, which is currently done in Mantis, and is partly/mostly handled by the Janitors group. You might want to bring up the idea on that list, although I guess it's worth pondering here for a bit.
Sure. Seems to me that we want the process to be as seamless as possible, while allowing maintainers control.
So I propose for each package, things get merged into 39a from its master repository, which can be Submissions if there's nothing else, or the maintainers repository otherwise.
Ok. (I'm not sure if we have to be rigid about *all* other changes coming from the Submissions repo, but we can try it. I guess there are some advantages to doing that.)
We make sure that in the image, each package includes its master repository, so things get submitted to the right place.
Absolutely, I forgot to mention this earlier. For example, the SM-base package in the 3.9a image should list both the 39a repository and also Goran's master repository in the Monticello browser. (And the Submissions repo for the other ones.) I will add this to the 3.9a-6676 image, or possibly add it as a DoIt update.
If the maintainer sees something interesting for his package that got to Submissions by mistake, he merges it (or not), and again, nobody else needs to do anything special - just merge from the maintainer every so often.
And from everyones point of view (Joe R. Fixit, Maintainer, update stream writer (are those still called harvesters?)), everything is quite uniform. I get my code for a package from one predefined repository, and merge and save what I think is right into another.
Sounds cool. :)
- Doug
Sure, I will work on it next week, once we're back.
Cheers, Alexandre
On Tue, Jul 05, 2005 at 04:37:27PM -0400, Doug Way wrote:
On Tue, 5 Jul 2005 10:20:58 +0200, "Alexandre Bergel" bergel@iam.unibe.ch said:
It works fine.
I have some changes related to the changeset notification. how should I proceed to submit it ?
I loaded my changeset, and the System is modified. Shall I save the package System?
As far as I understood, people willing to submit fixes, have to do it using changeset, and harvester are the only ones who can update the 39a.
Ok, here is where things get interesting. :)
Basically, most of the old rules are now gone. You won't have one group of harvesters controlling everything (so that everyone else has to send them changesets) anymore.
We need to get small groups of 2-6 maintainers for all of the 35 packages making up the base image. Easier said than done, of course, but we need to make a stab at it.
In this case, it sounds like all of your changes are in the System package? (When you make your changes, does only the System package have a "*" by it in the MC Browser?)
If that is the case, since you are doing significant work in the System package, I would suggest that you be one of the co-maintainers of the master System package. Various folks at Berne have made significant changes to System in the last few years (KCP team etc), so it makes sense for you or someone else there to be one of the co-maintainers.
With an important package like System, it's probably good to have at least 3 maintainers, so that the group is self-sustaining if somebody leaves, they can review each other's changes, etc. One of these people would be the "lead" maintainer. More trivial packages such as VersionNumber could probably just have one maintainer. (This is really a topic for a whole other email.)
Anyway, ideally we should have a master System package repository set up somewhere. Could be at source.sqf.org or squeaksource.com or somewhere else. As one of the co-maintainers, you would simply commit your changes to the System package there. Periodically, we (the v3.9 team) would add the latest version of the System package to the 39a repository as well, and then we'd have your changes.
(The cool thing about the master vs release repository separation is that if the maintainers of the System package don't do their job, someone else could create a competing System package in a different repository, and the v3.9 team could simply switch to using that version instead.)
So, the master repository vs release repository aspect is a little bit complicated, but it seems like the only real way to move to a system of truly distributed package development. We do already have master repositories for some packages, such as MC, MCConfigurations, SM-Base, etc. Now we need them for all the other packages.
In the short term, one thing we could do is simply make changes to the System package and other core packages directly in the 39a repository, since we don't have master repositories for these packages yet. I hesitate to suggest this, though, as this is still too centralized a model. But if we have trouble finding package maintainers, we may end up having to do this for some packages in the short term.
It is still true that an outsider submitting a bug fix should submit a changeset via Mantis or whatever current process. The package maintainers would be checking Mantis periodically for fixes from the community, and incorporating them into their package.
If what I understood is correct, I will contribute to the swiki page Doug started once I am back in bern next week.
So, you got that? ;-)
For now, I'm not sure if you should go ahead and make the change in the 39a System package, or if you should wait until we do a real maintainership drive, and create all the remaining master packages. Thoughts from anyone else?
- Doug
On Jul 4, 2005, at 1:09 AM, Doug Way wrote:
... However, there's a problem with the MCConfiguration install. It generally works and gets most of the way through it, but it hits an error trying to install the SMBase-dew.64 package. Looking at the debugger, it looks like it's trying to install an .mcd diff file for this. Ah, maybe the base version has to be in the same repository? (which would be SMBase-gk.63)
Forgot to attach the exception stack earlier... attached below.
The string parameter passed further down the stack in the method MCHttpRepository(MCFileBasedRepository)>>loadVersionFromFileNamed: is 'SMBase-dew.64(gk.63).mcd'.
I'll be looking at this problem shortly.
- Doug
----------------------------------------
4 July 2005 12:22:24 pm
VM: Mac OS - a SmalltalkImage Image: Squeak3.9alpha [latest update: #6675]
SecurityManager state: Restricted: false FileAccess: true SocketAccess: true Working Dir Macintosh HD:Users:dway:Squeak:Squeak 3.8 Trusted Dir Macintosh HD:Users:dway:Squeak:Squeak 3.8 Untrusted Dir foobar:tooBar:forSqueak:bogus
ZipArchive(Object)>>error: Receiver: a ZipArchive Arguments and temporary variables: aString: 'can''t find EOCD position' Receiver's instance variables: members: an OrderedCollection() centralDirectorySize: nil centralDirectoryOffsetWRTStartingDiskNumber: nil zipFileComment: '' writeCentralDirectoryOffset: 0 writeEOCDOffset: 0
ZipArchive>>readFrom: Receiver: a ZipArchive Arguments and temporary variables: aStreamOrFileName: a RWBinaryOrTextStream a ByteArray(60 33 68 79 67 84 89 80 6...etc... stream: a RWBinaryOrTextStream a ByteArray(60 33 68 79 67 84 89 80 69 32 72 84 ...etc... name: 'a stream' eocdPosition: 0 Receiver's instance variables: members: an OrderedCollection() centralDirectorySize: nil centralDirectoryOffsetWRTStartingDiskNumber: nil zipFileComment: '' writeCentralDirectoryOffset: 0 writeEOCDOffset: 0
MCMcdReader(MCMczReader)>>zip Receiver: a MCMcdReader Arguments and temporary variables:
Receiver's instance variables: stream: a RWBinaryOrTextStream a ByteArray(60 33 68 79 67 84 89 80 69 32 72 84 ...etc... package: nil info: nil definitions: nil dependencies: nil stepChildren: nil zip: a ZipArchive infoCache: nil baseInfo: nil patch: nil
MCMcdReader(MCMczReader)>>parseMember: Receiver: a MCMcdReader Arguments and temporary variables: fileName: 'package' tokens: nil Receiver's instance variables: stream: a RWBinaryOrTextStream a ByteArray(60 33 68 79 67 84 89 80 69 32 72 84 ...etc... package: nil info: nil definitions: nil dependencies: nil stepChildren: nil zip: a ZipArchive infoCache: nil baseInfo: nil patch: nil
--- The full stack --- ZipArchive(Object)>>error: ZipArchive>>readFrom: MCMcdReader(MCMczReader)>>zip MCMcdReader(MCMczReader)>>parseMember: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - MCMcdReader(MCMczReader)>>loadPackage MCMcdReader(MCVersionReader)>>package MCMcdReader>>basicVersion MCMcdReader(MCVersionReader)>>version [] in MCHttpRepository(MCFileBasedRepository)>>loadVersionFromFileNamed: {[:r | r version]} [] in MCHttpRepository(MCFileBasedRepository)>>versionReaderForFileNamed:do: {[:class | aBlock value: (class on: s fileName: aString)]} MCMcdReader class(Object)>>ifNotNilDo: [] in MCHttpRepository(MCFileBasedRepository)>>versionReaderForFileNamed:do: {[:s | (MCVersionReader readerClassForFileNamed: aString) ifNotNilDo: [:cla...]} MCHttpRepository>>readStreamForFileNamed:do: MCHttpRepository(MCFileBasedRepository)>>versionReaderForFileNamed:do: MCHttpRepository(MCFileBasedRepository)>>loadVersionFromFileNamed: [] in MCHttpRepository(MCFileBasedRepository)>>versionFromFileNamed: {[self loadVersionFromFileNamed: aString]} Dictionary>>at:ifAbsent: MCHttpRepository(MCFileBasedRepository)>>versionFromFileNamed: MCConfiguration>>versionNamed:for:from: [] in MCConfiguration>>depsSatisfying:versionDo:displayingProgress: {[:dep | ver := dep versionInfo name. repo := repoMap at: ver ifAbs...]} [] in OrderedCollection(SequenceableCollection)>>do:displayingProgress: {[:each :i | bar value: i. aBlock value: each]} OrderedCollection(SequenceableCollection)>>withIndexDo: [] in OrderedCollection(SequenceableCollection)>>do:displayingProgress: {[:bar | self withIndexDo: [:each :i | bar value: i. aBlock value: e...]} [] in ProgressInitiationException>>defaultMorphicAction {[result := workBlock value: progress]} BlockContext>>ensure: ProgressInitiationException>>defaultMorphicAction ProgressInitiationException>>defaultAction UndefinedObject>>handleSignal: MethodContext(ContextPart)>>handleSignal: MethodContext(ContextPart)>>handleSignal: ProgressInitiationException(Exception)>>signal ProgressInitiationException>>display:at:from:to:during: ProgressInitiationException class>>display:at:from:to:during: ByteString(String)>>displayProgressAt:from:to:during: OrderedCollection(SequenceableCollection)>>do:displayingProgress: MCConfiguration>>depsSatisfying:versionDo:displayingProgress: ...etc...
On Mon, 4 Jul 2005, Doug Way wrote:
On Jul 4, 2005, at 1:09 AM, Doug Way wrote:
... However, there's a problem with the MCConfiguration install. It generally works and gets most of the way through it, but it hits an error trying to install the SMBase-dew.64 package. Looking at the debugger, it looks like it's trying to install an .mcd diff file for this. Ah, maybe the base version has to be in the same repository? (which would be SMBase-gk.63)
Unfortunately, I can't look into this right now.
But there might be a bug in SqueakSource - the MCConf upgrade process tries to download the .mcd first (because that's much more efficient), and if that fails, it downloads the .mcz. So the server should return an error if it cannot build the diff. However, it might be that I never tried that because we usually have all package versions in the repository.
If that is not the problem, it might work if you just add another repository to the workingcopy which has the base version. But I'm not quite sure about that.
Forgot to attach the exception stack earlier... attached below.
The string parameter passed further down the stack in the method MCHttpRepository(MCFileBasedRepository)>>loadVersionFromFileNamed: is 'SMBase-dew.64(gk.63).mcd'.
I'll be looking at this problem shortly.
- Doug
4 July 2005 12:22:24 pm
VM: Mac OS - a SmalltalkImage Image: Squeak3.9alpha [latest update: #6675]
SecurityManager state: Restricted: false FileAccess: true SocketAccess: true Working Dir Macintosh HD:Users:dway:Squeak:Squeak 3.8 Trusted Dir Macintosh HD:Users:dway:Squeak:Squeak 3.8 Untrusted Dir foobar:tooBar:forSqueak:bogus
ZipArchive(Object)>>error: Receiver: a ZipArchive Arguments and temporary variables: aString: 'can''t find EOCD position' Receiver's instance variables: members: an OrderedCollection() centralDirectorySize: nil centralDirectoryOffsetWRTStartingDiskNumber: nil zipFileComment: '' writeCentralDirectoryOffset: 0 writeEOCDOffset: 0
ZipArchive>>readFrom: Receiver: a ZipArchive Arguments and temporary variables: aStreamOrFileName: a RWBinaryOrTextStream a ByteArray(60 33 68 79 67 84 89 80 6...etc... stream: a RWBinaryOrTextStream a ByteArray(60 33 68 79 67 84 89 80 69 32 72 84 ...etc... name: 'a stream' eocdPosition: 0 Receiver's instance variables: members: an OrderedCollection() centralDirectorySize: nil centralDirectoryOffsetWRTStartingDiskNumber: nil zipFileComment: '' writeCentralDirectoryOffset: 0 writeEOCDOffset: 0
MCMcdReader(MCMczReader)>>zip Receiver: a MCMcdReader Arguments and temporary variables:
Receiver's instance variables: stream: a RWBinaryOrTextStream a ByteArray(60 33 68 79 67 84 89 80 69 32 72 84 ...etc... package: nil info: nil definitions: nil dependencies: nil stepChildren: nil zip: a ZipArchive infoCache: nil baseInfo: nil patch: nil
MCMcdReader(MCMczReader)>>parseMember: Receiver: a MCMcdReader Arguments and temporary variables: fileName: 'package' tokens: nil Receiver's instance variables: stream: a RWBinaryOrTextStream a ByteArray(60 33 68 79 67 84 89 80 69 32 72 84 ...etc... package: nil info: nil definitions: nil dependencies: nil stepChildren: nil zip: a ZipArchive infoCache: nil baseInfo: nil patch: nil
--- The full stack --- ZipArchive(Object)>>error: ZipArchive>>readFrom: MCMcdReader(MCMczReader)>>zip MCMcdReader(MCMczReader)>>parseMember:
MCMcdReader(MCMczReader)>>loadPackage MCMcdReader(MCVersionReader)>>package MCMcdReader>>basicVersion MCMcdReader(MCVersionReader)>>version [] in MCHttpRepository(MCFileBasedRepository)>>loadVersionFromFileNamed: {[:r | r version]} [] in MCHttpRepository(MCFileBasedRepository)>>versionReaderForFileNamed:do: {[:class | aBlock value: (class on: s fileName: aString)]} MCMcdReader class(Object)>>ifNotNilDo: [] in MCHttpRepository(MCFileBasedRepository)>>versionReaderForFileNamed:do: {[:s | (MCVersionReader readerClassForFileNamed: aString) ifNotNilDo: [:cla...]} MCHttpRepository>>readStreamForFileNamed:do: MCHttpRepository(MCFileBasedRepository)>>versionReaderForFileNamed:do: MCHttpRepository(MCFileBasedRepository)>>loadVersionFromFileNamed: [] in MCHttpRepository(MCFileBasedRepository)>>versionFromFileNamed: {[self loadVersionFromFileNamed: aString]} Dictionary>>at:ifAbsent: MCHttpRepository(MCFileBasedRepository)>>versionFromFileNamed: MCConfiguration>>versionNamed:for:from: [] in MCConfiguration>>depsSatisfying:versionDo:displayingProgress: {[:dep | ver := dep versionInfo name. repo := repoMap at: ver ifAbs...]} [] in OrderedCollection(SequenceableCollection)>>do:displayingProgress: {[:each :i | bar value: i. aBlock value: each]} OrderedCollection(SequenceableCollection)>>withIndexDo: [] in OrderedCollection(SequenceableCollection)>>do:displayingProgress: {[:bar | self withIndexDo: [:each :i | bar value: i. aBlock value: e...]} [] in ProgressInitiationException>>defaultMorphicAction {[result := workBlock value: progress]} BlockContext>>ensure: ProgressInitiationException>>defaultMorphicAction ProgressInitiationException>>defaultAction UndefinedObject>>handleSignal: MethodContext(ContextPart)>>handleSignal: MethodContext(ContextPart)>>handleSignal: ProgressInitiationException(Exception)>>signal ProgressInitiationException>>display:at:from:to:during: ProgressInitiationException class>>display:at:from:to:during: ByteString(String)>>displayProgressAt:from:to:during: OrderedCollection(SequenceableCollection)>>do:displayingProgress: MCConfiguration>>depsSatisfying:versionDo:displayingProgress: ...etc...
On Jul 4, 2005, at 3:47 PM, bert@impara.de wrote:
On Jul 4, 2005, at 1:09 AM, Doug Way wrote:
... However, there's a problem with the MCConfiguration install. It generally works and gets most of the way through it, but it hits an error trying to install the SMBase-dew.64 package. ...
Unfortunately, I can't look into this right now.
But there might be a bug in SqueakSource - the MCConf upgrade process tries to download the .mcd first (because that's much more efficient), and if that fails, it downloads the .mcz. So the server should return an error if it cannot build the diff. However, it might be that I never tried that because we usually have all package versions in the repository. ...
Thanks Bert, that was enough for me to go on, at least. I came up with a fix in MCConfiguration>>versionNamed:for:from: so that it checks if the base version is in the repository first. It may be better to fix in SqueakSource, but this seems like a reasonable fix for now.
And of course, I used the new process to commit the fix in 39a. :-) So, there is a new MonticelloConfigurations-dew.32 in the 39a repository at source.squeakfoundation.org with the fix. I also changed the MCConfig update #6676 to include this new version. (Since it was broken before, I just overwrote it in place.) You can grab the change for the "master" MCConfigurations repository (at Impara?) if it looks good to you.
So anyway, the new 3.9alpha update stream is working! It installs SMBase and everything else just fine.
One catch, though... the resulting image is dramatically larger than the original. 3.8-6665 was 11.9MB, the new 3.9a-6676 is 27.7MB... oof. Not sure if this is a known problem or if it's something that can be easily cleaned up, I've haven't looked into it yet. The .changes file is the same size.
In any case, I put this new 3.9a-6676 image + changes in a zip file at http://update.squeakfoundation.org/external/Squeak3.9a-6676.zip for now. Check it out. We may want to hold off on making this more publicly available until we look into why the image is so large.
- Doug
Am 05.07.2005 um 06:08 schrieb Doug Way:
On Jul 4, 2005, at 3:47 PM, bert@impara.de wrote:
On Jul 4, 2005, at 1:09 AM, Doug Way wrote:
... However, there's a problem with the MCConfiguration install. It generally works and gets most of the way through it, but it hits an error trying to install the SMBase-dew.64 package. ...
Unfortunately, I can't look into this right now.
But there might be a bug in SqueakSource - the MCConf upgrade process tries to download the .mcd first (because that's much more efficient), and if that fails, it downloads the .mcz. So the server should return an error if it cannot build the diff. However, it might be that I never tried that because we usually have all package versions in the repository. ...
Thanks Bert, that was enough for me to go on, at least. I came up with a fix in MCConfiguration>>versionNamed:for:from: so that it checks if the base version is in the repository first. It may be better to fix in SqueakSource, but this seems like a reasonable fix for now.
And of course, I used the new process to commit the fix in 39a. :-) So, there is a new MonticelloConfigurations-dew.32 in the 39a repository at source.squeakfoundation.org with the fix. I also changed the MCConfig update #6676 to include this new version. (Since it was broken before, I just overwrote it in place.) You can grab the change for the "master" MCConfigurations repository (at Impara?) if it looks good to you.
Ok. I'll be back to Squeak work next week so I can look.
So anyway, the new 3.9alpha update stream is working! It installs SMBase and everything else just fine.
One catch, though... the resulting image is dramatically larger than the original. 3.8-6665 was 11.9MB, the new 3.9a-6676 is 27.7MB... oof. Not sure if this is a known problem or if it's something that can be easily cleaned up, I've haven't looked into it yet. The .changes file is the same size.
In any case, I put this new 3.9a-6676 image + changes in a zip file at http://update.squeakfoundation.org/external/Squeak3.9a-6676.zip for now. Check it out. We may want to hold off on making this more publicly available until we look into why the image is so large.
Make sure you flush the Monticello caches (there is a menu option for that in the MC browser's repository list menu).
- Bert -
On Tue, 5 Jul 2005 11:31:26 +0200, "Bert Freudenberg" bert@impara.de said:
Am 05.07.2005 um 06:08 schrieb Doug Way:
One catch, though... the resulting image is dramatically larger than the original. 3.8-6665 was 11.9MB, the new 3.9a-6676 is 27.7MB... oof. Not sure if this is a known problem or if it's something that can be easily cleaned up, I've haven't looked into it yet. ...
Make sure you flush the Monticello caches (there is a menu option for that in the MC browser's repository list menu).
Ahh, yes, when I do this, I get a little dialog saying "35 versions flushed, 30,322,492 bytes reclaimed". Whew. :) The saved image is back down to about 13MB.
I'll upload this smaller image/changes pair as a zip file later tonight.
I suppose I could even add an update to do this cache flushing, just for this initial loading of all packages.
- Doug
Am 05.07.2005 um 16:51 schrieb Doug Way:
On Tue, 5 Jul 2005 11:31:26 +0200, "Bert Freudenberg" bert@impara.de said:
Am 05.07.2005 um 06:08 schrieb Doug Way:
One catch, though... the resulting image is dramatically larger than the original. 3.8-6665 was 11.9MB, the new 3.9a-6676 is 27.7MB... oof. Not sure if this is a known problem or if it's something that can be easily cleaned up, I've haven't looked into it yet. ...
Make sure you flush the Monticello caches (there is a menu option for that in the MC browser's repository list menu).
Ahh, yes, when I do this, I get a little dialog saying "35 versions flushed, 30,322,492 bytes reclaimed". Whew. :) The saved image is back down to about 13MB.
Well, the size guestimate might not be quite correct, but it adds up to quite a lot memory.\
I'll upload this smaller image/changes pair as a zip file later tonight.
I suppose I could even add an update to do this cache flushing, just for this initial loading of all packages.
Yes ... we might also add a preference to flush the caches before saving the image ... Let's hear Avi's opinion on that.
- Bert -
Hi doug
I got a look at the image. Thanks for the work. Now how do we proceed? - Do we try to shrink a bit the packages because some of them are huge? - Do we try to get used to it by treating the pending fixes first? - to create a new package should I create a project on source.squeakfoundation.org? I guess so. - I gave a try I loaded 6517KCPRemoveComponent1.cs and republish Morphic. - I got an error HTTP unauthorized 401 at the end of the publish
- I removed from the 6517KCPRemoveComponent1 from update.list (this one a really simple clean that I did and that was in the 3.9 alpha stream).
- then does it means that I should recreate a new cs configuration changing Morphic-dew.1 into Morphi-sd.2? stef
On Tue, 5 Jul 2005 22:53:17 +0200, "stéphane ducasse" ducasse@iam.unibe.ch said:
Hi doug
I got a look at the image. Thanks for the work. Now how do we proceed?
- Do we try to shrink a bit the packages because some of them are huge?
- Do we try to get used to it by treating the pending fixes first?
Probably we should just work on a few pending fixes first, so we get used to the new process.
- to create a new package should I create a project on
source.squeakfoundation.org? I guess so.
Would this be for one of the 35 packages which doesn't have a master repository yet? (such as System or Kernel) We might want to wait on that for a bit until we know who will be the maintainers.
- I gave a try I loaded 6517KCPRemoveComponent1.cs and republish Morphic. - I got an error HTTP unauthorized 401 at the end of the publish
Ah, here you just need to click on the 39a repository in the right pane of the MC browser and "edit repository info" to include your source.sqf.org userid and password. Since you are set as a developer on the 39a project, you should then have access to save/publish a new version.
I guess we can do a little bit of testing like this but we shouldn't go too crazy with a lot of changes directly in the 39a repository right away. As an admin, it looks like I have access to delete versions, so if there's an obvious mistake, we can always clean it up.
- I removed from the 6517KCPRemoveComponent1 from update.list (this
one a really simple clean that I did and that was in the 3.9 alpha stream).
Actually you don't have to worry about editing any of that old stuff in updates.list now, I marked that whole section of the file as "#Squeak3.9alphaold" so all of those 6517 etc updates are now orphaned.
- then does it means that I should recreate a new cs configuration
changing Morphic-dew.1 into Morphic-sd.2?
Actually no, we only need to do that if we have a DoIt which has to go in the image as an update. See Andreas' original email listed on this page: http://minnow.cc.gatech.edu/squeak/5718
We should just simply load all of the latest packages after checking the update stream. Probably we should directly tweak the "update code from server" command so that this is done automatically after checking the update stream. Um, Andreas mentions that the latest packages should be loaded in a "well-defined order", but I'm not sure where that is defined or how it is done. Is the most recent MCConfiguration load order still available?
- Doug
- then does it means that I should recreate a new cs configuration
changing Morphic-dew.1 into Morphic-sd.2?
Actually no, we only need to do that if we have a DoIt which has to go in the image as an update. See Andreas' original email listed on this page: http://minnow.cc.gatech.edu/squeak/5718
We should just simply load all of the latest packages after checking the update stream. Probably we should directly tweak the "update code from server" command so that this is done automatically after checking the update stream. Um, Andreas mentions that the latest packages should be loaded in a "well-defined order", but I'm not sure where that is defined or how it is done. Is the most recent MCConfiguration load order still available?
Indeed this can be a key aspect of load.... the order ;)
- Doug
We should just simply load all of the latest packages after checking the update stream. Probably we should directly tweak the "update code from server" command so that this is done automatically after checking the update stream. Um, Andreas mentions that the latest packages should be loaded in a "well-defined order", but I'm not sure where that is defined or how it is done. Is the most recent MCConfiguration load order still available?
In Tweak we have basically a fixed load order (==configuration) which originates from the configuration used in the original build. But the main reason for wanting to have a well-defined load order is that typically, if there are sets of cross-package changes, they follow already established dependencies. Meaning that if I change a base class to include a new property and then some higher-level code to use it the load-order simply ensures that by default we load the dependent package version first. It may work without the load order, too, but the way it is right now is just clean and simple.
Cheers, - Andreas
On Wed, 06 Jul 2005 08:13:59 +0200, "Andreas Raab" andreas.raab@gmx.de said:
In Tweak we have basically a fixed load order (==configuration) which originates from the configuration used in the original build. But the main reason for wanting to have a well-defined load order is that typically, if there are sets of cross-package changes, they follow already established dependencies.
Yes, makes sense. That should reduce the need for special update changesets to load packages in a special order.
Any suggestions on how I should implement this? I guess we could just have a method defined somewhere (in the load-updates code) which has a hardcoded, ordered list of the 35 package names. I guess this list won't change very often. (The 3.9a-6676 image doesn't seem to be hanging onto the MCConfiguration instance that was loaded in update #6676, otherwise I might try to use that.)
Meaning that if I change a base class to include a new property and then some higher-level code to use it the load-order simply ensures that by default we load the dependent package version first.
The first half of your sentence makes it sound like the prerequisite/base packages (e.g. Kernel) should be loaded first, which I think sounds right. Then the outer ("dependent") packages later, not earlier, right? ("dependent" is the opposite of "prerequisite" AFAIK.)
- Doug
Hi Doug -
In Tweak we have basically a fixed load order (==configuration) which originates from the configuration used in the original build. But the main reason for wanting to have a well-defined load order is that typically, if there are sets of cross-package changes, they follow already established dependencies.
Yes, makes sense. That should reduce the need for special update changesets to load packages in a special order.
Yes. That's another implicit advantage - most of the time stuff just "works" by loading the latest package versions.
Any suggestions on how I should implement this? I guess we could just have a method defined somewhere (in the load-updates code) which has a hardcoded, ordered list of the 35 package names. I guess this list won't change very often. (The 3.9a-6676 image doesn't seem to be hanging onto the MCConfiguration instance that was loaded in update #6676, otherwise I might try to use that.)
What we've done so far is to have a global which stores the latest version of a configuration and then just works issuing a "updateFromRepositories". It's very simple stuff - if you are curious check out CProjectBuilder>>loadUpdates and updateFromRepositories.
Meaning that if I change a base class to include a new property and then some higher-level code to use it the load-order simply ensures that by default we load the dependent package version first.
The first half of your sentence makes it sound like the prerequisite/base packages (e.g. Kernel) should be loaded first, which I think sounds right. Then the outer ("dependent") packages later, not earlier, right? ("dependent" is the opposite of "prerequisite" AFAIK.)
Yes.
Cheers, - Andreas
Am 07.07.2005 um 08:35 schrieb Andreas Raab:
Hi Doug -
In Tweak we have basically a fixed load order (==configuration) which originates from the configuration used in the original build. But the main reason for wanting to have a well-defined load order is that typically, if there are sets of cross-package changes, they follow already established dependencies.
Yes, makes sense. That should reduce the need for special update changesets to load packages in a special order.
Yes. That's another implicit advantage - most of the time stuff just "works" by loading the latest package versions.
Indeed - given that you have a "managed" repository, which has a linear numbering scheme and everyone agrees to merge in the latest version (so that when you upload v.n+1 it has v.n as ancestor). Maybe one could use squeaksource's blessing feature to enforce that.
So if you have cross-package changes, you just post all of them, and the load order is defined by the stored configuration (see below).
Any suggestions on how I should implement this? I guess we could just have a method defined somewhere (in the load-updates code) which has a hardcoded, ordered list of the 35 package names. I guess this list won't change very often. (The 3.9a-6676 image doesn't seem to be hanging onto the MCConfiguration instance that was loaded in update #6676, otherwise I might try to use that.)
That "hardcoded, ordered list of the 35 package names" would just be like the literal configuration that you issue as changeset (without the #upgrade call), but put into a method. It is precisely that literal representation why the configuration is as simple as it is now (compared to a full-blown MCVersion). It just defines a load order for a few packages, that's all :)
What we've done so far is to have a global which stores the latest version of a configuration and then just works issuing a "updateFromRepositories". It's very simple stuff - if you are curious check out CProjectBuilder>>loadUpdates and updateFromRepositories.
Yep, this works nicely in Tweak. Pasting the methods below for easier reference. I stripped out the progress display and logging.
loadUpdates TweakUpdateStreamManager loadUpdates. "from update stream" self updateFromRepositories.
updateFromRepositories | map | map := self defaultConfiguration. [map updateFromRepositories. map upgrade. DefaultConfiguration == nil] whileTrue. "repeat if we have a new map"
defaultConfiguration ^DefaultConfiguration ifNil: [ DefaultConfiguration := MCConfiguration fromArray: #( ... )]
There is a hook to set DefaultConfiguration to nil whenever the #defaultConfiguration method is recompiled (but this relies on Tweak semantics). Alternatively, you might have a class-side #initialize method that contains the new configuration, so it would automatically get overwritten after an update, and you test for map identity in #updateFromRepositories rather than testing for nil.
The call to #updateFromRepositories in #loadUpdates might possibly be guarded by a "bleeding edge" preference. That way, normal users only would get the "stable" configuratons explicitly posted to the stream, while the system developers would get all the latest packages from the repository.
- Bert -
On Jul 7, 2005, at 6:16 AM, Bert Freudenberg wrote:
Am 07.07.2005 um 08:35 schrieb Andreas Raab:
Any suggestions on how I should implement this? I guess we could just have a method defined somewhere (in the load-updates code) which has a hardcoded, ordered list of the 35 package names. I guess this list won't change very often. (The 3.9a-6676 image doesn't seem to be hanging onto the MCConfiguration instance that was loaded in update #6676, otherwise I might try to use that.)
That "hardcoded, ordered list of the 35 package names" would just be like the literal configuration that you issue as changeset (without the #upgrade call), but put into a method. It is precisely that literal representation why the configuration is as simple as it is now (compared to a full-blown MCVersion). It just defines a load order for a few packages, that's all :)
Right, sounds good.
I haven't tried adding this yet to the update stream.
This may become obvious when I look into it, but I assume this MC configuration will load the very latest versions of each package, not necessarily the specified versions? (That's what I was looking for... a simple "load order" configuration and not a configuration that only loaded specific versions.) I think that's what the #upgrade does if I remember right, but I'll have to try it.
I assume the CProjectBuilder class below is in a particular Tweak package. Meaning that if you wanted to change the list of packages in #defaultConfiguration, the #defaultConfiguration method in that package would be updated. (In other words, that particular package would sort of store the configuration.) But I guess it wouldn't change that often, if it was simply used for the load order (and didn't have to be updated every time one package in the list bumped versions).
- Doug
What we've done so far is to have a global which stores the latest version of a configuration and then just works issuing a "updateFromRepositories". It's very simple stuff - if you are curious check out CProjectBuilder>>loadUpdates and updateFromRepositories.
Yep, this works nicely in Tweak. Pasting the methods below for easier reference. I stripped out the progress display and logging.
loadUpdates TweakUpdateStreamManager loadUpdates. "from update stream" self updateFromRepositories.
updateFromRepositories | map | map := self defaultConfiguration. [map updateFromRepositories. map upgrade. DefaultConfiguration == nil] whileTrue. "repeat if we have a new map"
defaultConfiguration ^DefaultConfiguration ifNil: [ DefaultConfiguration := MCConfiguration fromArray: #( ... )]
There is a hook to set DefaultConfiguration to nil whenever the #defaultConfiguration method is recompiled (but this relies on Tweak semantics). Alternatively, you might have a class-side #initialize method that contains the new configuration, so it would automatically get overwritten after an update, and you test for map identity in #updateFromRepositories rather than testing for nil.
The call to #updateFromRepositories in #loadUpdates might possibly be guarded by a "bleeding edge" preference. That way, normal users only would get the "stable" configuratons explicitly posted to the stream, while the system developers would get all the latest packages from the repository.
Am 10.07.2005 um 06:17 schrieb Doug Way:
This may become obvious when I look into it, but I assume this MC configuration will load the very latest versions of each package, not necessarily the specified versions?
No, installing always uses the version specified in the map:
MCConfiguration>>load loads exactly the versions specified in the configuration, overwriting even newer versions.
MCConfiguration>>merge merges exactly the versions specified in the configuration into the image.
MCConfiguration>>upgrade installs (*) exactly the versions specified in the configuration. However, it does not install a version if the one in the image is newer (that is, the version in the config map is an ancestor of the on in the image). (*) The preference #upgradeIsMerge governs the method for installing, that is, loading or merging. Normal users will want to use load, whereas system developers most surely do not want their local versions clobbered by an upgrade, so they'll enable that preference.
So if you look into a changeset installing a config map, it creates a config map and installs (using the #upgrade method) exactly those versions. You need that for various scenarios, just installing the latest packages when you are way behind would almost be guaranteed to not work.
(That's what I was looking for... a simple "load order" configuration and not a configuration that only loaded specific versions.) I think that's what the #upgrade does if I remember right, but I'll have to try it.
MCConfiguration>>updateFromRepositories modifies the configuration to point to the latest version it can find in the repositories. That is, it takes the old load order and plugs in the newest version of each package. It does not install any version.
So, to use an existing config map specifying the load order, but load the newest stuff, you'ld do:
map updateFromRepositories. map upgrade.
(incidentally, that's what the code below says ;-)
I assume the CProjectBuilder class below is in a particular Tweak package. Meaning that if you wanted to change the list of packages in #defaultConfiguration, the #defaultConfiguration method in that package would be updated. (In other words, that particular package would sort of store the configuration.)
Yes, the class holding that should go into the "core" package or even into its own.
But I guess it wouldn't change that often, if it was simply used for the load order (and didn't have to be updated every time one package in the list bumped versions).
Yes, it changes rather rarely.
- Doug
What we've done so far is to have a global which stores the latest version of a configuration and then just works issuing a "updateFromRepositories". It's very simple stuff - if you are curious check out CProjectBuilder>>loadUpdates and updateFromRepositories.
Yep, this works nicely in Tweak. Pasting the methods below for easier reference. I stripped out the progress display and logging.
loadUpdates TweakUpdateStreamManager loadUpdates. "from update stream" self updateFromRepositories.
updateFromRepositories | map | map := self defaultConfiguration. [map updateFromRepositories. map upgrade. DefaultConfiguration == nil] whileTrue. "repeat if we have a new map"
defaultConfiguration ^DefaultConfiguration ifNil: [ DefaultConfiguration := MCConfiguration fromArray: #( ... )]
There is a hook to set DefaultConfiguration to nil whenever the #defaultConfiguration method is recompiled (but this relies on Tweak semantics). Alternatively, you might have a class-side #initialize method that contains the new configuration, so it would automatically get overwritten after an update, and you test for map identity in #updateFromRepositories rather than testing for nil.
The call to #updateFromRepositories in #loadUpdates might possibly be guarded by a "bleeding edge" preference. That way, normal users only would get the "stable" configuratons explicitly posted to the stream, while the system developers would get all the latest packages from the repository.
- Bert -
Alright, that all makes sense below. I'm trying to create a #loadLatestMCPackages method along the lines of what you describe.
Earlier this week, as a test, I tried to run MCConfiguration>>updateFromRepositories on the MCConfiguration defined in update #6676. It hung, and I didn't have time to look into it further earlier, but I have a bit more time now. :)
So, the main problem here is that it tries to load an .mcd file for the Kernel package: Kernel-md.5(dew.1).mcd, which doesn't exist. Both the original Kernel-dew. and the newer Kernel-md.5 versions are in the 39a repository. (Marcus has been porting a few fixes to the new 3.9a packages, thus the new versions... cool stuff!)
There are two sub-problems here:
1. It just hangs when it tries to get a file which doesn't exist on SqueakSource. It should return some sort of not found error instead. (You mention this exact problem in your 7/12 email... apparently the impara.de SqueakSource server doesn't have this problem.) Help from Avi/Cees or a SqueakSource guy? :-)
2. The .mcd file does not exist. How is this intended to work? A. Is the SqueakSource server supposed to generate the .mcd file on the fly, or maybe upon the saving of a new version? (Makes sense, that's a lot less bandwidth for these big packages.) B. Should the MCConfig code in the image not be trying to download .mcd files? C. Do we need to be saving .mcd files and not .mcz when we save new versions to the repository?
(I'm pretty sure the answer isn't C, that would be dumb. I'm guessing A.)
Anyway, once we get this working, I'll create a #loadLatestMCPackages method and incorporate that as part of the Utilities>>updateFromServer code, and issue that as a new update. Then we should be ready to go!
(I'll probably do something like the class-side #initialize containing the configuration, so that it gets overwritten after an update, and then test it for equality, which you mentioned earlier.)
Mmm, I may just put this "configuration" in the System package for now (where the rest of the updating code is), since it's really just a non-version-specific load order which would not change often. If that becomes an issue we can move it into its own package.
(Speaking of non-version-specific, it might be nice to be able to define a non-version-specific MCConfiguration for this purpose, so that it's clear in the method that the version numbers aren't meant to be used. E.g. map _ MCConfiguration fromArray: #(repository ('http://source.squeakfoundation.org/39a') dependency ('Kernel') dependency ('Collections') ... etc. Although perhaps the initial versions or uuid's are needed for ancestry info? Anyway, not a big deal.)
- Doug
On Jul 10, 2005, at 5:40 AM, Bert Freudenberg wrote:
Am 10.07.2005 um 06:17 schrieb Doug Way:
This may become obvious when I look into it, but I assume this MC configuration will load the very latest versions of each package, not necessarily the specified versions?
No, installing always uses the version specified in the map:
MCConfiguration>>load loads exactly the versions specified in the configuration, overwriting even newer versions.
MCConfiguration>>merge merges exactly the versions specified in the configuration into the image.
MCConfiguration>>upgrade installs (*) exactly the versions specified in the configuration. However, it does not install a version if the one in the image is newer (that is, the version in the config map is an ancestor of the on in the image). (*) The preference #upgradeIsMerge governs the method for installing, that is, loading or merging. Normal users will want to use load, whereas system developers most surely do not want their local versions clobbered by an upgrade, so they'll enable that preference.
So if you look into a changeset installing a config map, it creates a config map and installs (using the #upgrade method) exactly those versions. You need that for various scenarios, just installing the latest packages when you are way behind would almost be guaranteed to not work.
(That's what I was looking for... a simple "load order" configuration and not a configuration that only loaded specific versions.) I think that's what the #upgrade does if I remember right, but I'll have to try it.
MCConfiguration>>updateFromRepositories modifies the configuration to point to the latest version it can find in the repositories. That is, it takes the old load order and plugs in the newest version of each package. It does not install any version.
So, to use an existing config map specifying the load order, but load the newest stuff, you'ld do:
map updateFromRepositories. map upgrade.
(incidentally, that's what the code below says ;-)
I assume the CProjectBuilder class below is in a particular Tweak package. Meaning that if you wanted to change the list of packages in #defaultConfiguration, the #defaultConfiguration method in that package would be updated. (In other words, that particular package would sort of store the configuration.)
Yes, the class holding that should go into the "core" package or even into its own.
But I guess it wouldn't change that often, if it was simply used for the load order (and didn't have to be updated every time one package in the list bumped versions).
Yes, it changes rather rarely.
Am 17.07.2005 um 00:48 schrieb Doug Way:
- The .mcd file does not exist. How is this intended to work?
A. Is the SqueakSource server supposed to generate the .mcd file on the fly, or maybe upon the saving of a new version? (Makes sense, that's a lot less bandwidth for these big packages.) B. Should the MCConfig code in the image not be trying to download .mcd files? C. Do we need to be saving .mcd files and not .mcz when we save new versions to the repository?
(I'm pretty sure the answer isn't C, that would be dumb. I'm guessing A.)
Yes it's A, a diff is generated on the fly when it is requested first, and then cached. Building a diff for version n against the previous n-1 versions would be O(n^2), and we might never need all. So it's only built on demand.
(Speaking of non-version-specific, it might be nice to be able to define a non-version-specific MCConfiguration for this purpose, so that it's clear in the method that the version numbers aren't meant to be used. E.g. map _ MCConfiguration fromArray: #(repository ('http://source.squeakfoundation.org/39a') dependency ('Kernel') dependency ('Collections') ... etc. Although perhaps the initial versions or uuid's are needed for ancestry info? Anyway, not a big deal.)
Well, if it's solely used to define the load order, and the versions are named after the package, that should indeed be sufficient.
The reason a config map item has the package, version name, and uuid is that the version name could be anything, although in practice we usually rely on it matching the package name. The UUID should be used to make sure we load the correct version, because version names are not guaranteed to be unique (however, I think at the moment the code does not actually verify that).
- Bert -
On Jul 17, 2005, at 4:43 PM, Bert Freudenberg wrote:
Am 17.07.2005 um 00:48 schrieb Doug Way:
- The .mcd file does not exist. How is this intended to work?
A. Is the SqueakSource server supposed to generate the .mcd file on the fly, or maybe upon the saving of a new version? (Makes sense, that's a lot less bandwidth for these big packages.) B. Should the MCConfig code in the image not be trying to download .mcd files? C. Do we need to be saving .mcd files and not .mcz when we save new versions to the repository?
(I'm pretty sure the answer isn't C, that would be dumb. I'm guessing A.)
Yes it's A, a diff is generated on the fly when it is requested first, and then cached. Building a diff for version n against the previous n-1 versions would be O(n^2), and we might never need all. So it's only built on demand.
Makes sense.
So, the problem is that the squeaksource repository at sqf.org is not generating any .mcd files for whatever reason. It looks like this .mcd generation is a relatively new feature... is it something that was just added in the impara.de version of squeaksource? I see that http://squeaksource.com at berne does not seem to have the .mcd capability either, although at least it returns a "not found" error rather than hanging.
Actually, it's pretty easy to plug the URLs into a browser to test the various squeaksource repositories, since it's just a simple HTTP get. These links show how the various repositories behave:
works: http://source.impara.de/mc/Monticello-bf.260.mcz http://source.impara.de/mc/Monticello-bf.261.mcz http://source.impara.de/mc/Monticello-bf.261%28bf.260%29.mcd
works: http://source.squeakfoundation.org/39a/Kernel-dew.1.mcz http://source.squeakfoundation.org/39a/Kernel-md.5.mcz
doesn't work: (hangs!) http://source.squeakfoundation.org/39a/Kernel-md.5%28dew.1%29.mcd
works: http://squeaksource.com/Monticello/Monticello-avi.231.mcz http://squeaksource.com/Monticello/Monticello-avi.232.mcz
doesn't work: ("not found" error) http://squeaksource.com/Monticello/Monticello-avi.232%28avi.231%29.mcd
- Doug
Am 18.07.2005 um 06:00 schrieb Doug Way:
On Jul 17, 2005, at 4:43 PM, Bert Freudenberg wrote:
Am 17.07.2005 um 00:48 schrieb Doug Way:
- The .mcd file does not exist. How is this intended to work?
A. Is the SqueakSource server supposed to generate the .mcd file on the fly, or maybe upon the saving of a new version? (Makes sense, that's a lot less bandwidth for these big packages.) B. Should the MCConfig code in the image not be trying to download .mcd files? C. Do we need to be saving .mcd files and not .mcz when we save new versions to the repository?
(I'm pretty sure the answer isn't C, that would be dumb. I'm guessing A.)
Yes it's A, a diff is generated on the fly when it is requested first, and then cached. Building a diff for version n against the previous n-1 versions would be O(n^2), and we might never need all. So it's only built on demand.
Makes sense.
So, the problem is that the squeaksource repository at sqf.org is not generating any .mcd files for whatever reason. It looks like this .mcd generation is a relatively new feature... is it something that was just added in the impara.de version of squeaksource?
Yes, I implemented the mcd stuff after we installed the Monticello- based update stream for Tweak. It became unbearably slow, updating sometimes took 10 minutes. While downloading the packages was actually quite fast (we're all on broadband), the diffing against the in-image version was taking most of the time. So the actual time saver is that the diff is loaded almost like a changeset, if the in- image version is unmodified and the diff's base version matches it.
So it's actually two parts, one on the client side (the optimization for loading mcds without diffing) and the server side (building the right mcd to match the in-image version).
I see that http://squeaksource.com at berne does not seem to have the .mcd capability either, although at least it returns a "not found" error rather than hanging.
Actually, it's pretty easy to plug the URLs into a browser to test the various squeaksource repositories, since it's just a simple HTTP get. These links show how the various repositories behave:
works: http://source.impara.de/mc/Monticello-bf.260.mcz http://source.impara.de/mc/Monticello-bf.261.mcz http://source.impara.de/mc/Monticello-bf.261%28bf.260%29.mcd
works: http://source.squeakfoundation.org/39a/Kernel-dew.1.mcz http://source.squeakfoundation.org/39a/Kernel-md.5.mcz
doesn't work: (hangs!) http://source.squeakfoundation.org/39a/Kernel-md.5%28dew.1%29.mcd
works: http://squeaksource.com/Monticello/Monticello-avi.231.mcz http://squeaksource.com/Monticello/Monticello-avi.232.mcz
doesn't work: ("not found" error) http://squeaksource.com/Monticello/Monticello-avi.232%28avi.231%29.mcd
Did you compare the package versions I sent around?
- Bert -
On Mon, 18 Jul 2005 12:36:38 +0200, "Bert Freudenberg" bert@impara.de said:
Am 18.07.2005 um 06:00 schrieb Doug Way:
... works: http://source.impara.de/mc/Monticello-bf.260.mcz http://source.impara.de/mc/Monticello-bf.261.mcz http://source.impara.de/mc/Monticello-bf.261%28bf.260%29.mcd
works: http://source.squeakfoundation.org/39a/Kernel-dew.1.mcz http://source.squeakfoundation.org/39a/Kernel-md.5.mcz
doesn't work: (hangs!) http://source.squeakfoundation.org/39a/Kernel-md.5%28dew.1%29.mcd ...
Did you compare the package versions I sent around?
Actually, I didn't set up the SqueakSource installation at source.squeakfoundation.org... Cees & Avi did. So I'm not sure which versions are in that install. I'll bug them about it.
- Doug
On Jul 18, 2005, at 12:20 PM, Doug Way wrote:
On Mon, 18 Jul 2005 12:36:38 +0200, "Bert Freudenberg" bert@impara.de said:
doesn't work: (hangs!) http://source.squeakfoundation.org/39a/Kernel-md.5%28dew.1%29.mcd ...
Did you compare the package versions I sent around?
Actually, I didn't set up the SqueakSource installation at source.squeakfoundation.org... Cees & Avi did. So I'm not sure which versions are in that install. I'll bug them about it.
Hmm, haven't heard from them yet, they may be busy or on summer holiday.
I'll see about getting access to the SqueakSource install and playing around with it myself, or setting up a new install. (Probably wouldn't hurt to learn how to install it.)
Basically, 3.9alpha is pretty much at a standstill until we get this diff generation fixed. Should be doable, Impara's install does it, we just need the same configuration.
(Well, it's not completely at a standstill, we can do some harvesting of a few things into the new packages. But the automatic updating of packages via mcconfig/mcd files won't work. I don't think we should make the new process totally public on squeak-dev until we get that side of it working. Alternatively we could just have it install the full .mcz packages every time, but that kills a lot of bandwidth and would probably hammer the SqueakSource server at sqf.org... we might as well just get the .mcd generation working.)
If anyone else knowledgable with SqueakSource wants to help with this, that would be great.
- Doug
On 7/23/05, Doug Way dway@mailcan.com wrote:
Actually, I didn't set up the SqueakSource installation at source.squeakfoundation.org... Cees & Avi did. So I'm not sure which versions are in that install. I'll bug them about it.
Hmm, haven't heard from them yet, they may be busy or on summer holiday.
Hey Doug,
Yeah, I unfortunately don't have a lot of bandwidth until I get back to Vancouver on Aug 16th. So if you want stuff done before then, it may be worthwhile for you to build or modify the SqueakSource install yourself. If there's something I can do to make access easier for you, though, let me know.
Avi
Am 05.07.2005 um 11:31 schrieb Bert Freudenberg:
Am 05.07.2005 um 06:08 schrieb Doug Way:
On Jul 4, 2005, at 3:47 PM, bert@impara.de wrote:
On Jul 4, 2005, at 1:09 AM, Doug Way wrote:
... However, there's a problem with the MCConfiguration install. It generally works and gets most of the way through it, but it hits an error trying to install the SMBase-dew.64 package. ...
Unfortunately, I can't look into this right now.
But there might be a bug in SqueakSource - the MCConf upgrade process tries to download the .mcd first (because that's much more efficient), and if that fails, it downloads the .mcz. So the server should return an error if it cannot build the diff. However, it might be that I never tried that because we usually have all package versions in the repository. ...
Thanks Bert, that was enough for me to go on, at least. I came up with a fix in MCConfiguration>>versionNamed:for:from: so that it checks if the base version is in the repository first. It may be better to fix in SqueakSource, but this seems like a reasonable fix for now.
And of course, I used the new process to commit the fix in 39a. :-) So, there is a new MonticelloConfigurations-dew.32 in the 39a repository at source.squeakfoundation.org with the fix. I also changed the MCConfig update #6676 to include this new version. (Since it was broken before, I just overwrote it in place.) You can grab the change for the "master" MCConfigurations repository (at Impara?) if it looks good to you.
Ok. I'll be back to Squeak work next week so I can look.
It's next week now :)
So I just tried this. There must be something different on your squeaksource installation than on our's.
When I access "http://source.squeakfoundation.org/39a/ MonticelloConfigurations-dew.31(bf.30).mcd" it just hangs. On our server, for "http://source.impara.de/iSqueak/MonticelloConfigurations- dew.31(bf.30).mcd'" I immediately get a 404 not found.
Here's the current list of my versions:
DynamicBindings (DynamicBindings-gk.1) KomHttpServer (KomHttpServer-gk.6) KomServices (KomServices-gk.2) Mewa (Mewa-al.13) Monticello (Monticello-bf.254) MonticelloConfigurations (MonticelloConfigurations-bf.26) PackageInfo-Base (PackageInfo-Base-bf.22) SMBase (SMBase-gk.63) SMLoader (SMLoader-gk.24) Seaside2 (Seaside2-avi.86) SqueakSource (SqueakSource-bf.146) SystemFixes (SystemFixes-bf.1) TinyWiki (TinyWiki-lr.10) TweakMC (TweakMC-bf.11)
- Bert -
On Jul 4, 2005, at 1:09 AM, Doug Way wrote:
So, there is a new MonticelloConfigurations-dew.32 in the 39a repository at source.squeakfoundation.org
Hmm, actually, there are two ... if you look into the history of MonticelloConfigurations-dew.32.mcz, it has a version of the same name (but a different UUID) as its ancestor. That is going to be problematic as in squeaksource versions are not referenced by UUID but by name. So the second version overwrote the dictionary entry of the first. It might work if you upload the first version (with dew.31 as ancestor) again.
Btw, we copied your initial set of packages to our repository so we can start our 3.8 changes from there, which should make it easier to merge back later.
- Bert -
On Jul 12, 2005, at 5:57 AM, Bert Freudenberg wrote:
On Jul 4, 2005, at 1:09 AM, Doug Way wrote:
So, there is a new MonticelloConfigurations-dew.32 in the 39a repository at source.squeakfoundation.org
Hmm, actually, there are two ... if you look into the history of MonticelloConfigurations-dew.32.mcz, it has a version of the same name (but a different UUID) as its ancestor. That is going to be problematic as in squeaksource versions are not referenced by UUID but by name. So the second version overwrote the dictionary entry of the first. It might work if you upload the first version (with dew.31 as ancestor) again.
Yes, sorry about that. MonticelloConfigurations-bf.31 was the shared ancestor. Back when I made the change, I think I accidentally saved an initial MonticelloConfigurations-dew.32 without any changes, then I wanted to overwrite/replace that one with a new MonticelloConfigurations-dew.32 which actually did have the changes, so when it went to save MonticelloConfigurations-dew.33, I renamed it to .32 in the commit log dialog. I wondered at the time if this wasn't really proper. (Although it sounds like MC can handle it, but squeaksource can't.)
In any case, it's probably best just to let the automatic numbering happen and not worry about an extra version (with no changes) being in there.
Now that I think about it, I should have gone back and re-loaded MonticelloConfigurations-bf.31 and made the change, that would have had the desired effect. (Hm, should I still go ahead and do that?)
Btw, we copied your initial set of packages to our repository so we can start our 3.8 changes from there, which should make it easier to merge back later.
Sounds good!
- Doug
On Jul 12, 2005, at 11:46 PM, Doug Way wrote:
On Jul 12, 2005, at 5:57 AM, Bert Freudenberg wrote:
Hmm, actually, there are two ... if you look into the history of MonticelloConfigurations-dew.32.mcz, it has a version of the same name (but a different UUID) as its ancestor. That is going to be problematic as in squeaksource versions are not referenced by UUID but by name. So the second version overwrote the dictionary entry of the first. It might work if you upload the first version (with dew.31 as ancestor) again.
Yes, sorry about that. MonticelloConfigurations-bf.31 was the shared ancestor. Back when I made the change, I think I accidentally saved an initial MonticelloConfigurations-dew.32 without any changes, then I wanted to overwrite/replace that one with a new MonticelloConfigurations-dew.32 which actually did have the changes, so when it went to save MonticelloConfigurations-dew.33, I renamed it to .32 in the commit log dialog. I wondered at the time if this wasn't really proper. (Although it sounds like MC can handle it, but squeaksource can't.)
Ok, I went ahead and saved a new version MonticelloConfigurations-dew.33 in the 39a repository which should have a clean ancestry (going straight from .31). Try using that one if .32 gave you problems.
- Doug
Am 12.07.2005 um 11:42 schrieb Bert Freudenberg:
Am 05.07.2005 um 11:31 schrieb Bert Freudenberg:
Am 05.07.2005 um 06:08 schrieb Doug Way:
On Jul 4, 2005, at 3:47 PM, bert@impara.de wrote:
On Jul 4, 2005, at 1:09 AM, Doug Way wrote:
... However, there's a problem with the MCConfiguration install. It generally works and gets most of the way through it, but it hits an error trying to install the SMBase-dew.64 package. ...
Unfortunately, I can't look into this right now.
But there might be a bug in SqueakSource - the MCConf upgrade process tries to download the .mcd first (because that's much more efficient), and if that fails, it downloads the .mcz. So the server should return an error if it cannot build the diff. However, it might be that I never tried that because we usually have all package versions in the repository. ...
Thanks Bert, that was enough for me to go on, at least. I came up with a fix in MCConfiguration>>versionNamed:for:from: so that it checks if the base version is in the repository first. It may be better to fix in SqueakSource, but this seems like a reasonable fix for now.
And of course, I used the new process to commit the fix in 39a. :-) So, there is a new MonticelloConfigurations-dew.32 in the 39a repository at source.squeakfoundation.org with the fix. I also changed the MCConfig update #6676 to include this new version. (Since it was broken before, I just overwrote it in place.) You can grab the change for the "master" MCConfigurations repository (at Impara?) if it looks good to you.
Ok. I'll be back to Squeak work next week so I can look.
It's next week now :)
So I just tried this. There must be something different on your squeaksource installation than on our's.
When I access "http://source.squeakfoundation.org/39a/ MonticelloConfigurations-dew.31(bf.30).mcd" it just hangs. On our server, for "http://source.impara.de/iSqueak/ MonticelloConfigurations-dew.31(bf.30).mcd'" I immediately get a 404 not found.
Here's the current list of my versions:
DynamicBindings (DynamicBindings-gk.1) KomHttpServer (KomHttpServer-gk.6) KomServices (KomServices-gk.2) Mewa (Mewa-al.13) Monticello (Monticello-bf.254) MonticelloConfigurations (MonticelloConfigurations-bf.26) PackageInfo-Base (PackageInfo-Base-bf.22) SMBase (SMBase-gk.63) SMLoader (SMLoader-gk.24) Seaside2 (Seaside2-avi.86) SqueakSource (SqueakSource-bf.146) SystemFixes (SystemFixes-bf.1) TinyWiki (TinyWiki-lr.10) TweakMC (TweakMC-bf.11)
I just fixed something else ... your Multilingual package contains source code with embedded "0" characters in GreekEnvironment>>supportedLanguages. When trying to build the diff, the parser crashes on this method, and I had no error handling for this (it should not happen, anyway ;-). So now I just catch any error that occurs during diff-building and fail properly (that is, answer a 404 and never try to build that particular diff again).
- Bert -
Hi Bert. Ok, Cees set me up with access to the machine at http://source.squeakfoundation.org. Maybe I'll be able to fix this after all, possibly even before Avi gets back. ;)
Looking at the SqueakSource image that was running, it does have a few differences in the package list from yours. Here's the list:
DynamicBindings (DynamicBindings-gk.1) * KomHttpServer (KomHttpServer-gk.6) KomServices (KomServices-gk.2) Mewa (Mewa-al.13) * Monticello (Monticello-avi.182) SMBase (SMBase-gk.63) SMLoader (SMLoader-gk.24) Scheduler (Scheduler-jrp.18) Seaside2 (Seaside2-avi.86) * SqueakSource (SqueakSource-bf.146) TinyWiki (TinyWiki-lr.10)
Probably the Monticello & MonticelloConfiguration packages are the main issue. I'll try to track down the other few packages too, though.
I may see if I can get it working locally first, and then try to replace the one at sqf.org. (Hopefully the old accounts/data files would still be okay after replacing the squeaksource image. It looks like the .mcz files are all in separate dirs so those should be fine, which is the most important thing.)
- Doug
On Jul 12, 2005, at 5:42 AM, Bert Freudenberg wrote:
... So I just tried this. There must be something different on your squeaksource installation than on our's.
When I access "http://source.squeakfoundation.org/39a/MonticelloConfigurations- dew.31(bf.30).mcd" it just hangs. On our server, for "http://source.impara.de/iSqueak/MonticelloConfigurations- dew.31(bf.30).mcd'" I immediately get a 404 not found.
Here's the current list of my versions:
DynamicBindings (DynamicBindings-gk.1) KomHttpServer (KomHttpServer-gk.6) KomServices (KomServices-gk.2) Mewa (Mewa-al.13) Monticello (Monticello-bf.254) MonticelloConfigurations (MonticelloConfigurations-bf.26) PackageInfo-Base (PackageInfo-Base-bf.22) SMBase (SMBase-gk.63) SMLoader (SMLoader-gk.24) Seaside2 (Seaside2-avi.86) SqueakSource (SqueakSource-bf.146) SystemFixes (SystemFixes-bf.1) TinyWiki (TinyWiki-lr.10) TweakMC (TweakMC-bf.11)
- Bert -
Am 02.07.2005 um 20:30 schrieb Doug Way:
Or is there a way to delete your packages?
Yes, you can delete versions in the SqueakSource interface - select the version, then this appears as option in the Actions menu on the top left (you must have write-access to see the option, and admin- access to execute it). You can delete one versions or all versions (= delete package). Internally, this just marks the version as deleted, it doesn't touch the files.
- Bert -
Hello,
Right now I'm loading all the packages from the 39a repository into a test image, so I can put together an MCConfiguration.
I have a silly question, but why do we need MCConfiguration? Why not to have a package named SqueakPackages that will depends on all the package of the system. One will just have to load this package to install all the other.
Doug, how did you load "all the packages from the 39a repository" ? One by one ? Manually ?
Cheers, Alexandre
I have a silly question, but why do we need MCConfiguration? Why not to have a package named SqueakPackages that will depends on all the package of the system.
Because you couldn't use that to interleave MC with other updates. What you're proposing is fine if you're only concerned in giving people access to the latest source code but it ain't working very well if you consider the kind of incremental updating process that we've usually provided. MCConfs give us the ability to go through the same steps again, if for example, someone wants to take a 3.8 image and upgrage it to a 3.9 image. If all that's involved is source-code changes that would work with a single "master package" as well but if there are intermediate stages, where certain versions of packages need to be present for the update to make sense you'll have a hard time to make that work.
Cheers, - Andreas
On 6/27/05, Avi Bryant avi.bryant@gmail.com wrote:
Ok, it's back up, and I'm slowly feeding it packages. For now you'll have to access it at port 9090 (http://source.squeakfoundation.org:9090/). Cees, when you get a chance, can you change the ProxyPass lines in the apache config to drop the /seaside/ss part?
Done.
Am 27.06.2005 um 17:59 schrieb Doug Way:
On Mon, 27 Jun 2005 17:15:16 +0200, "Avi Bryant" avi.bryant@gmail.com said:
On 6/27/05, Alexandre Bergel bergel@iam.unibe.ch wrote:
But the Project 39a is still empty...
Ah, yes, it is. We do need to commit stuff there. I just didn't understand the purpose of a script that created new packages when we already have a changeset from Andreas that does so.
Right, we can just use Andreas' changeset to do this partitioning. (which is very similar to Alexandre's script anyway, I think)
Except that the major portion of work was the recategorization. (for anyone who missed the original thread, have a look at the wiki page on http://source.impara.de/iSqueak.html).
I'll populate that project shortly.
Sounds good. By the way, does a SqueakSource "Developer" have access to do this initial populating (adding new packages), or do you need to be an Admin?
You just "populate" by saving versions in monticello.
- Bert -
Except that the major portion of work was the recategorization. (for anyone who missed the original thread, have a look at the wiki page on http://source.impara.de/iSqueak.html).
right. I guess you are referring to the Reorganize.3.8-6662.cs. There are just some categorizations. Bert, I did not deeply checked but I feel that actually it does not assign new category to classes/methods. For instance, the first line of the changeset is
SystemOrganization changeFromString: '(''Kernel-Chronology'' ChronologyConstants Date DateAndTime Duration Month Schedule Stopwatch Time TimeStamp Timespan TimeZone Week Year)
These classes are already included into the class category 'Kernel-Chronology'! Andreas worked on the system change notification, but I do not know what he did (I do not have time to dive into the iSqueak image). I suspect that it uses events triggered by the recategoryzation to create packages.
Alexandre
Am 27.06.2005 um 18:18 schrieb Alexandre Bergel:
Except that the major portion of work was the recategorization. (for anyone who missed the original thread, have a look at the wiki page on http://source.impara.de/iSqueak.html).
right. I guess you are referring to the Reorganize.3.8-6662.cs. There are just some categorizations. Bert, I did not deeply checked but I feel that actually it does not assign new category to classes/ methods. For instance, the first line of the changeset is
SystemOrganization changeFromString: '(''Kernel-Chronology'' ChronologyConstants Date DateAndTime Duration Month Schedule Stopwatch Time TimeStamp Timespan TimeZone Week Year)
These classes are already included into the class category 'Kernel- Chronology'! Andreas worked on the system change notification, but I do not know what he did (I do not have time to dive into the iSqueak image). I suspect that it uses events triggered by the recategoryzation to create packages.
You need to inspect the change set more closely. It may well have redundant categorization of classes where nothing changed - this does not hurt anybody. But there are also many recategorizations of methods. I guess Andreas made the changeset by just filing out the categorization for everything in the system.
- Bert -
On Mon, 27 Jun 2005 18:26:21 +0200, "Bert Freudenberg" bert@impara.de said:
Am 27.06.2005 um 18:18 schrieb Alexandre Bergel:
right. I guess you are referring to the Reorganize.3.8-6662.cs. There are just some categorizations. Bert, I did not deeply checked but I feel that actually it does not assign new category to classes/ methods. For instance, the first line of the changeset is
SystemOrganization changeFromString: '(''Kernel-Chronology'' ChronologyConstants Date DateAndTime Duration Month Schedule Stopwatch Time TimeStamp Timespan TimeZone Week Year)
These classes are already included into the class category 'Kernel- Chronology'! Andreas worked on the system change notification, but I do not know what he did (I do not have time to dive into the iSqueak image). I suspect that it uses events triggered by the recategoryzation to create packages.
You need to inspect the change set more closely. It may well have redundant categorization of classes where nothing changed - this does not hurt anybody. But there are also many recategorizations of methods. I guess Andreas made the changeset by just filing out the categorization for everything in the system.
It is difficult to see exactly what changed in the recategorization, because the changeset includes the whole system as you say.
However, I did see that the Tests package was recategorized into KernelTests, CollectionTests, NetworkTests, etc., which is a significant change and seems like a good change.
Just curious, were any methods actually recategorized into mc-based extension method categories? (I didn't see any.) That would be a bigger change, as it would essentially move the method into a different package as an extension, even though it's still on the same class. A classic example would be moving String>>asUrl from the "converting" to the "*Network" method category, so that it ends up in the Network package. That's more of a real detangling/refactoring than a mere recategorization.
- Doug
Avi,
I do not know if I miss something from some previous mail, but on which server did you post the configuration ?
A classic example would be moving String>>asUrl from the "converting" to the "*Network" method category, so that it ends up in the Network package. That's more of a real detangling/refactoring than a mere recategorization.
Yes. But I think it is important to have a clear process first. Remodularization is something that should come after I think.
I also think that most of the package are too big. For instance, it would be great to have the Flash player dissociated from Balloon. PDA, Postscript From Morphic...
Alexandre
Hi -
[Just subscribed to this list]
It is difficult to see exactly what changed in the recategorization, because the changeset includes the whole system as you say.
Here are all the changes:
a) Change the categories from "Foo-Bar-Tests" to "FooTests-Bar", e.g, "Kernel-Chronology-Tests" => "KernelTests-Chronology" "Collections-Streams-Tests" => "CollectionTests-Streams" The reason being that I want to be able to bootstrap a system without circular dependencies and moving the tests into their own categories makes it easy to load everything in the right order.
b) Merging of some remnant packages which need to be folded into Morphic and which never have been real standalone packages ("CustomEvents", "FlexibleVocabularies", ...)
c) Some arbitrary new top-level packages to get the size of the "System" package down to a reasonable level, including "Files", "Compiler", and "Compression"
d) Some reclassification to avoid remnants such as HTTPClient (a lone class) from "Framework-Download", FancyMailComposition from "EToy-Download" and then some.
The method reclassifications were all due b) e.g., removal of remnant packages.
Just curious, were any methods actually recategorized into mc-based extension method categories?
No. The idea was to get an *initial* structure going not to deal with fine-tuning package boundaries.
Cheers, - Andreas
On 28 juin 05, at 6:13, Andreas Raab wrote:
a) Change the categories from "Foo-Bar-Tests" to "FooTests-Bar", e.g, "Kernel-Chronology-Tests" => "KernelTests-Chronology" "Collections-Streams-Tests" => "CollectionTests-Streams" The reason being that I want to be able to bootstrap a system without circular dependencies and moving the tests into their own categories makes it easy to load everything in the right order.
Hi andreas
I understand and I agree that having separated tests is good and avoid dependencies. Now I do not understand the naming conventions. I miss something but I do not know what. Why the tests of Kernel-Chronology are not simply KernelChronology-Tests? Because MC would put them under Kernel? but we could have a separate package for KernelChronologyTests?
Stef
I understand and I agree that having separated tests is good and avoid dependencies. Now I do not understand the naming conventions. I miss something but I do not know what. Why the tests of Kernel-Chronology are not simply KernelChronology-Tests? Because MC would put them under Kernel? but we could have a separate package for KernelChronologyTests?
Because creating fifty packages for each and every category ending with "-Test" wasn't TSTTCPW. The simplest thing was to reclassify the classes so we have a "Kernel" package and a "KernelTests" package, a "Collection" package and a "CollectionTests" package etc.
Again keep in mind that the goal of this exercise was to *fully* packagize the system - and because of this some compromises had to be made. If you want to make all of these separate packages, by all means, feel free to do so. But that was out of my scope.
Cheers, - Andreas
Ok this was what I wanted to know and I will certainly repackage them separately.
Stef
I understand and I agree that having separated tests is good and avoid dependencies. Now I do not understand the naming conventions. I miss something but I do not know what. Why the tests of Kernel-Chronology are not simply KernelChronology-Tests? Because MC would put them under Kernel? but we could have a separate package for KernelChronologyTests?
Because creating fifty packages for each and every category ending with "-Test" wasn't TSTTCPW. The simplest thing was to reclassify the classes so we have a "Kernel" package and a "KernelTests" package, a "Collection" package and a "CollectionTests" package etc.
Again keep in mind that the goal of this exercise was to *fully* packagize the system - and because of this some compromises had to be made. If you want to make all of these separate packages, by all means, feel free to do so. But that was out of my scope.
Cheers,
- Andreas
Realistically, it's going to take us a little while to get everything working smoothly with this new process.
It depends. I think it is important to have a clear how-to page that describe the process.
But we might as well dive in and get started. It's a big change, but I think it's flexible enough that we will be able to adjust the process if necessary, and we won't end up in a 3.3alpha modules situation.
Squeak evolved a lot since I started to use it. I do not thing that big changes are bad if they make life of people easier.
I will set up a new swiki page covering the new process, as a point of reference.
I am willing to help.
alexandre
On Mon, 27 Jun 2005 18:22:11 +0200, "Alexandre Bergel" bergel@iam.unibe.ch said:
Realistically, it's going to take us a little while to get everything working smoothly with this new process.
It depends. I think it is important to have a clear how-to page that describe the process.
...
I will set up a new swiki page covering the new process, as a point of reference.
I am willing to help.
Thanks Alexandre. Ok, I've set up the swiki page for the new process:
http://minnow.cc.gatech.edu/squeak/5718
It's just an overview at this point, we can add some more specific how-to details soon. Also, it has the somewhat clumsy name "3.9 Packages+Update Stream Process", but I guess that's good enough for now.
- Doug
I think it is a good start. I will definitely contribute to it once we have something that runs.
Alexandre
On Tue, Jun 28, 2005 at 02:02:28PM -0400, Doug Way wrote:
On Mon, 27 Jun 2005 18:22:11 +0200, "Alexandre Bergel" bergel@iam.unibe.ch said:
Realistically, it's going to take us a little while to get everything working smoothly with this new process.
It depends. I think it is important to have a clear how-to page that describe the process.
...
I will set up a new swiki page covering the new process, as a point of reference.
I am willing to help.
Thanks Alexandre. Ok, I've set up the swiki page for the new process:
http://minnow.cc.gatech.edu/squeak/5718
It's just an overview at this point, we can add some more specific how-to details soon. Also, it has the somewhat clumsy name "3.9 Packages+Update Stream Process", but I guess that's good enough for now.
- Doug
I cannot access
source.squeakfoundation.org may be this is my settings but I get The proxy server received an invalid response from an upstream server.
On 27 juin 05, at 17:59, Doug Way wrote:
On Mon, 27 Jun 2005 17:15:16 +0200, "Avi Bryant" avi.bryant@gmail.com said:
On 6/27/05, Alexandre Bergel bergel@iam.unibe.ch wrote:
But the Project 39a is still empty...
Ah, yes, it is. We do need to commit stuff there. I just didn't understand the purpose of a script that created new packages when we already have a changeset from Andreas that does so.
Right, we can just use Andreas' changeset to do this partitioning. (which is very similar to Alexandre's script anyway, I think)
I'll populate that project shortly.
Sounds good. By the way, does a SqueakSource "Developer" have access to do this initial populating (adding new packages), or do you need to be an Admin?
I have many changes related to the system change notification that are waiting...
Great.
Yes, that should give the new process a good workout. Hmm, how many packages do these change-notification changes cut across? :-)
Realistically, it's going to take us a little while to get everything working smoothly with this new process. But we might as well dive in and get started. It's a big change, but I think it's flexible enough that we will be able to adjust the process if necessary, and we won't end up in a 3.3alpha modules situation.
I will set up a new swiki page covering the new process, as a point of reference.
- Doug
http://source.squeakfoundation.org:9090/ does not work, and http://source.squeakfoundation.org/ => "The proxy server received an invalid response from an upstream server."
Alexandre
On Tue, Jun 28, 2005 at 09:44:59AM +0200, stéphane ducasse wrote:
I cannot access
source.squeakfoundation.org may be this is my settings but I get The proxy server received an invalid response from an upstream server.
On 27 juin 05, at 17:59, Doug Way wrote:
On Mon, 27 Jun 2005 17:15:16 +0200, "Avi Bryant" avi.bryant@gmail.com said:
On 6/27/05, Alexandre Bergel bergel@iam.unibe.ch wrote:
But the Project 39a is still empty...
Ah, yes, it is. We do need to commit stuff there. I just didn't understand the purpose of a script that created new packages when we already have a changeset from Andreas that does so.
Right, we can just use Andreas' changeset to do this partitioning. (which is very similar to Alexandre's script anyway, I think)
I'll populate that project shortly.
Sounds good. By the way, does a SqueakSource "Developer" have access to do this initial populating (adding new packages), or do you need to be an Admin?
I have many changes related to the system change notification that are waiting...
Great.
Yes, that should give the new process a good workout. Hmm, how many packages do these change-notification changes cut across? :-)
Realistically, it's going to take us a little while to get everything working smoothly with this new process. But we might as well dive in and get started. It's a big change, but I think it's flexible enough that we will be able to adjust the process if necessary, and we won't end up in a 3.3alpha modules situation.
I will set up a new swiki page covering the new process, as a point of reference.
- Doug
On 6/28/05, Alexandre Bergel bergel@iam.unibe.ch wrote:
http://source.squeakfoundation.org:9090/ does not work, and http://source.squeakfoundation.org/ => "The proxy server received an invalid response from an upstream server."
I've restarted the image.
Avi - do you want to investigate the crashes or shall I just hang the thing under daemontools' supervision so it automagically restarts?
Am 28.06.2005 um 10:36 schrieb Cees De Groot:
On 6/28/05, Alexandre Bergel bergel@iam.unibe.ch wrote:
http://source.squeakfoundation.org:9090/ does not work, and http://source.squeakfoundation.org/ => "The proxy server received an invalid response from an upstream server."
I've restarted the image.
Avi - do you want to investigate the crashes or shall I just hang the thing under daemontools' supervision so it automagically restarts?
What VM are you running? Anyway, I, too, run it in a loop, just in case (hadn't much time to investigate the crashes, yet):
#!/bin/sh DIR=`dirname $0` exec nohup sh -c "while true; do $DIR/runss ; sleep 5; done"
Also, if it hangs, you can just kill it. I renamed the vm to make it stand out in the process listing for that.
- Bert -
I suggest you take a peek at daemontools. It does more-or-less the same, but much cleaner ;-)
The machine is running 3.7-7
On 6/28/05, Bert Freudenberg bert@impara.de wrote:
What VM are you running? Anyway, I, too, run it in a loop, just in case (hadn't much time to investigate the crashes, yet):
#!/bin/sh DIR=`dirname $0` exec nohup sh -c "while true; do $DIR/runss ; sleep 5; done"
Also, if it hangs, you can just kill it. I renamed the vm to make it stand out in the process listing for that.
- Bert -
I know, I use them at home. But on this server I'm not the boss.
- Bert -
Am 28.06.2005 um 11:44 schrieb Cees De Groot:
I suggest you take a peek at daemontools. It does more-or-less the same, but much cleaner ;-)
The machine is running 3.7-7
On 6/28/05, Bert Freudenberg bert@impara.de wrote:
What VM are you running? Anyway, I, too, run it in a loop, just in case (hadn't much time to investigate the crashes, yet):
#!/bin/sh DIR=`dirname $0` exec nohup sh -c "while true; do $DIR/runss ; sleep 5; done"
Also, if it hangs, you can just kill it. I renamed the vm to make it stand out in the process listing for that.
- Bert -
On 6/28/05, Cees De Groot cdegroot@gmail.com wrote:
On 6/28/05, Alexandre Bergel bergel@iam.unibe.ch wrote:
http://source.squeakfoundation.org:9090/ does not work, and http://source.squeakfoundation.org/ => "The proxy server received an invalid response from an upstream server."
I've restarted the image.
Avi - do you want to investigate the crashes or shall I just hang the thing under daemontools' supervision so it automagically restarts?
Well, I don't see that the latter precludes the former, so - first make sure it stays up, and we can figure why it goes down later. As long as it seems to be preserving the data ok... (does it?)
Avi
packages@lists.squeakfoundation.org