In the interests of revisiting Pavel Krivanek's work, and a long term goal of this community, I thought I'd use the Dependency Browser and dig out interpackage dependencies.
By scraping the DependencyBrowser's contents together with a bit of UI scripting I've constructed a dotfile of Trunk (attached). Turning this into a PNG results in an 11MB image! [1] Nodes near the top are nodes that aren't used by many things.
For instance, ReleaseBuilder's right at the top because nothing depends on it.
One thing to note is that XML-Parser and Nebraska are only used by Universes, and that Universes isn't used by anything else.
It occurs to me that we could thus remove these 3 packages from trunk and add the loading of these to ReleaseBuilderFor4dot5 [2], and still end up with a 4.5 that while apparently unchanged, actually has a smaller core.
What do you think of trying this out as an experiment? How would we unload these packages? (I should note: I've nothing against these packages. They're just packages that aren't woven into the guts of the image, and are thus easily removable.)
frank
[1] If you have dot installed, `dot -Tpng -o trunk-deps.dot trunk-deps.png` will do the trick. [2] Installer squeak package: 'trunk'; install: '39Deprecated-ar.19'; install: '311Deprecated-nice.2'; install: 'XML-Parser-ael.35'; install: 'Nebraska-ul.35'; install: 'Universes-nice.45'.
Doesn't sound like a bad idea. One concern though, might be to see how different e.g. XML-Parser is in trunk from whichever canonical repo hangs onto it. Push things that may have changed with it in the trunk upstream, then unload it.
Getting to a smaller core system is worth doing!
On Sat, Feb 9, 2013 at 3:51 PM, Frank Shearar frank.shearar@gmail.comwrote:
In the interests of revisiting Pavel Krivanek's work, and a long term goal of this community, I thought I'd use the Dependency Browser and dig out interpackage dependencies.
By scraping the DependencyBrowser's contents together with a bit of UI scripting I've constructed a dotfile of Trunk (attached). Turning this into a PNG results in an 11MB image! [1] Nodes near the top are nodes that aren't used by many things.
For instance, ReleaseBuilder's right at the top because nothing depends on it.
One thing to note is that XML-Parser and Nebraska are only used by Universes, and that Universes isn't used by anything else.
It occurs to me that we could thus remove these 3 packages from trunk and add the loading of these to ReleaseBuilderFor4dot5 [2], and still end up with a 4.5 that while apparently unchanged, actually has a smaller core.
What do you think of trying this out as an experiment? How would we unload these packages? (I should note: I've nothing against these packages. They're just packages that aren't woven into the guts of the image, and are thus easily removable.)
frank
[1] If you have dot installed, `dot -Tpng -o trunk-deps.dot trunk-deps.png` will do the trick. [2] Installer squeak package: 'trunk'; install: '39Deprecated-ar.19'; install: '311Deprecated-nice.2'; install: 'XML-Parser-ael.35'; install: 'Nebraska-ul.35'; install: 'Universes-nice.45'.
On Sat, Feb 09, 2013 at 11:51:21PM +0000, Frank Shearar wrote:
In the interests of revisiting Pavel Krivanek's work, and a long term goal of this community, I thought I'd use the Dependency Browser and dig out interpackage dependencies.
By scraping the DependencyBrowser's contents together with a bit of UI scripting I've constructed a dotfile of Trunk (attached). Turning this into a PNG results in an 11MB image! [1] Nodes near the top are nodes that aren't used by many things.
For instance, ReleaseBuilder's right at the top because nothing depends on it.
One thing to note is that XML-Parser and Nebraska are only used by Universes, and that Universes isn't used by anything else.
It occurs to me that we could thus remove these 3 packages from trunk and add the loading of these to ReleaseBuilderFor4dot5 [2], and still end up with a 4.5 that while apparently unchanged, actually has a smaller core.
What do you think of trying this out as an experiment? How would we unload these packages? (I should note: I've nothing against these packages. They're just packages that aren't woven into the guts of the image, and are thus easily removable.)
I would prefer to see the experiment focus on *reloadable* packages, in the sense of SmalltalkImage>>unloadAllKnownPackages, where the unloaded packages are supposedly distinct enough that they can be reloaded after having been removed from the image. Success would be defined as being able to unload a package, reload it, and verify that the image is equivalent to what you started with. I think that the packages you identified are probably good candidates for an initial experiment with this.
If we had some way to actually test that reloadable packages stay that way over time, presumably with the help of Jenkins, it would be a really good thing. I'm not entirely sure how to test for this, but it would be very worthwhile if someone could figure it out. I think that a verifiable set of truly reloadable packages would go a long way toward improving modularity.
I'm not so keen on the idea of just unloading things without some mechanism to verify that they stay reloadable. That's a fast track to bit rot.
frank
[1] If you have dot installed, `dot -Tpng -o trunk-deps.dot trunk-deps.png` will do the trick.
I think you meant 'dot -Tpng -o trunk-deps.png trunk-deps.dot'. Nice picture :)
[2] Installer squeak package: 'trunk'; install: '39Deprecated-ar.19'; install: '311Deprecated-nice.2'; install: 'XML-Parser-ael.35'; install: 'Nebraska-ul.35'; install: 'Universes-nice.45'.
On 2/9/13 9:43 PM, "David T. Lewis" lewis@mail.msen.com wrote:
On Sat, Feb 09, 2013 at 11:51:21PM +0000, Frank Shearar wrote:
In the interests of revisiting Pavel Krivanek's work, and a long term goal of this community, I thought I'd use the Dependency Browser and dig out interpackage dependencies.
By scraping the DependencyBrowser's contents together with a bit of UI scripting I've constructed a dotfile of Trunk (attached). Turning this into a PNG results in an 11MB image! [1] Nodes near the top are nodes that aren't used by many things.
For instance, ReleaseBuilder's right at the top because nothing depends on it.
One thing to note is that XML-Parser and Nebraska are only used by Universes, and that Universes isn't used by anything else.
It occurs to me that we could thus remove these 3 packages from trunk and add the loading of these to ReleaseBuilderFor4dot5 [2], and still end up with a 4.5 that while apparently unchanged, actually has a smaller core.
What do you think of trying this out as an experiment? How would we unload these packages? (I should note: I've nothing against these packages. They're just packages that aren't woven into the guts of the image, and are thus easily removable.)
I would prefer to see the experiment focus on *reloadable* packages, in the sense of SmalltalkImage>>unloadAllKnownPackages, where the unloaded packages are supposedly distinct enough that they can be reloaded after having been removed from the image. Success would be defined as being able to unload a package, reload it, and verify that the image is equivalent to what you started with. I think that the packages you identified are probably good candidates for an initial experiment with this.
+1 to both , Frank and David.
On 2/10/13, David T. Lewis lewis@mail.msen.com wrote:
On Sat, Feb 09, 2013 at 11:51:21PM +0000, Frank Shearar wrote:
In the interests of revisiting Pavel Krivanek's work, and a long term goal of this community, I thought I'd use the Dependency Browser and dig out interpackage dependencies.
By scraping the DependencyBrowser's contents together with a bit of UI scripting I've constructed a dotfile of Trunk (attached). Turning this into a PNG results in an 11MB image! [1] Nodes near the top are nodes that aren't used by many things.
For instance, ReleaseBuilder's right at the top because nothing depends on it.
One thing to note is that XML-Parser and Nebraska are only used by Universes, and that Universes isn't used by anything else.
It occurs to me that we could thus remove these 3 packages from trunk and add the loading of these to ReleaseBuilderFor4dot5 [2], and still end up with a 4.5 that while apparently unchanged, actually has a smaller core.
What do you think of trying this out as an experiment? How would we unload these packages? (I should note: I've nothing against these packages. They're just packages that aren't woven into the guts of the image, and are thus easily removable.)
I would prefer to see the experiment focus on *reloadable* packages, in the sense of SmalltalkImage>>unloadAllKnownPackages, where the unloaded packages are supposedly distinct enough that they can be reloaded after having been removed from the image. Success would be defined as being able to unload a package, reload it, and verify that the image is equivalent to what you started with. I think that the packages you identified are probably good candidates for an initial experiment with this.
Yes.
If we had some way to actually test that reloadable packages stay that way over time, presumably with the help of Jenkins, it would be a really good thing.
Yes, the question is how do we define success (i.e. construct a test) for a use case like
1) take the full image 2) unload a package 3) reload the package 4) check if we the full image is still OK
I'm not entirely sure how to test for this, but
it would be very worthwhile if someone could figure it out. I think that a verifiable set of truly reloadable packages would go a long way toward improving modularity.
Yes,
I'm not so keen on the idea of just unloading things without some mechanism to verify that they stay reloadable.
Sure, I understand that Frank wants the same thing.
He indentified
XML-Parser and Nebraska
as candidate for unloadable packages.
That's a fast track
to bit rot.
I think the way out is to keep the current full image and construct tests for unloading and reloading which can run on Jenkins.
--Hannes
frank
[1] If you have dot installed, `dot -Tpng -o trunk-deps.dot trunk-deps.png` will do the trick.
I think you meant 'dot -Tpng -o trunk-deps.png trunk-deps.dot'. Nice picture :)
[2] Installer squeak package: 'trunk'; install: '39Deprecated-ar.19'; install: '311Deprecated-nice.2'; install: 'XML-Parser-ael.35'; install: 'Nebraska-ul.35'; install: 'Universes-nice.45'.
On 10 February 2013 00:43, David T. Lewis lewis@mail.msen.com wrote:
On Sat, Feb 09, 2013 at 11:51:21PM +0000, Frank Shearar wrote:
In the interests of revisiting Pavel Krivanek's work, and a long term goal of this community, I thought I'd use the Dependency Browser and dig out interpackage dependencies.
By scraping the DependencyBrowser's contents together with a bit of UI scripting I've constructed a dotfile of Trunk (attached). Turning this into a PNG results in an 11MB image! [1] Nodes near the top are nodes that aren't used by many things.
For instance, ReleaseBuilder's right at the top because nothing depends on it.
One thing to note is that XML-Parser and Nebraska are only used by Universes, and that Universes isn't used by anything else.
It occurs to me that we could thus remove these 3 packages from trunk and add the loading of these to ReleaseBuilderFor4dot5 [2], and still end up with a 4.5 that while apparently unchanged, actually has a smaller core.
What do you think of trying this out as an experiment? How would we unload these packages? (I should note: I've nothing against these packages. They're just packages that aren't woven into the guts of the image, and are thus easily removable.)
I would prefer to see the experiment focus on *reloadable* packages, in the sense of SmalltalkImage>>unloadAllKnownPackages, where the unloaded packages are supposedly distinct enough that they can be reloaded after having been removed from the image. Success would be defined as being able to unload a package, reload it, and verify that the image is equivalent to what you started with. I think that the packages you identified are probably good candidates for an initial experiment with this.
If we had some way to actually test that reloadable packages stay that way over time, presumably with the help of Jenkins, it would be a really good thing. I'm not entirely sure how to test for this, but it would be very worthwhile if someone could figure it out. I think that a verifiable set of truly reloadable packages would go a long way toward improving modularity.
I'm not so keen on the idea of just unloading things without some mechanism to verify that they stay reloadable. That's a fast track to bit rot.
I don't see how you're disagreeing with me. I realised after my initial post that the packages I'd taken as examples are actually unloadable, in the sense that they're in the list in #unloadAllKnownPackages.
What I'm suggesting is the inverse of #unloadAllKnownPackages: strip these out of Trunk, and add them in as part of preparing a release. The only difference to the current Trunk then - in theory - is that we'd want to take a ReleaseTrunk artifact and run all its tests (because the SqueakTrunk job obviously wouldn't run the tests).
Or, we expand the ExternalPackage-Foo jobs to cover all the packages added by the ReleaseBuilder.
(One problem: Installer's marked as being unloadable, and it's probably one of the last packages we'd want to unload. The ExternalPackage-Foo scripts I'm writing use Installer. But then, I could always have those scripts load Installer as a prerequisite.)
frank
frank
[1] If you have dot installed, `dot -Tpng -o trunk-deps.dot trunk-deps.png` will do the trick.
I think you meant 'dot -Tpng -o trunk-deps.png trunk-deps.dot'. Nice picture :)
[2] Installer squeak package: 'trunk'; install: '39Deprecated-ar.19'; install: '311Deprecated-nice.2'; install: 'XML-Parser-ael.35'; install: 'Nebraska-ul.35'; install: 'Universes-nice.45'.
On 2/12/13, Frank Shearar frank.shearar@gmail.com wrote:
(One problem: Installer's marked as being unloadable, and it's probably one of the last packages we'd want to unload.
Yes, the Installer package was probably included to demonstrate that it is indeed unloadable.
The
ExternalPackage-Foo scripts I'm writing use Installer. But then, I could always have those scripts load Installer as a prerequisite.)
For what is the aim now. Unloading packages and building a release by reloading them again on the integration server the package 'Installer' should be excluded from the list.
Are there other candidates which should stay?
--Hannes
code snippet taken from SmalltalkImage>>unloadAllKnownPackages
"Go unloading" #( 'ReleaseBuilder' 'ScriptLoader' '311Deprecated' '39Deprecated' 'Universes' 'SMLoader' 'SMBase' 'Installer-Core' 'VersionNumberTests' 'VersionNumber' 'Services-Base' 'PreferenceBrowser' 'Nebraska' 'ToolBuilder-MVC' 'ST80' 'CollectionsTests' 'GraphicsTests' 'KernelTests' 'MorphicTests' 'MultilingualTests' 'NetworkTests' 'ToolsTests' 'TraitsTests' 'SystemChangeNotification-Tests' 'FlexibleVocabularies' 'EToys' 'Protocols' 'XML-Parser' 'Tests' 'SUnitGUI' 'Help-Squeak' 'HelpSystem' 'SystemReporter' ) do: [:pkgName| (MCPackage named: pkgName) unload. MCMcmUpdater disableUpdatesOfPackage: pkgName. ].
On Tue, Feb 12, 2013 at 01:40:21PM +0000, Frank Shearar wrote:
On 10 February 2013 00:43, David T. Lewis lewis@mail.msen.com wrote:
On Sat, Feb 09, 2013 at 11:51:21PM +0000, Frank Shearar wrote:
In the interests of revisiting Pavel Krivanek's work, and a long term goal of this community, I thought I'd use the Dependency Browser and dig out interpackage dependencies.
By scraping the DependencyBrowser's contents together with a bit of UI scripting I've constructed a dotfile of Trunk (attached). Turning this into a PNG results in an 11MB image! [1] Nodes near the top are nodes that aren't used by many things.
For instance, ReleaseBuilder's right at the top because nothing depends on it.
One thing to note is that XML-Parser and Nebraska are only used by Universes, and that Universes isn't used by anything else.
It occurs to me that we could thus remove these 3 packages from trunk and add the loading of these to ReleaseBuilderFor4dot5 [2], and still end up with a 4.5 that while apparently unchanged, actually has a smaller core.
What do you think of trying this out as an experiment? How would we unload these packages? (I should note: I've nothing against these packages. They're just packages that aren't woven into the guts of the image, and are thus easily removable.)
I would prefer to see the experiment focus on *reloadable* packages, in the sense of SmalltalkImage>>unloadAllKnownPackages, where the unloaded packages are supposedly distinct enough that they can be reloaded after having been removed from the image. Success would be defined as being able to unload a package, reload it, and verify that the image is equivalent to what you started with. I think that the packages you identified are probably good candidates for an initial experiment with this.
If we had some way to actually test that reloadable packages stay that way over time, presumably with the help of Jenkins, it would be a really good thing. I'm not entirely sure how to test for this, but it would be very worthwhile if someone could figure it out. I think that a verifiable set of truly reloadable packages would go a long way toward improving modularity.
I'm not so keen on the idea of just unloading things without some mechanism to verify that they stay reloadable. That's a fast track to bit rot.
I don't see how you're disagreeing with me. I realised after my initial post that the packages I'd taken as examples are actually unloadable, in the sense that they're in the list in #unloadAllKnownPackages.
I'm not disagreeing with you :)
Dave
What I'm suggesting is the inverse of #unloadAllKnownPackages: strip these out of Trunk, and add them in as part of preparing a release. The only difference to the current Trunk then - in theory - is that we'd want to take a ReleaseTrunk artifact and run all its tests (because the SqueakTrunk job obviously wouldn't run the tests).
Or, we expand the ExternalPackage-Foo jobs to cover all the packages added by the ReleaseBuilder.
(One problem: Installer's marked as being unloadable, and it's probably one of the last packages we'd want to unload. The ExternalPackage-Foo scripts I'm writing use Installer. But then, I could always have those scripts load Installer as a prerequisite.)
frank
frank
[1] If you have dot installed, `dot -Tpng -o trunk-deps.dot trunk-deps.png` will do the trick.
I think you meant 'dot -Tpng -o trunk-deps.png trunk-deps.dot'. Nice picture :)
[2] Installer squeak package: 'trunk'; install: '39Deprecated-ar.19'; install: '311Deprecated-nice.2'; install: 'XML-Parser-ael.35'; install: 'Nebraska-ul.35'; install: 'Universes-nice.45'.
I've never yet been convinced that *un*loading packages as a general technique is a good idea; unload them once to make the small core image and make the packages *load* nicely. Being able to unload seems likely to require tracking what was changed so you can restore it - and not just once but potentially many times with odd combinations for multi layers of loaded packages. I'm not sure it is practical to have a system that avoids any possibility of interactions but I'd certainly be happy to be wrong.
My suspicion is that we will need some very good tools to help make, maintain, test and care for loadable packages. It's so easy for things to slide into cruftulescense.
tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim "How many tnuctip does it take to change a lightbulb?" "Depends what you want them to change it into."
...unload them once to make the small core image and make the packages *load* nicely. Being able to unload seems likely to require tracking what was changed so you can restore it - and not just once but potentially many times with odd combinations for multi layers of loaded packages.
Hear, hear. :)
-C
-- Craig Latta www.netjam.org/resume +31 6 2757 7177 (SMS ok) + 1 415 287 3547 (no SMS)
On Tue, Feb 12, 2013 at 11:02 AM, tim Rowledge tim@rowledge.org wrote:
I've never yet been convinced that *un*loading packages as a general technique is a good idea; unload them once to make the small core image and make the packages *load* nicely. Being able to unload seems likely to require tracking what was changed so you can restore it - and not just once but potentially many times with odd combinations for multi layers of loaded packages. I'm not sure it is practical to have a system that avoids any possibility of interactions but I'd certainly be happy to be wrong.
In the absence of a full Spoon-like remote image development facility unloadable packages give one the ability to work on the core image with the full toolset. One works on the fiul image and then unloads to get the core. Without Spoon or unloadable packaes one can;t work on the core, one must construct it.
My suspicion is that we will need some very good tools to help make, maintain, test and care for loadable packages. It's so easy for things to slide into cruftulescense.
Smalltalk can do good tools.
tim
tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim "How many tnuctip does it take to change a lightbulb?" "Depends what you want them to change it into."
On 13-02-2013, at 10:27 AM, Eliot Miranda eliot.miranda@gmail.com wrote:
On Tue, Feb 12, 2013 at 11:02 AM, tim Rowledge tim@rowledge.org wrote: I've never yet been convinced that *un*loading packages as a general technique is a good idea; unload them once to make the small core image and make the packages *load* nicely. Being able to unload seems likely to require tracking what was changed so you can restore it - and not just once but potentially many times with odd combinations for multi layers of loaded packages. I'm not sure it is practical to have a system that avoids any possibility of interactions but I'd certainly be happy to be wrong.
In the absence of a full Spoon-like remote image development facility unloadable packages give one the ability to work on the core image with the full toolset. One works on the fiul image and then unloads to get the core. Without Spoon or unloadable packaes one can;t work on the core, one must construct it.
Fair point; I was pretty much assuming Spoon-feeding it and failed to be clear about that.
My suspicion is that we will need some very good tools to help make, maintain, test and care for loadable packages. It's so easy for things to slide into cruftulescense.
Smalltalk can do good tools.
All we need to do is work out what the good tools are and find the time to implement them whilst not starving in the process. Haven't we been there before ;-)
tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim "Bother" said Piglet, as Pooh smeared him in honey.
Hoi Eliot--
In the absence of a full Spoon-like remote image development facility unloadable packages give one the ability to work on the core image with the full toolset.
Have you tried the remote browsers in the Spoon releases? They include a mainstream Squeak image with remote browsers and a fully-functioning interpreter simulator with the Spoon VM changes applied.
http://netjam.org/spoon/releases/current
One works on the full image and then unloads to get the core. Without Spoon or unloadable packaes one can't work on the core, one must construct it.
But... we have Spoon, yes? :)
-C
-- Craig Latta www.netjam.org/resume +31 6 2757 7177 (SMS ok) + 1 415 287 3547 (no SMS)
Hi,
I recommend to follow the same way we use in Pharo: - firstly prepare the SqueakCore and clean it a bit. - make basic packages like Network and Monticello loadable. - make the rest of the system loadable using Monticello at once, ensure everything is working well - start to improve granularity of that bundle
Unfortunately, I have to say that Squeak is eons back from the point of modularity.
Cheers, -- Pavel
On Sun, Feb 10, 2013 at 12:51 AM, Frank Shearar frank.shearar@gmail.com wrote:
In the interests of revisiting Pavel Krivanek's work, and a long term goal of this community, I thought I'd use the Dependency Browser and dig out interpackage dependencies.
By scraping the DependencyBrowser's contents together with a bit of UI scripting I've constructed a dotfile of Trunk (attached). Turning this into a PNG results in an 11MB image! [1] Nodes near the top are nodes that aren't used by many things.
For instance, ReleaseBuilder's right at the top because nothing depends on it.
One thing to note is that XML-Parser and Nebraska are only used by Universes, and that Universes isn't used by anything else.
It occurs to me that we could thus remove these 3 packages from trunk and add the loading of these to ReleaseBuilderFor4dot5 [2], and still end up with a 4.5 that while apparently unchanged, actually has a smaller core.
What do you think of trying this out as an experiment? How would we unload these packages? (I should note: I've nothing against these packages. They're just packages that aren't woven into the guts of the image, and are thus easily removable.)
frank
[1] If you have dot installed, `dot -Tpng -o trunk-deps.dot trunk-deps.png` will do the trick. [2] Installer squeak package: 'trunk'; install: '39Deprecated-ar.19'; install: '311Deprecated-nice.2'; install: 'XML-Parser-ael.35'; install: 'Nebraska-ul.35'; install: 'Universes-nice.45'.
Hello Pavel
Thank you for your recommendation. As not having followed closely what you and others did in Pharo, may I ask you to give a short summary where you have reached now with Pharo core effort.
I have seen quite a number of smaller Pharo images on the Pharo integration server but I am not sure what the status of them is.
https://ci.inria.fr/pharo/view/Pharo-Kernel-2.0/
Please let me ask very basic questions.
1. Are they all derived from the the main current "full" Pharo2.0? 2. What is the quality of them? 3. How are they used by people?
Thank you in advance
--Hannes
On 2/11/13, Pavel Krivanek squeak1@continentalbrno.cz wrote:
Hi,
I recommend to follow the same way we use in Pharo:
- firstly prepare the SqueakCore and clean it a bit.
- make basic packages like Network and Monticello loadable.
- make the rest of the system loadable using Monticello at once,
ensure everything is working well
- start to improve granularity of that bundle
Unfortunately, I have to say that Squeak is eons back from the point of modularity.
Cheers, -- Pavel
On Sun, Feb 10, 2013 at 12:51 AM, Frank Shearar frank.shearar@gmail.com wrote:
In the interests of revisiting Pavel Krivanek's work, and a long term goal of this community, I thought I'd use the Dependency Browser and dig out interpackage dependencies.
By scraping the DependencyBrowser's contents together with a bit of UI scripting I've constructed a dotfile of Trunk (attached). Turning this into a PNG results in an 11MB image! [1] Nodes near the top are nodes that aren't used by many things.
For instance, ReleaseBuilder's right at the top because nothing depends on it.
One thing to note is that XML-Parser and Nebraska are only used by Universes, and that Universes isn't used by anything else.
It occurs to me that we could thus remove these 3 packages from trunk and add the loading of these to ReleaseBuilderFor4dot5 [2], and still end up with a 4.5 that while apparently unchanged, actually has a smaller core.
What do you think of trying this out as an experiment? How would we unload these packages? (I should note: I've nothing against these packages. They're just packages that aren't woven into the guts of the image, and are thus easily removable.)
frank
[1] If you have dot installed, `dot -Tpng -o trunk-deps.dot trunk-deps.png` will do the trick. [2] Installer squeak package: 'trunk'; install: '39Deprecated-ar.19'; install: '311Deprecated-nice.2'; install: 'XML-Parser-ael.35'; install: 'Nebraska-ul.35'; install: 'Universes-nice.45'.
On Mon, Feb 11, 2013 at 1:05 PM, H. Hirzel hannes.hirzel@gmail.com wrote:
Hello Pavel
Thank you for your recommendation. As not having followed closely what you and others did in Pharo, may I ask you to give a short summary where you have reached now with Pharo core effort.
I have seen quite a number of smaller Pharo images on the Pharo integration server but I am not sure what the status of them is.
https://ci.inria.fr/pharo/view/Pharo-Kernel-2.0/
Please let me ask very basic questions.
- Are they all derived from the the main current "full" Pharo2.0?
Yes, they are, the generation of this images is based on set of scripts that we modify to reflect latests changes in Pharo.
- What is the quality of them?
For the Pharo Kernel we have 3 Undeclared, no obsolete classes, 117 unimplemented calls and 7 failing tests. There is still a lot of space for improvements. Others are more dirty.
- How are they used by people?
I do not know about any real-life usage except usages by me. But we used them a lot for testing and improving of projects like Fuel/Tanker and bootstrapping. So we was able to produce full Morphic image generated by bootstrapping. Because we have a CI setting, we can see if some update has bad effect on modularity and UI independency of Pharo.
Primary goal is not to have minimal system but to have clean and modular system.
Cheers, -- Pavel
Thank you in advance
--Hannes
On 2/11/13, Pavel Krivanek squeak1@continentalbrno.cz wrote:
Hi,
I recommend to follow the same way we use in Pharo:
- firstly prepare the SqueakCore and clean it a bit.
- make basic packages like Network and Monticello loadable.
- make the rest of the system loadable using Monticello at once,
ensure everything is working well
- start to improve granularity of that bundle
Unfortunately, I have to say that Squeak is eons back from the point of modularity.
Cheers, -- Pavel
On Sun, Feb 10, 2013 at 12:51 AM, Frank Shearar frank.shearar@gmail.com wrote:
In the interests of revisiting Pavel Krivanek's work, and a long term goal of this community, I thought I'd use the Dependency Browser and dig out interpackage dependencies.
By scraping the DependencyBrowser's contents together with a bit of UI scripting I've constructed a dotfile of Trunk (attached). Turning this into a PNG results in an 11MB image! [1] Nodes near the top are nodes that aren't used by many things.
For instance, ReleaseBuilder's right at the top because nothing depends on it.
One thing to note is that XML-Parser and Nebraska are only used by Universes, and that Universes isn't used by anything else.
It occurs to me that we could thus remove these 3 packages from trunk and add the loading of these to ReleaseBuilderFor4dot5 [2], and still end up with a 4.5 that while apparently unchanged, actually has a smaller core.
What do you think of trying this out as an experiment? How would we unload these packages? (I should note: I've nothing against these packages. They're just packages that aren't woven into the guts of the image, and are thus easily removable.)
frank
[1] If you have dot installed, `dot -Tpng -o trunk-deps.dot trunk-deps.png` will do the trick. [2] Installer squeak package: 'trunk'; install: '39Deprecated-ar.19'; install: '311Deprecated-nice.2'; install: 'XML-Parser-ael.35'; install: 'Nebraska-ul.35'; install: 'Universes-nice.45'.
I recommend to follow the same way we use in Pharo:
- firstly prepare the SqueakCore and clean it a bit.
- make basic packages like Network and Monticello loadable.
- make the rest of the system loadable using Monticello at once,
ensure everything is working well
- start to improve granularity of that bundle
If I understand correctly, this would mean starting from another (smaller) image than the current trunk, and try to get to the trunk by adding packages. It seems to me that backward compatibility is jeopardized in this process (I know it's not a Pharo value, but it is a Squeak one).
I prefer the process of starting from the trunk and having more and more packages unloadable/loadable (in that order). This way the trunk is always there and backward compatibility can be verified at any time.
Stef
On Mon, Feb 11, 2013 at 3:18 PM, Stéphane Rollandin lecteur@zogotounga.net wrote:
I recommend to follow the same way we use in Pharo:
- firstly prepare the SqueakCore and clean it a bit.
- make basic packages like Network and Monticello loadable.
- make the rest of the system loadable using Monticello at once,
ensure everything is working well
- start to improve granularity of that bundle
If I understand correctly, this would mean starting from another (smaller) image than the current trunk, and try to get to the trunk by adding packages. It seems to me that backward compatibility is jeopardized in this process (I know it's not a Pharo value, but it is a Squeak one).
I prefer the process of starting from the trunk and having more and more packages unloadable/loadable (in that order). This way the trunk is always there and backward compatibility can be verified at any time.
Stef
Hi Stef,
it does not directly mean to start from another image because the smaller image would be generated from the trunk image for every update. It is only a way how to ensure that the packages are loadable back - only a more radical version of "unloadAllKnownPackages". It is a parallel process of checking the image quality and modularity. The way "from top" is very hard if you must deal with big dirty packages like Morphic and co.
Cheers, -- Pavel
On 11/02/13 9:18 AM, Stéphane Rollandin wrote:
... It seems to me that backward compatibility is jeopardized in this process (I know it's not a Pharo value, but it is a Squeak one).
<rant> What's up with some people in the Squeak "camp" trying to artificially draw distinctions between Squeak and Pharo - to the point of FUD, like the above statement.
Of course "backward compatibility" is of value to everybody. A less antagonistic way to state the Pharo point of view, is that: if backward compatibility would block forward progress, or is too costly to provide, then, choose to drop compatibility (and if proven wrong in that decision, put back the compatibility - which has occurred already).
How is that different from what Squeak has to do? IMHO, the difference is in the cost the community is willing to pay for compatibility, and the rate at which forward progress can be made. That's the choice individuals and communities choose, nothing wrong with that. </rant>
... It seems to me that backward compatibility is jeopardized in this process (I know it's not a Pharo value, but it is a Squeak one).
<rant> What's up with some people in the Squeak "camp" trying to artificially draw distinctions between Squeak and Pharo - to the point of FUD, like the above statement.
Keep cool, man. I just expressed a fact, and a worry as far as I am concerned, because I do need backward compatibility if I expect muO to grow with Squeak. I have no problem with Pharo project nor people, so please don't create trouble where there is none. This is a technical discussion.
Stef
Yanni,
I think Stéphane refers to the original Pharo manifesto which clearly states "no backward compatibility". http://code.google.com/p/pharo/
However the current Pharo web page has a mission statement http://www.pharo-project.org/about sets a much more moderate tone.
In any case in this thread we want to move on towards a Squeak core and learn from the Pharo experience as much as possible. Please let us not digress from this important topic.
Maybe we should follow both at the same time
Let me call it - the Pavel Krivanek approach and - the SmalltalkImage unloadAllKnownPackages approach
BTW #unloadAllKnownPackages
used to work in Squeak 4.1, see http://lists.squeakfoundation.org/pipermail/squeak-dev/2010-August/152427.ht...
So there is no reason why we should not manage to get it working again in Squeak 4.5alpha.
And Pavel's approach may be followed in parallel. Because fixing one thing will help the other and vice-verse.
--Hannes
On 2/11/13, Yanni Chiu yanni@rogers.com wrote:
On 11/02/13 9:18 AM, Stéphane Rollandin wrote:
... It seems to me that backward compatibility is jeopardized in this process (I know it's not a Pharo value, but it is a Squeak one).
<rant> What's up with some people in the Squeak "camp" trying to artificially draw distinctions between Squeak and Pharo - to the point of FUD, like the above statement.
Of course "backward compatibility" is of value to everybody. A less antagonistic way to state the Pharo point of view, is that: if backward compatibility would block forward progress, or is too costly to provide, then, choose to drop compatibility (and if proven wrong in that decision, put back the compatibility - which has occurred already).
How is that different from what Squeak has to do? IMHO, the difference is in the cost the community is willing to pay for compatibility, and the rate at which forward progress can be made. That's the choice individuals and communities choose, nothing wrong with that.
</rant>
On Mon, Feb 11, 2013 at 05:28:26PM +0000, H. Hirzel wrote:
Yanni,
I think St??phane refers to the original Pharo manifesto which clearly states "no backward compatibility". http://code.google.com/p/pharo/
However the current Pharo web page has a mission statement http://www.pharo-project.org/about sets a much more moderate tone.
In any case in this thread we want to move on towards a Squeak core and learn from the Pharo experience as much as possible. Please let us not digress from this important topic.
Maybe we should follow both at the same time
Let me call it
- the Pavel Krivanek approach and
- the SmalltalkImage unloadAllKnownPackages approach
BTW #unloadAllKnownPackages
used to work in Squeak 4.1, see http://lists.squeakfoundation.org/pipermail/squeak-dev/2010-August/152427.ht...
So there is no reason why we should not manage to get it working again in Squeak 4.5alpha.
And Pavel's approach may be followed in parallel. Because fixing one thing will help the other and vice-verse.
+1
Exactly so. That is what I intended when I mentioned #unloadAllKnownPackages. Thanks for stating it so clearly.
Dave
On 13 February 2013 11:22, David T. Lewis lewis@mail.msen.com wrote:
On Mon, Feb 11, 2013 at 05:28:26PM +0000, H. Hirzel wrote:
Yanni,
I think St??phane refers to the original Pharo manifesto which clearly states "no backward compatibility". http://code.google.com/p/pharo/
However the current Pharo web page has a mission statement http://www.pharo-project.org/about sets a much more moderate tone.
In any case in this thread we want to move on towards a Squeak core and learn from the Pharo experience as much as possible. Please let us not digress from this important topic.
Maybe we should follow both at the same time
Let me call it
- the Pavel Krivanek approach and
- the SmalltalkImage unloadAllKnownPackages approach
BTW #unloadAllKnownPackages
used to work in Squeak 4.1, see http://lists.squeakfoundation.org/pipermail/squeak-dev/2010-August/152427.ht...
So there is no reason why we should not manage to get it working again in Squeak 4.5alpha.
And Pavel's approach may be followed in parallel. Because fixing one thing will help the other and vice-verse.
+1
Exactly so. That is what I intended when I mentioned #unloadAllKnownPackages. Thanks for stating it so clearly.
While we're being clear about what's clear :) I'm wanting to _lose_ #unloadAllKnownPackages, and replace it with a #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload, and add a bunch of new jobs running those unloaded packages' test suites.
That way, the thing called Squeak4.5-nnnn.image still contains what Squeak4.5-${whatever_current_is}.image, only the essence of trunk - what SqueakTrunk produces - shrinks.
Dave
Frank
If I understand you correctly you want to do the following
1) unload some once packages from trunk. The packages are taken from what is in the list in #unloadAllKnownPackages.
2) add a trunk method #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload
3) add CI build jobs which do #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload and run tests on the loaded packages.
4) People use the result of 3) for regular work. This is the "full" 4.5 image as we have of now only that we have less in trunk and the unlodable packages reside in their own repositories.
Steps 2..4 are done repeatedly.
Is this what you mean?
--Hannes
On 2/13/13, Frank Shearar frank.shearar@gmail.com wrote:
On 13 February 2013 11:22, David T. Lewis lewis@mail.msen.com wrote:
On Mon, Feb 11, 2013 at 05:28:26PM +0000, H. Hirzel wrote:
Yanni,
I think St??phane refers to the original Pharo manifesto which clearly states "no backward compatibility". http://code.google.com/p/pharo/
However the current Pharo web page has a mission statement http://www.pharo-project.org/about sets a much more moderate tone.
In any case in this thread we want to move on towards a Squeak core and learn from the Pharo experience as much as possible. Please let us not digress from this important topic.
Maybe we should follow both at the same time
Let me call it
- the Pavel Krivanek approach and
- the SmalltalkImage unloadAllKnownPackages approach
BTW #unloadAllKnownPackages
used to work in Squeak 4.1, see
http://lists.squeakfoundation.org/pipermail/squeak-dev/2010-August/152427.ht...
So there is no reason why we should not manage to get it working again in Squeak 4.5alpha.
And Pavel's approach may be followed in parallel. Because fixing one thing will help the other and vice-verse.
+1
Exactly so. That is what I intended when I mentioned #unloadAllKnownPackages. Thanks for stating it so clearly.
While we're being clear about what's clear :) I'm wanting to _lose_ #unloadAllKnownPackages, and replace it with a #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload, and add a bunch of new jobs running those unloaded packages' test suites.
That way, the thing called Squeak4.5-nnnn.image still contains what Squeak4.5-${whatever_current_is}.image, only the essence of trunk - what SqueakTrunk produces - shrinks.
Dave
On 13 February 2013 12:45, H. Hirzel hannes.hirzel@gmail.com wrote:
Frank
If I understand you correctly you want to do the following
- unload some once packages from trunk. The packages are taken from
what is in the list in #unloadAllKnownPackages.
add a trunk method #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload
add CI build jobs which do
#loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload and run tests on the loaded packages.
- People use the result of 3) for regular work. This is the "full"
4.5 image as we have of now only that we have less in trunk and the unlodable packages reside in their own repositories.
Steps 2..4 are done repeatedly.
Is this what you mean?
Yes. Perhaps with a nicer name than #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload. Maybe #loadBasicPackages or something.
My main aim is to invert the approach we've had up until now, and _enforce_ the clean separation of these packages from the Core. I want it to be difficult to add a dependency from the Core to these packages.
frank
--Hannes
On 2/13/13, Frank Shearar frank.shearar@gmail.com wrote:
On 13 February 2013 11:22, David T. Lewis lewis@mail.msen.com wrote:
On Mon, Feb 11, 2013 at 05:28:26PM +0000, H. Hirzel wrote:
Yanni,
I think St??phane refers to the original Pharo manifesto which clearly states "no backward compatibility". http://code.google.com/p/pharo/
However the current Pharo web page has a mission statement http://www.pharo-project.org/about sets a much more moderate tone.
In any case in this thread we want to move on towards a Squeak core and learn from the Pharo experience as much as possible. Please let us not digress from this important topic.
Maybe we should follow both at the same time
Let me call it
- the Pavel Krivanek approach and
- the SmalltalkImage unloadAllKnownPackages approach
BTW #unloadAllKnownPackages
used to work in Squeak 4.1, see
http://lists.squeakfoundation.org/pipermail/squeak-dev/2010-August/152427.ht...
So there is no reason why we should not manage to get it working again in Squeak 4.5alpha.
And Pavel's approach may be followed in parallel. Because fixing one thing will help the other and vice-verse.
+1
Exactly so. That is what I intended when I mentioned #unloadAllKnownPackages. Thanks for stating it so clearly.
While we're being clear about what's clear :) I'm wanting to _lose_ #unloadAllKnownPackages, and replace it with a #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload, and add a bunch of new jobs running those unloaded packages' test suites.
That way, the thing called Squeak4.5-nnnn.image still contains what Squeak4.5-${whatever_current_is}.image, only the essence of trunk - what SqueakTrunk produces - shrinks.
Dave
Yes, thank you for the clarification, Frank. This makes a lot of sense to me and I hope that others follow this line of reasoning.
--Hannes
On 2/13/13, Frank Shearar frank.shearar@gmail.com wrote:
On 13 February 2013 12:45, H. Hirzel hannes.hirzel@gmail.com wrote:
Frank
If I understand you correctly you want to do the following
- unload some once packages from trunk. The packages are taken from
what is in the list in #unloadAllKnownPackages.
- add a trunk method
#loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload
- add CI build jobs which do
#loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload and run tests on the loaded packages.
- People use the result of 3) for regular work. This is the "full"
4.5 image as we have of now only that we have less in trunk and the unlodable packages reside in their own repositories.
Steps 2..4 are done repeatedly.
Is this what you mean?
Yes. Perhaps with a nicer name than #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload. Maybe #loadBasicPackages or something.
My main aim is to invert the approach we've had up until now, and _enforce_ the clean separation of these packages from the Core. I want it to be difficult to add a dependency from the Core to these packages.
frank
--Hannes
On 2/13/13, Frank Shearar frank.shearar@gmail.com wrote:
On 13 February 2013 11:22, David T. Lewis lewis@mail.msen.com wrote:
On Mon, Feb 11, 2013 at 05:28:26PM +0000, H. Hirzel wrote:
Yanni,
I think St??phane refers to the original Pharo manifesto which clearly states "no backward compatibility". http://code.google.com/p/pharo/
However the current Pharo web page has a mission statement http://www.pharo-project.org/about sets a much more moderate tone.
In any case in this thread we want to move on towards a Squeak core and learn from the Pharo experience as much as possible. Please let us not digress from this important topic.
Maybe we should follow both at the same time
Let me call it
- the Pavel Krivanek approach and
- the SmalltalkImage unloadAllKnownPackages approach
BTW #unloadAllKnownPackages
used to work in Squeak 4.1, see
http://lists.squeakfoundation.org/pipermail/squeak-dev/2010-August/152427.ht...
So there is no reason why we should not manage to get it working again in Squeak 4.5alpha.
And Pavel's approach may be followed in parallel. Because fixing one thing will help the other and vice-verse.
+1
Exactly so. That is what I intended when I mentioned #unloadAllKnownPackages. Thanks for stating it so clearly.
While we're being clear about what's clear :) I'm wanting to _lose_ #unloadAllKnownPackages, and replace it with a #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload, and add a bunch of new jobs running those unloaded packages' test suites.
That way, the thing called Squeak4.5-nnnn.image still contains what Squeak4.5-${whatever_current_is}.image, only the essence of trunk - what SqueakTrunk produces - shrinks.
Dave
We also now have builds for XML-Parser, Nebraska and Universes (for what it's worth: the code coverage is abysmal).
frank
On 13 February 2013 15:23, H. Hirzel hannes.hirzel@gmail.com wrote:
Yes, thank you for the clarification, Frank. This makes a lot of sense to me and I hope that others follow this line of reasoning.
--Hannes
On 2/13/13, Frank Shearar frank.shearar@gmail.com wrote:
On 13 February 2013 12:45, H. Hirzel hannes.hirzel@gmail.com wrote:
Frank
If I understand you correctly you want to do the following
- unload some once packages from trunk. The packages are taken from
what is in the list in #unloadAllKnownPackages.
- add a trunk method
#loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload
- add CI build jobs which do
#loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload and run tests on the loaded packages.
- People use the result of 3) for regular work. This is the "full"
4.5 image as we have of now only that we have less in trunk and the unlodable packages reside in their own repositories.
Steps 2..4 are done repeatedly.
Is this what you mean?
Yes. Perhaps with a nicer name than #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload. Maybe #loadBasicPackages or something.
My main aim is to invert the approach we've had up until now, and _enforce_ the clean separation of these packages from the Core. I want it to be difficult to add a dependency from the Core to these packages.
frank
--Hannes
On 2/13/13, Frank Shearar frank.shearar@gmail.com wrote:
On 13 February 2013 11:22, David T. Lewis lewis@mail.msen.com wrote:
On Mon, Feb 11, 2013 at 05:28:26PM +0000, H. Hirzel wrote:
Yanni,
I think St??phane refers to the original Pharo manifesto which clearly states "no backward compatibility". http://code.google.com/p/pharo/
However the current Pharo web page has a mission statement http://www.pharo-project.org/about sets a much more moderate tone.
In any case in this thread we want to move on towards a Squeak core and learn from the Pharo experience as much as possible. Please let us not digress from this important topic.
Maybe we should follow both at the same time
Let me call it
- the Pavel Krivanek approach and
- the SmalltalkImage unloadAllKnownPackages approach
BTW #unloadAllKnownPackages
used to work in Squeak 4.1, see
http://lists.squeakfoundation.org/pipermail/squeak-dev/2010-August/152427.ht...
So there is no reason why we should not manage to get it working again in Squeak 4.5alpha.
And Pavel's approach may be followed in parallel. Because fixing one thing will help the other and vice-verse.
+1
Exactly so. That is what I intended when I mentioned #unloadAllKnownPackages. Thanks for stating it so clearly.
While we're being clear about what's clear :) I'm wanting to _lose_ #unloadAllKnownPackages, and replace it with a #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload, and add a bunch of new jobs running those unloaded packages' test suites.
That way, the thing called Squeak4.5-nnnn.image still contains what Squeak4.5-${whatever_current_is}.image, only the essence of trunk - what SqueakTrunk produces - shrinks.
Dave
Great
where can I see what the script is doing for example for XML-Parser?
Which script does the job?
http://www.squeakci.org/job/ExternalPackage-XML-Parser/ws/
--Hannes
On 2/14/13, Frank Shearar frank.shearar@gmail.com wrote:
We also now have builds for XML-Parser, Nebraska and Universes (for what it's worth: the code coverage is abysmal).
frank
On 13 February 2013 15:23, H. Hirzel hannes.hirzel@gmail.com wrote:
Yes, thank you for the clarification, Frank. This makes a lot of sense to me and I hope that others follow this line of reasoning.
--Hannes
On 2/13/13, Frank Shearar frank.shearar@gmail.com wrote:
On 13 February 2013 12:45, H. Hirzel hannes.hirzel@gmail.com wrote:
Frank
If I understand you correctly you want to do the following
- unload some once packages from trunk. The packages are taken from
what is in the list in #unloadAllKnownPackages.
- add a trunk method
#loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload
- add CI build jobs which do
#loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload and run tests on the loaded packages.
- People use the result of 3) for regular work. This is the "full"
4.5 image as we have of now only that we have less in trunk and the unlodable packages reside in their own repositories.
Steps 2..4 are done repeatedly.
Is this what you mean?
Yes. Perhaps with a nicer name than #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload. Maybe #loadBasicPackages or something.
My main aim is to invert the approach we've had up until now, and _enforce_ the clean separation of these packages from the Core. I want it to be difficult to add a dependency from the Core to these packages.
frank
--Hannes
On 2/13/13, Frank Shearar frank.shearar@gmail.com wrote:
On 13 February 2013 11:22, David T. Lewis lewis@mail.msen.com wrote:
On Mon, Feb 11, 2013 at 05:28:26PM +0000, H. Hirzel wrote: > Yanni, > > I think St??phane refers to the original Pharo manifesto which > clearly > states "no backward compatibility". http://code.google.com/p/pharo/ > > However the current Pharo web page has a mission statement > http://www.pharo-project.org/about > sets a much more moderate tone. > > In any case in this thread we want to move on towards a Squeak core > and learn from the Pharo experience as much as possible. Please let > us > not digress from this important topic. > > Maybe we should follow both at the same time > > Let me call it > - the Pavel Krivanek approach and > - the > SmalltalkImage unloadAllKnownPackages > approach > > BTW > #unloadAllKnownPackages > > used to work in Squeak 4.1, see > > http://lists.squeakfoundation.org/pipermail/squeak-dev/2010-August/152427.ht... > > So there is no reason why we should not manage to get it working > again > in Squeak 4.5alpha. > > And Pavel's approach may be followed in parallel. Because fixing one > thing will help the other and vice-verse. >
+1
Exactly so. That is what I intended when I mentioned #unloadAllKnownPackages. Thanks for stating it so clearly.
While we're being clear about what's clear :) I'm wanting to _lose_ #unloadAllKnownPackages, and replace it with a #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload, and add a bunch of new jobs running those unloaded packages' test suites.
That way, the thing called Squeak4.5-nnnn.image still contains what Squeak4.5-${whatever_current_is}.image, only the essence of trunk - what SqueakTrunk produces - shrinks.
Dave
I.e. an updated README.md is very welcome :-)
On 2/14/13, H. Hirzel hannes.hirzel@gmail.com wrote:
Great
where can I see what the script is doing for example for XML-Parser?
Which script does the job?
http://www.squeakci.org/job/ExternalPackage-XML-Parser/ws/
--Hannes
On 2/14/13, Frank Shearar frank.shearar@gmail.com wrote:
We also now have builds for XML-Parser, Nebraska and Universes (for what it's worth: the code coverage is abysmal).
frank
On 13 February 2013 15:23, H. Hirzel hannes.hirzel@gmail.com wrote:
Yes, thank you for the clarification, Frank. This makes a lot of sense to me and I hope that others follow this line of reasoning.
--Hannes
On 2/13/13, Frank Shearar frank.shearar@gmail.com wrote:
On 13 February 2013 12:45, H. Hirzel hannes.hirzel@gmail.com wrote:
Frank
If I understand you correctly you want to do the following
- unload some once packages from trunk. The packages are taken from
what is in the list in #unloadAllKnownPackages.
- add a trunk method
#loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload
- add CI build jobs which do
#loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload and run tests on the loaded packages.
- People use the result of 3) for regular work. This is the "full"
4.5 image as we have of now only that we have less in trunk and the unlodable packages reside in their own repositories.
Steps 2..4 are done repeatedly.
Is this what you mean?
Yes. Perhaps with a nicer name than #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload. Maybe #loadBasicPackages or something.
My main aim is to invert the approach we've had up until now, and _enforce_ the clean separation of these packages from the Core. I want it to be difficult to add a dependency from the Core to these packages.
frank
--Hannes
On 2/13/13, Frank Shearar frank.shearar@gmail.com wrote:
On 13 February 2013 11:22, David T. Lewis lewis@mail.msen.com wrote: > On Mon, Feb 11, 2013 at 05:28:26PM +0000, H. Hirzel wrote: >> Yanni, >> >> I think St??phane refers to the original Pharo manifesto which >> clearly >> states "no backward compatibility". >> http://code.google.com/p/pharo/ >> >> However the current Pharo web page has a mission statement >> http://www.pharo-project.org/about >> sets a much more moderate tone. >> >> In any case in this thread we want to move on towards a Squeak core >> and learn from the Pharo experience as much as possible. Please let >> us >> not digress from this important topic. >> >> Maybe we should follow both at the same time >> >> Let me call it >> - the Pavel Krivanek approach and >> - the >> SmalltalkImage unloadAllKnownPackages >> approach >> >> BTW >> #unloadAllKnownPackages >> >> used to work in Squeak 4.1, see >> >> http://lists.squeakfoundation.org/pipermail/squeak-dev/2010-August/152427.ht... >> >> So there is no reason why we should not manage to get it working >> again >> in Squeak 4.5alpha. >> >> And Pavel's approach may be followed in parallel. Because fixing >> one >> thing will help the other and vice-verse. >> > > +1 > > Exactly so. That is what I intended when I mentioned > #unloadAllKnownPackages. > Thanks for stating it so clearly.
While we're being clear about what's clear :) I'm wanting to _lose_ #unloadAllKnownPackages, and replace it with a #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload, and add a bunch of new jobs running those unloaded packages' test suites.
That way, the thing called Squeak4.5-nnnn.image still contains what Squeak4.5-${whatever_current_is}.image, only the essence of trunk - what SqueakTrunk produces - shrinks.
> Dave
https://github.com/frankshearar/squeak-ci/blob/master/package-load-tests/XML...
They all get triggered by this guy, who I aim to replace with some Ruby (because I'd really, really, REALLY rather write Ruby than shell): https://github.com/frankshearar/squeak-ci/blob/master/run-test.sh
So the jobs only differ in their command: `./run-test.sh Control`, `./run-test.sh XML-Parser`, and so on.
frank
On 14 February 2013 16:34, H. Hirzel hannes.hirzel@gmail.com wrote:
Great
where can I see what the script is doing for example for XML-Parser?
Which script does the job?
http://www.squeakci.org/job/ExternalPackage-XML-Parser/ws/
--Hannes
On 2/14/13, Frank Shearar frank.shearar@gmail.com wrote:
We also now have builds for XML-Parser, Nebraska and Universes (for what it's worth: the code coverage is abysmal).
frank
On 13 February 2013 15:23, H. Hirzel hannes.hirzel@gmail.com wrote:
Yes, thank you for the clarification, Frank. This makes a lot of sense to me and I hope that others follow this line of reasoning.
--Hannes
On 2/13/13, Frank Shearar frank.shearar@gmail.com wrote:
On 13 February 2013 12:45, H. Hirzel hannes.hirzel@gmail.com wrote:
Frank
If I understand you correctly you want to do the following
- unload some once packages from trunk. The packages are taken from
what is in the list in #unloadAllKnownPackages.
- add a trunk method
#loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload
- add CI build jobs which do
#loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload and run tests on the loaded packages.
- People use the result of 3) for regular work. This is the "full"
4.5 image as we have of now only that we have less in trunk and the unlodable packages reside in their own repositories.
Steps 2..4 are done repeatedly.
Is this what you mean?
Yes. Perhaps with a nicer name than #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload. Maybe #loadBasicPackages or something.
My main aim is to invert the approach we've had up until now, and _enforce_ the clean separation of these packages from the Core. I want it to be difficult to add a dependency from the Core to these packages.
frank
--Hannes
On 2/13/13, Frank Shearar frank.shearar@gmail.com wrote:
On 13 February 2013 11:22, David T. Lewis lewis@mail.msen.com wrote: > On Mon, Feb 11, 2013 at 05:28:26PM +0000, H. Hirzel wrote: >> Yanni, >> >> I think St??phane refers to the original Pharo manifesto which >> clearly >> states "no backward compatibility". http://code.google.com/p/pharo/ >> >> However the current Pharo web page has a mission statement >> http://www.pharo-project.org/about >> sets a much more moderate tone. >> >> In any case in this thread we want to move on towards a Squeak core >> and learn from the Pharo experience as much as possible. Please let >> us >> not digress from this important topic. >> >> Maybe we should follow both at the same time >> >> Let me call it >> - the Pavel Krivanek approach and >> - the >> SmalltalkImage unloadAllKnownPackages >> approach >> >> BTW >> #unloadAllKnownPackages >> >> used to work in Squeak 4.1, see >> >> http://lists.squeakfoundation.org/pipermail/squeak-dev/2010-August/152427.ht... >> >> So there is no reason why we should not manage to get it working >> again >> in Squeak 4.5alpha. >> >> And Pavel's approach may be followed in parallel. Because fixing one >> thing will help the other and vice-verse. >> > > +1 > > Exactly so. That is what I intended when I mentioned > #unloadAllKnownPackages. > Thanks for stating it so clearly.
While we're being clear about what's clear :) I'm wanting to _lose_ #unloadAllKnownPackages, and replace it with a #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload, and add a bunch of new jobs running those unloaded packages' test suites.
That way, the thing called Squeak4.5-nnnn.image still contains what Squeak4.5-${whatever_current_is}.image, only the essence of trunk - what SqueakTrunk produces - shrinks.
> Dave
OK
https://github.com/frankshearar/squeak-ci/blob/master/run-test.sh
executes XML-Parser.st
which contains
Installer installUrl: 'http://source.squeak.org/trunk/XML-Parser-ael.35.mcz'.
HDTestReport runPackage: 'XML-Parser'.
"Throw away the dirty image." WorldState addDeferredUIMessage: [ SmalltalkImage current snapshot: false andQuit: true ].
I assume this operates on a trunk image which has the XML-Parser removed?
In which script does that happen?
--Hannes
On 2/14/13, Frank Shearar frank.shearar@gmail.com wrote:
https://github.com/frankshearar/squeak-ci/blob/master/package-load-tests/XML...
They all get triggered by this guy, who I aim to replace with some Ruby (because I'd really, really, REALLY rather write Ruby than shell): https://github.com/frankshearar/squeak-ci/blob/master/run-test.sh
So the jobs only differ in their command: `./run-test.sh Control`, `./run-test.sh XML-Parser`, and so on.
frank
On 14 February 2013 16:34, H. Hirzel hannes.hirzel@gmail.com wrote:
Great
where can I see what the script is doing for example for XML-Parser?
Which script does the job?
http://www.squeakci.org/job/ExternalPackage-XML-Parser/ws/
--Hannes
On 2/14/13, Frank Shearar frank.shearar@gmail.com wrote:
We also now have builds for XML-Parser, Nebraska and Universes (for what it's worth: the code coverage is abysmal).
frank
On 13 February 2013 15:23, H. Hirzel hannes.hirzel@gmail.com wrote:
Yes, thank you for the clarification, Frank. This makes a lot of sense to me and I hope that others follow this line of reasoning.
--Hannes
On 2/13/13, Frank Shearar frank.shearar@gmail.com wrote:
On 13 February 2013 12:45, H. Hirzel hannes.hirzel@gmail.com wrote:
Frank
If I understand you correctly you want to do the following
- unload some once packages from trunk. The packages are taken from
what is in the list in #unloadAllKnownPackages.
- add a trunk method
#loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload
- add CI build jobs which do
#loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload and run tests on the loaded packages.
- People use the result of 3) for regular work. This is the "full"
4.5 image as we have of now only that we have less in trunk and the unlodable packages reside in their own repositories.
Steps 2..4 are done repeatedly.
Is this what you mean?
Yes. Perhaps with a nicer name than #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload. Maybe #loadBasicPackages or something.
My main aim is to invert the approach we've had up until now, and _enforce_ the clean separation of these packages from the Core. I want it to be difficult to add a dependency from the Core to these packages.
frank
--Hannes
On 2/13/13, Frank Shearar frank.shearar@gmail.com wrote: > On 13 February 2013 11:22, David T. Lewis lewis@mail.msen.com > wrote: >> On Mon, Feb 11, 2013 at 05:28:26PM +0000, H. Hirzel wrote: >>> Yanni, >>> >>> I think St??phane refers to the original Pharo manifesto which >>> clearly >>> states "no backward compatibility". >>> http://code.google.com/p/pharo/ >>> >>> However the current Pharo web page has a mission statement >>> http://www.pharo-project.org/about >>> sets a much more moderate tone. >>> >>> In any case in this thread we want to move on towards a Squeak >>> core >>> and learn from the Pharo experience as much as possible. Please >>> let >>> us >>> not digress from this important topic. >>> >>> Maybe we should follow both at the same time >>> >>> Let me call it >>> - the Pavel Krivanek approach and >>> - the >>> SmalltalkImage unloadAllKnownPackages >>> approach >>> >>> BTW >>> #unloadAllKnownPackages >>> >>> used to work in Squeak 4.1, see >>> >>> http://lists.squeakfoundation.org/pipermail/squeak-dev/2010-August/152427.ht... >>> >>> So there is no reason why we should not manage to get it working >>> again >>> in Squeak 4.5alpha. >>> >>> And Pavel's approach may be followed in parallel. Because fixing >>> one >>> thing will help the other and vice-verse. >>> >> >> +1 >> >> Exactly so. That is what I intended when I mentioned >> #unloadAllKnownPackages. >> Thanks for stating it so clearly. > > While we're being clear about what's clear :) I'm wanting to _lose_ > #unloadAllKnownPackages, and replace it with a > #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload, and add a > bunch of new jobs running those unloaded packages' test suites. > > That way, the thing called Squeak4.5-nnnn.image still contains what > Squeak4.5-${whatever_current_is}.image, only the essence of trunk - > what SqueakTrunk produces - shrinks. > >> Dave > >
No, this takes the Trunk image as is and runs the job. No stripping has occurred. Noone's actually said "oh, that's a sane idea someone should do that" yet (except possibly for you, by virtue of not saying "are you crazy?" :) ), so I haven't done it yet.
So the installation is essentially a no-op.
With green lights on these builds, we can now verify that removing these packages won't break anything. At least, to the limit of the tests, which isn't much at all. So if we _do_ actually strip these guys out of Trunk, the builds should stay green. If they don't, we'll need to roll back the change and investigate.
But I don't actually know how to strip them out of Trunk. I _suspect_ it'll be via a Monticello config map of some description.
frank
On 14 February 2013 16:44, H. Hirzel hannes.hirzel@gmail.com wrote:
OK
https://github.com/frankshearar/squeak-ci/blob/master/run-test.shexecutes XML-Parser.st
which contains
Installer installUrl: 'http://source.squeak.org/trunk/XML-Parser-ael.35.mcz'.
HDTestReport runPackage: 'XML-Parser'.
"Throw away the dirty image." WorldState addDeferredUIMessage: [ SmalltalkImage current snapshot: false andQuit: true ].
I assume this operates on a trunk image which has the XML-Parser removed?
In which script does that happen?
--Hannes
On 2/14/13, Frank Shearar frank.shearar@gmail.com wrote:
https://github.com/frankshearar/squeak-ci/blob/master/package-load-tests/XML...
They all get triggered by this guy, who I aim to replace with some Ruby (because I'd really, really, REALLY rather write Ruby than shell): https://github.com/frankshearar/squeak-ci/blob/master/run-test.sh
So the jobs only differ in their command: `./run-test.sh Control`, `./run-test.sh XML-Parser`, and so on.
frank
On 14 February 2013 16:34, H. Hirzel hannes.hirzel@gmail.com wrote:
Great
where can I see what the script is doing for example for XML-Parser?
Which script does the job?
http://www.squeakci.org/job/ExternalPackage-XML-Parser/ws/
--Hannes
On 2/14/13, Frank Shearar frank.shearar@gmail.com wrote:
We also now have builds for XML-Parser, Nebraska and Universes (for what it's worth: the code coverage is abysmal).
frank
On 13 February 2013 15:23, H. Hirzel hannes.hirzel@gmail.com wrote:
Yes, thank you for the clarification, Frank. This makes a lot of sense to me and I hope that others follow this line of reasoning.
--Hannes
On 2/13/13, Frank Shearar frank.shearar@gmail.com wrote:
On 13 February 2013 12:45, H. Hirzel hannes.hirzel@gmail.com wrote: > Frank > > If I understand you correctly you want to do the following > > 1) unload some once packages from trunk. The packages are taken from > what is in the list in #unloadAllKnownPackages. > > 2) add a trunk method > #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload > > 3) add CI build jobs which do > #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload and run tests > on the loaded packages. > > 4) People use the result of 3) for regular work. This is the "full" > 4.5 image as we have of now only that we have less in trunk and the > unlodable packages reside in their own repositories. > > Steps 2..4 are done repeatedly. > > Is this what you mean?
Yes. Perhaps with a nicer name than #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload. Maybe #loadBasicPackages or something.
My main aim is to invert the approach we've had up until now, and _enforce_ the clean separation of these packages from the Core. I want it to be difficult to add a dependency from the Core to these packages.
frank
> --Hannes > > On 2/13/13, Frank Shearar frank.shearar@gmail.com wrote: >> On 13 February 2013 11:22, David T. Lewis lewis@mail.msen.com >> wrote: >>> On Mon, Feb 11, 2013 at 05:28:26PM +0000, H. Hirzel wrote: >>>> Yanni, >>>> >>>> I think St??phane refers to the original Pharo manifesto which >>>> clearly >>>> states "no backward compatibility". >>>> http://code.google.com/p/pharo/ >>>> >>>> However the current Pharo web page has a mission statement >>>> http://www.pharo-project.org/about >>>> sets a much more moderate tone. >>>> >>>> In any case in this thread we want to move on towards a Squeak >>>> core >>>> and learn from the Pharo experience as much as possible. Please >>>> let >>>> us >>>> not digress from this important topic. >>>> >>>> Maybe we should follow both at the same time >>>> >>>> Let me call it >>>> - the Pavel Krivanek approach and >>>> - the >>>> SmalltalkImage unloadAllKnownPackages >>>> approach >>>> >>>> BTW >>>> #unloadAllKnownPackages >>>> >>>> used to work in Squeak 4.1, see >>>> >>>> http://lists.squeakfoundation.org/pipermail/squeak-dev/2010-August/152427.ht... >>>> >>>> So there is no reason why we should not manage to get it working >>>> again >>>> in Squeak 4.5alpha. >>>> >>>> And Pavel's approach may be followed in parallel. Because fixing >>>> one >>>> thing will help the other and vice-verse. >>>> >>> >>> +1 >>> >>> Exactly so. That is what I intended when I mentioned >>> #unloadAllKnownPackages. >>> Thanks for stating it so clearly. >> >> While we're being clear about what's clear :) I'm wanting to _lose_ >> #unloadAllKnownPackages, and replace it with a >> #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload, and add a >> bunch of new jobs running those unloaded packages' test suites. >> >> That way, the thing called Squeak4.5-nnnn.image still contains what >> Squeak4.5-${whatever_current_is}.image, only the essence of trunk - >> what SqueakTrunk produces - shrinks. >> >>> Dave >> >> >
On 14 February 2013 16:50, Frank Shearar frank.shearar@gmail.com wrote:
No, this takes the Trunk image as is and runs the job. No stripping has occurred. Noone's actually said "oh, that's a sane idea someone should do that" yet (except possibly for you, by virtue of not saying "are you crazy?" :) ), so I haven't done it yet.
So the installation is essentially a no-op.
With green lights on these builds, we can now verify that removing these packages won't break anything. At least, to the limit of the tests, which isn't much at all. So if we _do_ actually strip these guys out of Trunk, the builds should stay green. If they don't, we'll need to roll back the change and investigate.
But I don't actually know how to strip them out of Trunk. I _suspect_ it'll be via a Monticello config map of some description.
Anyone? Bert? How do we strip packages out of Trunk?
frank
On 14 February 2013 16:44, H. Hirzel hannes.hirzel@gmail.com wrote:
OK
https://github.com/frankshearar/squeak-ci/blob/master/run-test.shexecutes XML-Parser.st
which contains
Installer installUrl: 'http://source.squeak.org/trunk/XML-Parser-ael.35.mcz'.
HDTestReport runPackage: 'XML-Parser'.
"Throw away the dirty image." WorldState addDeferredUIMessage: [ SmalltalkImage current snapshot: false andQuit: true ].
I assume this operates on a trunk image which has the XML-Parser removed?
In which script does that happen?
--Hannes
On 2/14/13, Frank Shearar frank.shearar@gmail.com wrote:
https://github.com/frankshearar/squeak-ci/blob/master/package-load-tests/XML...
They all get triggered by this guy, who I aim to replace with some Ruby (because I'd really, really, REALLY rather write Ruby than shell): https://github.com/frankshearar/squeak-ci/blob/master/run-test.sh
So the jobs only differ in their command: `./run-test.sh Control`, `./run-test.sh XML-Parser`, and so on.
frank
On 14 February 2013 16:34, H. Hirzel hannes.hirzel@gmail.com wrote:
Great
where can I see what the script is doing for example for XML-Parser?
Which script does the job?
http://www.squeakci.org/job/ExternalPackage-XML-Parser/ws/
--Hannes
On 2/14/13, Frank Shearar frank.shearar@gmail.com wrote:
We also now have builds for XML-Parser, Nebraska and Universes (for what it's worth: the code coverage is abysmal).
frank
On 13 February 2013 15:23, H. Hirzel hannes.hirzel@gmail.com wrote:
Yes, thank you for the clarification, Frank. This makes a lot of sense to me and I hope that others follow this line of reasoning.
--Hannes
On 2/13/13, Frank Shearar frank.shearar@gmail.com wrote: > On 13 February 2013 12:45, H. Hirzel hannes.hirzel@gmail.com wrote: >> Frank >> >> If I understand you correctly you want to do the following >> >> 1) unload some once packages from trunk. The packages are taken from >> what is in the list in #unloadAllKnownPackages. >> >> 2) add a trunk method >> #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload >> >> 3) add CI build jobs which do >> #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload and run tests >> on the loaded packages. >> >> 4) People use the result of 3) for regular work. This is the "full" >> 4.5 image as we have of now only that we have less in trunk and the >> unlodable packages reside in their own repositories. >> >> Steps 2..4 are done repeatedly. >> >> Is this what you mean? > > Yes. Perhaps with a nicer name than > #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload. Maybe > #loadBasicPackages or something. > > My main aim is to invert the approach we've had up until now, and > _enforce_ the clean separation of these packages from the Core. I want > it to be difficult to add a dependency from the Core to these > packages. > > frank > >> --Hannes >> >> On 2/13/13, Frank Shearar frank.shearar@gmail.com wrote: >>> On 13 February 2013 11:22, David T. Lewis lewis@mail.msen.com >>> wrote: >>>> On Mon, Feb 11, 2013 at 05:28:26PM +0000, H. Hirzel wrote: >>>>> Yanni, >>>>> >>>>> I think St??phane refers to the original Pharo manifesto which >>>>> clearly >>>>> states "no backward compatibility". >>>>> http://code.google.com/p/pharo/ >>>>> >>>>> However the current Pharo web page has a mission statement >>>>> http://www.pharo-project.org/about >>>>> sets a much more moderate tone. >>>>> >>>>> In any case in this thread we want to move on towards a Squeak >>>>> core >>>>> and learn from the Pharo experience as much as possible. Please >>>>> let >>>>> us >>>>> not digress from this important topic. >>>>> >>>>> Maybe we should follow both at the same time >>>>> >>>>> Let me call it >>>>> - the Pavel Krivanek approach and >>>>> - the >>>>> SmalltalkImage unloadAllKnownPackages >>>>> approach >>>>> >>>>> BTW >>>>> #unloadAllKnownPackages >>>>> >>>>> used to work in Squeak 4.1, see >>>>> >>>>> http://lists.squeakfoundation.org/pipermail/squeak-dev/2010-August/152427.ht... >>>>> >>>>> So there is no reason why we should not manage to get it working >>>>> again >>>>> in Squeak 4.5alpha. >>>>> >>>>> And Pavel's approach may be followed in parallel. Because fixing >>>>> one >>>>> thing will help the other and vice-verse. >>>>> >>>> >>>> +1 >>>> >>>> Exactly so. That is what I intended when I mentioned >>>> #unloadAllKnownPackages. >>>> Thanks for stating it so clearly. >>> >>> While we're being clear about what's clear :) I'm wanting to _lose_ >>> #unloadAllKnownPackages, and replace it with a >>> #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload, and add a >>> bunch of new jobs running those unloaded packages' test suites. >>> >>> That way, the thing called Squeak4.5-nnnn.image still contains what >>> Squeak4.5-${whatever_current_is}.image, only the essence of trunk - >>> what SqueakTrunk produces - shrinks. >>> >>>> Dave >>> >>> >> > >
On 2013-02-25, at 15:33, Frank Shearar frank.shearar@gmail.com wrote:
On 14 February 2013 16:50, Frank Shearar frank.shearar@gmail.com wrote:
No, this takes the Trunk image as is and runs the job. No stripping has occurred. Noone's actually said "oh, that's a sane idea someone should do that" yet (except possibly for you, by virtue of not saying "are you crazy?" :) ), so I haven't done it yet.
So the installation is essentially a no-op.
With green lights on these builds, we can now verify that removing these packages won't break anything. At least, to the limit of the tests, which isn't much at all. So if we _do_ actually strip these guys out of Trunk, the builds should stay green. If they don't, we'll need to roll back the change and investigate.
But I don't actually know how to strip them out of Trunk. I _suspect_ it'll be via a Monticello config map of some description.
Anyone? Bert? How do we strip packages out of Trunk?
Manually once, when you prepare the Core image. Otherwise, if we put it in the trunk update stream, the packages would be unloaded from everyone's working images when updating.
- Bert -
frank
On 14 February 2013 16:44, H. Hirzel hannes.hirzel@gmail.com wrote:
OK
https://github.com/frankshearar/squeak-ci/blob/master/run-test.sh
executes XML-Parser.st
which contains
Installer installUrl: 'http://source.squeak.org/trunk/XML-Parser-ael.35.mcz'.
HDTestReport runPackage: 'XML-Parser'.
"Throw away the dirty image." WorldState addDeferredUIMessage: [ SmalltalkImage current snapshot: false andQuit: true ].
I assume this operates on a trunk image which has the XML-Parser removed?
In which script does that happen?
--Hannes
On 2/14/13, Frank Shearar frank.shearar@gmail.com wrote:
https://github.com/frankshearar/squeak-ci/blob/master/package-load-tests/XML...
They all get triggered by this guy, who I aim to replace with some Ruby (because I'd really, really, REALLY rather write Ruby than shell): https://github.com/frankshearar/squeak-ci/blob/master/run-test.sh
So the jobs only differ in their command: `./run-test.sh Control`, `./run-test.sh XML-Parser`, and so on.
frank
On 14 February 2013 16:34, H. Hirzel hannes.hirzel@gmail.com wrote:
Great
where can I see what the script is doing for example for XML-Parser?
Which script does the job?
http://www.squeakci.org/job/ExternalPackage-XML-Parser/ws/
--Hannes
On 2/14/13, Frank Shearar frank.shearar@gmail.com wrote:
We also now have builds for XML-Parser, Nebraska and Universes (for what it's worth: the code coverage is abysmal).
frank
On 13 February 2013 15:23, H. Hirzel hannes.hirzel@gmail.com wrote: > Yes, thank you for the clarification, Frank. This makes a lot of sense > to me and I hope that others follow this line of reasoning. > > --Hannes > > On 2/13/13, Frank Shearar frank.shearar@gmail.com wrote: >> On 13 February 2013 12:45, H. Hirzel hannes.hirzel@gmail.com wrote: >>> Frank >>> >>> If I understand you correctly you want to do the following >>> >>> 1) unload some once packages from trunk. The packages are taken from >>> what is in the list in #unloadAllKnownPackages. >>> >>> 2) add a trunk method >>> #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload >>> >>> 3) add CI build jobs which do >>> #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload and run tests >>> on the loaded packages. >>> >>> 4) People use the result of 3) for regular work. This is the "full" >>> 4.5 image as we have of now only that we have less in trunk and the >>> unlodable packages reside in their own repositories. >>> >>> Steps 2..4 are done repeatedly. >>> >>> Is this what you mean? >> >> Yes. Perhaps with a nicer name than >> #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload. Maybe >> #loadBasicPackages or something. >> >> My main aim is to invert the approach we've had up until now, and >> _enforce_ the clean separation of these packages from the Core. I want >> it to be difficult to add a dependency from the Core to these >> packages. >> >> frank >> >>> --Hannes >>> >>> On 2/13/13, Frank Shearar frank.shearar@gmail.com wrote: >>>> On 13 February 2013 11:22, David T. Lewis lewis@mail.msen.com >>>> wrote: >>>>> On Mon, Feb 11, 2013 at 05:28:26PM +0000, H. Hirzel wrote: >>>>>> Yanni, >>>>>> >>>>>> I think St??phane refers to the original Pharo manifesto which >>>>>> clearly >>>>>> states "no backward compatibility". >>>>>> http://code.google.com/p/pharo/ >>>>>> >>>>>> However the current Pharo web page has a mission statement >>>>>> http://www.pharo-project.org/about >>>>>> sets a much more moderate tone. >>>>>> >>>>>> In any case in this thread we want to move on towards a Squeak >>>>>> core >>>>>> and learn from the Pharo experience as much as possible. Please >>>>>> let >>>>>> us >>>>>> not digress from this important topic. >>>>>> >>>>>> Maybe we should follow both at the same time >>>>>> >>>>>> Let me call it >>>>>> - the Pavel Krivanek approach and >>>>>> - the >>>>>> SmalltalkImage unloadAllKnownPackages >>>>>> approach >>>>>> >>>>>> BTW >>>>>> #unloadAllKnownPackages >>>>>> >>>>>> used to work in Squeak 4.1, see >>>>>> >>>>>> http://lists.squeakfoundation.org/pipermail/squeak-dev/2010-August/152427.ht... >>>>>> >>>>>> So there is no reason why we should not manage to get it working >>>>>> again >>>>>> in Squeak 4.5alpha. >>>>>> >>>>>> And Pavel's approach may be followed in parallel. Because fixing >>>>>> one >>>>>> thing will help the other and vice-verse. >>>>>> >>>>> >>>>> +1 >>>>> >>>>> Exactly so. That is what I intended when I mentioned >>>>> #unloadAllKnownPackages. >>>>> Thanks for stating it so clearly. >>>> >>>> While we're being clear about what's clear :) I'm wanting to _lose_ >>>> #unloadAllKnownPackages, and replace it with a >>>> #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload, and add a >>>> bunch of new jobs running those unloaded packages' test suites. >>>> >>>> That way, the thing called Squeak4.5-nnnn.image still contains what >>>> Squeak4.5-${whatever_current_is}.image, only the essence of trunk - >>>> what SqueakTrunk produces - shrinks. >>>> >>>>> Dave >>>> >>>> >>> >> >> >
On 25 February 2013 14:49, Bert Freudenberg bert@freudenbergs.de wrote:
On 2013-02-25, at 15:33, Frank Shearar frank.shearar@gmail.com wrote:
On 14 February 2013 16:50, Frank Shearar frank.shearar@gmail.com wrote:
No, this takes the Trunk image as is and runs the job. No stripping has occurred. Noone's actually said "oh, that's a sane idea someone should do that" yet (except possibly for you, by virtue of not saying "are you crazy?" :) ), so I haven't done it yet.
So the installation is essentially a no-op.
With green lights on these builds, we can now verify that removing these packages won't break anything. At least, to the limit of the tests, which isn't much at all. So if we _do_ actually strip these guys out of Trunk, the builds should stay green. If they don't, we'll need to roll back the change and investigate.
But I don't actually know how to strip them out of Trunk. I _suspect_ it'll be via a Monticello config map of some description.
Anyone? Bert? How do we strip packages out of Trunk?
Manually once, when you prepare the Core image. Otherwise, if we put it in the trunk update stream, the packages would be unloaded from everyone's working images when updating.
Given that we're early in 4.5's release cycle, if we do this kind of thing - stripping unnecessary things out of trunk - we ought to do it now rather than later. And announce loudly what we're doing, why, and how people can load in their favourite not-Core packages.
We have CI tests for the packages (well, the packages I chose as the first victims) showing that they can be loaded into a Trunk image, and while the test coverage is dismal, that just means we don't know whether they work while in Trunk.
I'm _not_ about to go and start unloading packages without significant buy-in, but I really do think we ought to actually start working towards a lean, minimal, modular core. That will, of necessity, cause disruption to people updating from trunk. The ReleaseBuilder script or CI job can re-add these packages so that people downloading an image see no change.
frank
- Bert -
frank
On 14 February 2013 16:44, H. Hirzel hannes.hirzel@gmail.com wrote:
OK
https://github.com/frankshearar/squeak-ci/blob/master/run-test.sh
executes XML-Parser.st
which contains
Installer installUrl: 'http://source.squeak.org/trunk/XML-Parser-ael.35.mcz'.
HDTestReport runPackage: 'XML-Parser'.
"Throw away the dirty image." WorldState addDeferredUIMessage: [ SmalltalkImage current snapshot: false andQuit: true ].
I assume this operates on a trunk image which has the XML-Parser removed?
In which script does that happen?
--Hannes
On 2/14/13, Frank Shearar frank.shearar@gmail.com wrote:
https://github.com/frankshearar/squeak-ci/blob/master/package-load-tests/XML...
They all get triggered by this guy, who I aim to replace with some Ruby (because I'd really, really, REALLY rather write Ruby than shell): https://github.com/frankshearar/squeak-ci/blob/master/run-test.sh
So the jobs only differ in their command: `./run-test.sh Control`, `./run-test.sh XML-Parser`, and so on.
frank
On 14 February 2013 16:34, H. Hirzel hannes.hirzel@gmail.com wrote:
Great
where can I see what the script is doing for example for XML-Parser?
Which script does the job?
http://www.squeakci.org/job/ExternalPackage-XML-Parser/ws/
--Hannes
On 2/14/13, Frank Shearar frank.shearar@gmail.com wrote: > We also now have builds for XML-Parser, Nebraska and Universes (for > what it's worth: the code coverage is abysmal). > > frank > > On 13 February 2013 15:23, H. Hirzel hannes.hirzel@gmail.com wrote: >> Yes, thank you for the clarification, Frank. This makes a lot of sense >> to me and I hope that others follow this line of reasoning. >> >> --Hannes >> >> On 2/13/13, Frank Shearar frank.shearar@gmail.com wrote: >>> On 13 February 2013 12:45, H. Hirzel hannes.hirzel@gmail.com wrote: >>>> Frank >>>> >>>> If I understand you correctly you want to do the following >>>> >>>> 1) unload some once packages from trunk. The packages are taken from >>>> what is in the list in #unloadAllKnownPackages. >>>> >>>> 2) add a trunk method >>>> #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload >>>> >>>> 3) add CI build jobs which do >>>> #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload and run tests >>>> on the loaded packages. >>>> >>>> 4) People use the result of 3) for regular work. This is the "full" >>>> 4.5 image as we have of now only that we have less in trunk and the >>>> unlodable packages reside in their own repositories. >>>> >>>> Steps 2..4 are done repeatedly. >>>> >>>> Is this what you mean? >>> >>> Yes. Perhaps with a nicer name than >>> #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload. Maybe >>> #loadBasicPackages or something. >>> >>> My main aim is to invert the approach we've had up until now, and >>> _enforce_ the clean separation of these packages from the Core. I want >>> it to be difficult to add a dependency from the Core to these >>> packages. >>> >>> frank >>> >>>> --Hannes >>>> >>>> On 2/13/13, Frank Shearar frank.shearar@gmail.com wrote: >>>>> On 13 February 2013 11:22, David T. Lewis lewis@mail.msen.com >>>>> wrote: >>>>>> On Mon, Feb 11, 2013 at 05:28:26PM +0000, H. Hirzel wrote: >>>>>>> Yanni, >>>>>>> >>>>>>> I think St??phane refers to the original Pharo manifesto which >>>>>>> clearly >>>>>>> states "no backward compatibility". >>>>>>> http://code.google.com/p/pharo/ >>>>>>> >>>>>>> However the current Pharo web page has a mission statement >>>>>>> http://www.pharo-project.org/about >>>>>>> sets a much more moderate tone. >>>>>>> >>>>>>> In any case in this thread we want to move on towards a Squeak >>>>>>> core >>>>>>> and learn from the Pharo experience as much as possible. Please >>>>>>> let >>>>>>> us >>>>>>> not digress from this important topic. >>>>>>> >>>>>>> Maybe we should follow both at the same time >>>>>>> >>>>>>> Let me call it >>>>>>> - the Pavel Krivanek approach and >>>>>>> - the >>>>>>> SmalltalkImage unloadAllKnownPackages >>>>>>> approach >>>>>>> >>>>>>> BTW >>>>>>> #unloadAllKnownPackages >>>>>>> >>>>>>> used to work in Squeak 4.1, see >>>>>>> >>>>>>> http://lists.squeakfoundation.org/pipermail/squeak-dev/2010-August/152427.ht... >>>>>>> >>>>>>> So there is no reason why we should not manage to get it working >>>>>>> again >>>>>>> in Squeak 4.5alpha. >>>>>>> >>>>>>> And Pavel's approach may be followed in parallel. Because fixing >>>>>>> one >>>>>>> thing will help the other and vice-verse. >>>>>>> >>>>>> >>>>>> +1 >>>>>> >>>>>> Exactly so. That is what I intended when I mentioned >>>>>> #unloadAllKnownPackages. >>>>>> Thanks for stating it so clearly. >>>>> >>>>> While we're being clear about what's clear :) I'm wanting to _lose_ >>>>> #unloadAllKnownPackages, and replace it with a >>>>> #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload, and add a >>>>> bunch of new jobs running those unloaded packages' test suites. >>>>> >>>>> That way, the thing called Squeak4.5-nnnn.image still contains what >>>>> Squeak4.5-${whatever_current_is}.image, only the essence of trunk - >>>>> what SqueakTrunk produces - shrinks. >>>>> >>>>>> Dave >>>>> >>>>> >>>> >>> >>> >> > >
On Mon, Feb 25, 2013 at 03:01:18PM +0000, Frank Shearar wrote:
On 25 February 2013 14:49, Bert Freudenberg bert@freudenbergs.de wrote:
On 2013-02-25, at 15:33, Frank Shearar frank.shearar@gmail.com wrote:
On 14 February 2013 16:50, Frank Shearar frank.shearar@gmail.com wrote:
No, this takes the Trunk image as is and runs the job. No stripping has occurred. Noone's actually said "oh, that's a sane idea someone should do that" yet (except possibly for you, by virtue of not saying "are you crazy?" :) ), so I haven't done it yet.
So the installation is essentially a no-op.
With green lights on these builds, we can now verify that removing these packages won't break anything. At least, to the limit of the tests, which isn't much at all. So if we _do_ actually strip these guys out of Trunk, the builds should stay green. If they don't, we'll need to roll back the change and investigate.
But I don't actually know how to strip them out of Trunk. I _suspect_ it'll be via a Monticello config map of some description.
Anyone? Bert? How do we strip packages out of Trunk?
Manually once, when you prepare the Core image. Otherwise, if we put it in the trunk update stream, the packages would be unloaded from everyone's working images when updating.
Given that we're early in 4.5's release cycle, if we do this kind of thing - stripping unnecessary things out of trunk - we ought to do it now rather than later. And announce loudly what we're doing, why, and how people can load in their favourite not-Core packages.
NO.
If Squeak developers work with images that contain just "their favourite non-Core packages" then none of the non-Core packages get maintained.
It is important to work on making packages loadable and unloadable, but you do not need to break the trunk image in order to accomplish this.
Also, please recognize that not everybody wants to do all their work in a clean image built by a robot. I for example do most of my work in a Squeak3.11alpha image that has been continuously updated from the update stream for quite a while. I also periodically check things against a fresh 4.4 image, but my usual pattern is to use one image over a long period of time with the trunk update stream, and most of my updates to trunk come from that old image.
Dave
We have CI tests for the packages (well, the packages I chose as the first victims) showing that they can be loaded into a Trunk image, and while the test coverage is dismal, that just means we don't know whether they work while in Trunk.
I'm _not_ about to go and start unloading packages without significant buy-in, but I really do think we ought to actually start working towards a lean, minimal, modular core. That will, of necessity, cause disruption to people updating from trunk. The ReleaseBuilder script or CI job can re-add these packages so that people downloading an image see no change.
frank
- Bert -
frank
On 14 February 2013 16:44, H. Hirzel hannes.hirzel@gmail.com wrote:
OK
https://github.com/frankshearar/squeak-ci/blob/master/run-test.sh
executes XML-Parser.st
which contains
Installer installUrl: 'http://source.squeak.org/trunk/XML-Parser-ael.35.mcz'.
HDTestReport runPackage: 'XML-Parser'.
"Throw away the dirty image." WorldState addDeferredUIMessage: [ SmalltalkImage current snapshot: false andQuit: true ].
I assume this operates on a trunk image which has the XML-Parser removed?
In which script does that happen?
--Hannes
On 2/14/13, Frank Shearar frank.shearar@gmail.com wrote:
https://github.com/frankshearar/squeak-ci/blob/master/package-load-tests/XML...
They all get triggered by this guy, who I aim to replace with some Ruby (because I'd really, really, REALLY rather write Ruby than shell): https://github.com/frankshearar/squeak-ci/blob/master/run-test.sh
So the jobs only differ in their command: `./run-test.sh Control`, `./run-test.sh XML-Parser`, and so on.
frank
On 14 February 2013 16:34, H. Hirzel hannes.hirzel@gmail.com wrote: > Great > > where can I see what the script is doing for example for XML-Parser? > > Which script does the job? > > http://www.squeakci.org/job/ExternalPackage-XML-Parser/ws/ > > --Hannes > > On 2/14/13, Frank Shearar frank.shearar@gmail.com wrote: >> We also now have builds for XML-Parser, Nebraska and Universes (for >> what it's worth: the code coverage is abysmal). >> >> frank >> >> On 13 February 2013 15:23, H. Hirzel hannes.hirzel@gmail.com wrote: >>> Yes, thank you for the clarification, Frank. This makes a lot of sense >>> to me and I hope that others follow this line of reasoning. >>> >>> --Hannes >>> >>> On 2/13/13, Frank Shearar frank.shearar@gmail.com wrote: >>>> On 13 February 2013 12:45, H. Hirzel hannes.hirzel@gmail.com wrote: >>>>> Frank >>>>> >>>>> If I understand you correctly you want to do the following >>>>> >>>>> 1) unload some once packages from trunk. The packages are taken from >>>>> what is in the list in #unloadAllKnownPackages. >>>>> >>>>> 2) add a trunk method >>>>> #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload >>>>> >>>>> 3) add CI build jobs which do >>>>> #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload and run tests >>>>> on the loaded packages. >>>>> >>>>> 4) People use the result of 3) for regular work. This is the "full" >>>>> 4.5 image as we have of now only that we have less in trunk and the >>>>> unlodable packages reside in their own repositories. >>>>> >>>>> Steps 2..4 are done repeatedly. >>>>> >>>>> Is this what you mean? >>>> >>>> Yes. Perhaps with a nicer name than >>>> #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload. Maybe >>>> #loadBasicPackages or something. >>>> >>>> My main aim is to invert the approach we've had up until now, and >>>> _enforce_ the clean separation of these packages from the Core. I want >>>> it to be difficult to add a dependency from the Core to these >>>> packages. >>>> >>>> frank >>>> >>>>> --Hannes >>>>> >>>>> On 2/13/13, Frank Shearar frank.shearar@gmail.com wrote: >>>>>> On 13 February 2013 11:22, David T. Lewis lewis@mail.msen.com >>>>>> wrote: >>>>>>> On Mon, Feb 11, 2013 at 05:28:26PM +0000, H. Hirzel wrote: >>>>>>>> Yanni, >>>>>>>> >>>>>>>> I think St??phane refers to the original Pharo manifesto which >>>>>>>> clearly >>>>>>>> states "no backward compatibility". >>>>>>>> http://code.google.com/p/pharo/ >>>>>>>> >>>>>>>> However the current Pharo web page has a mission statement >>>>>>>> http://www.pharo-project.org/about >>>>>>>> sets a much more moderate tone. >>>>>>>> >>>>>>>> In any case in this thread we want to move on towards a Squeak >>>>>>>> core >>>>>>>> and learn from the Pharo experience as much as possible. Please >>>>>>>> let >>>>>>>> us >>>>>>>> not digress from this important topic. >>>>>>>> >>>>>>>> Maybe we should follow both at the same time >>>>>>>> >>>>>>>> Let me call it >>>>>>>> - the Pavel Krivanek approach and >>>>>>>> - the >>>>>>>> SmalltalkImage unloadAllKnownPackages >>>>>>>> approach >>>>>>>> >>>>>>>> BTW >>>>>>>> #unloadAllKnownPackages >>>>>>>> >>>>>>>> used to work in Squeak 4.1, see >>>>>>>> >>>>>>>> http://lists.squeakfoundation.org/pipermail/squeak-dev/2010-August/152427.ht... >>>>>>>> >>>>>>>> So there is no reason why we should not manage to get it working >>>>>>>> again >>>>>>>> in Squeak 4.5alpha. >>>>>>>> >>>>>>>> And Pavel's approach may be followed in parallel. Because fixing >>>>>>>> one >>>>>>>> thing will help the other and vice-verse. >>>>>>>> >>>>>>> >>>>>>> +1 >>>>>>> >>>>>>> Exactly so. That is what I intended when I mentioned >>>>>>> #unloadAllKnownPackages. >>>>>>> Thanks for stating it so clearly. >>>>>> >>>>>> While we're being clear about what's clear :) I'm wanting to _lose_ >>>>>> #unloadAllKnownPackages, and replace it with a >>>>>> #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload, and add a >>>>>> bunch of new jobs running those unloaded packages' test suites. >>>>>> >>>>>> That way, the thing called Squeak4.5-nnnn.image still contains what >>>>>> Squeak4.5-${whatever_current_is}.image, only the essence of trunk - >>>>>> what SqueakTrunk produces - shrinks. >>>>>> >>>>>>> Dave >>>>>> >>>>>> >>>>> >>>> >>>> >>> >> >> >
On 25 February 2013 15:35, David T. Lewis lewis@mail.msen.com wrote:
On Mon, Feb 25, 2013 at 03:01:18PM +0000, Frank Shearar wrote:
On 25 February 2013 14:49, Bert Freudenberg bert@freudenbergs.de wrote:
On 2013-02-25, at 15:33, Frank Shearar frank.shearar@gmail.com wrote:
On 14 February 2013 16:50, Frank Shearar frank.shearar@gmail.com wrote:
No, this takes the Trunk image as is and runs the job. No stripping has occurred. Noone's actually said "oh, that's a sane idea someone should do that" yet (except possibly for you, by virtue of not saying "are you crazy?" :) ), so I haven't done it yet.
So the installation is essentially a no-op.
With green lights on these builds, we can now verify that removing these packages won't break anything. At least, to the limit of the tests, which isn't much at all. So if we _do_ actually strip these guys out of Trunk, the builds should stay green. If they don't, we'll need to roll back the change and investigate.
But I don't actually know how to strip them out of Trunk. I _suspect_ it'll be via a Monticello config map of some description.
Anyone? Bert? How do we strip packages out of Trunk?
Manually once, when you prepare the Core image. Otherwise, if we put it in the trunk update stream, the packages would be unloaded from everyone's working images when updating.
Given that we're early in 4.5's release cycle, if we do this kind of thing - stripping unnecessary things out of trunk - we ought to do it now rather than later. And announce loudly what we're doing, why, and how people can load in their favourite not-Core packages.
NO.
If Squeak developers work with images that contain just "their favourite non-Core packages" then none of the non-Core packages get maintained.
Clearly we disagree quite a bit on this, but I don't think anyone should actually _work_ on a Core image. _Noone_ should be able to work effectively in a minimal image because such a thing wouldn't even have a Browser!
It is important to work on making packages loadable and unloadable, but you do not need to break the trunk image in order to accomplish this.
Who's breaking trunk? I want to remove packages that are already unloadable, and have CI jobs that run every time Trunk updates that verifies that the packages can (a) reload and (b) run as well as we already know now (which is not very). To be precise: Universes (for example) might be smashed beyond recognition, and there is no way to know, because it has something like 23% method coverage.
Removing these packages from Trunk _will_ affect people updating from Trunk, I agree. How else do we make Trunk simpler, if not by removing stuff?
(It's also simple enough to reinstall your favourite packages: Installer squeakmap update; install: 'WhateverItIs (someVersion)'.)
Also, please recognize that not everybody wants to do all their work in a clean image built by a robot. I for example do most of my work in a Squeak3.11alpha image that has been continuously updated from the update stream for quite a while. I also periodically check things against a fresh 4.4 image, but my usual pattern is to use one image over a long period of time with the trunk update stream, and most of my updates to trunk come from that old image.
OK, but then we can't have a minimal core without an entire reboot. I mean, seriously, making a minimal core by unloading packages is an order of magnitude harder when you cannot ever actually unload the packages!
So, given the resistance to my slow, gradual unwinding of the tangle, what's your counterproposal?
frank
Dave
We have CI tests for the packages (well, the packages I chose as the first victims) showing that they can be loaded into a Trunk image, and while the test coverage is dismal, that just means we don't know whether they work while in Trunk.
I'm _not_ about to go and start unloading packages without significant buy-in, but I really do think we ought to actually start working towards a lean, minimal, modular core. That will, of necessity, cause disruption to people updating from trunk. The ReleaseBuilder script or CI job can re-add these packages so that people downloading an image see no change.
frank
- Bert -
frank
On 14 February 2013 16:44, H. Hirzel hannes.hirzel@gmail.com wrote:
OK
https://github.com/frankshearar/squeak-ci/blob/master/run-test.sh
executes XML-Parser.st
which contains
Installer installUrl: 'http://source.squeak.org/trunk/XML-Parser-ael.35.mcz'.
HDTestReport runPackage: 'XML-Parser'.
"Throw away the dirty image." WorldState addDeferredUIMessage: [ SmalltalkImage current snapshot: false andQuit: true ].
I assume this operates on a trunk image which has the XML-Parser removed?
In which script does that happen?
--Hannes
On 2/14/13, Frank Shearar frank.shearar@gmail.com wrote: > https://github.com/frankshearar/squeak-ci/blob/master/package-load-tests/XML... > > They all get triggered by this guy, who I aim to replace with some > Ruby (because I'd really, really, REALLY rather write Ruby than > shell): https://github.com/frankshearar/squeak-ci/blob/master/run-test.sh > > So the jobs only differ in their command: `./run-test.sh Control`, > `./run-test.sh XML-Parser`, and so on. > > frank > > On 14 February 2013 16:34, H. Hirzel hannes.hirzel@gmail.com wrote: >> Great >> >> where can I see what the script is doing for example for XML-Parser? >> >> Which script does the job? >> >> http://www.squeakci.org/job/ExternalPackage-XML-Parser/ws/ >> >> --Hannes >> >> On 2/14/13, Frank Shearar frank.shearar@gmail.com wrote: >>> We also now have builds for XML-Parser, Nebraska and Universes (for >>> what it's worth: the code coverage is abysmal). >>> >>> frank >>> >>> On 13 February 2013 15:23, H. Hirzel hannes.hirzel@gmail.com wrote: >>>> Yes, thank you for the clarification, Frank. This makes a lot of sense >>>> to me and I hope that others follow this line of reasoning. >>>> >>>> --Hannes >>>> >>>> On 2/13/13, Frank Shearar frank.shearar@gmail.com wrote: >>>>> On 13 February 2013 12:45, H. Hirzel hannes.hirzel@gmail.com wrote: >>>>>> Frank >>>>>> >>>>>> If I understand you correctly you want to do the following >>>>>> >>>>>> 1) unload some once packages from trunk. The packages are taken from >>>>>> what is in the list in #unloadAllKnownPackages. >>>>>> >>>>>> 2) add a trunk method >>>>>> #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload >>>>>> >>>>>> 3) add CI build jobs which do >>>>>> #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload and run tests >>>>>> on the loaded packages. >>>>>> >>>>>> 4) People use the result of 3) for regular work. This is the "full" >>>>>> 4.5 image as we have of now only that we have less in trunk and the >>>>>> unlodable packages reside in their own repositories. >>>>>> >>>>>> Steps 2..4 are done repeatedly. >>>>>> >>>>>> Is this what you mean? >>>>> >>>>> Yes. Perhaps with a nicer name than >>>>> #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload. Maybe >>>>> #loadBasicPackages or something. >>>>> >>>>> My main aim is to invert the approach we've had up until now, and >>>>> _enforce_ the clean separation of these packages from the Core. I want >>>>> it to be difficult to add a dependency from the Core to these >>>>> packages. >>>>> >>>>> frank >>>>> >>>>>> --Hannes >>>>>> >>>>>> On 2/13/13, Frank Shearar frank.shearar@gmail.com wrote: >>>>>>> On 13 February 2013 11:22, David T. Lewis lewis@mail.msen.com >>>>>>> wrote: >>>>>>>> On Mon, Feb 11, 2013 at 05:28:26PM +0000, H. Hirzel wrote: >>>>>>>>> Yanni, >>>>>>>>> >>>>>>>>> I think St??phane refers to the original Pharo manifesto which >>>>>>>>> clearly >>>>>>>>> states "no backward compatibility". >>>>>>>>> http://code.google.com/p/pharo/ >>>>>>>>> >>>>>>>>> However the current Pharo web page has a mission statement >>>>>>>>> http://www.pharo-project.org/about >>>>>>>>> sets a much more moderate tone. >>>>>>>>> >>>>>>>>> In any case in this thread we want to move on towards a Squeak >>>>>>>>> core >>>>>>>>> and learn from the Pharo experience as much as possible. Please >>>>>>>>> let >>>>>>>>> us >>>>>>>>> not digress from this important topic. >>>>>>>>> >>>>>>>>> Maybe we should follow both at the same time >>>>>>>>> >>>>>>>>> Let me call it >>>>>>>>> - the Pavel Krivanek approach and >>>>>>>>> - the >>>>>>>>> SmalltalkImage unloadAllKnownPackages >>>>>>>>> approach >>>>>>>>> >>>>>>>>> BTW >>>>>>>>> #unloadAllKnownPackages >>>>>>>>> >>>>>>>>> used to work in Squeak 4.1, see >>>>>>>>> >>>>>>>>> http://lists.squeakfoundation.org/pipermail/squeak-dev/2010-August/152427.ht... >>>>>>>>> >>>>>>>>> So there is no reason why we should not manage to get it working >>>>>>>>> again >>>>>>>>> in Squeak 4.5alpha. >>>>>>>>> >>>>>>>>> And Pavel's approach may be followed in parallel. Because fixing >>>>>>>>> one >>>>>>>>> thing will help the other and vice-verse. >>>>>>>>> >>>>>>>> >>>>>>>> +1 >>>>>>>> >>>>>>>> Exactly so. That is what I intended when I mentioned >>>>>>>> #unloadAllKnownPackages. >>>>>>>> Thanks for stating it so clearly. >>>>>>> >>>>>>> While we're being clear about what's clear :) I'm wanting to _lose_ >>>>>>> #unloadAllKnownPackages, and replace it with a >>>>>>> #loadAllPackagesThatUnloadAllKnownPackagesUsedToUnload, and add a >>>>>>> bunch of new jobs running those unloaded packages' test suites. >>>>>>> >>>>>>> That way, the thing called Squeak4.5-nnnn.image still contains what >>>>>>> Squeak4.5-${whatever_current_is}.image, only the essence of trunk - >>>>>>> what SqueakTrunk produces - shrinks. >>>>>>> >>>>>>>> Dave >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>> >>> >>> >> > >
On 2013-02-25, at 17:07, Frank Shearar frank.shearar@gmail.com wrote:
On 25 February 2013 15:35, David T. Lewis lewis@mail.msen.com wrote:
On Mon, Feb 25, 2013 at 03:01:18PM +0000, Frank Shearar wrote:
On 25 February 2013 14:49, Bert Freudenberg bert@freudenbergs.de wrote:
On 2013-02-25, at 15:33, Frank Shearar frank.shearar@gmail.com wrote:
On 14 February 2013 16:50, Frank Shearar frank.shearar@gmail.com wrote:
No, this takes the Trunk image as is and runs the job. No stripping has occurred. Noone's actually said "oh, that's a sane idea someone should do that" yet (except possibly for you, by virtue of not saying "are you crazy?" :) ), so I haven't done it yet.
So the installation is essentially a no-op.
With green lights on these builds, we can now verify that removing these packages won't break anything. At least, to the limit of the tests, which isn't much at all. So if we _do_ actually strip these guys out of Trunk, the builds should stay green. If they don't, we'll need to roll back the change and investigate.
But I don't actually know how to strip them out of Trunk. I _suspect_ it'll be via a Monticello config map of some description.
Anyone? Bert? How do we strip packages out of Trunk?
Manually once, when you prepare the Core image. Otherwise, if we put it in the trunk update stream, the packages would be unloaded from everyone's working images when updating.
Given that we're early in 4.5's release cycle, if we do this kind of thing - stripping unnecessary things out of trunk - we ought to do it now rather than later. And announce loudly what we're doing, why, and how people can load in their favourite not-Core packages.
NO.
If Squeak developers work with images that contain just "their favourite non-Core packages" then none of the non-Core packages get maintained.
Clearly we disagree quite a bit on this, but I don't think anyone should actually _work_ on a Core image. _Noone_ should be able to work effectively in a minimal image because such a thing wouldn't even have a Browser!
We don't disagree on this at all. People should be working in a full image.
It is important to work on making packages loadable and unloadable, but you do not need to break the trunk image in order to accomplish this.
Who's breaking trunk? I want to remove packages that are already unloadable,
Yes, but not from *user's* images, from the *core* image that is only used by the CI to build a full image.
and have CI jobs that run every time Trunk updates that verifies that the packages can (a) reload and (b) run as well as we already know now (which is not very). To be precise: Universes (for example) might be smashed beyond recognition, and there is no way to know, because it has something like 23% method coverage.
Right.
Removing these packages from Trunk _will_ affect people updating from Trunk, I agree. How else do we make Trunk simpler, if not by removing stuff?
I think we may be talking past each other. There are two types of packages we want to "remove from trunk":
full = core packages + unloadable packages + deprecated packages
So there are unloadable packages which should still exist in a full image that everyone is using. And there are deprecated packages which should not exist in a full image.
Both core packages and unloadable packages are in Trunk. The only packages that we actually want to remove for good are the deprecated ones. The mechanics of unloading a deprecated package is by committing an empty version (which will remove all the classes and methods from user's images when updating), and eventually removing the now empty package from the update map (and bumping the squeak-version package to keep update numbers continuous).
Maybe that's what you were asking about?
(It's also simple enough to reinstall your favourite packages: Installer squeakmap update; install: 'WhateverItIs (someVersion)'.)
Sure, that's how you would re-install a deprecated package.
Also, please recognize that not everybody wants to do all their work in a clean image built by a robot. I for example do most of my work in a Squeak3.11alpha image that has been continuously updated from the update stream for quite a while. I also periodically check things against a fresh 4.4 image, but my usual pattern is to use one image over a long period of time with the trunk update stream, and most of my updates to trunk come from that old image.
OK, but then we can't have a minimal core without an entire reboot. I mean, seriously, making a minimal core by unloading packages is an order of magnitude harder when you cannot ever actually unload the packages!
So, given the resistance to my slow, gradual unwinding of the tangle, what's your counterproposal?
frank
I still fail to see the problem. For purposes of the build server, you once make a Core image by unloading all possible packages. This becomes the new "master image" from which all subsequent releases are built. Then you use that as the base to run tests, build full images, etc.
Does that sound reasonable?
- Bert -
On Mon, Feb 25, 2013 at 05:37:38PM +0100, Bert Freudenberg wrote:
I still fail to see the problem. For purposes of the build server, you once make a Core image by unloading all possible packages. This becomes the new "master image" from which all subsequent releases are built. Then you use that as the base to run tests, build full images, etc.
Does that sound reasonable?
Yes. This provides a mechanism for creating and maintaining a core image, and for verifying on an ongoing basis that the full image built from core matches a full image that is tracking the trunk update stream. At image release time, two should be essentially the same. Meanwhile the full trunk image remains in the update stream so that all developers can see the impact of their changes on all packages.
Dave
On 25 February 2013 16:37, Bert Freudenberg bert@freudenbergs.de wrote:
On 2013-02-25, at 17:07, Frank Shearar frank.shearar@gmail.com wrote:
On 25 February 2013 15:35, David T. Lewis lewis@mail.msen.com wrote:
On Mon, Feb 25, 2013 at 03:01:18PM +0000, Frank Shearar wrote:
On 25 February 2013 14:49, Bert Freudenberg bert@freudenbergs.de wrote:
On 2013-02-25, at 15:33, Frank Shearar frank.shearar@gmail.com wrote:
On 14 February 2013 16:50, Frank Shearar frank.shearar@gmail.com wrote: > No, this takes the Trunk image as is and runs the job. No stripping > has occurred. Noone's actually said "oh, that's a sane idea someone > should do that" yet (except possibly for you, by virtue of not saying > "are you crazy?" :) ), so I haven't done it yet. > > So the installation is essentially a no-op. > > With green lights on these builds, we can now verify that removing > these packages won't break anything. At least, to the limit of the > tests, which isn't much at all. So if we _do_ actually strip these > guys out of Trunk, the builds should stay green. If they don't, we'll > need to roll back the change and investigate. > > But I don't actually know how to strip them out of Trunk. I _suspect_ > it'll be via a Monticello config map of some description.
Anyone? Bert? How do we strip packages out of Trunk?
Manually once, when you prepare the Core image. Otherwise, if we put it in the trunk update stream, the packages would be unloaded from everyone's working images when updating.
Given that we're early in 4.5's release cycle, if we do this kind of thing - stripping unnecessary things out of trunk - we ought to do it now rather than later. And announce loudly what we're doing, why, and how people can load in their favourite not-Core packages.
NO.
If Squeak developers work with images that contain just "their favourite non-Core packages" then none of the non-Core packages get maintained.
Clearly we disagree quite a bit on this, but I don't think anyone should actually _work_ on a Core image. _Noone_ should be able to work effectively in a minimal image because such a thing wouldn't even have a Browser!
We don't disagree on this at all. People should be working in a full image.
OK, cool. Finding out points of agreement helps in finding out where we disagree :)
It is important to work on making packages loadable and unloadable, but you do not need to break the trunk image in order to accomplish this.
Who's breaking trunk? I want to remove packages that are already unloadable,
Yes, but not from *user's* images, from the *core* image that is only used by the CI to build a full image.
and have CI jobs that run every time Trunk updates that verifies that the packages can (a) reload and (b) run as well as we already know now (which is not very). To be precise: Universes (for example) might be smashed beyond recognition, and there is no way to know, because it has something like 23% method coverage.
Right.
Removing these packages from Trunk _will_ affect people updating from Trunk, I agree. How else do we make Trunk simpler, if not by removing stuff?
I think we may be talking past each other. There are two types of packages we want to "remove from trunk":
full = core packages + unloadable packages + deprecated packages
So there are unloadable packages which should still exist in a full image that everyone is using. And there are deprecated packages which should not exist in a full image.
Both core packages and unloadable packages are in Trunk. The only packages that we actually want to remove for good are the deprecated ones. The mechanics of unloading a deprecated package is by committing an empty version (which will remove all the classes and methods from user's images when updating), and eventually removing the now empty package from the update map (and bumping the squeak-version package to keep update numbers continuous).
Maybe that's what you were asking about?
(It's also simple enough to reinstall your favourite packages: Installer squeakmap update; install: 'WhateverItIs (someVersion)'.)
Sure, that's how you would re-install a deprecated package.
Also, please recognize that not everybody wants to do all their work in a clean image built by a robot. I for example do most of my work in a Squeak3.11alpha image that has been continuously updated from the update stream for quite a while. I also periodically check things against a fresh 4.4 image, but my usual pattern is to use one image over a long period of time with the trunk update stream, and most of my updates to trunk come from that old image.
OK, but then we can't have a minimal core without an entire reboot. I mean, seriously, making a minimal core by unloading packages is an order of magnitude harder when you cannot ever actually unload the packages!
So, given the resistance to my slow, gradual unwinding of the tangle, what's your counterproposal?
frank
I still fail to see the problem. For purposes of the build server, you once make a Core image by unloading all possible packages. This becomes the new "master image" from which all subsequent releases are built. Then you use that as the base to run tests, build full images, etc.
Does that sound reasonable?
Ah! OK, right. So, to see if I understand correctly: * We _don't_ issue an empty version for the packages we're interested in unloading. (Just for argument's sake, I'll talk about Universes.) * If at some point in the future we decide to deprecate Universes, _then_ we commit the empty version, and update the map. That decision may be deferred for an arbitrary period after unloading it from Core. * We take a clean Trunk image from CI, unload all unloadable packages, and store that as our new master 4.5 image. (This will likely be a manual step, and the clean image committed to the squeak-ci script repository.) * The stripped master remains stripped, and the existing SqueakTrunk job publishes updated copies of the core image as per normal. * We adjust the ReleaseSqueakTrunk CI job to, via ReleaseBuilderForSqueak4dot5, loads the unloadable packages from SqueakTrunk's output to create a "full fat" version.
Dave, sound good?
frank
- Bert -
On Mon, Feb 25, 2013 at 04:57:54PM +0000, Frank Shearar wrote:
On 25 February 2013 16:37, Bert Freudenberg bert@freudenbergs.de wrote:
On 2013-02-25, at 17:07, Frank Shearar frank.shearar@gmail.com wrote:
On 25 February 2013 15:35, David T. Lewis lewis@mail.msen.com wrote:
On Mon, Feb 25, 2013 at 03:01:18PM +0000, Frank Shearar wrote:
On 25 February 2013 14:49, Bert Freudenberg bert@freudenbergs.de wrote:
On 2013-02-25, at 15:33, Frank Shearar frank.shearar@gmail.com wrote:
> On 14 February 2013 16:50, Frank Shearar frank.shearar@gmail.com wrote: >> No, this takes the Trunk image as is and runs the job. No stripping >> has occurred. Noone's actually said "oh, that's a sane idea someone >> should do that" yet (except possibly for you, by virtue of not saying >> "are you crazy?" :) ), so I haven't done it yet. >> >> So the installation is essentially a no-op. >> >> With green lights on these builds, we can now verify that removing >> these packages won't break anything. At least, to the limit of the >> tests, which isn't much at all. So if we _do_ actually strip these >> guys out of Trunk, the builds should stay green. If they don't, we'll >> need to roll back the change and investigate. >> >> But I don't actually know how to strip them out of Trunk. I _suspect_ >> it'll be via a Monticello config map of some description. > > Anyone? Bert? How do we strip packages out of Trunk?
Manually once, when you prepare the Core image. Otherwise, if we put it in the trunk update stream, the packages would be unloaded from everyone's working images when updating.
Given that we're early in 4.5's release cycle, if we do this kind of thing - stripping unnecessary things out of trunk - we ought to do it now rather than later. And announce loudly what we're doing, why, and how people can load in their favourite not-Core packages.
NO.
If Squeak developers work with images that contain just "their favourite non-Core packages" then none of the non-Core packages get maintained.
Clearly we disagree quite a bit on this, but I don't think anyone should actually _work_ on a Core image. _Noone_ should be able to work effectively in a minimal image because such a thing wouldn't even have a Browser!
We don't disagree on this at all. People should be working in a full image.
OK, cool. Finding out points of agreement helps in finding out where we disagree :)
It is important to work on making packages loadable and unloadable, but you do not need to break the trunk image in order to accomplish this.
Who's breaking trunk? I want to remove packages that are already unloadable,
Yes, but not from *user's* images, from the *core* image that is only used by the CI to build a full image.
and have CI jobs that run every time Trunk updates that verifies that the packages can (a) reload and (b) run as well as we already know now (which is not very). To be precise: Universes (for example) might be smashed beyond recognition, and there is no way to know, because it has something like 23% method coverage.
Right.
Removing these packages from Trunk _will_ affect people updating from Trunk, I agree. How else do we make Trunk simpler, if not by removing stuff?
I think we may be talking past each other. There are two types of packages we want to "remove from trunk":
full = core packages + unloadable packages + deprecated packages
So there are unloadable packages which should still exist in a full image that everyone is using. And there are deprecated packages which should not exist in a full image.
Both core packages and unloadable packages are in Trunk. The only packages that we actually want to remove for good are the deprecated ones. The mechanics of unloading a deprecated package is by committing an empty version (which will remove all the classes and methods from user's images when updating), and eventually removing the now empty package from the update map (and bumping the squeak-version package to keep update numbers continuous).
Maybe that's what you were asking about?
(It's also simple enough to reinstall your favourite packages: Installer squeakmap update; install: 'WhateverItIs (someVersion)'.)
Sure, that's how you would re-install a deprecated package.
Also, please recognize that not everybody wants to do all their work in a clean image built by a robot. I for example do most of my work in a Squeak3.11alpha image that has been continuously updated from the update stream for quite a while. I also periodically check things against a fresh 4.4 image, but my usual pattern is to use one image over a long period of time with the trunk update stream, and most of my updates to trunk come from that old image.
OK, but then we can't have a minimal core without an entire reboot. I mean, seriously, making a minimal core by unloading packages is an order of magnitude harder when you cannot ever actually unload the packages!
So, given the resistance to my slow, gradual unwinding of the tangle, what's your counterproposal?
frank
I still fail to see the problem. For purposes of the build server, you once make a Core image by unloading all possible packages. This becomes the new "master image" from which all subsequent releases are built. Then you use that as the base to run tests, build full images, etc.
Does that sound reasonable?
Ah! OK, right. So, to see if I understand correctly:
- We _don't_ issue an empty version for the packages we're interested
in unloading. (Just for argument's sake, I'll talk about Universes.)
- If at some point in the future we decide to deprecate Universes,
_then_ we commit the empty version, and update the map. That decision may be deferred for an arbitrary period after unloading it from Core.
- We take a clean Trunk image from CI, unload all unloadable packages,
and store that as our new master 4.5 image. (This will likely be a manual step, and the clean image committed to the squeak-ci script repository.)
- The stripped master remains stripped, and the existing SqueakTrunk
job publishes updated copies of the core image as per normal.
- We adjust the ReleaseSqueakTrunk CI job to, via
ReleaseBuilderForSqueak4dot5, loads the unloadable packages from SqueakTrunk's output to create a "full fat" version.
Dave, sound good?
Yes :)
Dave
frank
- Bert -
On 25 February 2013 18:51, David T. Lewis lewis@mail.msen.com wrote:
On Mon, Feb 25, 2013 at 04:57:54PM +0000, Frank Shearar wrote:
On 25 February 2013 16:37, Bert Freudenberg bert@freudenbergs.de wrote:
On 2013-02-25, at 17:07, Frank Shearar frank.shearar@gmail.com wrote:
On 25 February 2013 15:35, David T. Lewis lewis@mail.msen.com wrote:
On Mon, Feb 25, 2013 at 03:01:18PM +0000, Frank Shearar wrote:
On 25 February 2013 14:49, Bert Freudenberg bert@freudenbergs.de wrote: > > On 2013-02-25, at 15:33, Frank Shearar frank.shearar@gmail.com wrote: > >> On 14 February 2013 16:50, Frank Shearar frank.shearar@gmail.com wrote: >>> No, this takes the Trunk image as is and runs the job. No stripping >>> has occurred. Noone's actually said "oh, that's a sane idea someone >>> should do that" yet (except possibly for you, by virtue of not saying >>> "are you crazy?" :) ), so I haven't done it yet. >>> >>> So the installation is essentially a no-op. >>> >>> With green lights on these builds, we can now verify that removing >>> these packages won't break anything. At least, to the limit of the >>> tests, which isn't much at all. So if we _do_ actually strip these >>> guys out of Trunk, the builds should stay green. If they don't, we'll >>> need to roll back the change and investigate. >>> >>> But I don't actually know how to strip them out of Trunk. I _suspect_ >>> it'll be via a Monticello config map of some description. >> >> Anyone? Bert? How do we strip packages out of Trunk? > > Manually once, when you prepare the Core image. Otherwise, if we put it in the trunk update stream, the packages would be unloaded from everyone's working images when updating.
Given that we're early in 4.5's release cycle, if we do this kind of thing - stripping unnecessary things out of trunk - we ought to do it now rather than later. And announce loudly what we're doing, why, and how people can load in their favourite not-Core packages.
NO.
If Squeak developers work with images that contain just "their favourite non-Core packages" then none of the non-Core packages get maintained.
Clearly we disagree quite a bit on this, but I don't think anyone should actually _work_ on a Core image. _Noone_ should be able to work effectively in a minimal image because such a thing wouldn't even have a Browser!
We don't disagree on this at all. People should be working in a full image.
OK, cool. Finding out points of agreement helps in finding out where we disagree :)
It is important to work on making packages loadable and unloadable, but you do not need to break the trunk image in order to accomplish this.
Who's breaking trunk? I want to remove packages that are already unloadable,
Yes, but not from *user's* images, from the *core* image that is only used by the CI to build a full image.
and have CI jobs that run every time Trunk updates that verifies that the packages can (a) reload and (b) run as well as we already know now (which is not very). To be precise: Universes (for example) might be smashed beyond recognition, and there is no way to know, because it has something like 23% method coverage.
Right.
Removing these packages from Trunk _will_ affect people updating from Trunk, I agree. How else do we make Trunk simpler, if not by removing stuff?
I think we may be talking past each other. There are two types of packages we want to "remove from trunk":
full = core packages + unloadable packages + deprecated packages
So there are unloadable packages which should still exist in a full image that everyone is using. And there are deprecated packages which should not exist in a full image.
Both core packages and unloadable packages are in Trunk. The only packages that we actually want to remove for good are the deprecated ones. The mechanics of unloading a deprecated package is by committing an empty version (which will remove all the classes and methods from user's images when updating), and eventually removing the now empty package from the update map (and bumping the squeak-version package to keep update numbers continuous).
Maybe that's what you were asking about?
(It's also simple enough to reinstall your favourite packages: Installer squeakmap update; install: 'WhateverItIs (someVersion)'.)
Sure, that's how you would re-install a deprecated package.
Also, please recognize that not everybody wants to do all their work in a clean image built by a robot. I for example do most of my work in a Squeak3.11alpha image that has been continuously updated from the update stream for quite a while. I also periodically check things against a fresh 4.4 image, but my usual pattern is to use one image over a long period of time with the trunk update stream, and most of my updates to trunk come from that old image.
OK, but then we can't have a minimal core without an entire reboot. I mean, seriously, making a minimal core by unloading packages is an order of magnitude harder when you cannot ever actually unload the packages!
So, given the resistance to my slow, gradual unwinding of the tangle, what's your counterproposal?
frank
I still fail to see the problem. For purposes of the build server, you once make a Core image by unloading all possible packages. This becomes the new "master image" from which all subsequent releases are built. Then you use that as the base to run tests, build full images, etc.
Does that sound reasonable?
Ah! OK, right. So, to see if I understand correctly:
- We _don't_ issue an empty version for the packages we're interested
in unloading. (Just for argument's sake, I'll talk about Universes.)
- If at some point in the future we decide to deprecate Universes,
_then_ we commit the empty version, and update the map. That decision may be deferred for an arbitrary period after unloading it from Core.
- We take a clean Trunk image from CI, unload all unloadable packages,
and store that as our new master 4.5 image. (This will likely be a manual step, and the clean image committed to the squeak-ci script repository.)
- The stripped master remains stripped, and the existing SqueakTrunk
job publishes updated copies of the core image as per normal.
- We adjust the ReleaseSqueakTrunk CI job to, via
ReleaseBuilderForSqueak4dot5, loads the unloadable packages from SqueakTrunk's output to create a "full fat" version.
Dave, sound good?
Yes :)
OK, so I thought I'd start the ball rolling with inspiration from SmalltalkImage >> #unloadAllKnownPackages -
#('Nebraska' 'Universes' 'XML-Parser') do: [:pkgName| (MCPackage named: pkgName) unload. MCMcmUpdater disableUpdatesOfPackage: pkgName].
I've stored this as Squeak4.5.image in the squeak-ci repository. This is the new base that we update from trunk. ReleaseSqueakTrunk then loads these packages back into the image.
That's the theory. In practice, the above is insufficient because it leaves obsolete classes lying around. So I added
MCWorkingCopy flushObsoletePackageInfos. MCFileBasedRepository flushAllCaches. MCDefinition clearInstances. Behavior flushObsoleteSubclasses. ChangeSet current clear. ChangeSet current name: 'Unnamed1'. Smalltalk flushClassNameCache.
This too is insufficient: The SqueakCI-Benchmarking package happens to depend on XML-Parser, so I reload the latest version of that package... and things break because there are AnObsoleteXMLWriter instances.
What have I missed?
frank
Dave
frank
- Bert -
On Sat, Mar 30, 2013 at 4:52 AM, Frank Shearar frank.shearar@gmail.comwrote:
On 25 February 2013 18:51, David T. Lewis lewis@mail.msen.com wrote:
On Mon, Feb 25, 2013 at 04:57:54PM +0000, Frank Shearar wrote:
On 25 February 2013 16:37, Bert Freudenberg bert@freudenbergs.de
wrote:
On 2013-02-25, at 17:07, Frank Shearar frank.shearar@gmail.com
wrote:
On 25 February 2013 15:35, David T. Lewis lewis@mail.msen.com
wrote:
On Mon, Feb 25, 2013 at 03:01:18PM +0000, Frank Shearar wrote: > On 25 February 2013 14:49, Bert Freudenberg bert@freudenbergs.de
wrote:
>> >> On 2013-02-25, at 15:33, Frank Shearar frank.shearar@gmail.com
wrote:
>> >>> On 14 February 2013 16:50, Frank Shearar <
frank.shearar@gmail.com> wrote:
>>>> No, this takes the Trunk image as is and runs the job. No
stripping
>>>> has occurred. Noone's actually said "oh, that's a sane idea
someone
>>>> should do that" yet (except possibly for you, by virtue of not
saying
>>>> "are you crazy?" :) ), so I haven't done it yet. >>>> >>>> So the installation is essentially a no-op. >>>> >>>> With green lights on these builds, we can now verify that
removing
>>>> these packages won't break anything. At least, to the limit of
the
>>>> tests, which isn't much at all. So if we _do_ actually strip
these
>>>> guys out of Trunk, the builds should stay green. If they don't,
we'll
>>>> need to roll back the change and investigate. >>>> >>>> But I don't actually know how to strip them out of Trunk. I
_suspect_
>>>> it'll be via a Monticello config map of some description. >>> >>> Anyone? Bert? How do we strip packages out of Trunk? >> >> Manually once, when you prepare the Core image. Otherwise, if we
put it in the trunk update stream, the packages would be unloaded from everyone's working images when updating.
> > Given that we're early in 4.5's release cycle, if we do this kind
of
> thing - stripping unnecessary things out of trunk - we ought to do
it
> now rather than later. And announce loudly what we're doing, why,
and
> how people can load in their favourite not-Core packages.
NO.
If Squeak developers work with images that contain just "their
favourite
non-Core packages" then none of the non-Core packages get
maintained.
Clearly we disagree quite a bit on this, but I don't think anyone should actually _work_ on a Core image. _Noone_ should be able to
work
effectively in a minimal image because such a thing wouldn't even
have
a Browser!
We don't disagree on this at all. People should be working in a full
image.
OK, cool. Finding out points of agreement helps in finding out where we disagree :)
It is important to work on making packages loadable and unloadable,
but
you do not need to break the trunk image in order to accomplish
this.
Who's breaking trunk? I want to remove packages that are already unloadable,
Yes, but not from *user's* images, from the *core* image that is only
used by the CI to build a full image.
and have CI jobs that run every time Trunk updates that verifies that the packages can (a) reload and (b) run as well as we already know now (which is not very). To be precise: Universes (for example) might be smashed beyond recognition, and there is no way to know, because it has something like 23% method coverage.
Right.
Removing these packages from Trunk _will_ affect people updating from Trunk, I agree. How else do we make Trunk simpler, if not by removing stuff?
I think we may be talking past each other. There are two types of
packages we want to "remove from trunk":
full = core packages + unloadable packages + deprecated packages
So there are unloadable packages which should still exist in a full
image that everyone is using. And there are deprecated packages which should not exist in a full image.
Both core packages and unloadable packages are in Trunk. The only
packages that we actually want to remove for good are the deprecated ones. The mechanics of unloading a deprecated package is by committing an empty version (which will remove all the classes and methods from user's images when updating), and eventually removing the now empty package from the update map (and bumping the squeak-version package to keep update numbers continuous).
Maybe that's what you were asking about?
(It's also simple enough to reinstall your favourite packages: Installer squeakmap update; install: 'WhateverItIs (someVersion)'.)
Sure, that's how you would re-install a deprecated package.
Also, please recognize that not everybody wants to do all their
work in
a clean image built by a robot. I for example do most of my work in
a
Squeak3.11alpha image that has been continuously updated from the
update
stream for quite a while. I also periodically check things against a fresh 4.4 image, but my usual pattern is to use one image over a
long
period of time with the trunk update stream, and most of my updates
to
trunk come from that old image.
OK, but then we can't have a minimal core without an entire reboot. I mean, seriously, making a minimal core by unloading packages is an order of magnitude harder when you cannot ever actually unload the packages!
So, given the resistance to my slow, gradual unwinding of the tangle, what's your counterproposal?
frank
I still fail to see the problem. For purposes of the build server,
you once make a Core image by unloading all possible packages. This becomes the new "master image" from which all subsequent releases are built. Then you use that as the base to run tests, build full images, etc.
Does that sound reasonable?
Ah! OK, right. So, to see if I understand correctly:
- We _don't_ issue an empty version for the packages we're interested
in unloading. (Just for argument's sake, I'll talk about Universes.)
- If at some point in the future we decide to deprecate Universes,
_then_ we commit the empty version, and update the map. That decision may be deferred for an arbitrary period after unloading it from Core.
- We take a clean Trunk image from CI, unload all unloadable packages,
and store that as our new master 4.5 image. (This will likely be a manual step, and the clean image committed to the squeak-ci script repository.)
- The stripped master remains stripped, and the existing SqueakTrunk
job publishes updated copies of the core image as per normal.
- We adjust the ReleaseSqueakTrunk CI job to, via
ReleaseBuilderForSqueak4dot5, loads the unloadable packages from SqueakTrunk's output to create a "full fat" version.
Dave, sound good?
Yes :)
OK, so I thought I'd start the ball rolling with inspiration from SmalltalkImage >> #unloadAllKnownPackages -
#('Nebraska' 'Universes' 'XML-Parser') do: [:pkgName| (MCPackage named: pkgName) unload. MCMcmUpdater disableUpdatesOfPackage: pkgName].
I've stored this as Squeak4.5.image in the squeak-ci repository. This is the new base that we update from trunk. ReleaseSqueakTrunk then loads these packages back into the image.
That's the theory. In practice, the above is insufficient because it leaves obsolete classes lying around. So I added
MCWorkingCopy flushObsoletePackageInfos. MCFileBasedRepository flushAllCaches. MCDefinition clearInstances. Behavior flushObsoleteSubclasses. ChangeSet current clear. ChangeSet current name: 'Unnamed1'. Smalltalk flushClassNameCache.
This too is insufficient: The SqueakCI-Benchmarking package happens to depend on XML-Parser, so I reload the latest version of that package... and things break because there are AnObsoleteXMLWriter instances.
What have I missed?
This is when the tedium starts. Use PointerFinder to track down the reference path to the obsolete XMLWriter, then determine (e.g. find or invent) the initialization(s) that eliminate that path (either by deleting the thing because it shouldn't be referred to, or evaluating something that replaces it).
frank
Dave
frank
- Bert -
[Re-send because of message size] Dear all,
Am 10.02.2013 um 00:51 schrieb Frank Shearar frank.shearar@gmail.com:
In the interests of revisiting Pavel Krivanek's work, and a long term goal of this community, I thought I'd use the Dependency Browser and dig out interpackage dependencies.
By scraping the DependencyBrowser's contents together with a bit of UI scripting I've constructed a dotfile of Trunk (attached). Turning this into a PNG results in an 11MB image! [1] Nodes near the top are nodes that aren't used by many things.
So I played around with the data a bit.
I ran the dot file through tred (part of graphviz), to eliminate dependencies that are already satisfied through transitive relations. (deps-simplified) It turns out we have many cycles, and tred complaints about: cycle involves edge Files -> System
I then used cluster (also part of graphviz) to identify clusters in the dependency graph (found three around Kernel, Collections, and Tests) (deps-simplified-reclustered) and manually added a "morphic" and a "HelpSystem" cluster (deps-simplified-reclustered-manucluster)
Note the big circle from System over ST80 to Morphic and from there via Ballon/Collections back to System, or via Monticello/Kernel/Collection back to System. Apart from that, note a direct circl (MonticelloMocks<->Tests) but also a small indirect circles (System->Environments->Compiler->System).
If we aim for modularity, we should invest time to investigate those circles.
Best -Tobias [Find the formerly attached zip at http://netshed.de/trunk-deps.zip ]
On 12 February 2013 13:54, Tobias Pape Das.Linux@gmx.de wrote:
[Re-send because of message size] Dear all,
Am 10.02.2013 um 00:51 schrieb Frank Shearar frank.shearar@gmail.com:
In the interests of revisiting Pavel Krivanek's work, and a long term goal of this community, I thought I'd use the Dependency Browser and dig out interpackage dependencies.
By scraping the DependencyBrowser's contents together with a bit of UI scripting I've constructed a dotfile of Trunk (attached). Turning this into a PNG results in an 11MB image! [1] Nodes near the top are nodes that aren't used by many things.
So I played around with the data a bit.
Wow, thanks Tobias! Great stuff!
I ran the dot file through tred (part of graphviz), to eliminate dependencies that are already satisfied through transitive relations. (deps-simplified) It turns out we have many cycles, and tred complaints about: cycle involves edge Files -> System
I then used cluster (also part of graphviz) to identify clusters in the dependency graph (found three around Kernel, Collections, and Tests) (deps-simplified-reclustered) and manually added a "morphic" and a "HelpSystem" cluster (deps-simplified-reclustered-manucluster)
Note the big circle from System over ST80 to Morphic and from there via Ballon/Collections back to System, or via Monticello/Kernel/Collection back to System.
That's one circle we'd want to lose. I think we can break that just by moving MailComposition >> #addAttachment to Tools (thus breaking Network's usage of Tools).
(There might be a bunch of things to do, but that's definitely a dependency we want to break.)
Apart from that, note a direct circl (MonticelloMocks<->Tests) but also a small indirect circles (System->Environments->Compiler->System).
If we aim for modularity, we should invest time to investigate those circles.
Some of those circles we may never resolve. Craig Latta might have some insight here, because he had to deal with these issues in the creation of his micro-images.
Craig?
frank
Best -Tobias [Find the formerly attached zip at http://netshed.de/trunk-deps.zip ]
Hi Frank--
Some of those circles we may never resolve. Craig Latta might have some insight here, because he had to deal with these issues in the creation of his micro-images.
Craig?
With the current technique[1], I don't have to deal with those issues at all to make minimal object memories. I wrote a variant of the garbage collector that collects methods not run since a certain point in time. I reset a mark on all methods, run unit tests for the stuff I want to keep, and everything else vanishes. It takes a few seconds.
Before I started doing that, I was ripping things out manually with a remote browser, so I didn't have to think about most of the circularities (e.g., breaking the target's GUI was fine because I wasn't using it).
Generally, I advocate creating modules by imprinting[2] unit tests onto a minimal memory. We can divide the work of rounding out the unit tests using the current class categories, and prioritize by what we miss most when using a minimal system. I expect we have lots of opinions about that, so there should be plenty of volunteers. :)
I volunteer to coordinate this. I'll start by making an example of a simple "echo" network server[3]. I'll provide a package for Squeak 4.4 that installs imprinting support, and another that installs the echo server code and an imprinting unit test. You imprint the unit test's code onto a minimal memory, then get data echoed back to you from a newly-created echo server running there. We can move onto more interesting things, like graphics (or WebDAV, or...).
I advocate a new all-live-objects module system as well[4], but the organizational info gleaned from this work would lead to useful artifacts with any of the other ones (e.g., Monticello).
As for the DependencyBrowser data, I think it would be better to express the connections directly between classes, and ignore the class categories for now. (Actually, I'd like to drop the class categories permanently, and use tags instead[5].) Class references are the basis of the dependency graph, the categories are a distraction. Let's identify the methods causing reference circularities between classes.
thanks,
-C
[1] http://tinyurl.com/bz3eyzq (thiscontext.wordpress.com) [2] http://tinyurl.com/ajttt52 (thiscontext.wordpress.com) [3] https://tools.ietf.org/html/rfc862 [4] http://tinyurl.com/aoudtnq (thiscontext.wordpress.com) [5] http://netjam.org/spoon/naiad (sorry, no pretty blog version yet)
-- Craig Latta www.netjam.org/resume +31 6 2757 7177 (SMS ok) + 1 415 287 3547 (no SMS)
Hello Craig
Thank you for your mail which gives a lot of useful detailed explanations.
On 2/12/13, Craig Latta craig@netjam.org wrote:
Hi Frank--
Some of those circles we may never resolve. Craig Latta might have some insight here, because he had to deal with these issues in the creation of his micro-images.
Craig?
With the current technique[1], I don't have to deal with thoseissues at all to make minimal object memories. I wrote a variant of the garbage collector that collects methods not run since a certain point in time. I reset a mark on all methods, run unit tests for the stuff I want to keep, and everything else vanishes. It takes a few seconds.
Is thei garbage collector available as a loadable package?
Before I started doing that, I was ripping things out manually witha remote browser, so I didn't have to think about most of the circularities (e.g., breaking the target's GUI was fine because I wasn't using it).
Generally, I advocate creating modules by imprinting[2] unit testsonto a minimal memory. We can divide the work of rounding out the unit tests using the current class categories, and prioritize by what we miss most when using a minimal system. I expect we have lots of opinions about that, so there should be plenty of volunteers. :)
OK, fine.
I volunteer to coordinate this. I'll start by making an example ofa simple "echo" network server[3]. I'll provide a package for Squeak 4.4 that installs imprinting support, and another that installs the echo server code and an imprinting unit test.
Good
You imprint the unit test's
code onto a minimal memory,
With 'minimal memory' I assume you mean a minimal image while it is in memory?
then get data echoed back to you from a newly-created echo server running there. We can move onto more interesting things, like graphics (or WebDAV, or...).
With graphics you mean an image which executes graphics commands?
Yes, what other candidates do you propose as a test?
I advocate a new all-live-objects module system as well[4], but theorganizational info gleaned from this work would lead to useful artifacts with any of the other ones (e.g., Monticello).
As for the DependencyBrowser data, I think it would be better toexpress the connections directly between classes, and ignore the class categories for now. (Actually, I'd like to drop the class categories permanently, and use tags instead[5].) Class references are the basis of the dependency graph, the categories are a distraction. Let's identify the methods causing reference circularities between classes.
Makes sense to me
Regards --Hannes
[1] http://tinyurl.com/bz3eyzq (thiscontext.wordpress.com) [2] http://tinyurl.com/ajttt52 (thiscontext.wordpress.com) [3] https://tools.ietf.org/html/rfc862 [4] http://tinyurl.com/aoudtnq (thiscontext.wordpress.com) [5] http://netjam.org/spoon/naiad (sorry, no pretty blog version yet)
-- Craig Latta www.netjam.org/resume +31 6 2757 7177 (SMS ok)
- 1 415 287 3547 (no SMS)
On Tue, Feb 12, 2013 at 5:54 AM, Tobias Pape Das.Linux@gmx.de wrote:
Note the big circle from System over ST80 to Morphic and from there via Ballon/Collections back to System, or via Monticello/Kernel/Collection back to System. Apart from that, note a direct circl (MonticelloMocks<->Tests) but also a small indirect circles (System->Environments->Compiler->System).
Yes! We should especially look at System->GetText
One thing that seems odd is that many packages depend on Tests, but very few depend on SUnit. You would think that all those TestCase subclasses would create dependencies on SUnit, not Tests.
Sorting out all the test packages might be an easy place to start.
Colin
On 12 February 2013 20:24, Colin Putney colin@wiresong.com wrote:
On Tue, Feb 12, 2013 at 5:54 AM, Tobias Pape Das.Linux@gmx.de wrote:
Note the big circle from System over ST80 to Morphic and from there via Ballon/Collections back to System, or via Monticello/Kernel/Collection back to System. Apart from that, note a direct circl (MonticelloMocks<->Tests) but also a small indirect circles (System->Environments->Compiler->System).
Yes! We should especially look at System->GetText
One thing that seems odd is that many packages depend on Tests, but very few depend on SUnit. You would think that all those TestCase subclasses would create dependencies on SUnit, not Tests.
The arrows might be pointing in an unintuitive direction then: a TestCase subclass of course implies a dependency on SUnit, and the arrows might actually say "is used by" rather than "uses" (or the other way around; I've only just woken up).
frank
Sorting out all the test packages might be an easy place to start.
Colin
Hey Colin
Am 12.02.2013 um 21:24 schrieb Colin Putney colin@wiresong.com:
On Tue, Feb 12, 2013 at 5:54 AM, Tobias Pape Das.Linux@gmx.de wrote:
Note the big circle from System over ST80 to Morphic and from there via Ballon/Collections back to System, or via Monticello/Kernel/Collection back to System. Apart from that, note a direct circl (MonticelloMocks<->Tests) but also a small indirect circles (System->Environments->Compiler->System).
Yes! We should especially look at System->GetText
One thing that seems odd is that many packages depend on Tests, but very few depend on SUnit. You would think that all those TestCase subclasses would create dependencies on SUnit, not Tests.
The point here is the Transitivity of dependencies and the *BIG* loop via Tests->Collections->System->ST80->Morphic->Monitcello->ToolBuilder-Kernel->SUnit(->Kernel->Collections)
In the original deps, most of the Packages depending on Tests also directly depend on SUnit. BUT: ToolBuilder-Kernel also directly depends on SUnit but _not_ on Tests, hence SUnit “moved” in the part of the circle with Toolbuilder and Kernel.
So Colin, you made me play around again. I removed the direct dependencies of ToolBuilder-Kernel Compression ST-80 EToys on SUnit and got a much cleaner picture now. (deps-simplified-reclustered-colored.pdf) We got exactly 6 Clusters now and they seem more sensible. I did not manually add clusters now.
Hope there's insight with this ;)
Best -Tobias
[Find the new pictures at http://netshed.de/trunk-deps2.zip ]
On 2013-02-12, at 11:54 PM, Tobias Pape Das.Linux@gmx.de wrote:
In the original deps, most of the Packages depending on Tests also directly depend on SUnit.
Oh right, I forgot about that detail. So that graph really shows the dependencies that would *have* to be addressed to make packages unloadable.
Thanks for doing this analysis.
Colin
On 2013-02-12, at 21:24, Colin Putney colin@wiresong.com wrote:
We should especially look at System->GetText
IMHO #translated and co. should be part of the System package, because senders are necessarily sprinkled all over the place. There should be a translator registry and if no translator were registered it would return the string untranslated. GetText would hook into that.
- Bert -
squeak-dev@lists.squeakfoundation.org
-
Bert Freudenberg -
Casey Ransberger -
Colin Putney -
Craig Latta -
David T. Lewis -
Edgar J. De Cleene -
Eliot Miranda -
Frank Shearar -
H. Hirzel -
Pavel Krivanek -
Stéphane Rollandin -
tim Rowledge -
Tobias Pape -
Yanni Chiu