Folks -
I just had a very good (off-list) discussion where it was brought to my attention that one might look at the proposal from the point of view that it replaces Installer, Bob & Co. in situ and that there is no role for these tools in the proposed process. Nothing could be further from the truth.
What I am imagining goes along these lines: Consider Bob running daily/weekly builds and then checks if any new fixes have appeared on Mantis. If there is anything new, it loads the fix, runs the tests inside the image, runs the tests attached to the bug. If this works out Bob puts his "stamp of approval" on the fix or commits it directly to the repository. In effect, Bob becomes our main harvester doing all the boring integration stuff that we can reasonably automate.
This is actually quite similar to what Bob does today - except that Bob isn't producing an image but rather commits to the trunk. In any case, from my perspective there is a lot of room for Bob and Installer - be that in providing up-to-date images from weekly builds, running automated test suites, and so on.
I have no intention to get rid of Bob and Installer. I hope that with Keith's help we can make maximum use of both since these are valuable tools in a successful software development project.
Cheers, - Andreas
Andreas Raab wrote:
Folks -
I just had a very good (off-list) discussion where it was brought to my attention that one might look at the proposal from the point of view that it replaces Installer, Bob & Co. in situ and that there is no role for these tools in the proposed process. Nothing could be further from the truth.
What I am imagining goes along these lines: Consider Bob running daily/weekly builds and then checks if any new fixes have appeared on Mantis. If there is anything new, it loads the fix, runs the tests inside the image, runs the tests attached to the bug. If this works out Bob puts his "stamp of approval" on the fix or commits it directly to the repository. In effect, Bob becomes our main harvester doing all the boring integration stuff that we can reasonably automate.
This is actually quite similar to what Bob does today - except that Bob isn't producing an image but rather commits to the trunk. In any case, from my perspective there is a lot of room for Bob and Installer
- be that in providing up-to-date images from weekly builds, running
automated test suites, and so on.
I have no intention to get rid of Bob and Installer. I hope that with Keith's help we can make maximum use of both since these are valuable tools in a successful software development project.
Cheers,
- Andreas
Bob generates and maintains a repository with each build target that it makes.
Given that there may be 5 of us running our own bob servers why bother with a centralised (read slow) repository as part of the process. If we run a build once a day, the repository with that build target tracks the changes to that build target day by day. The tricky part is putting meaningful comments into the repository package comments.
ftp://squeak:viewpoints@squeak.safeprayer.com/bob/output/BuildBase/Squeak3.10.2-build/_source/
A centralised repository can be used for packages that have been pulled out of the kernel, to be managed externally. We have that repository already. squeaksource/311
The whole point of the way the process works is that there would be several projects going on at the same time in parallel. i.e. 3.10 + closures is a project going on at the same time as 3.10 + mantis fixes. Each project starts with a fixed starting point. There isn't one repository that represents the final output, until that final output, the integration of the chosen projects e.g. 3.10 + closures + fixes is generated.
It is a point of policy that the build of 3.11 has to be made up of bits that could be applied to other images. The task that builds 3.11 collects the knowledge of what went in to 3.11, and in doing so makes it available for other forks to see what we did and follow suit. They can do this by subclassing our 3.11 generating task and adapting it there. When they do this, we get to see what knowledge they captured in their subclass.
You suggest that bob's output should be to commit things to trunk... but bob is supposed to build images. The reason for this is that he not only builds the 3.11 image, but he also simultaneously builds all of the derived images (e.g. the developer image, the full image etc), which you need to test against.
It looks to me that you are still looking at the production of the final image as an incremental process. Its not supposed to be an incremental process. It is supposed to be an atomic process. However it may take several goes (one per day for a month) to get that atomic process right. Technically we should be able to make new releases once a month with ease, but practically the demand for that number of releases per year would not be there.
Keith
Before:
The procedure for 3.10 was - upload your fix to mantis and it will be integrated by the build server whose name used to be Edgar, but is now Bob. i.e. there were 2 inputs.
1. the design of the image (which for 3.10 included the removal of a couple of packages) 2. the fixes from mantis integrated manually.
After Bob:
1. the design of the image, a task which loads the latest packages, reorganises the whole image into a DIFFERENT package structure. (how are you going to support that with mcms?) 2. the fixes from mantis integrated automatically.
Your scheme:
1. People can contribute to trunk... 2. there is no image design because bob is not building images anymore. 3. People can contribute to mantis and those fixes get integrated into trunk as you go along. So in your scheme the mantis fixes are now being applied to a moving target. Which means that other people who want to see what good a fix is for to see if it might be useful to them, have no idea what state 3.11 trunk was in when the fix was added to it.
So basically the whole commit to trunk idea is not tenable.. for the following reasons.
1. There is no design of the final image published and coded in advance. It is now a free for all for the committers that have been let loose on trunk
2. The packages are going to be reorganised in the process of the image design.
a) "System" needs to be broken up b) The Implementation and Tests need to be logically and tidily organsied for once.
Keith
On Sat, Jul 04, 2009 at 11:42:31PM +0100, Keith Hodges wrote:
Before:
The procedure for 3.10 was - upload your fix to mantis and it will be integrated by the build server whose name used to be Edgar, but is now Bob. i.e. there were 2 inputs.
- the design of the image (which for 3.10 included the removal of a
couple of packages) 2. the fixes from mantis integrated manually.
After Bob:
- the design of the image, a task which loads the latest packages,
reorganises the whole image into a DIFFERENT package structure. (how are you going to support that with mcms?) 2. the fixes from mantis integrated automatically.
Your scheme:
- People can contribute to trunk...
- there is no image design because bob is not building images anymore.
Bob does not "design images". neither do you
- People can contribute to mantis and those fixes get integrated into
trunk as you go along. So in your scheme the mantis fixes are now being applied to a moving target. Which means that other people who want to see what good a fix is for to see if it might be useful to them, have no idea what state 3.11 trunk was in when the fix was added to it.
You want every contribution to have to flow thru the bottleneck of mantis? So, instead of
1. fix a bug 2. merge in any changes that may have happened since I started 3. press save in Monticello
You want us to
1. fix a bug 2. file out a changeset 3. Grab the latest image from Bob 4. Reaply the changeset to that image, making sure it works 5. upload that changeset to mantis5.
We have a great tool already for keeping track of code that has changed. It's called Monticello. You use it all the time. Bob can load .mcm or .mcz packages into an image just as easily as it can load changesets from mantis.
Bob is not a taskmaster who controls the image, demeaning our jobs to just submitting changesets for his approval. We, the community, build the image, and bob relieves us of the drudgery of making sure the packages load order is reproducable, and from waiting 5 hours for the unit tests to finish.
So basically the whole commit to trunk idea is not tenable.. for the following reasons.
- There is no design of the final image published and coded in advance.
It is now a free for all for the committers that have been let loose on trunk
if there was a final image published and coded in advance, there would be no reason to be bothering about a release, since it had already happened. This makes no sense.
- The packages are going to be reorganised in the process of the image
design.
they are? first I've heard about it.
a) "System" needs to be broken up b) The Implementation and Tests need to be logically and tidily organsied for once.
Bob is not going to do that. We are.
Hmm. What keith is saying is something I wrote, with his help, 9 months ago: http://installer.pbworks.com/Squeak311Proposal
I have to decide if I still agree with that proposal we came up with. My arguments below seem to indicate that I find serious flaws in it now.
On Sat, Jul 04, 2009 at 07:15:57PM -0400, Matthew Fulmer wrote:
On Sat, Jul 04, 2009 at 11:42:31PM +0100, Keith Hodges wrote:
Before:
The procedure for 3.10 was - upload your fix to mantis and it will be integrated by the build server whose name used to be Edgar, but is now Bob. i.e. there were 2 inputs.
- the design of the image (which for 3.10 included the removal of a
couple of packages) 2. the fixes from mantis integrated manually.
After Bob:
- the design of the image, a task which loads the latest packages,
reorganises the whole image into a DIFFERENT package structure. (how are you going to support that with mcms?) 2. the fixes from mantis integrated automatically.
Your scheme:
- People can contribute to trunk...
- there is no image design because bob is not building images anymore.
Bob does not "design images". neither do you
- People can contribute to mantis and those fixes get integrated into
trunk as you go along. So in your scheme the mantis fixes are now being applied to a moving target. Which means that other people who want to see what good a fix is for to see if it might be useful to them, have no idea what state 3.11 trunk was in when the fix was added to it.
You want every contribution to have to flow thru the bottleneck of mantis? So, instead of
- fix a bug
- merge in any changes that may have happened since I started
- press save in Monticello
You want us to
- fix a bug
- file out a changeset
- Grab the latest image from Bob
- Reaply the changeset to that image, making sure it works
- upload that changeset to mantis5.
We have a great tool already for keeping track of code that has changed. It's called Monticello. You use it all the time. Bob can load .mcm or .mcz packages into an image just as easily as it can load changesets from mantis.
Bob is not a taskmaster who controls the image, demeaning our jobs to just submitting changesets for his approval. We, the community, build the image, and bob relieves us of the drudgery of making sure the packages load order is reproducable, and from waiting 5 hours for the unit tests to finish.
So basically the whole commit to trunk idea is not tenable.. for the following reasons.
- There is no design of the final image published and coded in advance.
It is now a free for all for the committers that have been let loose on trunk
if there was a final image published and coded in advance, there would be no reason to be bothering about a release, since it had already happened. This makes no sense.
- The packages are going to be reorganised in the process of the image
design.
they are? first I've heard about it.
a) "System" needs to be broken up b) The Implementation and Tests need to be logically and tidily organsied for once.
Bob is not going to do that. We are.
-- Matthew Fulmer -- http://mtfulmer.wordpress.com/
release@lists.squeakfoundation.org