[squeak-dev] Responses to IRC Questions arising

Keith Hodges keith_hodges at yahoo.co.uk
Sun May 11 02:23:53 UTC 2008


I am writing this note to answer some questions/issues that were 
expressed in todays irc logs about issues pertaining to installer and 
LPF etc.

=========
Item 1
=========
kencausey: yeah, that's one solution to an explosion of custom images, 
custom scripts instead
[20:35] matthewf: is lots of custom images bad?
[20:36] matthewf: I think not
[20:36] kencausey: It's bad for me trying to help people who come here 
asking questions.
...
kencausey: matthewf: yes of course, but it's a pain in the *** to have 
to try to replicate that to answer a single question.
[20:45]

My response:

I have started a "bzr" repository with 160Gb of space that can/will be 
used to manage images what have been built with these custom scripts. 
The automatic building tool "Bob" has yet to be written, but will be 
based upon Sake.

So when a user is reading a particular "Squeak By Example" edition or 
(insert favourite tutorial here) this may be associated with a specific 
"check out" from the repository. Those attempting to answer beginners 
questions in any script-built-image will be able to download the same 
image that the beginner is using via simple bzr checkout.
=========
Item 2
=========

[10:59] edgardec: So, release team now was myself, jerome, ken and matthew

Do I get credit for causing all of this problem in the first place?

=========
Item 3
=========

snake_mountain: I am uisng Pidgin for IRC.  How's about IRCe?  Will not 
install.
snake_mountain: Ah, oh.  "Error:  Unable to find function address>".
snake_mountain: ExternalWebBrowserUnizMozilla(Object)>>error.
....
edgardec: I bet you use Universes for having IRC installed ?
[13:30] snake_mountain: It was recommended last night right here on irc
edgardec: Maybe who build the package have some more in mind, but the 
fact is you DON'T need any external
[13:32]

My response:

So here we have a package in universes which needs fixing. I think the 
3.10 universe has been frozen, perhaps the maintainer nominated in the 
universes browser could take a look at it. However this is where 
Sake/Packages comes into its own. Sake/Packages has the same universes 
package definitions, but encoded as methods on a Class. Because it is a 
class you can override these methods to your hearts content in the 
subclass,  in order to provide something with does work for you. If it 
works you can then post the mcz back onto squeaksource for others to 
take advantage of the new improved definition.

Anyone can edit these packages on squeaksource so the whole enterprise 
is to be organic and socially policed, I have proposed 
packages at lists@squeakfoundation.org as a forum for this, however the 
current owner is not responding to emails (Marcus D?). To whom it may 
concern, please could I volunteer to adopt that list?

So... I have now done this, fixing the IRCe package definition in 
Packages-Squeak310

Installer sake addPackage: 'IRCe'; install.  "should now work"
Installer universe addPackage: 'IRCe'; install.  "remains broken 
awaiting the maintainers attention"

=========
Item 4
=========
wBryce: What's the list of fixes planned for 3.10.1?
[14:54]

The current list according to the LPF collection is here

http://installer.pbwiki.com/MinorFixes-Squeak3:10

and here

http://installer.pbwiki.com/MinorFixesUnstable-Squeak3:10

The later being the current candidate list includes many very minor 
items that could be safely included. (the images I personally use 
include all of these)

==========
Item 5
==========
wBryce: edgardec, It may be worthwhile doing a quick 3.10.1 followed by 
a 3.10.2 with a longer UAT/gamma period.
[15:03] edgardec: 3.10.2 ?
....
[15:03] edgardec: I wish do 3.11 instead

My response:

At present I propose that 3.11 be assembled as

LPF + Latest (MinorFixes, MajorFixes, PackageUpgrades) + Clean (remove 
more packages)

i.e. generated by something like

squeak squeak3.10-7159gamma http:://installer.pbwiki.com/f/LPF.st 
Installer do="Latest;Clean" SmalltalkImage save=squeak3.11a.image +quit

Edgar, you have considerable experience of removing packages form the 
image, and it would be nice if you might help work on the "Clean" script 
so that it may be applied all versions of squeak 3.7 through to 3.10 
etc, in order to help everyone to tighten the belts of their production 
images. How about it?

========
Item 6
========
wBryce: edgardec, I'd suggest figuring out a more automated process of 
applying changes for 3.11. Even if it means you've got to be more fussy 
about how you accept changes.
[15:10] wBryce: Also, what's the scope of the 3.11 release? Any big 
changes, or another maintenence release?

My response: The brainstorm page that matthew and I are using for 
throwing ideas around is at:  http://installer.pbwiki.com/311 you are 
welcome to comment and contribute there if you wish.

[15:10] edgardec: wBryce: I agree, but how we automate mor ?
 
My response: Since I have been working for more than 18 months on 
"automating this more", and we currently have a completely transparent 
automated process with Installer + LPF + installer wiki + mantis etc, 
what more automation could you possibly be asking for? I dont think it 
is possible for squeak to fix itself, there is nothing more that can 
possibly be automated so I am amazed by the question. Either that or I 
am completely missing the point somewhere?

wBryce: Keith's got code to load Mantis issues. I wonder if that could 
be used to automatically update an image.
[15:33]

My response: indeed.

wBryce: You'd want to make sure that the image building code was easy to 
unload before a final release.

My Response: This is why Sake/Packages and Sake/Tasks are implemented as 
simple methods on a class.
All you have to do to remove image building code is to remove the class.

Part of the Sake/Packages design is to support the idea that the 
Sake/Packages package definitions be loaded temporarily for the purpose 
of building an image and disposed of prior to deployment. It can be 
loaded again on demand if needed.

=======
Item 7
=======

kencausey: One thing I don't care for about Keith's solution is that it 
is easily hackable, so doing anything automatic with it, for general 
release, is not a good idea.
....
kencausey: Well, it reads from web pages that are world writable, as 
simple as that

My Response:

A) installer.pbwiki.com is currently password protected, and can have 
any level of security you desire to make it non "hackable"
B) Use another wiki with more watertight security, Installer is not 
hardwired to installer.pbwiki.com
Bb) Use local files if you want to, we can mange these in any SCM, I 
dont think this would have much advantage over the wiki.
C) Installer mantis scripts can protect themselves from people hacking 
fixes with new code by specifying a date. If the date doesn't match a 
warning is given.
D) In theory you can download a zip archive of the whole site and use it 
locally
E) If instead of Installer install: 'MinorFixes', you run Installer 
view: 'MinorFixes' you get a worksace with the entire script generated 
from the wiki. This can be executed as a doit or saved for use as a FileIn.
F) installer.pbwiki.com is NOT intended to be the final release 
generating mechanism. It is a knowledge collection tool for collecting 
knowledge and fixes from the community and experimenting. There are two 
plans for generating the final release.
Plan 1: Using DS, DS will record the progress of all the scripts, the DS 
can then be used to generate cs for update streams.
Plan 2: Sake/MCTasks run after the scripts saving the whole image as mcz 
packages. The final golden master release image is generated from a 
fresh image by loading the mcz's (with atomic loading of course).

=======
Item 8
=======
kencausey: Well, number one, Mantis issues have all sort of stuff 
attached to them
[15:38] kencausey: Sometimes examples of what not to do
[15:38] kencausey: So some human is going to have to designate what to test
 
My Response: This is what Installer mantis does... it picks out the 
script placed on the bug report page, the script specifies the actual 
fix, and tests for that fix. Mantis users are able to write and adjust 
the contents of the script as much as they wish.

kencausey: I could maybe even add something to Mantis to make it easier

My Response: Its fine, its not perfect but it has been working most 
effectively for over a year. (If it aint broke...)
The current system whereby user-notes are used for the Installer mantis 
script is a little tricky since one person cant edit a script posted by 
someone else. Currently this is handled by appending a new script. Would 
it be desirable/possible to have a mutually editable note for this purpose?

=======
Item 9
=======

wBryce: I was just idlely speculating if there was any way to automate 
as much of the drudge work out of the release team's job.
[15:42] kencausey: Personally I think that the drudge work is useful as 
it requires human interaction and opportunities for thought, examination 
of the issue, the code, etc.
[15:42] kencausey: As it is bad code repeatedly slips through.
[15:42] kencausey: How is that going to improve with further automation 
without more eyeballs?

My Response: I was once a passenger in a car whose driver would insist 
on driving at 90 mph everywhere in the black mountains of Wales. His 
excuse being that the faster he went in the wrong direction the sooner 
he would realise that he was lost.

Removing the drugery is the whole point of using LPF and Installer image 
building tools. With installer.wiki we have a framework which is A) 
Public B) Provides space for Documentation of design intent and design 
decisions C) Open to all interested parties D) Anyone can build the 
latest image and test it locally for their platform E) Anyone can 
dry-run their own variants and experiments as they wish.

Using code snippets that are publically visible on 
http://installer.pbwiki.com lots of people can contribute to the release 
in a small way or a large way as appropriate for them. i.e. Lots more 
eyeballs. Take Nicolas Cellier, and look at all the tiny little 
contributions he has made to FixesUnstable. Its about harnessing the 
creativity of the community by creating opportunities to be inclusive, 
rather than defaulting to an exclusive team.

The update stream is exclusive, being in the grip of a privileged few, 
so thats a non starter IMHO.
Previous release teams though desiring to be inclusive have not 
succeeded as they might have wished.

Here we are planning for inclusiveness, and we are including users of 
past images as well as present ones. If someone wants to take 3.11 work 
and migrate it to 3.7 the mechanism is there, now that we have a "Level 
Playing Field" to start with.

======
Item 10
======
kencausey: The problem with relying on tests as it requires reasonable 
test coverage
[15:43] kencausey: Which we are not even close to
[15:43] kencausey: And I'm afraid the closer we get the more annoying 
testing is going to become

My response: 14 months ago I wrote code for automated testing, this 
applied the "pareto principle" whereby 80% of the result is achievable 
with 20% of the effort. I did this by recording the time taken by each 
test case, and running the fastest tests first. This way 90%-95% of the 
tests run in less than 5mins. I cant remember the exact figures, but it 
was a significant win. This also highlights the slowest tests for 
improvement, or less frequent running.

======
Item 11
======
wBryce: In my case, it would be worthwhile to have a build server that 
built a Exupery dev image then ran all tests. So I found out when 
something broke at the time, rather than when I next release.

My Response: you can do this now*

Write an Installer build script to build your image  exupery.st

#> squeak non-LPF-start.image http://installer.pbwiki.com/f/LPF.st 
Installer installUrl: 'file:/exupery.st' SmalltalkImage 
save=exupery.image TestReporter runAllSuites  SmalltalkImage +quit

*some of the Launcher code for running TestReporter from the command 
line has been lost and needs to be retrieved from earlier versions of 
Launcher


=============
sorry its been a long one


cheers

Keith











































More information about the Squeak-dev mailing list