3.6 Full release testing (was Re: [BUG]? Upgrade to full image script behavior)

Daniel Vainsencher danielv at netvision.net.il
Wed Aug 20 18:44:52 UTC 2003


I'm getting convinced that adding the Full packages using the update
stream is silly. I think Michael brought up something we were ignoring -
no amount of wiggling will change the fact that some thing have always
been better tested than others, and that testing depends on our
collective whim.

Richard's questions were very telling - "can I count on <something a
user might reasonably want to count on>", and the answer is
consistently, "no, we're not promising that". I think its time to face
the fact that the only way that newbie/endusers are going to get a
tested image is if people work with the alphas, in whatever
configuration suits them. I'm not saying people should do this - people
do what they want. But Richard (only as an articulate exemplar) cannot
reasonably expect the releases to be tested, and we shouldn't twist our
release procedures to create some illusion of testing. Only way to have
stabler releases is to work with alpha and work to fix anything that
breaks.

I know, everybody has a good reason not to update. We should just be
aware of the price.

Daniel

Doug Way <dway at riskmetrics.com> wrote:
> goran.krampe at bluefish.se wrote:
> 
> >Michael Rueger <michael at squeakland.org> wrote:
> >  
> >
> >>Richard A. O'Keefe wrote:
> >>
> >>    
> >>
> >>>- Can I be confident that updates to the Base image will have been
> >>>  *tested* with all the packages in the Full image, and that updates
> >>>  will check for the presence of packages they are incompatible with,
> >>>  so that I will be warned about and allowed to cancel an update that
> >>>  might break the Full image?
> >>>      
> >>>
> >>No, no, and not really doable in practice. Even MS with their gazillions 
> >>of testers fails to achieve this.
> >>    
> >>
> >
> >You should be confident that the updates at least "load" in a Full
> >image. I think Doug or whoever is in charge of the updatestream will
> >ensure that at least. And perhaps he may also run the available Unit
> >tests for the Base image that Marcus maintains. And perhaps even
> >existing unit tests for the packages in Full (are there any yet for the
> >famous 9?).
> >
> >But no manual testing will be performed - no time for that. Again, Doug
> >can tell you how much of the above he does today before pushing it out.
> >  
> >
> 
> Currently, I do make sure that any about-to-be-published updates 
> successfully load into a Basic image, of course.  Now that we are 
> talking about "supporting" Full images following the same update stream, 
> I will plan on also making sure the about-to-be-published updates 
> successfully load into a Full image too, at least.  (A Full image in 
> this case will be a Basic image with the "upgrade to full" script loaded 
> a few minutes prior.)
> 
> (The other thing I do test for right now is any conflicts between a 
> given update and more recent changes in the image and the other updates 
> in the batch.  The ConflictChecker does this, and it has caught a few 
> update conflicts over the last few months, which I haved fixed on the spot.)
> 
> I'm not currently running any Unit tests, although that's not a bad 
> idea.  Some of the famous 9 Full packages do have corresponding Unit 
> test packages, btw.  (Scamper, Celeste, Games, at least.)  I could give 
> this a try.  Although we've also talked about having a server out there 
> somewhere running Unit tests on a regular basis.
> 
> Right now I'm working on making my life easier by creating a little tool 
> which collects a list of potential updates in a window (added from the 
> BFAV), and then lets me number them as a group, load/check conflicts on 
> all of them at once, broadcast them all at once, and send out the 
> [update - xxxx] messages.  Right now I do these steps manually.  This 
> tool is partly done.
> 
> >>>- Will packages that are not part of the Base image also have some
> >>>  kind of update stream, or will they only be replaced in toto on
> >>>  SqueakMap?
> >>>      
> >>>
> >>That is very much up in the air right now, part of the reason for the 
> >>many messages that have been flying around.
> >>    
> >>
> >
> >Yes, this is in the air. I assume you mean the official Squeak packages
> >(currently the famous 9) and not *all* packages on SM. I also assume
> >that we will decide on a common strategy for these packages. But we
> >haven't done so yet. Andreas has made a hack available for setting up
> >updatestreams per packages that works with SM1. SM2 will include some
> >fields for this.
> >
> >But perhaps we will instead choose to use Monticello for these packages
> >and not "stream of a bunch of ChangeSets" at all.
> >
> 
> Yep. :-)  I'm not going to be spending any mental effort on this 
> particular issue anytime soon, there's enough other stuff to work on.  
> But perhaps a package owner may want to try setting up an update stream 
> for a package sometime soon. (probably after SM2 is out)  Update streams 
> may be overkill for some packages, but for others it might make sense.
> 
> >>>- Is there _anything_ I can do to keep a Full Squeak current, or do
> >>>  I just jump from official release to official release?
> >>>
> >>>To be perfectly honest, I have hitherto *preferred* "wait for the next
> >>>stable release" to "keep quite current", but I have been feeling a bit
> >>>guilty about that, because it's one of the main reasons why I haven't
> >>>done any bug fix testing.  The fact that a bug fix seems to work in my
> >>>image does NOT imply that it works in a fully patched image, so I've
> >>>felt that any bug testing I did would have been pointless.
> >>>      
> >>>
> >>So in practice not much is going to change for you then? ;-)
> >>Seriously, I've also preferred to not update while I'm in the middle of 
> >>some development and the squeakland plugin image is also getting new 
> >>releases twice a year at the most.
> >>As I said, the update stream is called alpha for a reason.
> >>    
> >>
> >
> >If you want to keep up with alpha/beta/gamma streams then just go ahead.
> >The stream is there for a reason. But it is currently not only used as a
> >general update mecanism for a stable release.
> >  
> >
> 
> You might not want to do large projects on top of a Full alpha image, 
> but it's not hard at all to just keep a Full alpha image (3.6) around 
> and update it periodically, in addition to a stable release image 
> (3.5).  If you see a bug in your stable image, you can check your 
> updated alpha image to see if the bug is also there, and then submit a 
> bug report/fix, and thus participate in the bug reporting/fixing process.
> 
> - Doug



More information about the Squeak-dev mailing list