Update stream ideas for 3.8 (was Re: Squeak 3.8 status)
Doug Way
dway at mailcan.com
Tue Aug 10 05:50:47 UTC 2004
There have been a number of suggestions lately on improving the speed
of the harvesting process/update stream.
After thinking about it for a bit, there are a few things we could
possibly try in 3.8alpha:
1. Start up the separate "internal" update stream as Michael suggested.
Except it wouldn't really be private/internal, it would just be very
bleeding-edge... it would be readable by anyone who wanted to file in a
special changeset to get access, or maybe just by switching a
preference setting. Periodically, these would be transferred to the
external/relatively stable stream, as Michael described below, maybe
every 2-4 weeks or so. (Maybe come up with new names for the
"internal" vs. "external" streams, maybe "unstable" vs. "stable"?
Although the regular stream won't really be all that stable... Maybe
"unstable" vs. "regular", I don't know.)
There would need to be some conventions followed for the unstable
update stream. For example, bug reports shouldn't be posted to
squeak-dev for something that's only in the unstable stream. Perhaps
there could be a separate mailing list with the very narrow topic of
fixing immediate problems in the unstable stream. And of course, Bruce
O'Neel would only upload images to the ftp site when the regular/stable
stream is updated.
The mechanism to do this is already in place, it just hasn't been used
in a while. I remember when the Guides started up we didn't want to
have a private update stream a la Squeak Central, but I think this sort
of semi-public/unstable stream could work.
2. (This one is my suggestion.) If we have this unstable update
stream, posting updates to this stream doesn't have to be as
coordinated as with the regular stream. So, we could allow harvesters
who [approve] an item in the BFAV to also immediately "broadcast" that
item as an update to the unstable stream. (The broadcasting mechanism
to do this is also already there from the SqC days, but needs some
fixing.)
If all harvesters have the ability to broadcast updates, we'd want to
make sure the same sort of automatic checks are happening before the
update is broadcasted, as when I publish updates. The broadcasting
code could run the ConflictChecker, could check to make sure method
timestamps aren't missing, check for linefeeds (I think it already does
that), etc. Plus it could be beefed up further, e.g. check to make
sure any new classes have a class comment, etc.
We should probably require that all SUnit tests are run and passed at
some point in this process, too. If not when updates are broadcast to
the unstable stream, then at least it should happen when they're moved
to the regular stream. (Something would need to be done with the
current known-to-fail tests, either fix or remove them, or categorize
them somehow.)
3. Let some harvesters/guides have direct "commit"/broadcast ability,
without going through the BFAV approval process, as some have
suggested. However, I think this should be limited to only certain
types of updates, probably just fixes and smallish
refactorings/cleanups. New enhancements or major redesigns would still
need to go through the approval process.
So... how do these ideas sound?
#2 and #3 would hopefully remove me as the bottleneck at the update
stream, which would be a bit of a relief at this point... ;)
There were also a couple of other things discussed:
- Enhancing BFAV to group posts by package and maybe by other
groupings. I think everyone agrees this is a good idea... I get sort
of annoyed seeing all of the posts in BFAV which are for the SM
package, or the VM, or Balloon3D... these posts can never proceed to
Approved/Update. I'm not sure exactly how this should be done, and I
probably won't have time to work on this, but I wanted to offer support
to the idea.
- Committing directly to a Monticello repository for the base image,
instead of using updates. This seems like more of a long-term goal,
not really something which can be achieved during 3.8. Actually, I'm
not sure there's any point in doing this until the base image is staked
out into packages (and stored on SM, with dependencies?) But perhaps
the goal was to stake out these areas relatively soon, even if many of
the packages are "co-dependent" in the dependency scheme. But in any
case, I think this is all less urgent as far as speeding up the
harvesting process goes. (Although I think we should still add
Monticello to Basic Squeak in 3.8 as discussed earlier... it is really
an add-on which should change existing behavior.)
Whew, that's it for now,
- Doug
On Thursday, August 5, 2004, at 03:44 PM, Michael Rueger wrote:
> Marcus Denker wrote:
>
>> I don't think that this would work... at least not as long as we use
>> an update-stream.
>> It's realy easy to destroy the system with an update. We had example
>> of faulty updates
>> that manage to slip into the stream and replacing the changesets on
>> the server is of
>> course possible, but it disrupts everything somehow...
>
> Back in the SqC days we had an internal and an external update stream.
> The internal one was pretty much unfiltered, living dangerous etc.
> EVeryone living on that stream was used to having backups of their
> image, ready to go back at any time. Then, when updates seemed ok for
> a while, they were transfered to the external stream. This way it was
> easy to retract updates, as they hadn't reached "everyone" yet, but
> only people who knew how to, and were actually willing to, roll back
> the changes.
>
> Re-establishing that separation might help speeding up the harvesting
> process as it would spread the actual testing of an update across all
> internal stream users.
> Maybe naming these streams differently could help with acceptance ;-)
>
> Michael
>
>
More information about the Squeak-dev
mailing list
|