location of package dependencies

goran.krampe at bluefish.se goran.krampe at bluefish.se
Sun Aug 3 21:25:07 UTC 2003


Hi all!

"Lex Spoon" <lex at cc.gatech.edu> wrote:
> Goran, I hate to complain about work that is perfectly fine and that is
> very valuable for Squeak.  But I will anyway.  :)  I want it to go as
> well as possible!

Sure, no problem.
 
> Basically, I am worried about one main issue.  It's the idea of "tested
> configurations".  I have some experience in Debian, and there it is HARD
> to assemble tested configurations.  It typically takes around a year
> (very roughly) to assemble a set of packages that is tested to work with
> each other.  Even then there are almost always updates as time goes by!
> 
> Unless there is something different about Squeak, it is entirely
> impractical for people to assemble tested configurations on a regular
> basis.  It *is* proven to be practical to simply depend-on the packages
> you need -- without mentioning specific versions -- and for everyone to
> use the newest version of everything during a development stream. 
> Usually everything works fine, and when there is a bug it gets fixed
> quickly because these are easy things to fix.
> 
> When the time is ripe, you certainly can build tested configurations by
> freezing all updates and focussing on fixing up the bugs with
> interactions between the packages.  However, this is a major affair and
> I don't see how it could be done on a regular basis when dozens of
> developers are all updating their packages all the time.

I don't think we have the same "picture" of how this is meant to work.

What I propose is to simply have a model that describes the information
at hand more exactly. I am not proposing that people should do more
"work" in testing configurations etc. Let me give you an example
(because it is easier to describe that way):

I am developing the SMServer. It uses the packages Comanche, HttpView
and VersionNumber. Instead of just saying "SMServer depends on Comanche,
HttpView and VersionNumber" it is more precise to actually say "SMServer
1.1 depends AFAIK on Comanche 5.1, HttpView 1.0 and VersionNumber 1.0",
because those are in fact the package releases I use currently and if I
were to release SMServer tomorrow I would publish that configuration as
the only one I know of at this point that I can say to some extent
works. Because I haven't tried any other releases yet.

Notice that there is no extra work involved here - I wouldn't do more
nor less testing given another dependency model.

It is also very important to remember that the model I propose also
includes one more attribute on package releases - the backwards
compatibility level.

Given this you get the exact same "slack" as you have in your model. So
instead of me guessing that SMServer works with Comanche 6.0 by listing
a dependency like "Comanche version 5.1 or newer" I find it much more
appropriate to say that I know it works with 5.1 and if the
compatibility level (you need to look at all releases from 5.1 up to
6.0) says that the external API of Comanche hasn't changed from 5.1 up
to 6.0 then the user can try using Comanche 6.0 if he/she so wants.

IMHO the proposed model is more precise and gives us more flexibility in
our tools.

> Following from this argument, I wonder who other than a package's
> maintainer is going to muck with dependencies?  First, it is hard enough

It doesn't have to be so hard! If I install SMServer and try it with
Comanche 6.0, given that my tools told me I would have a good chance of
success (see reasoning above) and it works then I can gladly post my
configuration with a short comment saying "it works for me". And if
there were SUnit tests to run I could even throw in some credibility
behind my configuration. :-)

Note that I truly intend these configurations to be very easy to
publish. After SM1.1 I want to extend the SM model so that we can have
"writes" to SM through tools inside Squeak (instead of as of today where
you need to "write" through the web UI).

This could then turn this into a very easy operation - select SMServer
in package loader, pull up menu, select "publish current configuration",
get an already filled out dialog (SM already knows what releases I have
installed and given already published configurations can give me a good
guess), make adjustments, enter a comment and push "publish".

Note that these configurations will be SMObjects (the general
superclass) and thus can have categories just like packages can today.
This means we can classify configurations - like "Hey it works for me"
all the way to "Has been running for years without a glitch".

These categories together with the identity of the publisher (is it the
package maintainer, someone I trust or a known clueless newbie? ;-) will
make it easier for users (and tools) to make educated choices.

> to find a single maintainer for most packages.  :)  Second, it is
> impractical to take a package and then mix it with other packages than
> the ones it was developed for.  Almost all Debian packages have at least

That would not be the typical scenario - the typical scenario is to use
a package with a newer release of a package it depends on like I
described above.

> one tweak to the *content* of the package, and not just the dependency
> information of a package.  Thus, if you want to take a package and put
> it into a different distribution, you may as well make a new package.

I don't really see how this corresponds to Squeak.

> On a smaller notes:
> 
> >> - Using operators and boolean algebra makes the dependencies hard to
> read, hard to get right and hard to analyze.
> 
> This is not true for those who have tried it.  Normally you don't use a
> version dependency at all.  Even when you do, you usually say simply
> "version >= 3.2".  It's not hard in practice.

Again - I think such a dependency is strange. Nobody can make such
promises.
The person has verified it works with 3.2 and is just making a guess
that it should work with newer versions. Why not simply let the model
reflect the information at hand instead? The only known fact is that it
works with 3.2. Leave it with that.

What I truly meant above is that it gets much harder to analyze. My
model will (Something I believe after making scenarios and playing
around with lots of arrows and circles :-) hopefully fit pretty nicely
with an engine that can - given a list of needed packages or package
releases - compute what I need to install. I still stand by the fact
that having a model with boolean algebra and operators makes it much
more complex.

> Also, by the way, increments in the major version are usually treated as
> creating new packages.  Thus you depend on things like "foolib3" instead
> of just "foolib".  This trick gets rid of the most common needs for
> version dependencies.

Right. I still can't see why you want a model that simply describes
less. :-)
 
> >> - It there is a conflict Debian can only barf at you and you essentially
> >> have no good options. Sure, it doesn't happen often, but Debian is more
> >> coarse grained and Debian can pretty easily have coexisting libraries
> >> etc.
> 
> This isn't a lovely situation, but how can you really solve it?  How is
> it different from any other bug?
> 
> There are two major answers to this kind of problem:
> 
> 	1. Stick with tested releases that have been verified to have no
> conflicts or major bugs.
> 
> 	2. Live with it, and try to get problems fixed as quickly as possible.
> 
> This is exactly what Debian does (sorry to harp, but Debian has the same
> issues at this level!).  They have "stable" and "unstable" releases
> corresponding exactly to these situations.

I know. What I was trying to say is that with my model, including the
compatibility level for the package releases, a rather simple
"dependency engine" can explain the problem and *give the user choices*.

Like "SMServer 1.2 has been verified by the maintainer to work with
Comanche 5.1. Others have verified it to work with 5.2 and 5.3. You
currently have 6.0 installed which has a compatibility level 4
(=external API unchanged) which means it could still work, will you take
the chance? Another choice is to downgrade Comanche to 5.3 because the
other dependent Seaside works with both 5.3 and 6.0 according to its
maintainer."

Sure, the tools would perhaps present these facts in less text but
anyway, you get the picture.

And also - since the engine can give me educated choices I will be more
tempted to actually try it with Comanche 6.0 - and thus I can later
publish a new configuration - it works with Comanche 6.0 too! And around
it goes.

> Finally, on a bigger note, what do you suggest for packages that want,
> say, any mail-sending utility and don't care which one?  You say you
> don't like dependencies such as "web-server" or "smtp-client".  What is
> to be done instead?  The smtp-client dependency is already happening in
> Squeak.

Well, what I mean is that dependencies should IMHO be precise and clear.
If someone says in a package "depends on 'webserver'" it can have
multiple meanings given different dependency models of course, let us
explore:

For example - if 'webserver' is a category on SM which only contains
packages that implement some kind of common webserver interface then it
would mean it should work with any of those packages. Ok, but which
version of the interface? Any? Again - how can anyone make such a
promise?

Wouldn't it be better then to have a "proper" package called "Webserver
Interface" or something that encodes the API? It would just be like any
other package but it would have pluggable implementations. And it would
have releases which you then can depend on. The "Webserver Interface"
would then have configurations - one for each package that works as an
implementation.

So... I am not sure what you mean with the smtp-client dependency? Could
you elaborate on how you see that as an example? 

> Lex

Anyway, I am superpleased that people are interested in this - just one
final important note:

SM1.1 will not include the dependency model. It will "only" include the
capabilities it needs for such a model to be built on top of it (package
releases, embedded resources, developer accounts, release compatibility
level).

In fact - I would very much like someone else to help me here! 

Given SM1.1 we will have the ability to build the dependency engine as a
separate tool. This is because I want SM to be targeted and tight - I
don't want it to "handle evertyhing" and when I realized that it should
only deal with maintaining a distributed model of package
metainformation then I also realized that a dependency model + engine is
just a client of on top of that model.

SqueakMap is just the map.

regards, Göran



More information about the Squeak-dev mailing list