Hello Lex and other interested readers,

first please excuse my late answer: but there is so much to do and the
things are very much in flux.
Currently I'm mostly thinking and writing: the intesting thing is, that
I'm writing docu about features of the planned system *before* any
coding! But this is very important in this area, since if some
dependency resolving system should have a chance to be accepted, people
have to understand, how to deal with it.

lex at cc.gatech.edu wrote:
> Hey Stephan, I enjoyed reading this.  By all means continue to explore
> in this direction and to refine your essay, so that people new to the
> topic can get going quickly!

Thanks for the motivation!

> 
> Here are a few quick thoughts on it.
> 

> First, if I'm not mistaken, your capabilities model seems to boil down,
> for each package, to this form:
> 
> 	(cap1 AND cap2 AND cap3)  IMPLIES  (cap4 AND cap5)
> 	(cap6 AND cap7)  IMPLIES  (cap8 AND cap9 AND cap10)
> 	
> Is this correct?  You'd ask the installer tool to find a way to get you
> cap10, and then it would trace backwards through all the packages to
> know about to see if it can satisfy this.

Yes.
But the Caps can be hidden, so that the user asking for installing a
package does not see them.
And the mechanism allows to express more complex dependencies.

> 	
> 
> Do you have any concrete examples where these transformers would be
> useful, Stephan?  It would help in thinking about them.

Good question ;-)

In general I want to have a mechanism which *allows* expressing complex
dependencies as a backend, without beeing forced to go into nasty
technical details in the frontend, at least for expressing simple
dependencies.

A concrete example is the possibility to express ORed dependencies via
logical Caps in extra Transformation rules: something like I need
package SocketStuffFastButRisky in this OR package
SocketStuffSlowButReliable in that version (assumed that both are OK for
me). But this is just the tip of the iceberg...

> 
> One aspect of baking in dependencies into version numbers is that people
> may well end up not wanting to use the dependency-accurate version
> numbers as their normal version numbers.  If am working on Chuck II Son
> of Chuck(y), then I just want version numbers that go 2.0, 2.1, 2.2,
> etc., but those will not capture the dependency information accurately. 
> This makes me wary of capturing *accurate* dependency information in the
> versions, though it may still be that the *intent* of the dependencies
> can still be encoded in them.

As Göran has already written, compatibility code and version number are
separated.

After some (to be honest many) thoughts and monster mail discussions
with Göran, current state of the art
(note: from my POV; I haven't gotten Görans feedback regarding it so far
(it is *very* new))
is visible in
  http://minnow.cc.gatech.edu/squeak/3792
, a Swiki page about so called 'compatibility codes' (CCs).
Note: this page is under construction, so many things already in my head
are missing, but what is in there should reflect my current thinking
accurately.

Though there is a 1-to-1 relation to some constant version number, in
opposite to the latter the CCs may be *changed* afterwards. So there is
a good reason to keep them separated (this insight is quite new to me,
before I've thought they could be used as version number). The
opportunity for these changes is necessary in a world not perfect...

> 
> 
> Along these lines, note that a conflict can be detected after the
> version number has already been decided.  What happens in that case?  If
> the version number only captures intent then there is no problem.  If it
> is supposed to be accurate, then that seems impossible.

There are some ways to go (writing more into this direction at the
'compatibility codes' Swiki page is in the queue):

1. a bug would be fixed resulting in a new package release without this
conflict, having a new CC: then the package could be installed;

2. the package wouldn't be changed, but its CC would be corrected (not
the *constant* version number): the dependency resolver would avoid it then;

3. a conflict would be expressed explicitely in the Transformation rules
(e.g. if a package maintainer is on holidays).

The last variant is the worst, since it requires extra rules for
describing the exception from the wished behavior.

Note: there may be more variants.

> 
> 
> 
> 
>>- The package name combined with the major number could
>>be seen as the - technically and socially - needed new name.

Now - after the separation of compatibility code and version number - I
would write:
"The package name combined with the minor <'minor' may change>
compatibility code number could be seen as the - technically and
socially - needed new name."

> 
> 
> You beat me to it.  For packaging purposes, it gives you a small
> simplification if Chuck1 and Chuck2 are simply different packages if
> they are in fact incompatible.  Additionally, putting them in separate
> packages means that you can arrange for *both* of them to be installed
> simultaneously....

If they don't conflict!
One important thing not to forget: that the API is compatible does *not*
mean, that the implementation is compatible, too!

> 
> 
> 
> 
>>('WebBrowserStandard_2.2', "minor release"
>>            'WebBrowserStandard_2.2.1',
>>            'WebBrowserStandard_2.2.1.1',
>>           'WebBrowserStandard_2.2.1.1.1')."
> 
> 
> Keep in mind, everyone, that accurate dependencies require extra work to
> happen somewhere.

I know ;-)

> I hope stuff like the above does not need to be
> maintained by humans!

I have good hope, that it is possible to hide it in most cases.

> And anyway, it makes me wonder in a lot of cases
> if it is not simpler to *fix* a conflict between two packages and upload
> a new version of the package, as opposed to trying to accurately report
> that Chuck 1.28 is incompatible with RB 2.11.

Yes, of course: variant 1. above.

> Note that we have to deal
> with bugs anyway, and that an incompatibility can often be considered
> simply a bug.

Correct.

> 
> 
> 
>>But here again: we *need* a versioning policy, which people are following!
> 
> 
> Yes, that sounds right to me.  Bugfixes happen, and we should have a
> system that can deal rationally with them.

> Versioning policies would
> let you easily (hopefully) express the claims about compatibility that
> people often have mentally.

I (and also Göran, of course) try to do my best to reach the 'easily'.

> 
> 
> . Since we are faced with a
> 
>>combinatorial explosion here, hints regarding
>>stable/unstable/bleedingEdge/etc. from the package maintainers would be
>>helpful though...
> 
> 

> I suggest we solve this through the idea of package universes.  You
> simply live in a bleeding edge universe or a stable one.  I don't see
> how you can combine a bleeding edge package and a stable package and get
> something meaningful.  The result is not stable, for sure.
> 
> There's no explosion in practice.  The simple "Stable" versus "Unstable"
> goes a long way.

Here I have a question following in another thread.

> 
> 
> 
> 
>>There also may be a logical package WebBrowser which just
>>depends on one of them. But who makes the choice? This is similar to the
>>question, who decides which stuff is worth enough to come into the
>>official distribution.
> 
> 
> Exactly!!  There is no one answer to this.  A package universe can be
> viewed as *one* answer to the question "what packages exist?"
> 
> 
> 
> 
> 

>>It would be nice to have a detection of class extension conflicts.
> 
> 
> This sounds like a job for the package loader and unloader, and if I'm
> not mistaken Monticello will do this already.

But here it is important, that the dependency resolver sees this
conflicht *before* installing! That is the problem.

> 
> 
> 
> 
> 

>>Don't try to be too perfect: KISS!
>>I have to admit that KISS seams to be quite complex in this domain...
> 
> 
> I am not convinced.  If we restrict attention to individual package
> universes, then simple unversioned dependencies seem to be sufficient. 
> It is worth trying to improve on them, but I don't see why we *have* to
> improve on them.  How much talking *about* our packages do we really
> want to be doing, as opposed to actually improving the packages
> themselves?  :)

It depends on what you want: if you want to have packages accessible in
different versions (for simplicity let's assume different stable
versions) and packages requiring different of these stable versions
(e.g. they need different APIs) accessible in one catalogue, then the
world is not so simple.

> 
> 
> 
> -Lex
> 
> 
> PS -- There is a non-trivial UVersion in the Universes prototype I
> posted, but it is way shy of the kind of stuff you are talking about.  I
> have made -- KISS remember  :) -- the assumption that incompatible
> packages have different names, and that newer versions of packages are
> prefered even if someone asked for an older version of the package.  Oh,
> and UVersion's are a total order; there are no branches.  But Universes
> is supposed to be a playground.  Feel free to put together an alternate
> universe browser that follows whatever dependency rules you like!!
> 
> 

Thanks for the pointer: but I don't have time to follow it...
But I *have* read the corresponding post!


Greetings
Stephan
-- 
Stephan Rudlof (sr at evolgo.de)
   "Genius doesn't work on an assembly line basis.
    You can't simply say, 'Today I will be brilliant.'"
    -- Kirk, "The Ultimate Computer", stardate 4731.3