Hi keith

This is several weeks that I want to seriously look at Sake.
Will you attend ESUG? Participate to the camp?
Because this would be occasion to do something?

Marcus will visit us from 10 to 23 of August. We could organize a  
Squeak day at Lille?

Stef


On Jun 27, 2008, at 5:21 AM, Keith Hodges wrote:

> Matthew wrote:
>> Sake/Packages is an attempt to address both of the above
>> concerns. It as yet has no UI. Unlike in SM or Universes, the
>> entire distribution is versioned as a whole, rather than the
>>
> This is a feature. If you want an old version of the s/p package  
> map, then everything is as it was on that past date. (if certain  
> criteria are adhered to)
>> individual packages, so editing one package means making a new
>> distribution version. This makes it easy to specify exactly what
>> version of the database your image is up-to-date with, but does
>> not scale if there are many contributions.
>>
> Currently s/p uses mcz files to store its package definitions as a  
> body of code.
>
> I do not know of any squeak project that currently has scalability  
> issues with too many contributors to the code repo. s/p is a body of  
> code like any other.
>
> If a scalability problem was to occur which I doubt very much  
> indeed, there are numerous solutions.
>
> 1. split the packages up into smaller granularity, i.e.  
> WebDevelopment packages, Dev Tools packages
>
> 2. adopt an editorial policy informally - i.e. load scripts only  
> load the latest "approved" version.
> (seaside users do this frequently, loading Lukas or Phillippe's  
> versions in preference to unknown contributors.)
>
> 3. adopt an editorial policy formally - close the repo to a small  
> number of approved editors.
>
> 4. spilt the package up, so that some packages load further package  
> definitions as needed.
>
> 5. adopt a different server from the basic mcz server, such as Magma  
> with more intelligent actions/merging capbability.
>
> 6. write a front end tool for publishing versions with some form of  
> process management, aka, Gjallar / map.squeak.org / SqueakMap3
>
> 7. Split the package definitions master repo up into smaller repo's  
> arbitrarily as needed. e.g. Database packages, Seaside, Kernel,  
> etc.  Use sake/tasks to auto generate combinations for specific  
> squeak images based upon metadata. i.e. collect all definitions with  
> the flag #worksInCroquet set into PackagesCroquet10
>
> So yes it is scalable, because the actual nuts and bolts of the  
> implementation of sake/packages is fairly orthogonal to how it is  
> used. If the usage pattern turns out not to be scalable then we  
> change the usage pattern.
>
> i.e. Sake/Package definitons could be distributed via DeltaStreams  
> rather than Monticello.
>
> Randal wrote:
>> Notice about 50 uploads a *day*.  And everything *works* in an
>> automated way, except for the "CPAN gods" exceptions listed above
> As I said before if the current solution cannot cope with 50 uploads  
> a day, then it is not the fault of s/p, this is a server issue or a  
> usage patterns issue. Squeaksource is not the only mcz server out  
> there, and it is not the only solution for distributing/managing  
> code. mcz can store diffs, magma might make a more intelligent back  
> end and mc2 is out there somewhere too. This issue is orthogonal to  
> s/p.
>
> The s/p model is intended to be like that of wikipedia etc. relying  
> upon open editorial policies. They cope with scalability issues  
> somehow.
>
> In addition those 50 uploads a day is quite high bandwidth. But I  
> think that this would consist of people writing their packages, and  
> every time they bump a single version, they have to republish so  
> that their users can keep up.
>
> Not so with s/p.
>
> s/p allows publishers to publish 'NON-SPECIFIC' "the latest version  
> that I have in my repo whatever it is". So package writers do not  
> have to constantly worry about publishing their work for their  
> bleeding edge users. This is a wonderful feature, you just publish  
> ONCE even at the beginning of your project, you dont even need to  
> wait until it is ready for release!
>
> Users or publishers can write definitions that are more specific  
> such as "the latest version that is authored by 'the master coder'. "
>
> or even more specific such as "the latest version that is published  
> on SqueakMap."
>
> It's only when users find that a specific version is needed for a  
> specific image that the definition needs to be updated with a  
> "SPECIFIC" definition.  "Squeak3.10 definitely works with version  
> 522 of Seaside2.8."
>
> So...
>
> A developer can publish a package, generating 100 versions as he  
> goes, as it reaches some level of maturity the users add the  
> definition that says version 100 definitely works in Squeak1.1.
>
> Beta users who want the latest all the time, use
>
> Packages beta load: 'ThePackageName'.
>
> Normal users what the definitions that have been conformed to work  
> BY OTHER USERS. They use..
>
> Packages current load: 'ThePackageName'.
>
> So... I doubt very much that the bandwidth for s/p will be anything  
> like CPAN for the same amount of business.
>> Sake/Packages - single point of update, making publishing hard
> Single point of update, making management easy! And putting the  
> power in the USERS hands, not the publishers.
>
> The publisher will not have used the package in the specific images  
> that the users have tested it in, so actually the Users collectively  
> have more knowledge about the use of the package than the publisher.  
> This model allows the user to publish his knowledge in the  
> appropriate place. i.e A fix for Squeak 3.7 does not need to  
> contaminate everyone elses definitions.
>
> I would like to  know how CPAN publishes version 23 of the database  
> driver for use on this version of Linux, with this Database loaded  
> and this specialised file system. I beleive the database driver  
> publishes his work and it is the USERS which decide how it will be  
> used. S/P is primarily a means by which USERS share "What Works"  
> information. SqueakMap and SqueakSource is where publishers can  
> publish "What is Available".
>
> Randal wrote:
>> recent Squeak releases for installation, so we have a bootstrapping  
>> problem.
>>
> I dont think s/p has a bootstrapping problem, since if your image is  
> capable of loading code, s/p is only code and can be loaded any  
> which way.  I surmse that s/p will have negligable bootstrapping  
> problems when compared with the SqueakMap problems of the past.  
> Furthermore LevelPlayingField is present to help with any  
> bootstrapping problems we may have for any package in the future.
>
> Randal wrote:
>> And all of these systems are missing dependency and automatic upgrade
>> management.
>>
> s/p models dependencies to whatever depth you wish.
>
> s/p has a rudimentary upgrade capability AND package UNLOAD  
> capability...
>
> If I want to unload Seaside it will unload in a sensible order every  
> package that has been loaded which depended upon Seaside, prior to  
> unloading Seaside (well that is the theory).
>
> Damien Wrote:
>
>> I don't see the point of having completely separate tools/websites  
>> for
>> code management (SqueakSource/MC) and package management (all  
>> others).
>> I want to configure an image as a user, then decide that I want to
>> contribute on such package, and upgrade it to the devel snapshot with
>> the same tool.
>>
>> Package management and code version control are really the same  
>> thing,
>> just at different levels of granularity.
>>
> Amen Brother!
>
> If you fit the paradigm to the task it makes things much easier I  
> think.
>
> Let a coder use code to determine what and how his code is loaded  
> into his image, and it will be a lot simpler than....
>
> Using a web front end to a database of meta data, describing code  
> files that are to be replicated all over the world for users that  
> are still using 1980's modems. etc etc.
>
> 90% of software engineering is solving the right problem. The  
> problem is actually very simple,
>
> "collect the necessary data, what works where and with what  
> dependencies."
>
> s/p collects that data in an executable form (and all of the  
> metadata is optional anyway), I dont see how it can get much simpler  
> than that.
>
> Randal wrote:
>> * CPAN installation is easy
>> * CPAN publishing is easy
> s/p publishing is probably easier than CPAN, since the package  
> writer only has to publish once ever, and it is easy to browse every  
> use case of a particular package...
>
> load:  Installer install: 'PacakgesAllVersions'.
>
> To find all versions of squeak which can load versions of seaside,  
> do "implementers of #Seaside."
> To find all packages that are dependant upon Seaside..
>
> Packages current Seaside allDependants.
>
>> Lowering this barrier in Squeak is important
>>
>> Merik
>
> It may not be perfect but it is lowered to the point of actually  
> being usable.
>
> Ralph worte:
>> A "feature" of Squeak that might differentiate it from perl is that
>> there are many versions of Squeak.  Packages that work with one image
>> might not work with another.  So, if you are using 3.8 then you do  
>> not
>> necessarily want the latest version of some package if that package
>> only works in 3.9 or 3.10.   PU addresses this problem by having a
>> different Universe for each version of the image, though I think  
>> there
>> is currently a Universe only for 3.9 and 3.10.  Even if the code for
>> PU could run in earlier images, it isn't useful until someone  
>> builds a
>> Universe for it.
> Got it exactly. CPAN does not address the real problems that we  
> actually face. i.e "WHAT works WHERE." as seen by package users.   
> And this is exactly why it is that Users of a particular image that  
> need to manage the package definitions for that image. The package  
> writers are often behind the enlightened users who are putting their  
> package to new and varied uses in different contexts.
>
>> Maybe PU is the closest to CPAN we have.  So the question is, do we  
>> keep
>> mutating PU into what I'm looking for?
> No, because PU will be no use in an image that doesnt have any  
> Network code, doesnt have XML parser, doesnt have Morphic, so if you  
> do improve PU someone will have to develop another solution for the  
> more stripped down image. If future squeaks are based upon more  
> stripped down images then we end up with two solutions to the same  
> problem.
>> Or do we first need to agree that the
>> CPAN might be a pretty darn good model?
>>
> Its not a good model because it is solving a problem that we don't  
> currently have. We are not distributing tar files for users around  
> the world that are using modems to access the internet at 1200 baud,  
> for use in a language with no reflexive capabilities. i.e. unix bash/ 
> csh etc
>
> (btw: Our packages do not have to be restricted to code, they could  
> include objects, projects, or anything loadable or creatable by  
> code, this includes mantis fixes.)
>
> Our problem is not with publishing packages, our problem is enabling  
> users to load what THEY want into their images and to share their  
> feedback as to what works with other users in a way that can be used.
>
> Ralph wrote:
>> CPAN is probably a good model.  However, a model that works well for
>> one language does not always work well for another.  How would you
>> think a CPAN-like system would handle the fact that people are using
>> 3.6, 3.7, 3.8, 3.9 and 3.10 and that some packages work for all of
>> these, but some packages are specialized for a particular image?
>>
> Exactly, that's why CPAN is not a darn good model for this problem.  
> I can only envisage that it would be a darn big complicated  
> unmanageable pain. The proposal is made with blinkers on, since  
> Randal refused to even look at s/p because of his concerns over  
> "scalability", which as I have described above is a complete red- 
> herring-issue.
>
> Randal wrote:
>> We do need something that says package X works with squeak Y and  
>> package X
>> depends on package Z.  so it's two dimensions, not just one.
>>
> Yes we do... and which omnipotent package publisher knows the answer  
> to that question in every context? Answer none, problem is solvable  
> with a user feedback mechanism, not a package publishing mechanism.
>
> Merik wrote:
>> And what about patches?
>>
>> When an Seaside website needs a patch of an Integer method, for  
>> example.
> Yes indeed , some images need patching before a particular package  
> will load. Sake/Packages handles this, it can load patches from  
> mantis using Installer ensureFix: 1234.  OR It can specify a  
> dependency upon the Sake/Tasks load mantis fix task, which documents  
> the fixes loaded as it goes.
>
> so Randal why you wont look at Sake/Packages is beyond me.
>
> regards
>
> Keith
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>