Last night I attended (and spoke at) the Seattle Perl User's Group meeting. Also in attendance were Brian Ingerson and Michael Schwern of Portland.
Before the meeting, I had a chance to talk with Brian Ingerson (more below). He's working on an interesting and relevant project.
<sidelight> Though not the subject of the main talk, I soon found myself answering questions about Squeak (and doing some promotion <g>). There were several people in the crowd who were interested.
After the meeting, we went to a pub for a beer or two. There I demonstrated Squeak to several interested people (note to self: get a good demo together to show to programmers).
I think that with the right kind of introduction and modular system we could convert a bunch of Perl programmers. </sidelight>
Anyway, one of the interesting ideas that came from this was from Brian.
He's working on an alternative to CPAN (the popular Perl repository/mirror/registry system). I'll let him describe what he's doing (I just sent him an invitation to join this list), but in summary he's working on a repository/registry system with metadata. Currently he's making it handle Perl and Ruby; there's some interest in Python from Michael Schwern. Especially since we're also evolving the SqueakMap system, it might be useful to talk with him about joining SqueakMap with FreePAN in some way. This would provide a single set of mirrored repositories covering all our languages, with immediate exposure to the "scripting language" community.
And it sounds like some of Göran's excellent work could be used with FreePAN (or at least inform it).
Anyway, I thought I'd open this up to discussion, as it's a timely and relevant issue for both our communities.
Ideas?
"Ned" == Ned Konz ned@bike-nomad.com writes:
Ned> <sidelight> Ned> Though not the subject of the main talk, I soon found myself answering Ned> questions about Squeak (and doing some promotion <g>). There were Ned> several people in the crowd who were interested.
Ned> After the meeting, we went to a pub for a beer or two. There I Ned> demonstrated Squeak to several interested people (note to self: get a Ned> good demo together to show to programmers).
Ned> I think that with the right kind of introduction and modular system we Ned> could convert a bunch of Perl programmers. Ned> </sidelight>
Works for me. :)
I seem to have mentioned Squeak at least once at every Perl or Linux group meeting I've attended.
My argument is "you don't know OO until you've used a pure OO language", and when people ask me how to fulfill that, I suggest they find a good Smalltalk or Eiffel or Ruby implementation. "Smalltalk - that's long dead, right?" and then I point them at Squeak. Neat.
Hi Brian and all!
Ned Konz ned@bike-nomad.com wrote:
Last night I attended (and spoke at) the Seattle Perl User's Group meeting. Also in attendance were Brian Ingerson and Michael Schwern of Portland.
Cool!
Before the meeting, I had a chance to talk with Brian Ingerson (more below). He's working on an interesting and relevant project.
<sidelight> Though not the subject of the main talk, I soon found myself answering questions about Squeak (and doing some promotion <g>). There were several people in the crowd who were interested.
After the meeting, we went to a pub for a beer or two. There I demonstrated Squeak to several interested people (note to self: get a good demo together to show to programmers).
Smart move Ned, hit them while they are intoxicated. :-)
I think that with the right kind of introduction and modular system we could convert a bunch of Perl programmers.
</sidelight>
Anyway, one of the interesting ideas that came from this was from Brian.
He's working on an alternative to CPAN (the popular Perl repository/mirror/registry system). I'll let him describe what he's doing (I just sent him an invitation to join this list), but in summary he's working on a repository/registry system with metadata. Currently he's making it handle Perl and Ruby; there's some interest in Python from Michael Schwern. Especially since we're also evolving the SqueakMap system, it might be useful to talk with him about joining SqueakMap with FreePAN in some way. This would provide a single set of mirrored repositories covering all our languages, with immediate exposure to the "scripting language" community.
Sounds of course interesting.
And it sounds like some of Göran's excellent work could be used with FreePAN (or at least inform it).
Ok.
Anyway, I thought I'd open this up to discussion, as it's a timely and relevant issue for both our communities.
Ideas?
Well, to let you know where I am right now:
Currently I am writing version 1.1 of SM which involves a whole bunch of changes. The current 1.0x version is quite simplistic in many ways, but that was *by design* to get it up and running and to make it easily accepted.
Basically SqueakMap is a metadata-catalog with "smart" incremental update-mechanism based on "transactions". It also has a domainmodel which is identical on the client and the server - in fact, in SM1.1 the client and server will more or less be equal entities.
Then on top of the domain model we have a bunch of slick Morphic UIs and a web UI that is darn ugly, but that was also *by design*. Next version will be a bit prettier but on the other hand next version will not really need it anymore (currently some operations can only be done trough the web UI) so people will not care.
Currently SM has a master and all the clients are just synched slave mirrors. In SM1.1 the "slave" mirrors will be able to have additional local content visibile to them and their slave mirrors. It will also mean that changes in the map can be applied locally and then "published" up the tree of servers.
Important notes:
- SqueakMap is completely written in Squeak. Not a single line of anything else. - The protocol between servers and clients is dead simple on top of http. I like to keep things as small as necessary - no need for SOAP or XML. I just use very simple commands in http requests and use Squeak as "language" for transmitting structures. - The domain model (both client and server) maintains a logfile with all the transactions logged a "evals" (or do-its as we Smalltalkers call them). This means the domain (server or client) is easily reloaded/recovered from the log and the server uses the logfile when calculating the incremental update to send to a client.
Anyway, I have no idea if this was interesting to read for anyone. ;-)
regards, Göran
On Thursday 19 December 2002 05:43 am, goran.hultgren@bluefish.se wrote:
- The protocol between servers and clients is dead simple on top of
http. I like to keep things as small as necessary - no need for SOAP or XML. I just use very simple commands in http requests and use Squeak as "language" for transmitting structures.
That is, what's going over the net looks like this (example is for a category):
(self newCategoryWithId: 'aaac9f7d-6d46-471d-96aa-522dc546cbee') created: 3206703930 updated:3206703930 name: 'Class libraries' summary: 'Class libraries for Squeak to use for development' url: '' mandatory: false parentId: nil!
Note that this could easily be represented in YAML...
On 19/12/02 09:50 -0800, Ned Konz wrote:
On Thursday 19 December 2002 05:43 am, goran.hultgren@bluefish.se wrote:
- The protocol between servers and clients is dead simple on top of
http. I like to keep things as small as necessary - no need for SOAP or XML. I just use very simple commands in http requests and use Squeak as "language" for transmitting structures.
That is, what's going over the net looks like this (example is for a category):
(self newCategoryWithId: 'aaac9f7d-6d46-471d-96aa-522dc546cbee') created: 3206703930 updated:3206703930 name: 'Class libraries' summary: 'Class libraries for Squeak to use for development' url: '' mandatory: false parentId: nil!
Note that this could easily be represented in YAML...
And it might look like this:
--- !newCategoryWithId =: aaac9f7d-6d46-471d-96aa-522dc546cbee created: 3206703930 updated: 3206703930 name: Class libraries summary: Class libraries for Squeak to use for development url: '' mandatory: false parentId:
BTW, Here's an example of the YAML meta data for a random FreePAN module:
dist_id: rskkserv version: 2.94.12 language: ruby description: |- rskkserv is an alternate version of skkserv implemented by Ruby. Please use ruby-tcpwrap library for security.
Now, supports multi skkdic and epwing dictionaries(*1).
*1: Needs ruby-eb library for epwing dictionary.
categories: - Application/Util license: GPL owner_id: YamashitaJunji update: 2001-08-15 01:11:52.000000 Z wrapped_content: rskkserv-2.94.12/
This file is at
http://freepan.org/ruby/by-owner/YamashitaJunji/rskkserv/index.yml
Cheers, Brian
On 18/12/02 09:31 -0800, Ned Konz wrote:
Last night I attended (and spoke at) the Seattle Perl User's Group meeting. Also in attendance were Brian Ingerson and Michael Schwern of Portland.
Before the meeting, I had a chance to talk with Brian Ingerson (more below). He's working on an interesting and relevant project.
Anyway, one of the interesting ideas that came from this was from Brian.
He's working on an alternative to CPAN (the popular Perl repository/mirror/registry system). I'll let him describe what he's doing (I just sent him an invitation to join this list), but in summary he's working on a repository/registry system with metadata. Currently he's making it handle Perl and Ruby; there's some interest in Python from Michael Schwern. Especially since we're also evolving the SqueakMap system, it might be useful to talk with him about joining SqueakMap with FreePAN in some way. This would provide a single set of mirrored repositories covering all our languages, with immediate exposure to the "scripting language" community.
Hi all. Brian Ingerson here. Thanks for inviting me to join the SqFd mailing list, Ned. I'll give a quick intro of myself and FreePAN.
I have been heavily involved in the Perl community for the past 3 years. The major contributions that I have made are Inline.pm:
And YAML.pm
Inline allows you to bind Perl to over 15 other programming languages. YAML is a cross programming language serialization language that uses plain text in an extremely human friendly way. These projects are neat in their own right, but what I find even more interesting is that they bring different programming languages into a more cooperative environment. And that turns me on!
This summer, (now my roommate) Michael Schwern and I were speaking at the O'Reilly Open Source Conference. We got to talking about the possibility of putting Python modules on Perl's CPAN. If you don't what CPAN is (cpan.org), it's basically the thing that makes Perl (a relatively non elegant language in its own right) such a powerhouse of a scripting language. It's a worldwide mirrored collection of over 3000 extensions complete with documentation and a common installation procedure.
We thought it would be nice to extend the resource to Python and Ruby, which are excellent languages but are just painfully behind in this regard. At first people (admittedly drunk ;) were agreeable to the idea. I'm talking about the people who run CPAN, who were also at the conference.
Then in November I went to the Ruby conference and ended up promising their community that I would help get them on CPAN. When I told the CPAN people about it, they decided it wasn't what they wanted to do. So that's when I started the FreePAN project.
FreePAN is a mirrored content archive for any software whose source code is freely distributable. It is organized by programming language and is cross-indexed by key fields. It has a consistent set of metadata for every component, including the license which that software is under.
The best way to describe it is to go to http://www.freepan.org
I just got this site up yesterday. It has about 50% of all the known Ruby modules. I want to start adding other languages soon. I put Squeak as one of the languages because I'm hopeful that we can do something interesting here. The key missing piece is an author registration and upload facility.
The effort is very collaborative. People have already started showing up on irc and asking if they could help. To which I'm very grateful. Plenty of work to do. I am currently looking for a Squeak Content Manager. Someone to make sure Squeak is well represented. Anybody interested?
Cheers, Brian
On Tue, 24 Dec 2002, Brian Ingerson wrote:
The effort is very collaborative. People have already started showing up on irc and asking if they could help. To which I'm very grateful. Plenty of work to do. I am currently looking for a Squeak Content Manager. Someone to make sure Squeak is well represented. Anybody interested?
Hi Brian,
How's the Squeak port of YAML going? ;-)
I have some concern about muddying the waters as to where to look for Squeak packages - as I think Ned mentioned, SqueakMap has gained a lot of momentum and mind share lately, and if FreePAN is going to work for the Squeak community we'll have to be careful to coordinate/integrate with SqueakMap, not compete with it.
Assuming we can resolve that issue, I'd be willing to make sure Squeak was properly represented on FreePAN.
Cheers, Avi
On 24/12/02 14:14 -0800, Avi Bryant wrote:
On Tue, 24 Dec 2002, Brian Ingerson wrote:
The effort is very collaborative. People have already started showing up on irc and asking if they could help. To which I'm very grateful. Plenty of work to do. I am currently looking for a Squeak Content Manager. Someone to make sure Squeak is well represented. Anybody interested?
Hi Brian,
How's the Squeak port of YAML going? ;-)
Ned told me he might be interested in doing that. :)
I have some concern about muddying the waters as to where to look for Squeak packages - as I think Ned mentioned, SqueakMap has gained a lot of momentum and mind share lately, and if FreePAN is going to work for the Squeak community we'll have to be careful to coordinate/integrate with SqueakMap, not compete with it.
Definitely. It seems that FreePAN just offers SqueakMap mirroring. This is something SqueakMap appears not to have. Same with Ruby and RAA.
I suspect that FreePAN may offer other features too, but I'd leave it up to the Language Managers as to how much they want.
Competition is not bad in itself. I am completely open to building bridges to other facilities. But I'm also interested in offering an alternative to CPAN for Perl, which in many ways has stagnated.
BTW, I'm trying very hard not to over-architect this project. I want maximum payback for effort. Especially my effort. I want it to be something the community maintains.
Assuming we can resolve that issue, I'd be willing to make sure Squeak was properly represented on FreePAN.
Great. Volunteered then! I'll put you down as the Squeak Content Manager.
One thing we're doing is writing all the tools to build freepan in different languages. It would be neat to see some Squeak tools. I need an upload server. Seems like something you're good at.
Find me on irc (see link on freepan.org) and let's discuss it further.
Cheers, Brian
On Tue, 24 Dec 2002, Brian Ingerson wrote:
One thing we're doing is writing all the tools to build freepan in different languages. It would be neat to see some Squeak tools. I need an upload server. Seems like something you're good at.
Find me on irc (see link on freepan.org) and let's discuss it further.
Will do, but probably not for the next coupla days. There's some kinda holiday my shikse girlfriend seems to expect me to stop working for.
Merry Christmas, all. Avi
On 24/12/02 19:16 -0800, Avi Bryant wrote:
On Tue, 24 Dec 2002, Brian Ingerson wrote:
One thing we're doing is writing all the tools to build freepan in different languages. It would be neat to see some Squeak tools. I need an upload server. Seems like something you're good at.
Find me on irc (see link on freepan.org) and let's discuss it further.
Will do, but probably not for the next coupla days. There's some kinda holiday my shikse girlfriend seems to expect me to stop working for.
OK. I might mock up a dummy application in html. Then you could make it real.
Cheers, Brian
Hi Brian and all!
Brian Ingerson ingy@ttul.org wrote: [SNIP]
I have some concern about muddying the waters as to where to look for Squeak packages - as I think Ned mentioned, SqueakMap has gained a lot of momentum and mind share lately, and if FreePAN is going to work for the Squeak community we'll have to be careful to coordinate/integrate with SqueakMap, not compete with it.
Definitely. It seems that FreePAN just offers SqueakMap mirroring. This is something SqueakMap appears not to have. Same with Ruby and RAA.
First of all - if I haven't already said so - I am all ears for coordination/integration/cooperation - that is always good. Mind though that SqueakMap may have an architecture (especially upcoming 1.1) that might make it a bit "incompatible" - I am not sure though just noting it for the record.
Well, in fact SqueakMap has mirroring - sort of - we just haven't set up any mirrors yet. :-) Ehrm. A mirror of SqueakMap is easily set up by simply setting up another server (identical code) and having it regularly update itself from the master and disabling the web UI for changing the model. No code to be written for this.
Setting one up is on my todo-list but so is also getting SM1.1 out of the door... And SM1.1 will offer more interesting replication with the added ability to have additional "local" content.
SM works by mirroring itself down to the client using a transaction model which makes it theoretically much more efficient than rsync. It also makes it "smartness capable" - the transactions are actually messages that get replayed at the client and the client modifies the local mirror accordingly. At that time it can "react" to the transactions etc.
In SM1.1 I plan to let a mirror be able to have local changes (like an added local package for example) that can optionally be "published" to a selected master higher up in the hierarchy.
This means that a person can have private packages in his/her local SM and a company or organisation can have companywide packages not visible outside the company etc.
It also means that modifications to the map can be applied locally and then published (compared to the current model where modifications to the map can only be applied on the master through the web UI).
Anyway, enough blabbering about that. :-)
I suspect that FreePAN may offer other features too, but I'd leave it up to the Language Managers as to how much they want.
Competition is not bad in itself. I am completely open to building bridges to other facilities. But I'm also interested in offering an alternative to CPAN for Perl, which in many ways has stagnated.
The best way to collaborate here I think is by:
1. Explaining our architectures to each other and learn good stuff. 2. Building a bridge from SqueakMap to FreePAN so that SqueakMap is mirrored into FreePAN. The other direction will be harder I think.
SqueakMap is an architecture heavily focused on Squeak and should remain so. But if there is interest in mirroring the SM content on FreePAN then of course, why not! Personally I am not sure though what the advantage would be - don't mistake me for being negative here - I just need to get the advantages explained to me. :-)
But I don't think I want to limit the development of SM to be constrained by a common possibly simpler model of packages - note though that I haven't read up enough on FreePAN to be sure it is simpler - it's just a guess. SM1.1 will for example have releases of packages maintained as separate "records", does FreePAN have something similar?
BTW, I'm trying very hard not to over-architect this project. I want maximum payback for effort. Especially my effort. I want it to be something the community maintains.
On 07/01/03 13:02 +0100, goran.hultgren@bluefish.se wrote:
Hi Brian and all!
Brian Ingerson ingy@ttul.org wrote: [SNIP]
I have some concern about muddying the waters as to where to look for Squeak packages - as I think Ned mentioned, SqueakMap has gained a lot of momentum and mind share lately, and if FreePAN is going to work for the Squeak community we'll have to be careful to coordinate/integrate with SqueakMap, not compete with it.
Definitely. It seems that FreePAN just offers SqueakMap mirroring. This is something SqueakMap appears not to have. Same with Ruby and RAA.
First of all - if I haven't already said so - I am all ears for coordination/integration/cooperation - that is always good. Mind though that SqueakMap may have an architecture (especially upcoming 1.1) that might make it a bit "incompatible" - I am not sure though just noting it for the record.
Well, in fact SqueakMap has mirroring - sort of - we just haven't set up any mirrors yet. :-) Ehrm. A mirror of SqueakMap is easily set up by simply setting up another server (identical code) and having it regularly update itself from the master and disabling the web UI for changing the model. No code to be written for this.
Setting one up is on my todo-list but so is also getting SM1.1 out of the door... And SM1.1 will offer more interesting replication with the added ability to have additional "local" content.
SM works by mirroring itself down to the client using a transaction model which makes it theoretically much more efficient than rsync. It also makes it "smartness capable" - the transactions are actually messages that get replayed at the client and the client modifies the local mirror accordingly. At that time it can "react" to the transactions etc.
In SM1.1 I plan to let a mirror be able to have local changes (like an added local package for example) that can optionally be "published" to a selected master higher up in the hierarchy.
This means that a person can have private packages in his/her local SM and a company or organisation can have companywide packages not visible outside the company etc.
It also means that modifications to the map can be applied locally and then published (compared to the current model where modifications to the map can only be applied on the master through the web UI).
Anyway, enough blabbering about that. :-)
I suspect that FreePAN may offer other features too, but I'd leave it up to the Language Managers as to how much they want.
Competition is not bad in itself. I am completely open to building bridges to other facilities. But I'm also interested in offering an alternative to CPAN for Perl, which in many ways has stagnated.
The best way to collaborate here I think is by:
- Explaining our architectures to each other and learn good stuff.
- Building a bridge from SqueakMap to FreePAN so that SqueakMap is
mirrored into FreePAN. The other direction will be harder I think.
Agreed.
SqueakMap is an architecture heavily focused on Squeak and should remain so. But if there is interest in mirroring the SM content on FreePAN then of course, why not! Personally I am not sure though what the advantage would be - don't mistake me for being negative here - I just need to get the advantages explained to me. :-)
I agree that FreePAN might not offer Squeak much more than a reliable mirror infrastructure, which you could probably attain on your own.
What I do think that it might add though is exposure. Squeak seems to be way below the radar of the masses. The nature of FreePAN is multilingual, and I hope it becomes a common kiosk where folks can not only shop and share their favorite language, but also learn about new ones.
But I don't think I want to limit the development of SM to be constrained by a common possibly simpler model of packages - note though that I haven't read up enough on FreePAN to be sure it is simpler - it's just a guess. SM1.1 will for example have releases of packages maintained as separate "records", does FreePAN have something similar?
The hallmark of FreePAN is simplicity. It's almost an exercise in under-design.
I see all the code packages as data. FreePAN tries to organize the data in a very logical way. But that's about it. If it's clean and logical people will build tools as needed.
BTW, I'm trying very hard not to over-architect this project. I want maximum payback for effort. Especially my effort. I want it to be something the community maintains.
From my experience with developing SM and the current success (I dare
say) of it I totally agree. SM1.0x is very simple and works mostly because of that AND the fact that it has a GUI inside Squeak - a very important feature for Squeakers. And now that it is easily bootstrapped from within stock Squeak it has gained even more momentum.
The community has helped out with addons, fixes and related stuff - and of course by publishing packages and using it. But noone (but me) has really dug into SM itself yet - I am waiting for that to happen still. I have a few volunteers lined up though, we just haven't had time yet to "sit down" and get cranking.
Assuming we can resolve that issue, I'd be willing to make sure Squeak was properly represented on FreePAN.
Great. Volunteered then! I'll put you down as the Squeak Content Manager.
One thing we're doing is writing all the tools to build freepan in different languages. It would be neat to see some Squeak tools. I need an upload server. Seems like something you're good at.
This last paragraph got me a bit puzzled - could you elaborate? Tools? An upload server is trivial in Squeak - but I am not sure I know what you mean by it.
FreePAN is two things.
1) There is a big static mirror of content built with symlinks, and metadata generated html pages. There are a bunch of little scripts that run to keep this up to date. Those are written in Perl and Ruby.
2) FreePAN needs a central upload server for authors to publish their content. This could be written in anything. Squeak might be nice :)
2a) There are also syncing sripts that pull down content from other archives. We do that now for RAA. Squeak might work this way too.
Find me on irc (see link on freepan.org) and let's discuss it further.
I will check if you are there later today. Unfortunately I am in Sweden so I normally don't "match" timezone-wise...
In closing, I'd say that Squeak is probably the oddest fit for FreePAN at the moment. But everything's still young. And I am very collaborative by nature. I think it's vital for our small languages to share brainpower when possible, if we are going to make a dent against the forces of silly corporate languages like Java.
But, of course, FreePAN welcomes open source Java code as well :)
Cheers, Brian
squeakfoundation@lists.squeakfoundation.org
-
Avi Bryant -
Brian Ingerson -
goran.hultgren＠bluefish.se -
ingy＠ttul.org
-
merlyn＠stonehenge.com -
Ned Konz