Squeak-dev November 2008

squeak-dev@lists.squeakfoundation.org

119 participants
135 discussions

second call for feedback on Naiad design
by Craig Latta 19 Jan '09

19 Jan '09

Hi-- This is another call for feedback on the design of Naiad[1], a Smalltalk module system I'm writing for Squeak as part of the Spoon project[2]. On the theory that I'll get more of a response by including the whole text rather than a link to it, here it is... :) *** 2008-10-20, 1946 GMT Copyright (c) 2008 Craig Latta. All rights reserved. Hi-- I've been on a quest to make Squeak smaller and more modular, the Spoon project[1]. Part one was making the object memory small. Part three is about making the virtual machine small. This message is about part two, making a module system suitable for adding new behavior to a minimal system in an organized way, and for transferring behavior accurately between running systems. Spoon's module system is called "Naiad", which is an acronym for "Name And Identity Are Distinct". It keeps track of the development history of a system (what the "sources" and "changes" files are for now), and makes it available for exchange with other systems. I think keeping classes' names and identities separate is critical for this. Following are some notes on its design and use, including the object model[1]. At this point I'd like to emphasize I am the author of this design, that I intend to release its implementation under an MIT-style license, and that I'd like to pursue a graduate degree with it (I'm open to invitations :). *** motivation A traditional Smalltalk system uses source code to express both development history and changes exchanged between systems. The precise meaning of source code depends on the current state of the system compiling it. Since a Smalltalk system is dynamic, source code is an inherently ambiguous medium across time. The most problematic system artifacts in light of this ambiguity are classes. All activity in a Smalltalk system is the result of sending messages to objects. The sending of a message invokes the execution of a method, a sequence of instructions for a virtual processor. Some of these instructions manipulate the state of the object receiving the message. Classes define the structure of that state. Therefore, when those class definitions change, the source code for the methods of those classes may become meaningless. One may confront this situation when trying to recompile source code for an old version of a method whose class definition has changed in the meantime. Similarly, source code from one system may not be meaningful on another, since corresponding class definitions on each system may change independently (or be removed entirely). This means that the accurate exchange of behavior requires manual labor, hindering the propagation of useful fixes and new code. It also means that interpretation and use of historical code is more difficult than necessary. So we pay twice for this problem: when learning the system, and when trying to share our work with others. By separating class name from identity, Naiad makes Smalltalk more approachable for newcomers, and more productive for developer and user communities. editions Using Naiad, each development system consists of two object memories: one containing developed code, and another containing "editions" which describe that code. I'll call the first one the "subject memory" and the other the "history memory". An Edition is a description of some artifact in the subject memory at some point in time, currently an author, comment, tag, class, method, module, checkpoint, or edit. Each edition has a reference to that artifact's next state in the future (the next edition) and in the past (the previous edition), as well as an author edition, a collection of licenses, and a timestamp. An Edit represents the activation of some edition at a point in time. For example, there may be a method created in 2005 that is removed in 2006 and reactivated in 2007. There would be an Edit for each of those three events, but only two method editions (one representing the method becoming active, and one representing it being removed). The history memory replaces the current changes and sources files. It has an instance of EditHistory corresponding to the subject memory, which records the active (current) editions for the classes, method, modules, and authors in the subject memory. It also keeps the subject memory's id and the last Edit made to the subject memory. Every time the subject memory adds, changes, or removes a class definition, method, author, comment, tag, or module, or makes a checkpoint (i.e., makes an edit), it adds the appropriate editions to the history memory via remote messages. The history memory snapshots itself after every edit, so as to provide crash recovery support. The subject memory keeps a remote reference to the history memory's instance of EditHistory as a class variable of the local EditHistory class, and interacts with it using utility messages sent to the local EditHistory class. The history memory also keeps that EditHistory instance as a class variable of its local EditHistory class, but as a local reference. An edition typically elides some of its references when it is transferred out of a history memory. For example, a transferred edition will usually omit the references to its next and previous editions. The requesting subject memory can calculate the ID of those editions and obtain them with a separate request, if necessary. A subject memory may elect to keep its EditHistory instance as a local object, such as in a situation where one wants some limited immutable history for debugging purposes, and no crash recovery support. Whether in this scenario or in normal development the same EditHistory utility messages suffice, since no special code need be written to support remote objects. If no edits will be made during deployment, and no history retrieval is required, one may simply jettison the history memory. One may always reconnect the subject and history memories at a later time and continue development. The subject memory has tools for browsing and activating the editions, wherever they are located. This means that no special tools are needed to browse the artifacts of multiple subject systems; one uses the same tools as for browsing the artifacts of the local subject memory. Each subject memory may connect to multiple history memories concurrently (if allowed). For that matter, the history memories of multiple systems may connect to each other directly, to aggregate editions from multiple people, for example. class and method IDs Each class in the subject memory has a universally-unique identifier[3], or UUID. The classes in the minimal subject memory are assigned UUIDs before the initial release, and all subsequent classes are assigned UUIDs when created. Rather than use the single word "class" to refer to either a metaclass or to its sole instance, Spoon introduces the term "protoclass". For example, (Array class) is a metaclass, and its sole instance, Array, is a protoclass. Each metaclass and protoclass has its own UUID, called a "base ID". This is supported by a new instance variable in ClassDescription. Each version of each class is identified by a ClassID, a byte array with segments for the class's baseID, author UUID, and a sixteen-bit version. This means we can uniquely identify, for each author, 65,535 versions of each class in the system. Since we identify authors by UUID, the number of possible authors is very large. Each version of each method is identified by a MethodID, a byte array which contains a ClassID and segments for the method's selector, author UUID, and a sixteen-bit version. This means we can uniquely identify, for each author, 65,535 versions of each method in each version of each class in the system. method editions and method literal markers Each MethodEdition holds a reference to the corresponding ClassEdition, the method source code, and the information needed to reconstruct the corresponding CompiledMethod directly, without need of the compiler (the method header, initial and final program-counter values, method literal markers, and instructions). If one will never use the history memory to install methods in a subject memory that lacks a compiler, one could drop the compiled method information to save space. Method literal markers are used to transmit a compiled method's literal frame values between object memories. There are method literal marker classes to support references to classes, class variables, other pool variables, and literal objects, and to support methods which perform class-side super-sends. Each method literal marker instance knows how to serialize itself as part of Spoon's remote messaging system. In particular, when a method literal that refers to a class transmits itself, it transmits the ClassID of that class, not the name of the class. This gets at the namesake concept of Naiad, "Name And Identity Are Distinct". When referring to a class, we never need to use its name. Each version of each class is an object with a distinct identity. By using ClassIDs to refer to each of them, we can avoid using class names at all when storing history or distributing code. This means that name of each class can be anything, as far as the system is concerned. With every class name unconstrained, there is no need for "namespaces" to distinguish between classes which happen to have same name at some point in time. Each class effectively has its own namespace, since it is uniquely identifiable regardless of its name. Developer tools armed with this information can resolve ambiguity for humans browsing and changing the system. If a developer writes a method which uses a name shared by multiple classes, the system can present more information about each of those classes (such as the author, time of creation, version, and module association), so that the developer can choose the intended one. When browsing such a method, the system can distinguish the aliased class name visually, indicating that there is disambiguating information available. class editions and shared variables Each ClassEdition holds the editions for all the method versions currently active in the corresponding class in the subject memory. Since every edition keeps a reference to its previous and next editions, one can trace the history of any method by starting at the active edition. Removed methods are represented by method editions which have the same MethodID as a normal previous method edition, but with the rest of the fields set to nil. Each ClassEdition also holds the information needed to reconstruct the corresponding class directly, without need of the class builder. For all classes, this includes the format, instance variable names, and superclass ID. For protoclasses, it also includes the class pool keys, class name, and received pool IDs. In Spoon, every shared variable pool is the responsibility of some class in the system. There is no global variables pool ("system dictionary"). Each class that defines a pool is said to "publish" that pool; classes which use that pool "receive" it. Spoon adds an instance variable to Class to map published pools to their names. Each ReceivedPoolID that a protoclass edition uses is a byte array which contains a class ID and a published pool name. checkpoints and modules A Checkpoint edition is simply a named marker of a particular point in time. A developer may use checkpoints to indicate various interesting states of development, and use the tools to regress or replay edits made before or after that time. The largest unit of work is represented by module editions. They are named collections of method IDs, indicating the specific versions of methods which comprise a module, along with sets of child, parent, prerequisite, and postrequisite module editions. When a module edition is transferred out of a history memory, those edition references are transmitted as ModuleIDs. Each module edition also has an "antimodule", a module edition calculated at installation time by a receiving system which, if applied, would undo the changes made by installing the original module. Finally, each module edition has a URI by which someone at a remote site may install the module. That URI represents a command to a Spoon system running on a requestor's local machine; it refers to a standard port on localhost. Its path is a text-encoded action, containing an instruction (in this case "install a module"), the hostname and port of a Spoon system providing the module, and the module's ID. The receiving system uses this information to request the module from a providing history memory, which then transmits editions as necessary. Exactly which editions are transmitted depends on the state of the receiving system; this is a two-way conversation between the providing and receiving systems. This is often more time and space efficient than simply providing all of a module's code, which is what happens with traditional static representations like change sets. The URIs may be cited on ordinary webpages, which are indexed by search engines like Google. A person in search of a module for a particular purpose can search for it with a web browser, using those search engines. Having found a module's URI, the person can click on it, establishing a connection to an embedded webserver in their local Spoon system, which carries out the URI's command. This mechanism for code distribution avoids storing code in static files. It's a deparature from Smalltalk's traditional "fileout" mechanism. The encoded URIs can serve other functions as well, such as listing a system's installed modules, removing an installed module, making a snapshot, and quitting the system. In this way one can use a web browser to interact with a Spoon system for several basic tasks; this is especially useful when the system is headless (e.g., in its initial minimal state). comments and tags Editions for authors, classes, methods, checkpoints, edits, and modules each have their own comment and tag editions. This means each one of those artifacts has a comment and tags, and the changes in both are recorded over time. Comments are as we've already been using them: they're explanatory prose about the artifacts. Tags may be familiar to you from the web; they are short semantic markers used for grouping similar artifacts. I intend for tags to replace class and method categories. Nominally, we've been using class and method categories to establish semantic hierarchies, but the hierarchies have turned out to be quite shallow. Although we can form hierarchies with tags as well, I think we would do better to apply the sorts of algorithms that search engines use, and not concern ourselves with memorizing an artifact's semantic markers. The computational cost this incurs for the tools might have been high in the early days of Smalltalk, but it is quite modest now. Thanks for reading! Please let me know of any questions or other feedback, and feel free to discuss this on the Spoon and Squeak-dev mailing lists. -C [1] http://netjam.org/spoon/naiad [2] http://netjam.org/spoon [3] http://en.wikipedia.org/wiki/Universally_Unique_Identifier -- Craig Latta improvisational musical informaticist www.netjam.org Smalltalkers do: [:it | All with: Class, (And love: it)]

6 7

Linux and ttyUSB0
by Jon Hylands 13 Jan '09

13 Jan '09

Hi everyone, On my current robotics project, I'm using a gumstix verdex to talk over a USB to serial converter chip (FT232) to a serial bus. My brother Dave made up a new kernel driver for the gumstix, and when I plug in the chip it registers itself as ttyUSB0. For work I've done previously on Squeak on the gumstix, I've used standard serial ports, which map to ttyS0, ttyS1, ttyS2, etc, which correspond to port 0, 1, and 2 when I open the Serial Port in Squeak. However, I have no idea which port number ttyUSB0 corresponds to... Anyone got any idea? Thanks, Jon -------------------------------------------------------------- Jon Hylands Jon(a)huv.com http://www.huv.com/jon Project: Micro Raptor (Small Biped Velociraptor Robot) http://www.huv.com/blog

11 24

Re: [squeak-dev] Kay's Little Blue Book, or, Let's Just Do It! (was Re: The Gospel of Alan)
by Ties Stuij 14 Dec '08

14 Dec '08

On Thu, Nov 27, 2008 at 3:54 AM, Jecel Assumpcao Jr <jecel(a)merlintec.com> wrote: > Tony Garnock-Jones wrote: >> Bert Freudenberg wrote: >> > Just a meta remark - I find it highly amusing how people dissect the >> > Gospel of Alan, even interpreting it literally. He must get quite a >> > chuckle from that ;) >> >> Well I hope so! :-) > > Given how happy he said he was to not have any disciples, he might be > less than amused to find out otherwise ;-) > > Going to the other extreme, where everyone's opinion is equally valid, > leads to situations like a guy explaining to me on comp.lang.lisp that > multiple dispatch is the most important feature of OOP and so CLOS and > C++ are true OO languages but Smalltalk is not. > >>http://groups.google.com/group/comp.lang.lisp/browse_thread/thread/b2aa1842… I know I shouldn't react, but could you just dissect for me how Alan's post, either the one before or the one after the post of yourself you linked here, makes the point you're suggesting him to be making? Is it the 'I've no idea where I'm going with this.' part perhaps? /Ties

3 3

The state of Squeak I18n support: let's improve it
by Katerina Barone-Adesi 09 Dec '08

09 Dec '08

There has been a lot of excellent work done on Unicode in Squeak. Unfortunately, not all of it works 'out of the box'. I've started working on trying to figure out how to enable the basic operations (input, display, and copy/paste) for several languages. With the kind help of several Russian squeakers, I've managed to create relatively simple instructions on how to do so for Russian, which I've posted here: http://wiki.squeak.org/squeak/5773 ; I've personally tested the 3.10 instructions under Linux. However, getting to this point involved days of experimentation, including quite a lot of image hangs. The reason I created the documentation was that, despite reading everything on squeak.org and the mailing list archives about Russian and Cyrillic support, I still couldn't get it working - it took combining the instructions of two people who had had similar experiences, and the patches of two others, to have everything work, although there are some problems with garbage characters with copy/paste sometimes. A side effect of these instructions is that various accented European characters (such as à, å, ò, ö, ĵ, ñ, ç, č, ć, š, đ, etc - everything I've tried) work. This cursory testing suggests that Italian/French/German/Spanish/Swedish/Esperanto/Slovenian/etc operations also work after following the Russian instructions; they do not work by default. Perhaps more interestingly, Thai and Greek also work; Arabic nearly does, but only displays in its 'general Unicode' form (rather than correctly changing what appears based on the position of each letter in the word: see http://en.wikipedia.org/wiki/Arabic_alphabet for a better description). I'm now going through the same process for Japanese. Installing 'Japanese Environment Installer' (JEI) from SqueakMap only gave me a working display of Japanese characters, without working input or copy/paste. Possibly interestingly, direct input while using a Japanese keymap (with no input method) to type kana works with the Russian patches, but not with the JEI patches. Nothing using extra input method software works anywhere in Squeak in any configuration I've tried. Deadkeys also don't work. I would like to request help from all Squeakers who can input, display, and copy/paste text in languages which have characters or writing systems that don't appear in the languages mentioned above, and especially of those who have everything working with Japanese. Any instructions on how you did it would be greatly appreciated. I intend to keep documenting the simplest ways to currently set things up, which will hopefully have the side effect of making a baseline set of patches to have things Just Work become clear. Any reports of errata in the documentation, platform differences, or bugs are also extremely welcome. Thank you; Katerina Barone-Adesi

4 11

Botsinc free
by stephane ducasse 04 Dec '08

04 Dec '08

Hi all I'm happy to announce that my book BotsInc http://smallwiki.unibe.ch/botsinc will be free soon (in fact it is since 23 september). I would like to thank ESUG http://www.esug.org for supporting financially the buying of the book rights and giving the book to the kid world. A catalan version will be available really soon thanks to jordi delgado! (Yes the french were not the first this time... We were the first for the etoy book and squeak by example. :)) If you want to participate to a translation please contact me. I will be setting up soon an infrastructure to support any language translation. Stef ------------------------ Stéphane Ducasse USTL - LILF - CNRS UMR 8022 - INRIA stephane.ducasse(a)inria.fr http://stephane.ducasse.free.fr "if you knew today was your last day on earth, what would you do different? ... especially if, by doing something different, today might not be your last day on earth" Calvin&Hobbes Tel 33 3 59 57 78 66 - Fax 33 3 59 57 78 50 40, avenue Halley, Parc Scientifique de la Haute Borne, Bât.A, Park Plaza, Villeneuve d'Ascq, FR-59650, France.

5 7

Rio: exceptions documentation
by Casimiro de Almeida Barreto 03 Dec '08

03 Dec '08

Hello, I had a code like: [ aStream := CrLfFileStream new open: 'aGivenFileName' forWrite: false ] on: FileDoesNotExistException do: [ :exception | <code ....> ]. It worked ok. But with: [ aStream := 'aGivenFileName' asRio reader ] on: FileDoesNotExistException do: [ :exception | <code ...> ]. things just don't work... Where I can find exceptions documentation (in particular for Rio) ? Besides, it seems that the atEnd message is delayed by one entry (in the first case 1025 lines are correctly read and in the second case it goes one line ahead in the file).

3 6

[BUG] signedLongAt: 1 put:−2,147,483,648
by John M McIntosh 02 Dec '08

02 Dec '08

I noted the the current VMs don't handle the case of storing −2,147,483,648 (primitive failure) via signed32BitValueOf: usually called via FFI signedLongAt:put: For plugin writers who used signed64BitValueOf: (primitive failure) and signed64BitIntegerFor: returned zero if passed −9,223,372,036,854,775,808. signed64BitIntegerFor: is the one that is nasty since your value of 0x8000000000000000 silently turns in 0. I reopened problem http://bugs.squeak.org/view.php?id=6987 and wrote some SUnits. I also provided a suggested fix, which allows me to run the SUnits. Lastly for Alien I added long long (signed/unsigned) support which is still missing in the FFI plugin so I could do some testing. -- = = = ======================================================================== John M. McIntosh <johnmci(a)smalltalkconsulting.com> Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com = = = ========================================================================

2 5

Cleaning up lava code and unsent messages.
by Jerome Peace 02 Dec '08

02 Dec '08

Cleaning up lava code and unsent messages. Hi all, This was an after thought from my previous post. It deserved its own topic. Lava code is code left over from historical times which has no use in the current image and yet no one is willing to take responsiblity for removing it. For the record squeak has historically had about one sixth of its messages as unsent methods. Some of which is lava code others of which are examples, probes, utilities and tests. It would be nice to clean things up so there are no unsent messages. This could be done simply by creating a (pseudo-recursive) message #legitamatelyUnsentMethods (one method say for each class.) That sends for any legitimate examples, etc. Then anything that shows up as unsent would be illegitamate and removeable. >From time to time you would look at the implementors of the legitamatelyUnsentMethods to clean out what no longer belongs. a legitamatelyUnsentMethods method would be packagable if care were take that it mentions only the unsent methods of that package. My proposal is that package maintainers should take the responsibility for doing this in their own packages and the release team should reject any code that contains unsent messages. An Sunit test could be provided to check. I am also hoping to recruit those who would be willing to sort out the unsent messages in core packages and either write legitamatelyUnsentMethods methods for them or flag them as lava code for removal. It is a good way to gain an insight into a lot of squeak code while helping the community. If this task appeals to you please start it by planting a seed on mantis. Yours in service and curiosity, --Jeorme Peace ____________________________________________________________________________________ Never miss a thing. Make Yahoo your home page. http://www.yahoo.com/r/hs

6 5

Missing Signers for Squeak 3.10 relicense
by Matthew Fulmer 01 Dec '08

01 Dec '08

I took a look through yoshiki's 3.10 history database [1], and saw 4 names I immediately recognized as not having signed the agreement: Matthew Fulmer Keith Hodges Ralph Johnson Bryce Kampjes oops. One of those is me :P. I know all these people are here now, so let's go sign that agreement: http://netjam.org/squeak/SqueakDistributionAgreement.pdf [1] Yoshiki's history database: http://lists.squeakfoundation.org/pipermail/squeak-dev/2008-September/13158… [2] Attached is the list of non-signers and their methods (missing.txt) -- Matthew Fulmer -- http://mtfulmer.wordpress.com/

4 8

Instance variable access in superclasses.
by Michael van der Gulik 30 Nov '08

30 Nov '08

Hi all. What would people's reaction be if a class was prevented from being able to directly access its superclass's instance variables? A subclass should use accessor methods to access a superclass's instance variables. Is there any particular reason subclasses get access to superclass instance variables? I think it breaks encapsulation. If this was implemented, it might be possible to avoid needing to recompile every subclass when you modify the instance variables of a class. Gulik. -- http://people.squeakfoundation.org/person/mikevdg http://gulik.pbwiki.com/

20 70

Jump to page: