Fast bytecode loader (was Re: [squeak-dev] Re: Squeak vision)

Eliot Miranda eliot.miranda at gmail.com
Thu Jul 16 16:38:38 UTC 2009


On Thu, Jul 16, 2009 at 4:40 AM, Bert Freudenberg <bert at freudenbergs.de>wrote:

> On 14.07.2009, at 03:01, osp at aloha.com wrote:
>
>  Maybe it is just me, but "a fast bytecode package loader, without the need
>> to compile Smalltalk code" combined with "such feature could make life
>> easier to third party Etoys developer and eventually attract more of them
>> to Etoys" sounds like an endorsement of closed-source commercial EToys.
>>
>
> No, that's is a severe misunderstanding you construct here.
>
>  If
>> commercial success is not the attraction, what is? Faster initial load
>> time? Reduced file transfer size? Perhaps I assumed too much.
>>
>
>
> Yes, both of these. The scenario we care most about is the one million kids
> using XO laptops. These machines are under-powered compared to recent PCs,
> and have rather little main memory. It's infeasible to have one image with
> all the extensions to Etoys, because the whole image is loaded into RAM. It
> would be very nice if these add-ons could be delivered as separate packages
> and loaded on demand.
>
> That's what the current Dr Geo II package is doing. But it is slow, because
> all the code is compiled to byte code when loading the package. The Squeak
> byte code compiler has not exactly been optimized for speed, because in
> usual development you only compile a method at a time. In any case, the
> *result* of loading a package that includes both source code and byte code
> would be exactly the same as compiling the source code, but way faster.
>
> This could also be used to speed up package loading in general. In fact,
> this would make the concept of "libraries" feasible in Squeak - packages
> loaded on demand. Think about pre-compiled lisp modules, or the .pyc files
> generated from .py Python files for speed. Note I'm not advocating this for
> Squeak but it would make it technically feasible.
>
> It's just not exactly trivial to do in a robust way. At the very least, all
> global references in the compiled methods will have to be updated. This
> includes classes which have to be referenced, Symbols need to get interned
> and unified. The source could be appended to the changes file and source
> pointers incremented, or the source could remain in separate files but we'd
> need a new source code scheme for that.


This has been done for Smalltalk before.  The Digitalk SLL mechanism loaded
code using a mechanism essentially isomorphic to image segments.  The
VisualWorks parcel mechanism is more like a reference stream but optimised
for speed of loading (and because its not tied to the VM's object format is
much more flexible and future-proof, and turned out to be very fast).

IIRC, the Digitalk SLL system didn't support source (the source needed to be
already present).
The VW parcel system doesn't need to append source to the changes file.
 Instead a parcel is a pair of files, a binary file and a source file.  The
SourcesFiles is replaced by a proper SourceFileManager that can register
multiple source files, handing out an index to a source file, so that on
load a CompiledMethod's source index is updated with the index the
SourceFileManager assigns to the registered source file.


> I think this all would be immensely useful in the long run, but truth is we
> have been getting by without for so long that there just does not seem to be
> anybody both motivated and skilled enough to do it.
>
> - Bert -
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20090716/4070ce6f/attachment.htm


More information about the Squeak-dev mailing list