Klaus D. Witzel wrote:
> On Fri, 15 Aug 2008 09:11:21 +0200, Joshua Gargus wrote:
>
>> (Wishing myself more success with writing this email before sending 
>> it :-) )
>>
>> Igor Stasenko wrote:
>>> Hi folks,
>>>
>>> soon Hydra will provide a support to instantiate new interpreter
>>> instance from current object memory, e.g. not based on images which
>>> residing on file system.
>>>
>>> The main focus for this feature is to create a tiny images, with
>>> limited behavior for off-loading processing from main interpreter to
>>> separate worker interpreter.
>>> Since Hydra already has mechanisms to transfer data between
>>> interpreters, the need in initially packing new image(s) with data is
>>> minimal.
>>> The most important (and interesting) IMO is to define and transfer a
>>> behavior (classes & their methods) which is minimal for solving some
>>> problem in its domain.
>>>
>> Not that it's immediately relevant, but keep in mind that we'll 
>> eventually want to be able to share behavior between images.
>
> This is my part of Igor's enterprise ;) we discuss my crazy cross-heap 
> pointerage approaches with Igor as sparring partner ;)
>
> The main attention got GC, which has (among others) these aspects:
>
> o distributing object allocation is very promising
>  (Guillermo Adrián Molina: distributing alloc is the main
>   application of parallel processing [native threads in
>   Huemul], email communication)
>
> o 90% of each Process (remember 1:1 to a thread) references
>   are in the same chunk of process generated objects, 9% of
>   the references are to globals, and 1% to other process
>   generated objects (Guillermo Adrián Molina, native threads
>   in Huemul, email communication)
>
> o local GC gets often in the way when doing things in parallel
>
> o garbage can by come cyclic cross-heap and so unreclaimable

That's all very interesting (especially the measurements about which 
objects a Process references).  I'll reluctanly resist the temptation to 
take the conversation in 10 different directions :-)

(big snip)

>> The technical details of your approach sound good to me (without 
>> having thought deeply enough to provide truly constructive 
>> criticism).  However...
>>
>> My main concern is that your argument against separate images is 
>> disingenuous.  They won't be slower if you store them as ByteArrays 
>> within the main image.
>
> But then they are always in the way when GC comes around :( This would 
> invalidate all the pointers of the parallel thread and require global 
> synchronization :(
>
> Not a good idea :( we want things to run in parallel independent of 
> each other's GC.
>
I think that there is a misunderstanding.  I'm saying that you can store 
a prototype of an image as a ByteArray in the main image, but you 
wouldn't actually run a spawned interpreter using the ByteArray as the 
object memory!  You would use it to populate a separate, newly-spawned 
HydraVM object memory.

It would actually be pretty funny to implement it the way you thought I 
meant, in the same way that Intercal and Lolcode are funny (except this 
would be more of an inside joke).  But certainly not practical!

(hmm, maybe we could combine them... you could spawn a new interpreter 
with the command "I can has new interpreter?"... what do you think?)

>> In fact, I believe that the opposite would be true; don't you agree?  
>> From a performance standpoint, it seems like separate images are the 
>> better option.
>
> When creation of bytearray versus creation of separate heap can be 
> ignored, there would be no difference in terms of performance (it's 
> all oops all the way down, anyways). Only that bytearrays are not 
> usable for parallel processing.
Now that the confusion above has been cleared up...

Wouldn't it be faster to spawn a new object-memory from an image in a 
ByteArray (which requires a memcpy() and a single pass through the image 
to relocate oops by a fixed amount) compared to the scheme that Igor 
describes?

(snip the rest, where we are in agreement)

Cheers,
Josh