[Vm-dev] NativeBoost / Callbacks and Multithreading (Eliot: This is not just NB, we need you here :P)

[stephane ducasse]

On 18 Apr 2015, at 19:36, Eliot Miranda <eliot.miranda at gmail.com> wrote:

> Hi Stef,
> 
> On Apr 18, 2015, at 10:13 AM, stepharo <stepharo at free.fr> wrote:
> 
>> I think that the idea of eliot is independent from NB but use FFI.
>> Esteban will resume his task to use NB syntax for FFI and NB as a back-end of the UnifiedFFI.
> 
> I hope we can revise this slightly to agree with the back end that Ronie, Clément and I propose, which is to use the unsafe bytecodes in Sista/Scorch to include Ronie's lowcode and do the marshaling, avoiding NB.  This gives us platform independence, the ability to function in the interpreter (the Sista interpreter can execute these bytecodes too), and security, since when jitted the code lives in the code zone which we can secure, and avoid making the whole heap executable.  Maybe you can discuss with Clément when he returns.

yes we do not want to be bound to NB. (what I menat with NB as a back-end of the UnifiedFFI was that if people want to use NB as a back-end they should be able to do it).
If Igor maintains it and port it to 64 bits we do not want to be against. 
What we need is a version of FFI that understand NB syntax running so that we can ship Spur.

Stef

> 
>> 
>> Stef
>> 
>> Le 18/4/15 18:24, Nicolai Hess a écrit :
>>>  
>>> 
>>> 
>>> Thank you Esteban,
>>> 
>>> 2015-04-18 9:36 GMT+02:00 Esteban Lorenzano <estebanlm at gmail.com>:
>>>  
>>> Hi, 
>>> 
>>> So… “multithread” is a complicated issue. Pharo as most Smalltalks is designed thinking is as monolithic so if you are trying to make the image to work in different processes… it will not be easy at all :)
>>> Time ago Eliot did a prototype (more than a prototype in fact, but still far from “production ready”) to have Threaded FFI and I’m quite sure this is the way to go, at least as a 1st milestone. 
>>> 
>>> I already have a process building this experimental VM:
>>> 
>>> https://ci.inria.fr/pharo/view/VM/job/CogMTVM/
>>> 
>>> But this one does not include NativeBoost support. Do we have a build with MT and NB?
>>>  
>>> 
>>> (No idea why windows build failed last two times… this can be a random fail).
>>> 
>>> I do not remember exactly the changes needed in the image to take advantage of it… it was just an instVar in Process, I think, but then FFI package was adapted to take advantage of this… no idea where it is now (if is already incorporated in latest FFI, which we have).
>>> 
>>> Then, after this… I imagine the callback mechanism can be adapted to work in multithread environments but probably we will need something like isolates from Dart to provide some degree of multithread processing without killing the image (but I’m just thinking in loud here, this can be a really bad idea, and probably there are other ways to do this better… I will let Eliot to explain better). 
>>> 
>>> TL;DR: Start for the CogMTVM, is the way to go.
>>> 
>>> cheers, 
>>> Esteban
>>> 
>>>> On 17 Apr 2015, at 09:06, Nicolai Hess <nicolaihess at web.de> wrote:
>>>> 
>>>> Hi, 
>>>> 
>>>> I've tried to build a pharovm with multithread support. I changed the config in the 
>>>> pharo generator.image to use the cogmt config and was able to build a vm.
>>>> But this vm crashes on startup. (windows7)
>>>> 
>>>> Is this the right way and did anyone got this to work (windows or linux)?
>>>> 
>>>> And can this work to make NB callbacks working for multithreaded libraries.
>>>> 
>>>> (I made some simple bindings for the gstreamer lib with NB, this works
>>>> for simple calls (create element/change state). But I guess this won't work
>>>> for any gstreamer function that requests a callback that may b e called from
>>>> a different thread)
>>>> 
>>>> Would this be the right way to do:
>>>> - build a vm with MT support
>>>> - guard the NB callback entry/leave code with the ownVM()/disownVM() call.
>>>> 
>>>> Or is there more to do.
>>>> 
>>>> Thanks in advance
>>>> 
>>>> 
>>>> nicolai
>>>> 
>>>> 
>>> 
>>> 
>>> 
>> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20150418/93544e03/attachment.htm