[Pharo-dev] [Vm-dev] Re: Random forest in Pharo

Thu Oct 22 07:47:19 UTC 2015

Eliot, you wrote this post on your phone? That's non-trivial. I'm hoping 
you used a bluetooth keyboard.

You asked me to join the VM development effort. Thank you so much for 
the welcome, I'd love to help where I can.

Best,
Robert

On 10/22/2015 03:05 AM, stepharo wrote:
>   
>
>
> +1
> (I'm amazed that you can type all that on an iPhone).
>
> Eliot I hope that the blog and the lecture clement is designing will 
> help people to get closer to the VM.
> I would love that Inria hires a VM researcher and that the rmod group 
> gets exposed and push the VM. It did not work with Stefan Marr
> which could have got a position but was home sick :).
> If you know people that have a good cv and want to live in France, we 
> can fight to get a permanent life researcher position.
>
> Stef
> Le 19/10/15 16:10, Eliot Miranda a écrit :
>>   
>>
>>
>> Hi Robert,
>>
>> _,,,^..^,,,_ (phone)
>>
>> On Oct 19, 2015, at 5:03 AM, Robert Withers 
>> <robert.w.withers at gmail.com> wrote:
>>
>>> Hi Esteban,
>>>
>>>
>>> On 10/19/2015 05:10 AM, Esteban Lorenzano wrote:
>>>>
>>>> Hi,
>>>>
>>>> just to be clear.
>>>> When we talk about MTVM, we talk about a MT-FFI, *not* a MTVM in 
>>>> general.
>>>> In general, a “common” approach to MT cannot be applied in Pharo 
>>>> (or Smalltalk in general) and to get a VM *and* an image working 
>>>> properly is an effort that makes what I called massive some mails 
>>>> above like a small stone compared to a mountain.
>>>
>>> Could you please help me by talking further about the different 
>>> models and scopes of what is meant by MT?
>>>
>>> a) MT-FFI I believe gives the developer a way to call and be invoked 
>>> on callback, asynchronously. Is it so?
>>
>> It's a bit more than that.  It is the sharing of the VM between 
>> different threads, but only allowing one thread to own the VM at any 
>> one time, changing ownership in call out or Smalltalk process switch 
>> time.  This approach provides interleaved concurrency but not 
>> parallelism in your Smalltalk code and it means the Smalltalk class 
>> library doesn't have to be made thread-safe, which as Esteban said is 
>> a huge task.
>>
>> See
>> http://lists.gforge.inria.fr/pipermail/pharo-project/2011-January/038943.html
>>
>> and google "eliot Miranda Simmons own thread" to find more messages.
>>
>>
>>> b) General MTVM means other system services are threaded, like I/O 
>>> events and scheduling and heartbeat.
>>
>> No; at least not in my opinion. In the standard single-threaded VM 
>> the heartbeat is ideally a thread (it can be an interval timer, but 
>> that's problematic; system calls get interrupted), and maybe an 
>> incremental global GC could be in its own thread.
>>
>> So I'm defining the MTVM to be the sharing of the VM between threads, 
>> and /not/ just the use of threads to implement non-Smalltalk sub 
>> tasks of the VM, and /not/ a full-blown multithreaded Smalltalk VM 
>> providing concurrent execution of Smalltalk processes in parallel.
>>
>>> I think that the right model (my stack/priQueue/pool intuition?) 
>>> will change a Herculean task into a fairly straightfoward task and 
>>> achievable. Change the problem, to get better answers.
>>
>> This has been well thought through and discussed.  The definition 
>> above is very useful.  It provides a system that can inter operate 
>> with concurrent code without having to implement a system that 
>> provides parallelism.  It is used in David Simmons' VMs for S# etc 
>> and a similar (but less performant) scheme is available in Python VMs.
>>
>> Please, let's get this scheme working first.  I'm not at all happy 
>> (read, extremely unhappy) that there is not much focus on working 
>> together to get our current VM to an excellent state and instead lots 
>> of work on other VMs that is speculative and a long way away from 
>> being production ready. We have a huge amount of work to do on Cog:
>>
>> - event-driven VM (that hence costs 0% processor time at idle)
>> - 64-bits (x64 and ARM and...?)
>> - Sista adaptive optimizer
>> - FFI via dynamic generation of marshaling code, as required for 
>> efficient and correct call outs on x64
>> - MTVM as defined above
>> - an incremental global mark-sweep GC for Spur
>> - running on Xen/Unikernels/containers
>> - providing a JavaScript plugin to proved rendering and events so we 
>> can run an efficient VM in a web browser
>> - a port of the Interpreter/Context VM to Spur
>>
>> IMO, things that can /and should/ wait are
>> - throwing away Slang and providing a true written-in-pure-Smalltalk 
>> VM that is self-bootstrapped a la Gerardo Richarte and Xavier Burroni
>> - a truly parallel multi/threaded VM
>>
>> and things we shouldn't go anywhere near are
>> - using libffi
>> - targeting JavaScript, Java or any other dynamic language de jour 
>> that happens to run in a web browser but either provides abysmal 
>> performance or doesn't support full Smalltalk semantics
>> - implementing the VM in other VM frameworks such as PyPy which 
>> simply strengthens that community and weakens our own
>>
>> Right now there are only a handful of people who make commits to the 
>> VM and three who are "full time", and we're all overloaded.  But the 
>> VM is the base of the pillar and if we want to provide high-quality 
>> solutions that people will pay money to use we have to have a 
>> high-quality VM.  In Spur we have a VM that is significantly faster 
>> that VW, and very reliable.  In Sista we will have a system that is 
>> much faster and can be improved upon for years to come and a system 
>> that can migrate to future VMs (because it is mostly Smalltalk), and 
>> useful support for a high quality FFI.  People like have stepped up 
>> and made significant contributions to give us what is a respectable 
>> VM that is on an arc to providing a really high-quality production 
>> Smalltalk VM written in Smalltalk produced by a very small community. 
>>  But it is now 2015 and Cog started 7 years ago. All the work on 
>> other VMs, deployment platforms etc, IMO dilutes and delays in 
>> delivering to our community a truly world-class VM that we can 
>> compete with against Java HotSpot, node.js v8, lua luajit, factor, 
>> swift et al.  Please get on board. We'd love the help and we can 
>> guarantee you'll have fun and you can guarantee you'll have an impact.
>>
>>>
>>> I appreciate you and this MT discussion.
>>>
>>>
>>>> Said that:
>>>>
>>>> - What is in plans is MT-FFI, and that will be available eventually.
>>>> - There is an approach I want to re-work, that would allow us 
>>>> profit of multicores without going multithread: the “hydra” 
>>>> experiment made some years ago by Igor creates a good basis to 
>>>> this. But is also a lot of of work (but a lot less than a complete 
>>>> MT), and not a real priority for now… I hope to resume work on that 
>>>> area some day… just not anytime soon.
>>>
>>> Yes, please. I recall those discussions. Hydra is cosmological.
>>>
>>> Regards,
>>> Robert
>>>
>>>>
>>>> Esteban
>>>>
>>>>> On 18 Oct 2015, at 17:56, Ben Coman <btc at openInWorld.com 
>>>>> <mailto:btc at openinworld.com>> wrote:
>>>>>
>>>>>
>>>>> On Sat, Oct 17, 2015 at 2:25 AM, Robert Withers
>>>>> <robert.w.withers at gmail.com> wrote:
>>>>>> Yes, exactly. I do realize I was consciously changing that effort
>>>>>> synchronization order.
>>>>>
>>>>> I see 64-bit being higher priority than multi-threaded for the wider
>>>>> community.  Dealing with larger in-Image data opens the door to more
>>>>> corporate project/funding opportunities. Also simplifying the install
>>>>> on modern Linux platforms without requiring additional 386 libraries
>>>>> will help acceptance there.
>>>>>
>>>>>> It is my humble opinion, without really knowing, that 64-bit 
>>>>>> would have to be redone after the MTVM completes.
>>>>>
>>>>> I would assume it was the other way around. Presuming that Eliot has
>>>>> sponsors influencing his priorities, it seems given that 64-bits will
>>>>> happen first.   I would guess any MTVM development on the old vm would
>>>>> then need to be reworked.
>>>>>
>>>>>> I was doing so with the idea in mind that I and others
>>>>>> might dig into working on the VM, for threading support, while Eliot
>>>>>> maintains focus on 64-bits...a tall order, I know.
>>>>>
>>>>> The usual downside of splitting resources applies.  There are not that
>>>>> many "others" and maybe they would be drawn away from helping with the
>>>>> 64-bit vm.  If the 64-bit vm goes slower for lack of resources then
>>>>> your footing for MTVM will shifting for a longer time.  You may
>>>>> ultimately get where you want to go faster by helping with the 64-bit
>>>>> vm.  The rapport built with other vm devs from working on 64-bit might
>>>>> could then be applied to MTVM.  (Of course, its your free time, so you
>>>>> should pursue what interests you.)
>>>>>
>>>>>> I was barely familiar with the VM, slang, interpreter, it years 
>>>>>> ago...
>>>>>> I'm totally unfamiliar with cog.
>>>>>
>>>>> The experience you gain from working beside Esteban and Eliot on
>>>>> 64-bit Cog/Spur could then be applied to a MTVM.
>>>>>
>>>>> btw, you may find these threads interesting...
>>>>> * 
>>>>> http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/2015-April/108648.html
>>>>> * 
>>>>> http://forum.world.st/Copy-on-write-for-a-multithreaded-VM-td4837905.html
>>>>>
>>>>> cheers -ben
>>>>>
>>>>>> I believe another item on that list ought to be modernizing slang. So
>>>>>> many big items!
>>>>>>
>>>>>> Robert
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 10/16/2015 12:48 PM, Stephan Eggermont wrote:
>>>>>>>
>>>>>>>
>>>>>>> On 16-10-15 14:05, Robert Withers wrote:
>>>>>>>>
>>>>>>>> Because of that assumption I've made and without the 
>>>>>>>> responsibilities
>>>>>>>> you have, Esteban, but recognizing modernizing NB to FFI, my 
>>>>>>>> desired
>>>>>>>> list is:
>>>>>>>
>>>>>>>
>>>>>>> I would expect the least total effort to be needed by keeping 
>>>>>>> the work
>>>>>>> of Esteban and Eliot as much as possible aligned. That is what 
>>>>>>> Esteban's
>>>>>>> list achieves.
>>>>>>>
>>>>>>> Stephan
>>>>>>>
>>>>>>>
>>>>>>
>>>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20151022/e2368b1b/attachment-0001.htm