[Pharo-dev] [Vm-dev] Re: Random forest in Pharo

Nicolas Cellier nicolas.cellier.aka.nice at gmail.com
Sat Oct 24 16:48:51 UTC 2015


2015-10-23 19:01 GMT+02:00 Eliot Miranda <eliot.miranda at gmail.com>:

>
>
>
> On Thu, Oct 22, 2015 at 12:05 AM, stepharo <stepharo at free.fr> wrote:
>
>>
>> +1
>> (I'm amazed that you can type all that on an iPhone).
>>
>> Eliot I hope that the blog and the lecture clement is designing will help
>> people to get closer to the VM.
>> I would love that Inria hires a VM researcher and that the rmod group
>> gets exposed and push the VM. It did not work with Stefan Marr
>> which could have got a position but was home sick :).
>> If you know people that have a good cv and want to live in France, we can
>> fight to get a permanent life researcher position.
>>
>
> Good to hear!  I'll keep my eyes peeled, as they say :-)
>

But please, let him write more code than papers ;)


>
> Stef
>> Le 19/10/15 16:10, Eliot Miranda a écrit :
>>
>>
>>
>> Hi Robert,
>>
>> _,,,^..^,,,_ (phone)
>>
>> On Oct 19, 2015, at 5:03 AM, Robert Withers <
>> <robert.w.withers at gmail.com>robert.w.withers at gmail.com> wrote:
>>
>> Hi Esteban,
>>
>>
>> On 10/19/2015 05:10 AM, Esteban Lorenzano wrote:
>>
>>
>> Hi,
>>
>>
>> just to be clear.
>>
>> When we talk about MTVM, we talk about a MT-FFI, *not* a MTVM in general.
>>
>> In general, a “common” approach to MT cannot be applied in Pharo (or
>> Smalltalk in general) and to get a VM *and* an image working properly is an
>> effort that makes what I called massive some mails above like a small stone
>> compared to a mountain.
>>
>>
>> Could you please help me by talking further about the different models
>> and scopes of what is meant by MT?
>>
>> a) MT-FFI I believe gives the developer a way to call and be invoked on
>> callback, asynchronously. Is it so?
>>
>>
>> It's a bit more than that.  It is the sharing of the VM between different
>> threads, but only allowing one thread to own the VM at any one time,
>> changing ownership in call out or Smalltalk process switch time.  This
>> approach provides interleaved concurrency but not parallelism in your
>> Smalltalk code and it means the Smalltalk class library doesn't have to be
>> made thread-safe, which as Esteban said is a huge task.
>>
>> See
>>
>> http://lists.gforge.inria.fr/pipermail/pharo-project/2011-January/038943.html
>>
>> and google "eliot Miranda Simmons own thread" to find more messages.
>>
>>
>> b) General MTVM means other system services are threaded, like I/O events
>> and scheduling and heartbeat.
>>
>>
>> No; at least not in my opinion. In the standard single-threaded VM the
>> heartbeat is ideally a thread (it can be an interval timer, but that's
>> problematic; system calls get interrupted), and maybe an incremental global
>> GC could be in its own thread.
>>
>> So I'm defining the MTVM to be the sharing of the VM between threads, and
>> /not/ just the use of threads to implement non-Smalltalk sub tasks of the
>> VM, and /not/ a full-blown multithreaded Smalltalk VM providing concurrent
>> execution of Smalltalk processes in parallel.
>>
>> I think that the right model (my stack/priQueue/pool intuition?) will
>> change a Herculean task into a fairly straightfoward task and achievable.
>> Change the problem, to get better answers.
>>
>>
>> This has been well thought through and discussed.  The definition above
>> is very useful.  It provides a system that can inter operate with
>> concurrent code without having to implement a system that provides
>> parallelism.  It is used in David Simmons' VMs for S# etc and a similar
>> (but less performant) scheme is available in Python VMs.
>>
>> Please, let's get this scheme working first.  I'm not at all happy (read,
>> extremely unhappy) that there is not much focus on working together to get
>> our current VM to an excellent state and instead lots of work on other VMs
>> that is speculative and a long way away from being production ready. We
>> have a huge amount of work to do on Cog:
>>
>> - event-driven VM (that hence costs 0% processor time at idle)
>> - 64-bits (x64 and ARM and...?)
>> - Sista adaptive optimizer
>> - FFI via dynamic generation of marshaling code, as required for
>> efficient and correct call outs on x64
>> - MTVM as defined above
>> - an incremental global mark-sweep GC for Spur
>> - running on Xen/Unikernels/containers
>> - providing a JavaScript plugin to proved rendering and events so we can
>> run an efficient VM in a web browser
>> - a port of the Interpreter/Context VM to Spur
>>
>> IMO, things that can /and should/ wait are
>> - throwing away Slang and providing a true written-in-pure-Smalltalk VM
>> that is self-bootstrapped a la Gerardo Richarte and Xavier Burroni
>> - a truly parallel multi/threaded VM
>>
>> and things we shouldn't go anywhere near are
>> - using libffi
>> - targeting JavaScript, Java or any other dynamic language de jour that
>> happens to run in a web browser but either provides abysmal performance or
>> doesn't support full Smalltalk semantics
>> - implementing the VM in other VM frameworks such as PyPy which simply
>> strengthens that community and weakens our own
>>
>> Right now there are only a handful of people who make commits to the VM
>> and three who are "full time", and we're all overloaded.  But the VM is the
>> base of the pillar and if we want to provide high-quality solutions that
>> people will pay money to use we have to have a high-quality VM.  In Spur we
>> have a VM that is significantly faster that VW, and very reliable.  In
>> Sista we will have a system that is much faster and can be improved upon
>> for years to come and a system that can migrate to future VMs (because it
>> is mostly Smalltalk), and useful support for a high quality FFI.  People
>> like have stepped up and made significant contributions to give us what is
>> a respectable VM that is on an arc to providing a really high-quality
>> production Smalltalk VM written in Smalltalk produced by a very small
>> community.  But it is now 2015 and Cog started 7 years ago. All the work on
>> other VMs, deployment platforms etc, IMO dilutes and delays in delivering
>> to our community a truly world-class VM that we can compete with against
>> Java HotSpot, node.js v8, lua luajit, factor, swift et al.  Please get on
>> board. We'd love the help and we can guarantee you'll have fun and you can
>> guarantee you'll have an impact.
>>
>>
>> I appreciate you and this MT discussion.
>>
>>
>> Said that:
>>
>>
>> - What is in plans is MT-FFI, and that will be available eventually.
>>
>> - There is an approach I want to re-work, that would allow us profit of
>> multicores without going multithread: the “hydra” experiment made some
>> years ago by Igor creates a good basis to this. But is also a lot of of
>> work (but a lot less than a complete MT), and not a real priority for now…
>> I hope to resume work on that area some day… just not anytime soon.
>>
>>
>> Yes, please. I recall those discussions. Hydra is cosmological.
>>
>> Regards,
>> Robert
>>
>>
>> Esteban
>>
>>
>> On 18 Oct 2015, at 17:56, Ben Coman <btc at openInWorld.com
>> <btc at openinworld.com>> wrote:
>>
>>
>>
>> On Sat, Oct 17, 2015 at 2:25 AM, Robert Withers
>>
>> < <robert.w.withers at gmail.com>robert.w.withers at gmail.com> wrote:
>>
>> Yes, exactly. I do realize I was consciously changing that effort
>>
>> synchronization order.
>>
>>
>> I see 64-bit being higher priority than multi-threaded for the wider
>>
>> community.  Dealing with larger in-Image data opens the door to more
>>
>> corporate project/funding opportunities. Also simplifying the install
>>
>> on modern Linux platforms without requiring additional 386 libraries
>>
>> will help acceptance there.
>>
>>
>> It is my humble opinion, without really knowing, that 64-bit would have
>> to be redone after the MTVM completes.
>>
>>
>> I would assume it was the other way around. Presuming that Eliot has
>>
>> sponsors influencing his priorities, it seems given that 64-bits will
>>
>> happen first.   I would guess any MTVM development on the old vm would
>>
>> then need to be reworked.
>>
>>
>> I was doing so with the idea in mind that I and others
>>
>> might dig into working on the VM, for threading support, while Eliot
>>
>> maintains focus on 64-bits...a tall order, I know.
>>
>>
>> The usual downside of splitting resources applies.  There are not that
>>
>> many "others" and maybe they would be drawn away from helping with the
>>
>> 64-bit vm.  If the 64-bit vm goes slower for lack of resources then
>>
>> your footing for MTVM will shifting for a longer time.  You may
>>
>> ultimately get where you want to go faster by helping with the 64-bit
>>
>> vm.  The rapport built with other vm devs from working on 64-bit might
>>
>> could then be applied to MTVM.  (Of course, its your free time, so you
>>
>> should pursue what interests you.)
>>
>>
>> I was barely familiar with the VM, slang, interpreter, it years ago...
>>
>> I'm totally unfamiliar with cog.
>>
>>
>> The experience you gain from working beside Esteban and Eliot on
>>
>> 64-bit Cog/Spur could then be applied to a MTVM.
>>
>>
>> btw, you may find these threads interesting...
>>
>> *
>> <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/2015-April/108648.html>
>> http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/2015-April/108648.html
>>
>> *
>> <http://forum.world.st/Copy-on-write-for-a-multithreaded-VM-td4837905.html>
>> http://forum.world.st/Copy-on-write-for-a-multithreaded-VM-td4837905.html
>>
>>
>> cheers -ben
>>
>>
>> I believe another item on that list ought to be modernizing slang. So
>>
>> many big items!
>>
>>
>> Robert
>>
>>
>>
>>
>> On 10/16/2015 12:48 PM, Stephan Eggermont wrote:
>>
>>
>>
>> On 16-10-15 14:05, Robert Withers wrote:
>>
>>
>> Because of that assumption I've made and without the responsibilities
>>
>> you have, Esteban, but recognizing modernizing NB to FFI, my desired
>>
>> list is:
>>
>>
>>
>> I would expect the least total effort to be needed by keeping the work
>>
>> of Esteban and Eliot as much as possible aligned. That is what Esteban's
>>
>> list achieves.
>>
>>
>> Stephan
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> --
> _,,,^..^,,,_
> best, Eliot
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20151024/12e04f3b/attachment-0001.htm


More information about the Vm-dev mailing list