[Pharo-dev] [Vm-dev] Re: Random forest in Pharo

Eliot Miranda eliot.miranda at gmail.com
Fri Oct 23 17:01:22 UTC 2015


On Thu, Oct 22, 2015 at 12:05 AM, stepharo <stepharo at free.fr> wrote:

>
> +1
> (I'm amazed that you can type all that on an iPhone).
>
> Eliot I hope that the blog and the lecture clement is designing will help
> people to get closer to the VM.
> I would love that Inria hires a VM researcher and that the rmod group gets
> exposed and push the VM. It did not work with Stefan Marr
> which could have got a position but was home sick :).
> If you know people that have a good cv and want to live in France, we can
> fight to get a permanent life researcher position.
>

Good to hear!  I'll keep my eyes peeled, as they say :-)

Stef
> Le 19/10/15 16:10, Eliot Miranda a écrit :
>
>
>
> Hi Robert,
>
> _,,,^..^,,,_ (phone)
>
> On Oct 19, 2015, at 5:03 AM, Robert Withers < <robert.w.withers at gmail.com>
> robert.w.withers at gmail.com> wrote:
>
> Hi Esteban,
>
>
> On 10/19/2015 05:10 AM, Esteban Lorenzano wrote:
>
>
> Hi,
>
>
> just to be clear.
>
> When we talk about MTVM, we talk about a MT-FFI, *not* a MTVM in general.
>
> In general, a “common” approach to MT cannot be applied in Pharo (or
> Smalltalk in general) and to get a VM *and* an image working properly is an
> effort that makes what I called massive some mails above like a small stone
> compared to a mountain.
>
>
> Could you please help me by talking further about the different models and
> scopes of what is meant by MT?
>
> a) MT-FFI I believe gives the developer a way to call and be invoked on
> callback, asynchronously. Is it so?
>
>
> It's a bit more than that.  It is the sharing of the VM between different
> threads, but only allowing one thread to own the VM at any one time,
> changing ownership in call out or Smalltalk process switch time.  This
> approach provides interleaved concurrency but not parallelism in your
> Smalltalk code and it means the Smalltalk class library doesn't have to be
> made thread-safe, which as Esteban said is a huge task.
>
> See
>
> http://lists.gforge.inria.fr/pipermail/pharo-project/2011-January/038943.html
>
> and google "eliot Miranda Simmons own thread" to find more messages.
>
>
> b) General MTVM means other system services are threaded, like I/O events
> and scheduling and heartbeat.
>
>
> No; at least not in my opinion. In the standard single-threaded VM the
> heartbeat is ideally a thread (it can be an interval timer, but that's
> problematic; system calls get interrupted), and maybe an incremental global
> GC could be in its own thread.
>
> So I'm defining the MTVM to be the sharing of the VM between threads, and
> /not/ just the use of threads to implement non-Smalltalk sub tasks of the
> VM, and /not/ a full-blown multithreaded Smalltalk VM providing concurrent
> execution of Smalltalk processes in parallel.
>
> I think that the right model (my stack/priQueue/pool intuition?) will
> change a Herculean task into a fairly straightfoward task and achievable.
> Change the problem, to get better answers.
>
>
> This has been well thought through and discussed.  The definition above is
> very useful.  It provides a system that can inter operate with concurrent
> code without having to implement a system that provides parallelism.  It is
> used in David Simmons' VMs for S# etc and a similar (but less performant)
> scheme is available in Python VMs.
>
> Please, let's get this scheme working first.  I'm not at all happy (read,
> extremely unhappy) that there is not much focus on working together to get
> our current VM to an excellent state and instead lots of work on other VMs
> that is speculative and a long way away from being production ready. We
> have a huge amount of work to do on Cog:
>
> - event-driven VM (that hence costs 0% processor time at idle)
> - 64-bits (x64 and ARM and...?)
> - Sista adaptive optimizer
> - FFI via dynamic generation of marshaling code, as required for efficient
> and correct call outs on x64
> - MTVM as defined above
> - an incremental global mark-sweep GC for Spur
> - running on Xen/Unikernels/containers
> - providing a JavaScript plugin to proved rendering and events so we can
> run an efficient VM in a web browser
> - a port of the Interpreter/Context VM to Spur
>
> IMO, things that can /and should/ wait are
> - throwing away Slang and providing a true written-in-pure-Smalltalk VM
> that is self-bootstrapped a la Gerardo Richarte and Xavier Burroni
> - a truly parallel multi/threaded VM
>
> and things we shouldn't go anywhere near are
> - using libffi
> - targeting JavaScript, Java or any other dynamic language de jour that
> happens to run in a web browser but either provides abysmal performance or
> doesn't support full Smalltalk semantics
> - implementing the VM in other VM frameworks such as PyPy which simply
> strengthens that community and weakens our own
>
> Right now there are only a handful of people who make commits to the VM
> and three who are "full time", and we're all overloaded.  But the VM is the
> base of the pillar and if we want to provide high-quality solutions that
> people will pay money to use we have to have a high-quality VM.  In Spur we
> have a VM that is significantly faster that VW, and very reliable.  In
> Sista we will have a system that is much faster and can be improved upon
> for years to come and a system that can migrate to future VMs (because it
> is mostly Smalltalk), and useful support for a high quality FFI.  People
> like have stepped up and made significant contributions to give us what is
> a respectable VM that is on an arc to providing a really high-quality
> production Smalltalk VM written in Smalltalk produced by a very small
> community.  But it is now 2015 and Cog started 7 years ago. All the work on
> other VMs, deployment platforms etc, IMO dilutes and delays in delivering
> to our community a truly world-class VM that we can compete with against
> Java HotSpot, node.js v8, lua luajit, factor, swift et al.  Please get on
> board. We'd love the help and we can guarantee you'll have fun and you can
> guarantee you'll have an impact.
>
>
> I appreciate you and this MT discussion.
>
>
> Said that:
>
>
> - What is in plans is MT-FFI, and that will be available eventually.
>
> - There is an approach I want to re-work, that would allow us profit of
> multicores without going multithread: the “hydra” experiment made some
> years ago by Igor creates a good basis to this. But is also a lot of of
> work (but a lot less than a complete MT), and not a real priority for now…
> I hope to resume work on that area some day… just not anytime soon.
>
>
> Yes, please. I recall those discussions. Hydra is cosmological.
>
> Regards,
> Robert
>
>
> Esteban
>
>
> On 18 Oct 2015, at 17:56, Ben Coman <btc at openInWorld.com
> <btc at openinworld.com>> wrote:
>
>
>
> On Sat, Oct 17, 2015 at 2:25 AM, Robert Withers
>
> < <robert.w.withers at gmail.com>robert.w.withers at gmail.com> wrote:
>
> Yes, exactly. I do realize I was consciously changing that effort
>
> synchronization order.
>
>
> I see 64-bit being higher priority than multi-threaded for the wider
>
> community.  Dealing with larger in-Image data opens the door to more
>
> corporate project/funding opportunities. Also simplifying the install
>
> on modern Linux platforms without requiring additional 386 libraries
>
> will help acceptance there.
>
>
> It is my humble opinion, without really knowing, that 64-bit would have to
> be redone after the MTVM completes.
>
>
> I would assume it was the other way around. Presuming that Eliot has
>
> sponsors influencing his priorities, it seems given that 64-bits will
>
> happen first.   I would guess any MTVM development on the old vm would
>
> then need to be reworked.
>
>
> I was doing so with the idea in mind that I and others
>
> might dig into working on the VM, for threading support, while Eliot
>
> maintains focus on 64-bits...a tall order, I know.
>
>
> The usual downside of splitting resources applies.  There are not that
>
> many "others" and maybe they would be drawn away from helping with the
>
> 64-bit vm.  If the 64-bit vm goes slower for lack of resources then
>
> your footing for MTVM will shifting for a longer time.  You may
>
> ultimately get where you want to go faster by helping with the 64-bit
>
> vm.  The rapport built with other vm devs from working on 64-bit might
>
> could then be applied to MTVM.  (Of course, its your free time, so you
>
> should pursue what interests you.)
>
>
> I was barely familiar with the VM, slang, interpreter, it years ago...
>
> I'm totally unfamiliar with cog.
>
>
> The experience you gain from working beside Esteban and Eliot on
>
> 64-bit Cog/Spur could then be applied to a MTVM.
>
>
> btw, you may find these threads interesting...
>
> *
> <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/2015-April/108648.html>
> http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/2015-April/108648.html
>
> *
> <http://forum.world.st/Copy-on-write-for-a-multithreaded-VM-td4837905.html>
> http://forum.world.st/Copy-on-write-for-a-multithreaded-VM-td4837905.html
>
>
> cheers -ben
>
>
> I believe another item on that list ought to be modernizing slang. So
>
> many big items!
>
>
> Robert
>
>
>
>
> On 10/16/2015 12:48 PM, Stephan Eggermont wrote:
>
>
>
> On 16-10-15 14:05, Robert Withers wrote:
>
>
> Because of that assumption I've made and without the responsibilities
>
> you have, Esteban, but recognizing modernizing NB to FFI, my desired
>
> list is:
>
>
>
> I would expect the least total effort to be needed by keeping the work
>
> of Esteban and Eliot as much as possible aligned. That is what Esteban's
>
> list achieves.
>
>
> Stephan
>
>
>
>
>
>
>
>


-- 
_,,,^..^,,,_
best, Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20151023/160d3b67/attachment-0001.htm


More information about the Vm-dev mailing list