[Vm-dev] Re: [Pharo-dev] Parsing Pharo syntax to C/C++

Tue Sep 16 22:00:33 UTC 2014

On Tue, Sep 16, 2014 at 12:56 AM, Göran Krampe <goran at krampe.se> wrote:

>
> Hi Eliot and all!
>
> Since I work with Ron at 3DICC and Cog is vital to us, I wanted to chime
> in here.
>
> On 09/15/2014 06:23 PM, Eliot Miranda wrote:
>
>> I find this whole discussion depressing.  It seems people would rather
>> put their energy in chasing quick fixes or other technologies instead of
>> contributing to the work that is being done in the existing VM.  People
>> discuss using LLVM as if the code generation capabilities inside Cog
>> were somehow poor or have no chance of competing.  Spur is around twice
>> as fast as the current memory manager, has much better support for the
>> FFI.  Clément and I, now with help from Ronie, are making excellent
>> progress towards an adaptive optimizer/speculative inliner that will
>> give us similar performance to V8 (the Google JavaScript VM, lead by
>> Lars Bak, who implemented the HotSpot VM (Smalltalk and Java)) et al.
>>
>
> One thing you need to understand Eliot is that most of us don't have the
> mind power or time to be able to contribute on that level.
>

Time is the issue.  I'm no brighter than anyone here, but I have my
passion.  And one can learn. Doug McPherson just contributed the
ThreadedARMPlugin having never read the ABI (because he never needed to)
before he started the project.

But still, a lot of us are tickled by ideas on the low level - and thus
> ideas like reusing LLVM, reusing some other base VM, cross compilation etc
> - pop up.
>
> Don't put too much into it - I am always toying with similar ideas in my
> head for "fun", it doesn't mean we don't also see/know that *real* VM work
> like Cog is the main road.
>
>    We are trying to get person-power for a high-quality FFI and have a
>> prototype for a non-blocking VM.  When we succeed C won't be any better
>> and so it won't be an interesting target.  One will be able to program
>> entirely in Smalltalk and get excellent performance.  But we need
>> effort.  Collaboration.
>>
>
> Let me just mention LuaJIT2 - besides very good performance, among other
> things it sports a *very* good FFI. Well, in fact Lua in general has
> several FFIs and tons of C++ bindings tools too - so IMHO anyone doing work
> in that area should take a sneak peek at LuaJIT2.
>
> And this is a truly "sore" area in Smalltalk since eternity. If we had
> something as solid as the stuff in the Lua community - then Cog and
> Smalltalk could go places where it haven't been before I suspect.
>
> If we look at the codebase we have at 3DICC - a very large part consists
> of complicated plugin code to external libraries and accompanying
> complicated Smalltalk glue.
>
> Also, if we compare the Lua community with the Squeak/Pharo community, it
> is quite obvious that the lack of really good FFI solutions leads us to
> "reinvent" stuff over and over, often poorly, while the Lua people simply
> wrap high quality external libraries and that's it. Done.
>

Well I hear you and think that the FFI is extremely important.  That's why
I implemented proper callbacks for Squeak, why Spur supports pinning, and
why I did the MT prototype, and one of the main areas the Pharo team is
working on.

Of course still also stems from the very different background and motives
> behind the two languages and their respective domains, but still.
>
>  Personally I feel so discouraged when people talk about using LLVM or
>> libffi or whatever instead of having the courage and energy to make our
>> system world-class.
>>
>
> Don't feel discouraged - its just that 99% of the community can't help
> you. :) Instead we should feel blessed that we have 1 Eliot, 1 Clement, 1
> Igor and 1 Ronie. Do we have more?

Collaborators <http://www.mirandabanda.org/cogblog/collaborators/>

 I have the confidence in our abilities to compete
>> with the best and am saddened that people in the community don't value
>> the technology we already have and can't show faith in our abilities to
>> improve it further.  Show some confidence and express support and above
>> all get involved.
>>
>
> Let me then make sure you know that 3DICC values *all* work in Cog
> *tremendously*.
>
> As soon as you have something stable on the Linux side - we would start
> trying it. Just let me know, on Linux (server) we run your upstream Cog "as
> is". In fact, I should probably update what we use at the moment :)
>
> Every bit of performance makes a big impact for us - but to be honest,
> what we would value even more than performance would be ... robustness. I
> mean, *really* robust. As in a freaking ROCK.
>
> An example deployment: More than 3000 users running the client on private
> laptops (all Windows variants and hw you can imagine, plus some macs) and
> the server side running on a SLEW of FAT EC2 servers. We are talking about
> a whole BUNCH of Cogs running 24x7 on a bunch of servers.
>

Without error reports, in fact, without an ability to debug in place (run
the assert VM for example, using the -blockonerror switch to freeze it when
an assert fails) there's nt a lot I can do.  We use a CI server to run
regressions at Cadence and my boss makes sure I fix VM bugs promptly when
the CI system shows them.  We deploy on linux and so reliability there-on
is important to us.  So perhaps we can discuss how to debug your server
issues.

We experience VM blow ups on the client side, both Win32 and OSX. OSX may
> be due to our current VM being built by clang, but I am not sure. Our Win32
> VM is old, we need to rebuild it ASAP. Hard to know if these are Cog
> related or more likely 3DICC plugin related, but still.
>

There are ways of finding out.

But the client side is still not the "painful" part - we also experience
> Linux server side Cogs going berserk (100% CPU, no response) or just
> locking up or suddenly failing to resolve localhost :) etc. I suspect the
> networking code in probably all these cases. Here we do NOT have special
> 3DICC plugins so no, here we blame Cog or more likely, Socket plugin.
> Often? No, but "sometimes" is often enough to be a big problem. In fact, a
> whole new networking layer would make sense to me.
>

So we should talk.

Also... we need to be able to use more RAM. We are now deploying to cloud
> servers more and more - and using instances with 16Gb RAM or more is
> normal. But our Cogs can't utilize it. I am not up to speed what Spur gives
> us or if we in fact need to go 64 bit for that.
>

yes.  Spur 32-bit will allow you to use a little more memory than 32-bit
Cog, but tens of percent, not large factors.  You'll need to go to 64-bit
Spur to be able to access more than 2 or perhaps 3 Gb at the outside.

> regards, Göran
>

-- 
best,
Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20140916/5ada6897/attachment.htm