[Vm-dev] Fwd: Memory mapped files and Pharo (aka Unreal Engine 4 integration with Pharo)

[Dimitris Chloupis]

I attach the forwarded post I posted in pharo-users but if you want the
summary its a collection of questions whether its possible to use memory
mapped files shared memory with Pharo VM for IPC. I thought to forward also
here since this may be a VM related too. Any help is appreciated

---------- Forwarded message ---------
From: Dimitris Chloupis <kilon.alios at gmail.com>
Date: Sun, Jan 17, 2016 at 6:41 PM
Subject: Memory mapped files and Pharo (aka Unreal Engine 4 integration
with Pharo)
To: Any question about pharo is welcome <pharo-users at lists.pharo.org>


An apology for the long post but this is not a simple issue and hence not a
simple question.

So I am looking into ways to integrate Pharo with Unreal to bring some
nuclear powered graphics to Pharo and make us first class citizents to 2D
and 3D world.

For those not aware this is unreal

https://www.unrealengine.com/what-is-unreal-engine-4

As suprising it may sound Unreal and Pharo share a lot in common

1) Both integrated languages with IDEs
2) Both promote live coding and live manipulation of data
3) Both promote visual coding
4) Both promote ease of use and wizard based development (aka RAD)

Unreal is indeed a C++ based project made to be used by C++ coders but it
comes with a very powerful visual coding language that can map visual nodes
to any C++ function/Method and another common ground with Pharo is the very
strict OO nature of both projects. But then OOP is very big in game
development anyway.

Of course my goal is not to force Pharoers to learn C++, but rather make a
Pharo API so Pharo can be used to do some simple graphics development in
the start and then we can continue importing more and more functionality.
Unreal is generally a heavy engine that requires quite a powerful GPU but
its rendering is just amazing.

So how I make Pharo talk to Unreal is the million dollar question.

The road of compiling Unreal as a set of DLLs to be loaded by Pharo via FFI
is a road full of thorns because its quite an undertaking cause Unreal is
HUGE and it will be nightmare to maintain since Unreal moves forward very
fast.

So we come to the subject of IPC, or Inter Process Communication. I am not
new to this as you know I have build a socket bridge between Pharo and
Python that allows Pharo to use Python libraries and in my implementation I
focus on Blender Python API.

But sockets are not exactly blazzing fast, calling functions is fine
because you can even get responses bellow 1 millisecond but if try to run
some heavy loops you will be in trouble. There are work arounds of course,
like sending the loop with the socket and language etc but they
overcomplicate something that at least in my opinion should remain simple.

Another IPC method is shared memory, it is what it says , basically
processes that share a place in memory where they can access the same data.
Extremely fast but it has its own traps for example how to make sure the
processes dont access the same data at the same time etc.

So after some reading I came across to a shared memory model called Memory
mapped files, basically what that means is essentially a virtual file that
resides on memory (it may also reside on the hard disk or other permanent
physical storage but its not necessary) that different processes can
access.

So in our case we can have memory mapped file that Unreal , Pharo and even
Blender can access where we can store command / messages that each
diffirent application must execute, let them share data and sky is the
limit.

Now I know this will not be a walk in the park specially to someone like me
a C++ noob but I am looking for the Pharo wisemen wisewomen to guide me at
least through the obstacles of this.

So the questions are the following

1) Do you think this is possible with the current Pharo ?

2) Will I be limited by the fact that the VM currently does not multithread
or cannot use multithreading libraries ? ( I have no intention of using
multithreading but some handling of process access to the data may be
necessary to make sure data is safe from concurent modification)

3) Is there anything else I should be aware of as common pitfalls for such
implementation ?

4) Can the current FFI help in this task or would I need to implement this
as a C DLL and load it from Pharo

PS: I have no intention of messing with the Pharo VM and I also want to
avoid the use of plugins as I want this to work with standard Pharo
distributions.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20160117/7d3ec9b3/attachment.htm