[Vm-dev] Overview of the OpenSmalltalk VM and its build process

Tue Oct 17 17:22:45 UTC 2017

2017-10-17 19:03 GMT+02:00 Nicolas Cellier <
nicolas.cellier.aka.nice at gmail.com>:

>
>
> 2017-10-17 16:00 GMT+02:00 Clément Bera <bera.clement at gmail.com>:
>
>>
>> Hi Raffaello,
>>
>> Answers inlined...
>>
>> On Tue, Oct 17, 2017 at 3:07 PM, Raffaello Giulietti <
>> raffaello.giulietti at lifeware.ch> wrote:
>>
>>>
>>> Hi,
>>>
>>> I'm interested in understanding the overall OpenSmalltalk VM structure
>>> and design and its build process.
>>>
>>> I read [1] and [2] to get a high level picture of the vm and its
>>> interesting strategy to development (Slang, VMMaker, etc.)
>>>
>>> Apart from these, are there other useful documents/sites/tutorials on
>>> the vm development process in general and about the underlying ideas in the
>>> build process? I'm aware that vm construction is a rather special topic but
>>> any useful reference or hint is welcome.
>>>
>>> I'm already familiar with the interesting [3] and [4] websites and their
>>> authors papers, talks and slides, but these do not seem to help much in
>>> clarifying the development process itself.
>>>
>>> In particular, I'm targeting Pharo 64 bit/Win 64 bit (x64-86). I hope to
>>> be able someday to actively participate in bringing Cog/Spur/Sista to a
>>> more mature status on this platform.
>>>
>>> The questions I would like to find an answer to are of the following
>>> kind:
>>> * Where should I start?
>>>
>>
>> A good start would be to load the Squeak VMMaker dev image (See here:
>> http://www.mirandabanda.org/cogblog/build-image/) and read the different
>> opened workspaces. In addition, ...
>>
>> *Compilation toolchain*
>>
>> To check you have everything installed to compile, you can go to the one
>> of the build folder (One of the windows build folder if you're on windows,
>> for example build.Win64x64, in most case we don't recommend cross
>> compilation), read the HowToBuild file, then go to one of the inner folder
>> and use the mvm script to compile a VM from pre-generated C sources.
>>
>
> See below though, on windows, we target mingw so it's kind of
> cross-compilation anyway.
>
>
>> If it does not work, check on the openSmalltalk-VM repo how the VM is
>> built using Travis CI and set-up your machine similarly to the Travis
>> slaves.
>>
>> *Changes in Slang (GC/Interpreter/JIT/...)*
>>
>> In general when you want to change something in the GC / Interpreter the
>> process is as follow:
>>
>> 1) Make your changes work with the StackVM (interpreter-only) simulator
>> 2) Make your changes work with the compiled StackVM (interpreter-only)
>> 3) Make your changes work with the CogVM simulator
>> 4) Make your changes work with the compiled CogVM
>>
>> In general when you want to change something in the JIT the process is as
>> follow:
>>
>> 1) Make your changes generate correct in-image machine code
>> 2) Make your changes work with the CogVM simulator
>> 3) Make your changes work with the compiled CogVM
>>
>> So a good place to start is to have the simulators working. You could use
>> this to help:
>> https://clementbera.wordpress.com/2016/05/30/simulating-the-cog-vm/
>>
>> Once the simulator works with your changes, you can generate C sources
>> with something like "VMMaker generateConfiguration"
>>
>> *Changes in Platform code*
>>
>> Go to the platform folder at the root and change the C files with a text
>> editor
>>
>>
>>> * Should I use Squaek or can I use Pharo for vm development purposes?
>>>
>>
>> Currently I still use Squeak but I plan to migrate to Pharo. I'd say it's
>> still a bit easier from Squeak due to package compatibility making it hard
>> to commit from Pharo without uncommitting something else, some scripts are
>> also present by default in the Squeak dev image, but everything should work
>> on both environments. No big deal in using one of the other.
>>
>>
>>> * What are the restrictions of Slang?
>>>
>>
>> ...
>> As long as the VM compiles and run, this is correct Slang code, else it's
>> not.
>> In general you should just think in C and write Smalltalk instead.
>> I don't know what to say. This is not dynamic at all. No dictionaries, no
>> look-up, no nothing. Even arrays look like Smalltalk arrays but like C
>> arrays you don't have the actual size.
>> Blocks work as long as they're inlined and removed when performing
>> inlining in the Slang to C compiler.
>>
>> * Where does VMMaker live?
>>>
>>
>> Half there:
>> https://github.com/OpenSmalltalk/opensmalltalk-vm
>> Accessible from git
>>
>> Half there:
>> http://source.squeak.org/VMMaker/
>> Accessible from Monticello
>>
>> * Which C toolchain shall I use?
>>
>> * Which release of gcc, ld, make are known to work?
>>> * What about clang?
>>
>>
>> Depends on the platform. The mvm script normally chooses for you.
>>
>> All VMs are built from Travis CI so you can check how to set up your
>> machine from the Travis set-up.
>>
>> On linux it's gcc by default. All recent gcc versions should work since
>> 4.4
>> On Mac, gcc is not supported anymore, so clang is used. I've used many
>> different versions of clang and they all worked.
>>
>> Specific compiler versions that don't work are mentioned in the
>> HowToBuild file in the platform folders (if any).
>>
>>
>>> * Which version of cygwin64 or mingw64 are OK?
>>>
>>
>> Arf. Both worked at some point, I don't know what the default script use
>> right now I would start with the version used by the mvm script / the
>> Travis build.
>>
>>
> gcc is broken for win64 (or at least not satisfying our expectations about
> stack management)
> So we have to use clang.
> For this reason, we prefer to compile thru cygwin which has prebuilt
> (downlodable) clang modules.
> It's much more difficult to get a working clang in a mingw environment, so
> this brand is not maintained anymore.
>
>
Ah, and there was another grief with mingw...
Currently the VM still uses DirectX-9.
cygwin provides support for this (maybe it's not very future proof now...)
mingw status is unclear, and it seems that recent distributions lack this
support...


But the target is a mingw target, not a cygwin target (we don't want VM to
> depend on cygwin layer, we want to stay minimal).
> So we use a cross compiler. See the line
>
> TOOLPREFIX:=x86_64-w64-mingw32-
>
> in https://github.com/OpenSmalltalk/opensmalltalk-
> vm/blob/Cog/build.win64x64/common/Makefile.tools
>
> Using cross-compiler means that we can eventually build a x86_64 .exe from
> a cygwin32.
> But cygwin32 does not provide a debugger (gdb) for x86_64, that's why we
> prefer developing from a cygwin64 environment.
>
> Eventually, I have a hand-assembled visual studio project for compiling
> the vm with MSVC (2015 if I remember...)
> Of course that kind of project should better be generated (as was possible
> via cmake) rather than hand-assembled...
> It's absolutely cumbersome to maintain such .vc(x)proj across several
> versions of MSVC.
> Anyway, it does not produce working VM yet.
> I retried with 2017 but encountered a code generation bug.
>
> We could try to adapt the gnu makefiles to use MSVC thru command line, but
> IMO the main interest of MSVC is to perform complementary analysis thru IDE.
>
>
>>
>>> * etc.
>>>
>>>
>>>
>>> Greetings
>>> Raffaello
>>>
>>> ---
>>>
>>> [1] http://www.vpri.org/pdf/tr1997001_backto.pdf
>>> [2] http://design.cs.iastate.edu/vmil/2011/papers/p03-miranda.pdf
>>> [3] http://www.mirandabanda.org/cogblog/
>>> [4] https://clementbera.wordpress.com/
>>>
>>
>>
>>
>> --
>> Clément Béra
>> Pharo consortium engineer
>> https://clementbera.wordpress.com/
>> Bâtiment B 40, avenue Halley 59650 Villeneuve d'Ascq
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20171017/f53c8cc4/attachment-0001.html>