[Vm-dev] Overview of the OpenSmalltalk VM and its build process

raffaello.giulietti at lifeware.ch raffaello.giulietti at lifeware.ch
Wed Oct 18 06:11:20 UTC 2017


Hi Clément and Nicolas,

thanks for the useful hints and background information.




On 17/10/17 19:22, Nicolas Cellier wrote:
>  
> 
> 
> 
> 
> 
> 2017-10-17 19:03 GMT+02:00 Nicolas Cellier
> <nicolas.cellier.aka.nice at gmail.com
> <mailto:nicolas.cellier.aka.nice at gmail.com>>:
> 
> 
> 
>     2017-10-17 16:00 GMT+02:00 Clément Bera <bera.clement at gmail.com
>     <mailto:bera.clement at gmail.com>>:
> 
>          
>         Hi Raffaello,
> 
>         Answers inlined...
> 
>         On Tue, Oct 17, 2017 at 3:07 PM, Raffaello Giulietti
>         <raffaello.giulietti at lifeware.ch
>         <mailto:raffaello.giulietti at lifeware.ch>> wrote:
> 
> 
>             Hi,
> 
>             I'm interested in understanding the overall OpenSmalltalk VM
>             structure and design and its build process.
> 
>             I read [1] and [2] to get a high level picture of the vm and
>             its interesting strategy to development (Slang, VMMaker, etc.)
> 
>             Apart from these, are there other useful
>             documents/sites/tutorials on the vm development process in
>             general and about the underlying ideas in the build process?
>             I'm aware that vm construction is a rather special topic but
>             any useful reference or hint is welcome.
> 
>             I'm already familiar with the interesting [3] and [4]
>             websites and their authors papers, talks and slides, but
>             these do not seem to help much in clarifying the development
>             process itself.
> 
>             In particular, I'm targeting Pharo 64 bit/Win 64 bit
>             (x64-86). I hope to be able someday to actively participate
>             in bringing Cog/Spur/Sista to a more mature status on this
>             platform.
> 
>             The questions I would like to find an answer to are of the
>             following kind:
>             * Where should I start?
> 
> 
>         A good start would be to load the Squeak VMMaker dev image (See
>         here: http://www.mirandabanda.org/cogblog/build-image/
>         <http://www.mirandabanda.org/cogblog/build-image/>) and read the
>         different opened workspaces. In addition, ... 
> 
>         *Compilation toolchain*
> 
>         To check you have everything installed to compile, you can go to
>         the one of the build folder (One of the windows build folder if
>         you're on windows, for example build.Win64x64, in most case we
>         don't recommend cross compilation), read the HowToBuild file,
>         then go to one of the inner folder and use the mvm script to
>         compile a VM from pre-generated C sources.
> 
> 
>     See below though, on windows, we target mingw so it's kind of
>     cross-compilation anyway.
> 
> 
>         If it does not work, check on the openSmalltalk-VM repo how the
>         VM is built using Travis CI and set-up your machine similarly to
>         the Travis slaves.
> 
>         *Changes in Slang (GC/Interpreter/JIT/...)*
> 
>         In general when you want to change something in the GC /
>         Interpreter the process is as follow:
> 
>         1) Make your changes work with the StackVM (interpreter-only)
>         simulator
>         2) Make your changes work with the compiled StackVM
>         (interpreter-only) 
>         3) Make your changes work with the CogVM simulator
>         4) Make your changes work with the compiled CogVM
> 
>         In general when you want to change something in the JIT the
>         process is as follow:
> 
>         1) Make your changes generate correct in-image machine code
>         2) Make your changes work with the CogVM simulator
>         3) Make your changes work with the compiled CogVM
> 
>         So a good place to start is to have the simulators working. You
>         could use this to help:
>         https://clementbera.wordpress.com/2016/05/30/simulating-the-cog-vm/
>         <https://clementbera.wordpress.com/2016/05/30/simulating-the-cog-vm/>
> 
>         Once the simulator works with your changes, you can generate C
>         sources with something like "VMMaker generateConfiguration"
> 
>         *Changes in Platform code*
>         *
>         *
>         Go to the platform folder at the root and change the C files
>         with a text editor
>          
> 
>             * Should I use Squaek or can I use Pharo for vm development
>             purposes?
> 
> 
>         Currently I still use Squeak but I plan to migrate to Pharo. I'd
>         say it's still a bit easier from Squeak due to package
>         compatibility making it hard to commit from Pharo without
>         uncommitting something else, some scripts are also present by
>         default in the Squeak dev image, but everything should work on
>         both environments. No big deal in using one of the other.
>          
> 
>             * What are the restrictions of Slang?
> 
> 
>         ...
>         As long as the VM compiles and run, this is correct Slang code,
>         else it's not.
>         In general you should just think in C and write Smalltalk instead. 
>         I don't know what to say. This is not dynamic at all. No
>         dictionaries, no look-up, no nothing. Even arrays look like
>         Smalltalk arrays but like C arrays you don't have the actual size.
>         Blocks work as long as they're inlined and removed when
>         performing inlining in the Slang to C compiler.
> 
>             * Where does VMMaker live?
> 
> 
>         Half there:
>         https://github.com/OpenSmalltalk/opensmalltalk-vm
>         <https://github.com/OpenSmalltalk/opensmalltalk-vm>
>         Accessible from git
> 
>         Half there:
>         http://source.squeak.org/VMMaker/
>         <http://source.squeak.org/VMMaker/>
>         Accessible from Monticello
> 
>             * Which C toolchain shall I use? 
> 
>             * Which release of gcc, ld, make are known to work?
>             * What about clang?
> 
> 
>         Depends on the platform. The mvm script normally chooses for you. 
> 
>         All VMs are built from Travis CI so you can check how to set up
>         your machine from the Travis set-up.
> 
>         On linux it's gcc by default. All recent gcc versions should
>         work since 4.4
>         On Mac, gcc is not supported anymore, so clang is used. I've
>         used many different versions of clang and they all worked.
> 
>         Specific compiler versions that don't work are mentioned in the
>         HowToBuild file in the platform folders (if any).
>          
> 
>             * Which version of cygwin64 or mingw64 are OK?
> 
> 
>         Arf. Both worked at some point, I don't know what the default
>         script use right now I would start with the version used by the
>         mvm script / the Travis build.
> 
> 
>     gcc is broken for win64 (or at least not satisfying our expectations
>     about stack management)
>     So we have to use clang.
>     For this reason, we prefer to compile thru cygwin which has prebuilt
>     (downlodable) clang modules.
>     It's much more difficult to get a working clang in a mingw
>     environment, so this brand is not maintained anymore.
> 
> 
> Ah, and there was another grief with mingw...
> Currently the VM still uses DirectX-9.
> cygwin provides support for this (maybe it's not very future proof now...)
> mingw status is unclear, and it seems that recent distributions lack
> this support...
> 
> 
>     But the target is a mingw target, not a cygwin target (we don't want
>     VM to depend on cygwin layer, we want to stay minimal).
>     So we use a cross compiler. See the line
> 
>     TOOLPREFIX:=x86_64-w64-mingw32-
> 
>     in
>     https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/build.win64x64/common/Makefile.tools
>     <https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/build.win64x64/common/Makefile.tools>
> 
>     Using cross-compiler means that we can eventually build a x86_64
>     .exe from a cygwin32.
>     But cygwin32 does not provide a debugger (gdb) for x86_64, that's
>     why we prefer developing from a cygwin64 environment.
> 
>     Eventually, I have a hand-assembled visual studio project for
>     compiling the vm with MSVC (2015 if I remember...)
>     Of course that kind of project should better be generated (as was
>     possible via cmake) rather than hand-assembled...
>     It's absolutely cumbersome to maintain such .vc(x)proj across
>     several versions of MSVC.
>     Anyway, it does not produce working VM yet.
>     I retried with 2017 but encountered a code generation bug.
> 
>     We could try to adapt the gnu makefiles to use MSVC thru command
>     line, but IMO the main interest of MSVC is to perform complementary
>     analysis thru IDE.
> 
>          
> 
>             * etc.
> 
> 
> 
>             Greetings
>             Raffaello
> 
>             ---
> 
>             [1] http://www.vpri.org/pdf/tr1997001_backto.pdf
>             <http://www.vpri.org/pdf/tr1997001_backto.pdf>
>             [2]
>             http://design.cs.iastate.edu/vmil/2011/papers/p03-miranda.pdf <http://design.cs.iastate.edu/vmil/2011/papers/p03-miranda.pdf>
>             [3] http://www.mirandabanda.org/cogblog/
>             <http://www.mirandabanda.org/cogblog/>
>             [4] https://clementbera.wordpress.com/
>             <https://clementbera.wordpress.com/>
> 
> 
> 
> 
>         -- 
>         Clément Béra
>         Pharo consortium engineer
>         https://clementbera.wordpress.com/
>         <https://clementbera.wordpress.com/>
>         Bâtiment B 40, avenue Halley 59650 Villeneuve d'Ascq
> 
> 
> 



More information about the Vm-dev mailing list