Hello,
I am working on removing most of windowing code from the VM, and in trying to unify the platform specific code of the VM. In the MinimalistHeadless branch of https://github.com/ronsaldo/opensmalltalk-vm I made the following changes:
- Unified standard CMake building scripts for Unixes. I hate autoconf. I want to use them in Windows too. - Refactoring the Unix entry points. I am trying to remove a lot of code for simplfying stuff. - Null window driver for true headless. - Optional SDL2 based traditional display backend for compatibility reasons, and because the extra Morphic worlds using OSWindow are a bit unstable.
So far I managed to run a standard Pharo 6 image using this custom VM in Linux. Windows and Mac are coming next. Hopefully this is going to fix the problems with duplicated events when using OSWindow in Windows.
I am also planning on making a standard interface for embedding the VM in an application, at least as a static library. With this new building system, this seems to be easy to do.
BTW. When I tried the minimalistic Pharo image, in a completely headless VM, I got the following error:
[ "Ugh .... now this is a biggie - a system that does not support any of the display depths at all." Smalltalk logError: 'Fatal error: This system has no support for any display depth at all.' inContext: thisContext. Smalltalk quitPrimitive "There is no way to continue from here" ] in DisplayScreen>>findAnyDisplayDepth DisplayScreen>>findAnyDisplayDepthIfNone: DisplayScreen>>findAnyDisplayDepth DisplayScreen>>setExtent:depth: DisplayScreen class>>startUp
As a workaround, I am doing the following for ioHasDisplayDepth with the null driver:
sqInt ioHasDisplayDepth(sqInt depth) { return true; }
In DisplayScreen class >> startUp we have the following: startUp "DisplayScreen startUp" Display setExtent: self actualScreenSize depth: Display nativeDepth. Display beDisplay
Does it make sense, to always trying to create a display in startup time?
Best regards, Ronie
Hi Ronie,
First, this is super cool! I want this to happen since a lot of time :) Second, observation below
On 24 Nov 2016, at 08:12, Ronie Salgado roniesalg@gmail.com wrote:
Hello,
I am working on removing most of windowing code from the VM, and in trying to unify the platform specific code of the VM. In the MinimalistHeadless branch of https://github.com/ronsaldo/opensmalltalk-vm https://github.com/ronsaldo/opensmalltalk-vm I made the following changes:
- Unified standard CMake building scripts for Unixes. I hate autoconf. I want to use them in Windows too.
you cannot unilaterally do this. I hate autoconf too, but in VM list we have a general agreement on how to manage the VM build process, and unless you manage to convince the rest of people working there (and specially Eliot, who works there more than the rest), you have literally zero chance this will be adopted (and then, your work will be a general lose of time…). In any case, you have my vote :)
- Refactoring the Unix entry points. I am trying to remove a lot of code for simplfying stuff.
- Null window driver for true headless.
- Optional SDL2 based traditional display backend for compatibility reasons, and because the extra Morphic worlds using OSWindow are a bit unstable.
So far I managed to run a standard Pharo 6 image using this custom VM in Linux. Windows and Mac are coming next. Hopefully this is going to fix the problems with duplicated events when using OSWindow in Windows.
this is sooo super :)
I am also planning on making a standard interface for embedding the VM in an application, at least as a static library. With this new building system, this seems to be easy to do.
even more super :)
where can I see the code?
Esteban
BTW. When I tried the minimalistic Pharo image, in a completely headless VM, I got the following error:
[ "Ugh .... now this is a biggie - a system that does not support any of the display depths at all." Smalltalk logError: 'Fatal error: This system has no support for any display depth at all.' inContext: thisContext. Smalltalk quitPrimitive "There is no way to continue from here" ] in DisplayScreen>>findAnyDisplayDepth DisplayScreen>>findAnyDisplayDepthIfNone: DisplayScreen>>findAnyDisplayDepth DisplayScreen>>setExtent:depth: DisplayScreen class>>startUp
As a workaround, I am doing the following for ioHasDisplayDepth with the null driver:
sqInt ioHasDisplayDepth(sqInt depth) { return true; }
In DisplayScreen class >> startUp we have the following: startUp "DisplayScreen startUp" Display setExtent: self actualScreenSize depth: Display nativeDepth. Display beDisplay
Does it make sense, to always trying to create a display in startup time?
Best regards, Ronie
On 24 Nov 2016, at 08:24, Esteban Lorenzano estebanlm@gmail.com wrote:
where can I see the code?
forget it, I was so overblown with your ANN that I just skip the link to your branch :P
Esteban
+10000000000000000000000000000000
amen to that.
especially this:
I am also planning on making a standard interface for embedding the VM in
an application, at least as a static library. With this new building system, this seems to be easy to do.
This is a holy grail!!! Go, Ronie, Go!!!
On 24 November 2016 at 09:24, Esteban Lorenzano estebanlm@gmail.com wrote:
Hi Ronie,
First, this is super cool! I want this to happen since a lot of time :) Second, observation below
On 24 Nov 2016, at 08:12, Ronie Salgado roniesalg@gmail.com wrote:
Hello,
I am working on removing most of windowing code from the VM, and in trying to unify the platform specific code of the VM. In the MinimalistHeadless branch of https://github.com/ronsaldo/opensmalltalk-vm I made the following changes:
- Unified standard CMake building scripts for Unixes. I hate autoconf. I
want to use them in Windows too.
you cannot unilaterally do this. I hate autoconf too, but in VM list we have a general agreement on how to manage the VM build process, and unless you manage to convince the rest of people working there (and specially Eliot, who works there more than the rest), you have literally zero chance this will be adopted (and then, your work will be a general lose of time…). In any case, you have my vote :)
- Refactoring the Unix entry points. I am trying to remove a lot of code
for simplfying stuff.
- Null window driver for true headless.
- Optional SDL2 based traditional display backend for compatibility
reasons, and because the extra Morphic worlds using OSWindow are a bit unstable.
So far I managed to run a standard Pharo 6 image using this custom VM in Linux. Windows and Mac are coming next. Hopefully this is going to fix the problems with duplicated events when using OSWindow in Windows.
this is sooo super :)
I am also planning on making a standard interface for embedding the VM in an application, at least as a static library. With this new building system, this seems to be easy to do.
even more super :)
where can I see the code?
Esteban
BTW. When I tried the minimalistic Pharo image, in a completely headless VM, I got the following error:
[ "Ugh .... now this is a biggie - a system that does not support any of the display depths at all." Smalltalk logError: 'Fatal error: This system has no support for any display depth at all.' inContext: thisContext. Smalltalk quitPrimitive "There is no way to continue from here" ] in DisplayScreen>>findAnyDisplayDepth DisplayScreen>>findAnyDisplayDepthIfNone: DisplayScreen>>findAnyDisplayDepth DisplayScreen>>setExtent:depth: DisplayScreen class>>startUp
As a workaround, I am doing the following for ioHasDisplayDepth with the null driver:
sqInt ioHasDisplayDepth(sqInt depth) { return true; }
In DisplayScreen class >> startUp we have the following: startUp "DisplayScreen startUp" Display setExtent: self actualScreenSize depth: Display nativeDepth. Display beDisplay
Does it make sense, to always trying to create a display in startup time?
Best regards, Ronie
Hi
2016-11-24 8:12 GMT+01:00 Ronie Salgado roniesalg@gmail.com:
Hello,
I am working on removing most of windowing code from the VM, and in trying to unify the platform specific code of the VM. In the MinimalistHeadless branch of https://github.com/ronsaldo/opensmalltalk-vm I made the following changes:
This is super cool. With Pavel we just tried open oswindow based world from headless image and it work almost perfect. No problems with keyboard and scrolling anymore. And it already provides very good approach for deploying desktop app.
- Unified standard CMake building scripts for Unixes. I hate autoconf. I
want to use them in Windows too.
- Refactoring the Unix entry points. I am trying to remove a lot of code
for simplfying stuff.
- Null window driver for true headless.
- Optional SDL2 based traditional display backend for compatibility
reasons, and because the extra Morphic worlds using OSWindow are a bit unstable.
Could you explain more what traditional display backend means? Because I am probably out of domain but want to know more.
So far I managed to run a standard Pharo 6 image using this custom VM in Linux. Windows and Mac are coming next. Hopefully this is going to fix the problems with duplicated events when using OSWindow in Windows.
I am also planning on making a standard interface for embedding the VM in an application, at least as a static library. With this new building system, this seems to be easy to do.
BTW. When I tried the minimalistic Pharo image, in a completely headless VM, I got the following error:
[ "Ugh .... now this is a biggie - a system that does not support any of the display depths at all." Smalltalk logError: 'Fatal error: This system has no support for any display depth at all.' inContext: thisContext. Smalltalk quitPrimitive "There is no way to continue from here" ] in DisplayScreen>>findAnyDisplayDepth DisplayScreen>>findAnyDisplayDepthIfNone: DisplayScreen>>findAnyDisplayDepth DisplayScreen>>setExtent:depth: DisplayScreen class>>startUp
As a workaround, I am doing the following for ioHasDisplayDepth with the null driver:
sqInt ioHasDisplayDepth(sqInt depth) { return true; }
I guess it is temp solution? Because no DisplayScreen should exist in headless mode.
you cannot unilaterally do this. I hate autoconf too, but in VM list we have a general agreement on how to manage the VM build process, and unless you manage to convince the rest of people working there (and specially Eliot, who works there more than the rest), you have literally zero chance this will be adopted (and then, your work will be a general lose of time…). In any case, you have my vote :)
This was something that I did for quickly testing and trying to remove the windowing code. I also did this, because the building system is not working all of the time in my linux machine. The scripts for loading the VMMaker image are most of the time not working at all in Linux. I got frustrated by not being able on keep working on Lowcode for supporting the 64 bits version, because of these problems.
Anyway, I do not care if this part gets integrated or not, as long as I can keep working on my stuff, and I don't have a bunch of manully written scripts in build.*/* . As long as there is a single script that can be configured for each target or flavour of the VM I am okay with it.
Now I will document these scripts:
For setting the flavour of the VM, there are the following options - SPUR_OBJECT_MODEL (On by default) - SISTA_OPTIMIZER (Off by default) - PHARO_BRANDING (On by default) - COG_JIT(On by default. If off, it will build a stack interpreter only VM)
There are platform specific check for choosing whether to build a 64 bits VM or not. When building in a x86_64 systems, there is the BUILD_I386_VERSION option (Off by default) which will set the appropiate flags for building a 32 bits VM in a 64 bits system (-m32)
These options are used for making a string with the vm sources direction(src ,stacksrc, spursrc, etc) and for selecting the sources of the vm. In this experimental branch I am moving some of the sources at platform/<platform_name/vm into the minheadless/source, for including the barest minimum code for compiling the VM. By the end of this, I should probably move back the platform specific sources into their original locations.
The plugins to build with the VM are defined in Plugins.cmake ( https://github.com/ronsaldo/opensmalltalk-vm/blob/MinimalisticHeadless/Plugi... )
Most of the time, adding a plugin is just a matter calling a macro:
# Basic internal plugins add_vm_plugin_auto(FilePlugin INTERNAL) add_vm_plugin_auto(LargeIntegers INTERNAL) add_vm_plugin_auto(LocalePlugin INTERNAL) add_vm_plugin_auto(MiscPrimitivePlugin INTERNAL) add_vm_plugin_auto(SecurityPlugin INTERNAL) add_vm_plugin_auto(SocketPlugin INTERNAL)
add_vm_plugin_auto(B2DPlugin INTERNAL) add_vm_plugin_auto(BitBltPlugin INTERNAL)
add_vm_plugin_auto(FloatArrayPlugin INTERNAL) add_vm_plugin_auto(FloatMathPlugin INTERNAL) add_vm_plugin_auto(Matrix2x3Plugin INTERNAL)
# Basic external plugins add_vm_plugin_auto(SurfacePlugin EXTERNAL)
For more selective plugins, such as the SqueakFFIPrims plugin there is the add_vm_plugin_sources macro:
add_vm_plugin_sources(SqueakFFIPrims EXTERNAL ${SqueakFFIPrims_Sources})
Later I will add another macro for linking a library with a plugin, which will perform the correct action depending whether the Plugin is internal or external. The macros for building a plugin add an option for specifying whether to build or not a plugin (e.g: BUILD_PLUGIN_FilePlugin, BUILD_PLUGIN_FT2Plugin).
For building the a default VM using cmake, the only commands are required in the top level source directory:
mkdir build_dir cd build_dir cmake .. make
For choosing interactively which options to enable or not, it is possible to use cmake-gui instead of cmake in the above sequence of commands. The built VM will be placed at build_dir/dist
Currently, I am not using the cmake configure capabilties for generating config.h and generating the HAVE_*_H defines. Later I will implement this, but taking special care in supporting the cross compilation scenario, where it is not possible to execute compiled program for probing the system for things such as sizeof(void*).
Could you explain more what traditional display backend means? Because I am
probably out of domain but want to know more.
If the SUPPORT_TRADITIONAL_DISPLAY cmake option is enabled, and cmake finds SDL2 (it will look for the SDL2 libraries using the names SDL2-2.0 SDL2), then a non-headless VM will be built using sqPlatformSpecific-SDL2Window.c ( https://github.com/ronsaldo/opensmalltalk-vm/blob/MinimalisticHeadless/minhe... ) instead of sqPlatformSpecific-NullWindow.c ( https://github.com/ronsaldo/opensmalltalk-vm/blob/MinimalisticHeadless/minhe... ).
This non-headless VM using SDL2 instead of the platform specific API, which should help in avoiding conflicts between the OSWindow event loop and the VM event loop. This works with a normal Pharo 6 image.
This SDL2 based backend creates a window for the VM, only when asked by the image and process events sent to the VM window in the traditional way of translating the events into squeak events; placing the translated event in a queue, and then signaling a semaphore. SDL2 events that are not related to the VM window are placed untranslated in a separate queue, and there are used signal a separate semaphore for allowing OSWindow to process the event. I have already implemented the required primitives and updated OSWindow for using them, when they exist. I still have to update the OSWindow configuration. This approach allows preventing conflicts between the events for the VM window and OSWindow.
Ronie, do you already have code to restore windows after image save? Any
issue opened?
Not yet for the extra morphic Worlds.
How much VM size it reduced?
I have not checked yet, but this is about reducing some complexity.
How dare you ?
Yesterday 03:00 am I finally managed to make my shared memory bridge between Pharo and C++ to work also on Windows after a ton of debugging and here comes you telling me that all my effort was for nothing. It now works and is tested on MacOS 10.12 , Ubuntu 15.04 and Windows 10 (all of them 64 bit).
That is not worthless at all. That is still an excellent mechanism for
fast IPC.
Have you got no mercy ?
No :) . Ask Thibault about the OpenGL bindings for the UFFI. That time I was cleaning OSWindow dependencies.
Suffice to say I am super interested in both Non GUI Pharo because I use
Unreal Game Engine and embedding Pharo because I use Unreal Game Engine.
Embedding Pharo is a must have for me because my shared memory solution would not be acceptable by Apple on iOS since it forbids an app from running two executable at the same time which is basic requirement for shared memory. Stupid Apple :/ . Android seems to be fine with this.
For now my game will target only PC where my shared memory solution works fine but sooner or later I will have to port to iOS and Android.
If there is anything I can do to help you out, dont hesitate to ask. I am going to give your build a try now :).
It is still needed to define a proper interface for embedding the VM. This interface should be a single .h file, with the highest level of abstraction possible. There are some other issues such as potential name clashes, and the fact that the VM symbols are public by default. At the bare minimum, it has to provide functions for initializing the VM, passing command line arguments, loading an image, running the image, and shutting down.
So far I managed to reduce main source code a lot in sqMain.c ( https://github.com/ronsaldo/opensmalltalk-vm/blob/MinimalisticHeadless/minhe... ) . I am probably moving most of that into a separate source code file that will be always linked with VM core library, and keeping the source file with main function only couple of lines long (Init vm, load image, interpret and shutdown )
BTW, now I am working for a chilean video game company using Unreal. I do not like Unreal because it takes a very long time to compile, even with a 44 cores machine (88 threads) that we have in the company for rendering and compiling. I am wondering how that integration is going to end.
The embedding interface is not in my priority list. My priority is getting OSWindow working well in Windows (required by the Bloc and Roassal people). My second priority is getting Lowcode working on Linux, Mac and Windows in both, 32-bits and 64 bits mode.
Best regards, Ronie
2016-11-24 5:59 GMT-03:00 Denis Kudriashov dionisiydk@gmail.com:
Hi
2016-11-24 8:12 GMT+01:00 Ronie Salgado roniesalg@gmail.com:
Hello,
I am working on removing most of windowing code from the VM, and in trying to unify the platform specific code of the VM. In the MinimalistHeadless branch of https://github.com/ronsaldo/opensmalltalk-vm I made the following changes:
This is super cool. With Pavel we just tried open oswindow based world from headless image and it work almost perfect. No problems with keyboard and scrolling anymore. And it already provides very good approach for deploying desktop app.
- Unified standard CMake building scripts for Unixes. I hate autoconf. I
want to use them in Windows too.
- Refactoring the Unix entry points. I am trying to remove a lot of code
for simplfying stuff.
- Null window driver for true headless.
- Optional SDL2 based traditional display backend for compatibility
reasons, and because the extra Morphic worlds using OSWindow are a bit unstable.
Could you explain more what traditional display backend means? Because I am probably out of domain but want to know more.
So far I managed to run a standard Pharo 6 image using this custom VM in Linux. Windows and Mac are coming next. Hopefully this is going to fix the problems with duplicated events when using OSWindow in Windows.
I am also planning on making a standard interface for embedding the VM in an application, at least as a static library. With this new building system, this seems to be easy to do.
BTW. When I tried the minimalistic Pharo image, in a completely headless VM, I got the following error:
[ "Ugh .... now this is a biggie - a system that does not support any of the display depths at all." Smalltalk logError: 'Fatal error: This system has no support for any display depth at all.' inContext: thisContext. Smalltalk quitPrimitive "There is no way to continue from here" ] in DisplayScreen>>findAnyDisplayDepth DisplayScreen>>findAnyDisplayDepthIfNone: DisplayScreen>>findAnyDisplayDepth DisplayScreen>>setExtent:depth: DisplayScreen class>>startUp
As a workaround, I am doing the following for ioHasDisplayDepth with the null driver:
sqInt ioHasDisplayDepth(sqInt depth) { return true; }
I guess it is temp solution? Because no DisplayScreen should exist in headless mode.
That is not worthless at all. That is still an excellent mechanism for fast
IPC.
Indeed it is, and it works now, so no complains, plus I learned a ton in
the process about UFFI, C++, Pharo and much more.
It is still needed to define a proper interface for embedding the VM. This interface should be a single .h file, with the highest level of abstraction possible. There are some other issues such as potential name clashes, and the fact that the VM symbols are public by default. At the bare minimum, it has to provide functions for initializing the VM, passing command line arguments, loading an image, running the image, and shutting down.
So far I managed to reduce main source code a lot in sqMain.c ( https://github.com/ronsaldo/opensmalltalk-vm/blob/MinimalisticHeadless/minhe... ) . I am probably moving most of that into a separate source code file that will be always linked with VM core library, and keeping the source file with main function only couple of lines long (Init vm, load image, interpret and shutdown )
Will give it a look and send you pull requests
BTW, now I am working for a chilean video game company using Unreal. I do not like Unreal because it takes a very long time to compile, even with a 44 cores machine (88 threads) that we have in the company for rendering and compiling. I am wondering how that integration is going to end.
Yeah why people do that ?
It took me a day to find a way to reduce compile times down to zero seconds and basically turn C++ to a live coding language following the advice of this brilliant C coder here
https://www.youtube.com/watch?v=WMSBRk5WG58
The idea is simple , generate an executable with a basic loop that the only things it does is just gives access to its memory via pointers, detects changes in code and auto compiles it to dynamically linked libraries that loads of course dynamically. Because it compiles only the dynamic library , compile times are close to zero seconds and also compiles happen in the background so you dont have to even wait for the compile to finish if a couple seconds compile times is too much.
The executable never stops executing , its just the dynamic libraries that modify and recompile.
This is very similar to what Unreal editor uses for hot reloading.
So its pretty easy to do this with your own unreal projects and eliminate the need for dealing with compile time and even avoid using a scripting language like pharo altogether if you prefer working only on C++.
I am going to use this technique because live coding in pharo for unreal wont be enough I will need to live code also in C++ because some of the code will have to be written on C++ for performance reason and compile times for than 2-3 seconds are unacceptable for my standards.
I did a test for this with C++ you can find the code here
https://github.com/kilon/LiveCPP
Worked like a charm.
But have not tested with Unreal yet because I was focused on my shared memory bridge. However I see no reason why this would not work well. It should work even for static compiles because you can static compile dynamic libraries.
there is also this library
https://github.com/RuntimeCompiledCPlusPlus/RuntimeCompiledCPlusPlus
which takes live coding C++ to a whole new level that I may integrate or at least use as inspiration.
I am not giving up on live coding any time soon this millennium :D.
On Thu, Nov 24, 2016 at 10:06 PM, Ronie Salgado roniesalg@gmail.com wrote:
It is still needed to define a proper interface for embedding the VM. This interface should be a single .h file, with the highest level of abstraction possible. There are some other issues such as potential name clashes, and the fact that the VM symbols are public by default. At the bare minimum, it has to provide functions for initializing the VM, passing command line arguments, loading an image, running the image, and shutting down.
I don't know if its really suitable, but since Lua is the incumbent embedded gaming language, perhaps echoing it would make it easier to substitute ours for theirs. https://www.lua.org/pil/24.2.html
cheers -ben
On 29 November 2016 at 14:43, Ben Coman btc@openinworld.com wrote:
On Thu, Nov 24, 2016 at 10:06 PM, Ronie Salgado roniesalg@gmail.com wrote:
It is still needed to define a proper interface for embedding the VM. This interface should be a single .h file, with the highest level of abstraction possible. There are some other issues such as potential name clashes, and the fact that the VM symbols are public by default. At the bare minimum, it has to provide functions for initializing the VM, passing command line arguments, loading an image, running the image, and shutting down.
I don't know if its really suitable, but since Lua is the incumbent embedded gaming language, perhaps echoing it would make it easier to substitute ours for theirs. https://www.lua.org/pil/24.2.html
cheers -ben
well, the problem is that we're in a bit more difficult situation that Lua.. in Lua there's no processes/contexts.. and if we would want implement an external message send i.e. send message - get result, things not so straightforward as in Lua. it's all because it was never designed to be run in a host environment and be open from that end. that's why it is much easier to implement FFI, that C -> smalltalk API.
well, the problem is that we're in a bit more difficult situation that Lua.. in Lua there's no processes/contexts.. and if we would want implement an external message send i.e. send message
- get result, things not so straightforward as in Lua.
it's all because it was never designed to be run in a host environment and be open from that end. that's why it is much easier to implement FFI, that C -> smalltalk API.
Currently, I am going to focus on making a standard interface for initializing the VM, loading the image, passing the command line arguments, running the interpreter and shutting down. This seems easy to do, and it allows me to unify the platform specific versions of the VM.
For more complicated stuffs such as calling Smalltalk from C, or a standard interface for defining primitives in C( for doing things such as moving unessential VM plugins outside of the main VM source code tree). We should have a proper discussion on the mailing list.
My priority with removing the windowing code is getting OSWindow working well in Windows. Currently it is having conflicts with the main loop provided by the VM, which makes it unusable.
2016-11-29 14:22 GMT-03:00 Igor Stasenko siguctua@gmail.com:
On 29 November 2016 at 14:43, Ben Coman btc@openinworld.com wrote:
On Thu, Nov 24, 2016 at 10:06 PM, Ronie Salgado roniesalg@gmail.com wrote:
It is still needed to define a proper interface for embedding the VM. This interface should be a single .h file, with the highest level of abstraction possible. There are some other issues such as potential name clashes, and the fact that the VM symbols are public by default. At the bare minimum, it has to provide functions for initializing the VM, passing command line arguments, loading an image, running the image, and shutting down.
I don't know if its really suitable, but since Lua is the incumbent embedded gaming language, perhaps echoing it would make it easier to substitute ours for theirs. https://www.lua.org/pil/24.2.html
cheers -ben
well, the problem is that we're in a bit more difficult situation that Lua.. in Lua there's no processes/contexts.. and if we would want implement an external message send i.e. send message
- get result, things not so straightforward as in Lua.
it's all because it was never designed to be run in a host environment and be open from that end. that's why it is much easier to implement FFI, that C -> smalltalk API.
-- Best regards, Igor Stasenko.
On 29 November 2016 at 19:30, Ronie Salgado roniesalg@gmail.com wrote:
well, the problem is that we're in a bit more difficult situation that Lua.. in Lua there's no processes/contexts.. and if we would want implement an external message send i.e. send message - get result, things not so straightforward as in Lua. it's all because it was never designed to be run in a host environment and be open from that end. that's why it is much easier to implement FFI, that C -> smalltalk API.
Currently, I am going to focus on making a standard interface for initializing the VM, loading the image, passing the command line arguments, running the interpreter and shutting down. This seems easy to do, and it allows me to unify the platform specific versions of the VM.
For more complicated stuffs such as calling Smalltalk from C, or a standard interface for defining primitives in C( for doing things such as moving unessential VM plugins outside of the main VM source code tree). We should have a proper discussion on the mailing list.
My priority with removing the windowing code is getting OSWindow working well in Windows. Currently it is having conflicts with the main loop provided by the VM, which makes it unusable.
right, this main loop concept is a first thing, that goes in a conflict
with host application, that wants to control what it wants to run or not and when.
2016-11-29 14:22 GMT-03:00 Igor Stasenko siguctua@gmail.com:
On 29 November 2016 at 14:43, Ben Coman btc@openinworld.com wrote:
On Thu, Nov 24, 2016 at 10:06 PM, Ronie Salgado roniesalg@gmail.com wrote:
It is still needed to define a proper interface for embedding the VM. This interface should be a single .h file, with the highest level of abstraction possible. There are some other issues such as potential name clashes, and the fact that the VM symbols are public by default. At the bare minimum, it has to provide functions for initializing the VM, passing command line arguments, loading an image, running the image, and shutting down.
I don't know if its really suitable, but since Lua is the incumbent embedded gaming language, perhaps echoing it would make it easier to substitute ours for theirs. https://www.lua.org/pil/24.2.html
cheers -ben
well, the problem is that we're in a bit more difficult situation that Lua.. in Lua there's no processes/contexts.. and if we would want implement an external message send i.e. send message - get result, things not so straightforward as in Lua. it's all because it was never designed to be run in a host environment and be open from that end. that's why it is much easier to implement FFI, that C -> smalltalk API.
-- Best regards, Igor Stasenko.
On 29 November 2016 at 19:42, Igor Stasenko siguctua@gmail.com wrote:
On 29 November 2016 at 19:30, Ronie Salgado roniesalg@gmail.com wrote:
well, the problem is that we're in a bit more difficult situation that Lua.. in Lua there's no processes/contexts.. and if we would want implement an external message send i.e. send message - get result, things not so straightforward as in Lua. it's all because it was never designed to be run in a host environment and be open from that end. that's why it is much easier to implement FFI, that C -> smalltalk API.
Currently, I am going to focus on making a standard interface for initializing the VM, loading the image, passing the command line arguments, running the interpreter and shutting down. This seems easy to do, and it allows me to unify the platform specific versions of the VM.
For more complicated stuffs such as calling Smalltalk from C, or a standard interface for defining primitives in C( for doing things such as moving unessential VM plugins outside of the main VM source code tree). We should have a proper discussion on the mailing list.
My priority with removing the windowing code is getting OSWindow working well in Windows. Currently it is having conflicts with the main loop provided by the VM, which makes it unusable.
right, this main loop concept is a first thing, that goes in a conflict
with host application, that wants to control what it wants to run or not and when.
that's why, for instance i was pursuing the idea of implementing process scheduling in a language itself, leaving very small part of required logic on VM side - mostly stripping a logic that switches process(es) to specially registered 'interrupt' process, when signal from semaphore arrived, while leaving the rest of logic on hands of image-side implementation.
On Wed, Nov 30, 2016 at 1:49 AM, Igor Stasenko siguctua@gmail.com wrote:
On 29 November 2016 at 19:42, Igor Stasenko siguctua@gmail.com wrote:
On 29 November 2016 at 19:30, Ronie Salgado roniesalg@gmail.com wrote:
well, the problem is that we're in a bit more difficult situation that Lua.. in Lua there's no processes/contexts.. and if we would want implement an external message send i.e. send message - get result, things not so straightforward as in Lua. it's all because it was never designed to be run in a host environment and be open from that end. that's why it is much easier to implement FFI, that C -> smalltalk API.
okay. thanks for the insight.
Currently, I am going to focus on making a standard interface for initializing the VM, loading the image, passing the command line arguments, running the interpreter and shutting down. This seems easy to do, and it allows me to unify the platform specific versions of the VM.
For more complicated stuffs such as calling Smalltalk from C, or a standard interface for defining primitives in C( for doing things such as moving unessential VM plugins outside of the main VM source code tree). We should have a proper discussion on the mailing list.
My priority with removing the windowing code is getting OSWindow working well in Windows. Currently it is having conflicts with the main loop provided by the VM, which makes it unusable.
right, this main loop concept is a first thing, that goes in a conflict with host application, that wants to control what it wants to run or not and when.
that's why, for instance i was pursuing the idea of implementing process scheduling in a language itself, leaving very small part of required logic on VM side
- mostly stripping a logic that switches process(es) to specially registered 'interrupt' process, when signal from semaphore arrived, while leaving the rest of logic on hands
of image-side implementation.
Would it make sense to mix that in with DelayScheduler running at priority 80? Use the same Semaphore that is already signalled from the VM.
cheers -ben
Whats so special about pharo context and process and how is that related to embedding pharo ?
On Tue, Nov 29, 2016 at 7:22 PM Igor Stasenko siguctua@gmail.com wrote:
On 29 November 2016 at 14:43, Ben Coman btc@openinworld.com wrote:
On Thu, Nov 24, 2016 at 10:06 PM, Ronie Salgado roniesalg@gmail.com wrote:
It is still needed to define a proper interface for embedding the VM. This interface should be a single .h file, with the highest level of abstraction possible. There are some other issues such as potential name clashes, and the fact that the VM symbols are public by default. At the bare minimum, it has to provide functions for initializing the VM, passing command line arguments, loading an image, running the image, and shutting down.
I don't know if its really suitable, but since Lua is the incumbent embedded gaming language, perhaps echoing it would make it easier to substitute ours for theirs. https://www.lua.org/pil/24.2.html
cheers -ben
well, the problem is that we're in a bit more difficult situation that Lua.. in Lua there's no processes/contexts.. and if we would want implement an external message send i.e. send message
- get result, things not so straightforward as in Lua.
it's all because it was never designed to be run in a host environment and be open from that end. that's why it is much easier to implement FFI, that C -> smalltalk API.
-- Best regards, Igor Stasenko.
On 29 November 2016 at 22:51, Dimitris Chloupis kilon.alios@gmail.com wrote:
Whats so special about pharo context and process and how is that related to embedding pharo ?
in short: scheduling , and top-level infinite loop.
On Tue, Nov 29, 2016 at 7:22 PM Igor Stasenko siguctua@gmail.com wrote:
On 29 November 2016 at 14:43, Ben Coman btc@openinworld.com wrote:
On Thu, Nov 24, 2016 at 10:06 PM, Ronie Salgado roniesalg@gmail.com wrote:
It is still needed to define a proper interface for embedding the VM. This interface should be a single .h file, with the highest level of abstraction possible. There are some other issues such as potential name clashes, and the fact that the VM symbols are public by default. At the bare minimum, it has to provide functions for initializing the VM, passing command line arguments, loading an image, running the image, and shutting down.
I don't know if its really suitable, but since Lua is the incumbent embedded gaming language, perhaps echoing it would make it easier to substitute ours for theirs. https://www.lua.org/pil/24.2.html
cheers -ben
well, the problem is that we're in a bit more difficult situation that Lua.. in Lua there's no processes/contexts.. and if we would want implement an external message send i.e. send message - get result, things not so straightforward as in Lua. it's all because it was never designed to be run in a host environment and be open from that end. that's why it is much easier to implement FFI, that C -> smalltalk API.
-- Best regards, Igor Stasenko.
2016-11-24 8:12 GMT+01:00 Ronie Salgado roniesalg@gmail.com:
So far I managed to run a standard Pharo 6 image using this custom VM in Linux.
How much VM size it reduced?
On Thu, Nov 24, 2016 at 9:12 AM Ronie Salgado roniesalg@gmail.com wrote:
.
I am also planning on making a standard interface for embedding the VM in an application, at least as a static library. With this new building system, this seems to be easy to do.
Best regards, Ronie
This negative energy surrounding you right now is me hating you Sith Lord style.
How dare you ?
Yesterday 03:00 am I finally managed to make my shared memory bridge between Pharo and C++ to work also on Windows after a ton of debugging and here comes you telling me that all my effort was for nothing. It now works and is tested on MacOS 10.12 , Ubuntu 15.04 and Windows 10 (all of them 64 bit).
Have you got no mercy ?
Suffice to say I am super interested in both Non GUI Pharo because I use Unreal Game Engine and embedding Pharo because I use Unreal Game Engine.
Embedding Pharo is a must have for me because my shared memory solution would not be acceptable by Apple on iOS since it forbids an app from running two executable at the same time which is basic requirement for shared memory. Stupid Apple :/ . Android seems to be fine with this.
For now my game will target only PC where my shared memory solution works fine but sooner or later I will have to port to iOS and Android.
If there is anything I can do to help you out, dont hesitate to ask. I am going to give your build a try now :)
2016-11-24 8:12 GMT+01:00 Ronie Salgado roniesalg@gmail.com:
So far I managed to run a standard Pharo 6 image using this custom VM in Linux. Windows and Mac are coming next. Hopefully this is going to fix the problems with duplicated events when using OSWindow in Windows.
Ronie, do you already have code to restore windows after image save? Any issue opened?
On 23-11-2016, at 11:12 PM, Ronie Salgado roniesalg@gmail.com wrote:
In DisplayScreen class >> startUp we have the following: startUp "DisplayScreen startUp" Display setExtent: self actualScreenSize depth: Display nativeDepth. Display beDisplay
Does it make sense, to always trying to create a display in startup time?
No, it doesn’t. Ideally no resource should be started up/connected/created/tweaked until and unless it is actually needed by the running system.
Obviously, the default setup for a Smalltalk tends to be the GUI IDE and it’s so very easy to slide into the assumption that graphics and sounds and windows must be started up no matter what. We all do that kind of thing, even when we know we shouldn’t. You can make some reasonable arguments that an image configured to be an IDE ought to open a window or two during start up since that is kind of its raison d’etre.
My small protest against this is/was the RISC OS VM not creating a UI window until the first display update was made. At least that meant that an image with no need of a graphical UI never needed to do anything other than not display anything on the Display.
I’d say that all the host window IO related stuff ought to go into an extend HostWindowPlugin and be removed from the core. I’d also prefer all the plugins to be external, even though I know there can be a tiny performance hit, depending upon the details of how the OS does late binding. The unix VM kinda-sorta does some of this with the C++ usage and vm-display-xxxxx modules etc, but I think a lot more needs doing. And of course, the image needs lots of work to make good use of it. Until you have images that don’t need display/sound/sockets/whatever, there isn’t a lot of point worrying about the vm side.
tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Strange OpCodes: LA: Lockout Access
On 24 November 2016 at 19:55, tim Rowledge tim@rowledge.org wrote:
On 23-11-2016, at 11:12 PM, Ronie Salgado roniesalg@gmail.com wrote:
In DisplayScreen class >> startUp we have the following: startUp "DisplayScreen startUp" Display setExtent: self actualScreenSize depth: Display nativeDepth. Display beDisplay
Does it make sense, to always trying to create a display in startup time?
No, it doesn’t. Ideally no resource should be started up/connected/created/tweaked until and unless it is actually needed by the running system.
Obviously, the default setup for a Smalltalk tends to be the GUI IDE and it’s so very easy to slide into the assumption that graphics and sounds and windows must be started up no matter what. We all do that kind of thing, even when we know we shouldn’t. You can make some reasonable arguments that an image configured to be an IDE ought to open a window or two during start up since that is kind of its raison d’etre.
My small protest against this is/was the RISC OS VM not creating a UI window until the first display update was made. At least that meant that an image with no need of a graphical UI never needed to do anything other than not display anything on the Display.
I’d say that all the host window IO related stuff ought to go into an extend HostWindowPlugin and be removed from the core. I’d also prefer all the plugins to be external, even though I know there can be a tiny performance hit, depending upon the details of how the OS does late binding. The unix VM kinda-sorta does some of this with the C++ usage and vm-display-xxxxx modules etc, but I think a lot more needs doing. And of course, the image needs lots of work to make good use of it. Until you have images that don’t need display/sound/sockets/whatever, there isn’t a lot of point worrying about the vm side.
In Pharo we did a special OSWindow layer, that allows platform-independent access to host window setup & manipulation etc. There are multiple drivers , each for own platform/implementation, like SDL2, as well as a special NullWindowDriver, that simply passing all UI-related requirests to /dev/null .. This is done so, to ensure that image could run fine even if VM is missing host window support or can't find any plugins etc, since historically too much code in image is done with assumption, that its always there.. In that case, we can always switch to null-window driver so all user-code would still function without knowing that there's actually no host window created etc.
tim
tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Strange OpCodes: LA: Lockout Access
This is super coooooool Embedding the Vm as a static lib is a kind of dream becoming true.
Stef
Hello,
I am working on removing most of windowing code from the VM, and in trying to unify the platform specific code of the VM. In the MinimalistHeadless branch of https://github.com/ronsaldo/%3Eopensmalltalk-vm I made the following changes:
- Unified standard CMake building scripts for Unixes. I hate autoconf. I
want to use them in Windows too.
- Refactoring the Unix entry points. I am trying to remove a lot of code
for simplfying stuff.
- Null window driver for true headless.
- Optional SDL2 based traditional display backend for compatibility
reasons, and because the extra Morphic worlds using OSWindow are a bit unstable.
So far I managed to run a standard Pharo 6 image using this custom VM in Linux. Windows and Mac are coming next. Hopefully this is going to fix the problems with duplicated events when using >OSWindow in Windows.
I am also planning on making a standard interface for embedding the VM in an application, at least as a static library. With this new building system, this seems to be easy to do.
BTW. When I tried the minimalistic Pharo image, in a completely headless VM, I got the following error:
[ "Ugh .... now this is a biggie - a system that does not support any of the display depths at all." Smalltalk logError: 'Fatal error: This system has no support for any display depth at all.' inContext: thisContext. Smalltalk quitPrimitive "There is no way to continue from here" ] in DisplayScreen>>findAnyDisplayDepth DisplayScreen>>findAnyDisplayDepthIfNone: DisplayScreen>>findAnyDisplayDepth DisplayScreen>>setExtent:depth: DisplayScreen class>>startUp
As a workaround, I am doing the following for ioHasDisplayDepth with the null driver:
sqInt ioHasDisplayDepth(sqInt depth) { return true; }
In DisplayScreen class >> startUp we have the following: startUp "DisplayScreen startUp" Display setExtent: self actualScreenSize depth: Display nativeDepth. Display beDisplay
Does it make sense, to always trying to create a display in startup time?
Best regards, Ronie
+100.
Great job!
Doru
On Nov 26, 2016, at 9:44 AM, stepharo stepharo@free.fr wrote:
This is super coooooool Embedding the Vm as a static lib is a kind of dream becoming true.
Stef
Hello,
I am working on removing most of windowing code from the VM, and in trying to unify the platform specific code of the VM. In the MinimalistHeadless branch of https://github.com/ronsaldo/opensmalltalk-vm I made the following changes:
- Unified standard CMake building scripts for Unixes. I hate autoconf. I want to use them in Windows too.
- Refactoring the Unix entry points. I am trying to remove a lot of code for simplfying stuff.
- Null window driver for true headless.
- Optional SDL2 based traditional display backend for compatibility reasons, and because the extra Morphic worlds using OSWindow are a bit unstable.
So far I managed to run a standard Pharo 6 image using this custom VM in Linux. Windows and Mac are coming next. Hopefully this is going to fix the problems with duplicated events when using OSWindow in Windows.
I am also planning on making a standard interface for embedding the VM in an application, at least as a static library. With this new building system, this seems to be easy to do.
BTW. When I tried the minimalistic Pharo image, in a completely headless VM, I got the following error:
[ "Ugh .... now this is a biggie - a system that does not support any of the display depths at all." Smalltalk logError: 'Fatal error: This system has no support for any display depth at all.' inContext: thisContext. Smalltalk quitPrimitive "There is no way to continue from here" ] in DisplayScreen>>findAnyDisplayDepth DisplayScreen>>findAnyDisplayDepthIfNone: DisplayScreen>>findAnyDisplayDepth DisplayScreen>>setExtent:depth: DisplayScreen class>>startUp
As a workaround, I am doing the following for ioHasDisplayDepth with the null driver:
sqInt ioHasDisplayDepth(sqInt depth) { return true; }
In DisplayScreen class >> startUp we have the following: startUp "DisplayScreen startUp" Display setExtent: self actualScreenSize depth: Display nativeDepth. Display beDisplay
Does it make sense, to always trying to create a display in startup time?
Best regards, Ronie
-- Using Opera's mail client: http://www.opera.com/mail/
-- www.tudorgirba.com www.feenk.com
"Be rather willing to give than demanding to get."
Hi Roni,
On Nov 23, 2016, at 11:12 PM, Ronie Salgado roniesalg@gmail.com wrote:
Hello,
I am working on removing most of windowing code from the VM, and in trying to unify the platform specific code of the VM. In the MinimalistHeadless branch of https://github.com/ronsaldo/opensmalltalk-vm I made the following changes:
Fantastic, except:
- Unified standard CMake building scripts for Unixes. I hate autoconf. I want to use them in Windows too.
We have agreed that we will use makefiles and restrict make to generating platform config.h files that define platform facilities. I have very good reasons for doing this. I have discussed this at length this year. Please don't make me argue this all over again. Are you on board? Would you like to discuss this on skype or somewhere? ... [goodness deleted]
Does it make sense, to always trying to create a display in startup time?
Good question. The answer should be "no" but that only makes sense if the system is written to not depend on a display when in headless mode.
Best regards, Ronie
Hi,
I am fighting Window VM compilation issues (e.g. Cairo with GCC5.x under
Mingw). If you have a GCC5 instead of the old 4.6, I am interested in your config.
Now I am building in Windows using both, MinGW version 4.9.3, and Visual Studio 2015 Community Edition.
I got working pharo,exe -headless Pharo-minimal.image eval "1+1" (the -headless parameter is for the image to quit after finishing). When running a normal Pharo.image, I am getting a FFI related crash (with Visual Studio and MinGW), when the image tries to read the APPDATA environment variable.
Right now I am using the same main function for both, the Unix and Win32 VM. This means, that we are close of having the embedded version.
In windows I am build two variants of the VM, pharo.exe (or squeak.exe) and pharow.exe (or squeakw.exe). The executable without the w is a console program, so it can use stdin, stdout and stderr. The executable with the w is a windows program that does not create a console window on startup, but it cannot use stdout, stderr, or stdin. This is the standard workaround for an annoying limitation of Windows itself (Python does the same in windows, with the same nomenclature).
Visual Studio is unstable than MinGW, but it has an excellent debugger. With Visual Studio, I am getting some false positives for an assertion because they get miscompiled. For some reason, the assert(GIV(remapBufferCount) == 0) gets compiled into the same as assert(GIV(remapBufferCount)). Apart from this assertion, these two versions of the VM are crashing in the same place.
Fantastic, except:
- Unified standard CMake building scripts for Unixes. I hate autoconf. I
want to use them in Windows too.
We have agreed that we will use makefiles and restrict make to generating platform config.h files that define platform facilities. I have very good reasons for doing this. I have discussed this at length this year. Please don't make me argue this all over again. ... [goodness deleted]
I cannot just use makefile, because I am also compiling in Windows using Visual Studio 2015 Community Edition, by generating the visual studio projects with CMake. I looked for the older discussions on the mailing lists, I could not find, a long discussion on the mailing list, except for several short ones. I have also seen some projects(LLVM, Boost) that have more than one build system, an official based on autoconf(or something else), and an alternative based on CMake.
As for the config.h, I can also generate it using CMake, and I will start doing it now because I think that it is related to the FFI crashing issues that I am experimenting on Windows. Currently I am already using CMake for generating the git version string. As long as git is in the path and cmake founds it, the header with the version string is generated correctly, even when using the Visual Studio projects.
Anyway, I am not using the old CMakeVMMaker that Igor did(and I just learned about Ian CMake work by finding in the mailing list). I just adapted the plain CMake scripts that I normally use for building my own projects to build the VM.
Are you on board? Would you like to discuss this on skype or somewhere?
Sure we can discuss it, I am on chile so we are not far apart in time zones.
As long as building the VM can be done automatically in Unixes by running a short number of commands, that work most of the time, I am OK with it. As for Windows, it has to be possible to use Visual Studio in addition to MinGW or Cygwin. Visual Studio is required to be able to build for the Universal Windows Platform, and getting the VM working in a retail XBox One ( https://msdn.microsoft.com/en-us/windows/uwp/xbox-apps/index ).
Best regards, Ronie
2016-11-26 14:09 GMT-03:00 Eliot Miranda eliot.miranda@gmail.com:
Hi Roni,
On Nov 23, 2016, at 11:12 PM, Ronie Salgado roniesalg@gmail.com wrote:
Hello,
I am working on removing most of windowing code from the VM, and in trying to unify the platform specific code of the VM. In the MinimalistHeadless branch of https://github.com/ronsaldo/opensmalltalk-vm I made the following changes:
Fantastic, except:
- Unified standard CMake building scripts for Unixes. I hate autoconf. I
want to use them in Windows too.
We have agreed that we will use makefiles and restrict make to generating platform config.h files that define platform facilities. I have very good reasons for doing this. I have discussed this at length this year. Please don't make me argue this all over again. Are you on board? Would you like to discuss this on skype or somewhere? ... [goodness deleted]
Does it make sense, to always trying to create a display in startup time?
Good question. The answer should be "no" but that only makes sense if the system is written to not depend on a display when in headless mode.
Best regards, Ronie
H Roniei, great work. I am mot sure if it can help but there is a decent free IDE for Windows called Codeblocks which can be configured to use (IIRC) various compilers and debuggers. Since you're into it maybe it worths paying a visit to their website to figure out its potential.
Cheers, vagy
On 27 Nov 2016, at 00:16, Ronie Salgado roniesalg@gmail.com wrote:
Hi,
I am fighting Window VM compilation issues (e.g. Cairo with GCC5.x under Mingw). If you have a GCC5 instead of the old 4.6, I am interested in your config.
Now I am building in Windows using both, MinGW version 4.9.3, and Visual Studio 2015 Community Edition.
I got working pharo,exe -headless Pharo-minimal.image eval "1+1" (the -headless parameter is for the image to quit after finishing). When running a normal Pharo.image, I am getting a FFI related crash (with Visual Studio and MinGW), when the image tries to read the APPDATA environment variable.
Right now I am using the same main function for both, the Unix and Win32 VM. This means, that we are close of having the embedded version.
In windows I am build two variants of the VM, pharo.exe (or squeak.exe) and pharow.exe (or squeakw.exe). The executable without the w is a console program, so it can use stdin, stdout and stderr. The executable with the w is a windows program that does not create a console window on startup, but it cannot use stdout, stderr, or stdin. This is the standard workaround for an annoying limitation of Windows itself (Python does the same in windows, with the same nomenclature).
Visual Studio is unstable than MinGW, but it has an excellent debugger. With Visual Studio, I am getting some false positives for an assertion because they get miscompiled. For some reason, the assert(GIV(remapBufferCount) == 0) gets compiled into the same as assert(GIV(remapBufferCount)). Apart from this assertion, these two versions of the VM are crashing in the same place.
Fantastic, except:
- Unified standard CMake building scripts for Unixes. I hate autoconf. I want to use them in Windows too.
We have agreed that we will use makefiles and restrict make to generating platform config.h files that define platform facilities. I have very good reasons for doing this. I have discussed this at length this year. Please don't make me argue this all over again. ... [goodness deleted]
I cannot just use makefile, because I am also compiling in Windows using Visual Studio 2015 Community Edition, by generating the visual studio projects with CMake. I looked for the older discussions on the mailing lists, I could not find, a long discussion on the mailing list, except for several short ones. I have also seen some projects(LLVM, Boost) that have more than one build system, an official based on autoconf(or something else), and an alternative based on CMake.
As for the config.h, I can also generate it using CMake, and I will start doing it now because I think that it is related to the FFI crashing issues that I am experimenting on Windows. Currently I am already using CMake for generating the git version string. As long as git is in the path and cmake founds it, the header with the version string is generated correctly, even when using the Visual Studio projects.
Anyway, I am not using the old CMakeVMMaker that Igor did(and I just learned about Ian CMake work by finding in the mailing list). I just adapted the plain CMake scripts that I normally use for building my own projects to build the VM.
Are you on board? Would you like to discuss this on skype or somewhere?
Sure we can discuss it, I am on chile so we are not far apart in time zones.
As long as building the VM can be done automatically in Unixes by running a short number of commands, that work most of the time, I am OK with it. As for Windows, it has to be possible to use Visual Studio in addition to MinGW or Cygwin. Visual Studio is required to be able to build for the Universal Windows Platform, and getting the VM working in a retail XBox One ( https://msdn.microsoft.com/en-us/windows/uwp/xbox-apps/index ).
Best regards, Ronie
2016-11-26 14:09 GMT-03:00 Eliot Miranda eliot.miranda@gmail.com:
Hi Roni,
On Nov 23, 2016, at 11:12 PM, Ronie Salgado roniesalg@gmail.com wrote:
Hello,
I am working on removing most of windowing code from the VM, and in trying to unify the platform specific code of the VM. In the MinimalistHeadless branch of https://github.com/ronsaldo/opensmalltalk-vm I made the following changes:
Fantastic, except:
- Unified standard CMake building scripts for Unixes. I hate autoconf. I want to use them in Windows too.
We have agreed that we will use makefiles and restrict make to generating platform config.h files that define platform facilities. I have very good reasons for doing this. I have discussed this at length this year. Please don't make me argue this all over again. Are you on board? Would you like to discuss this on skype or somewhere? ... [goodness deleted]
Does it make sense, to always trying to create a display in startup time?
Good question. The answer should be "no" but that only makes sense if the system is written to not depend on a display when in headless mode.
Best regards, Ronie
Ronie,
Is all of this in a branch somewhere so that I can have a look?
For cairo, I think that I can build it with another compiler than MinGW, it doesn't really matter as it just makes a dll (or well, I can just skip that and reuse an already built one).
-ltro is the flag that makes the whole thing trips up in GCC5 and requires additional flags, but adding them just kicked the can down the road a bit further as they were still unresolved symbols. Need to see why cairo build gives that but seems a cairo issue that others are facing as well. May end up building cairo with the old compiler but this defeats the purpose.
For VS2015, I've got it too and yes, the debugging facilities are much better, not to mention that the compilation times are much nicer. GCC is just dog slow on Windows compared to Linux. I read that it was because of sub-bar fork support in Windows.
Nice to give me an excuse to get an XBox One for XMas in order to try Pharo on it.
Best, Phil
On Sat, Nov 26, 2016 at 11:16 PM, Ronie Salgado roniesalg@gmail.com wrote:
Hi,
I am fighting Window VM compilation issues (e.g. Cairo with GCC5.x under
Mingw). If you have a GCC5 instead of the old 4.6, I am interested in your config.
Now I am building in Windows using both, MinGW version 4.9.3, and Visual Studio 2015 Community Edition.
I got working pharo,exe -headless Pharo-minimal.image eval "1+1" (the -headless parameter is for the image to quit after finishing). When running a normal Pharo.image, I am getting a FFI related crash (with Visual Studio and MinGW), when the image tries to read the APPDATA environment variable.
Right now I am using the same main function for both, the Unix and Win32 VM. This means, that we are close of having the embedded version.
In windows I am build two variants of the VM, pharo.exe (or squeak.exe) and pharow.exe (or squeakw.exe). The executable without the w is a console program, so it can use stdin, stdout and stderr. The executable with the w is a windows program that does not create a console window on startup, but it cannot use stdout, stderr, or stdin. This is the standard workaround for an annoying limitation of Windows itself (Python does the same in windows, with the same nomenclature).
Visual Studio is unstable than MinGW, but it has an excellent debugger. With Visual Studio, I am getting some false positives for an assertion because they get miscompiled. For some reason, the assert(GIV(remapBufferCount) == 0) gets compiled into the same as assert(GIV(remapBufferCount)). Apart from this assertion, these two versions of the VM are crashing in the same place.
Fantastic, except:
- Unified standard CMake building scripts for Unixes. I hate autoconf. I
want to use them in Windows too.
We have agreed that we will use makefiles and restrict make to generating platform config.h files that define platform facilities. I have very good reasons for doing this. I have discussed this at length this year. Please don't make me argue this all over again. ... [goodness deleted]
I cannot just use makefile, because I am also compiling in Windows using Visual Studio 2015 Community Edition, by generating the visual studio projects with CMake. I looked for the older discussions on the mailing lists, I could not find, a long discussion on the mailing list, except for several short ones. I have also seen some projects(LLVM, Boost) that have more than one build system, an official based on autoconf(or something else), and an alternative based on CMake.
As for the config.h, I can also generate it using CMake, and I will start doing it now because I think that it is related to the FFI crashing issues that I am experimenting on Windows. Currently I am already using CMake for generating the git version string. As long as git is in the path and cmake founds it, the header with the version string is generated correctly, even when using the Visual Studio projects.
Anyway, I am not using the old CMakeVMMaker that Igor did(and I just learned about Ian CMake work by finding in the mailing list). I just adapted the plain CMake scripts that I normally use for building my own projects to build the VM.
Are you on board? Would you like to discuss this on skype or somewhere?
Sure we can discuss it, I am on chile so we are not far apart in time zones.
As long as building the VM can be done automatically in Unixes by running a short number of commands, that work most of the time, I am OK with it. As for Windows, it has to be possible to use Visual Studio in addition to MinGW or Cygwin. Visual Studio is required to be able to build for the Universal Windows Platform, and getting the VM working in a retail XBox One ( https://msdn.microsoft.com/en-us/windows/uwp/xbox-apps/index ).
Best regards, Ronie
2016-11-26 14:09 GMT-03:00 Eliot Miranda eliot.miranda@gmail.com:
Hi Roni,
On Nov 23, 2016, at 11:12 PM, Ronie Salgado roniesalg@gmail.com wrote:
Hello,
I am working on removing most of windowing code from the VM, and in trying to unify the platform specific code of the VM. In the MinimalistHeadless branch of https://github.com/ronsaldo/opensmalltalk-vm I made the following changes:
Fantastic, except:
- Unified standard CMake building scripts for Unixes. I hate autoconf. I
want to use them in Windows too.
We have agreed that we will use makefiles and restrict make to generating platform config.h files that define platform facilities. I have very good reasons for doing this. I have discussed this at length this year. Please don't make me argue this all over again. Are you on board? Would you like to discuss this on skype or somewhere? ... [goodness deleted]
Does it make sense, to always trying to create a display in startup time?
Good question. The answer should be "no" but that only makes sense if the system is written to not depend on a display when in headless mode.
Best regards, Ronie
Hi Phil,
It is on the MinimalisticHeadless branch of ( https://github.com/ronsaldo/opensmalltalk-vm/tree/MinimalisticHeadless ).
With VS you are going to receive a lot of messages such as:
GIV(remapBufferCount) == 0 55758
These are false positives due to the macro expansion bug.
Nice to give me an excuse to get an XBox One for XMas in order to try Pharo on it.
For me it is also an excuse for XMas ;) .
Best regards, Ronie
2016-11-27 18:01 GMT-03:00 phil@highoctane.be phil@highoctane.be:
Ronie,
Is all of this in a branch somewhere so that I can have a look?
For cairo, I think that I can build it with another compiler than MinGW, it doesn't really matter as it just makes a dll (or well, I can just skip that and reuse an already built one).
-ltro is the flag that makes the whole thing trips up in GCC5 and requires additional flags, but adding them just kicked the can down the road a bit further as they were still unresolved symbols. Need to see why cairo build gives that but seems a cairo issue that others are facing as well. May end up building cairo with the old compiler but this defeats the purpose.
For VS2015, I've got it too and yes, the debugging facilities are much better, not to mention that the compilation times are much nicer. GCC is just dog slow on Windows compared to Linux. I read that it was because of sub-bar fork support in Windows.
Nice to give me an excuse to get an XBox One for XMas in order to try Pharo on it.
Best, Phil
On Sat, Nov 26, 2016 at 11:16 PM, Ronie Salgado roniesalg@gmail.com wrote:
Hi,
I am fighting Window VM compilation issues (e.g. Cairo with GCC5.x under
Mingw). If you have a GCC5 instead of the old 4.6, I am interested in your config.
Now I am building in Windows using both, MinGW version 4.9.3, and Visual Studio 2015 Community Edition.
I got working pharo,exe -headless Pharo-minimal.image eval "1+1" (the -headless parameter is for the image to quit after finishing). When running a normal Pharo.image, I am getting a FFI related crash (with Visual Studio and MinGW), when the image tries to read the APPDATA environment variable.
Right now I am using the same main function for both, the Unix and Win32 VM. This means, that we are close of having the embedded version.
In windows I am build two variants of the VM, pharo.exe (or squeak.exe) and pharow.exe (or squeakw.exe). The executable without the w is a console program, so it can use stdin, stdout and stderr. The executable with the w is a windows program that does not create a console window on startup, but it cannot use stdout, stderr, or stdin. This is the standard workaround for an annoying limitation of Windows itself (Python does the same in windows, with the same nomenclature).
Visual Studio is unstable than MinGW, but it has an excellent debugger. With Visual Studio, I am getting some false positives for an assertion because they get miscompiled. For some reason, the assert(GIV(remapBufferCount) == 0) gets compiled into the same as assert(GIV(remapBufferCount)). Apart from this assertion, these two versions of the VM are crashing in the same place.
Fantastic, except:
- Unified standard CMake building scripts for Unixes. I hate autoconf. I
want to use them in Windows too.
We have agreed that we will use makefiles and restrict make to generating platform config.h files that define platform facilities. I have very good reasons for doing this. I have discussed this at length this year. Please don't make me argue this all over again. ... [goodness deleted]
I cannot just use makefile, because I am also compiling in Windows using Visual Studio 2015 Community Edition, by generating the visual studio projects with CMake. I looked for the older discussions on the mailing lists, I could not find, a long discussion on the mailing list, except for several short ones. I have also seen some projects(LLVM, Boost) that have more than one build system, an official based on autoconf(or something else), and an alternative based on CMake.
As for the config.h, I can also generate it using CMake, and I will start doing it now because I think that it is related to the FFI crashing issues that I am experimenting on Windows. Currently I am already using CMake for generating the git version string. As long as git is in the path and cmake founds it, the header with the version string is generated correctly, even when using the Visual Studio projects.
Anyway, I am not using the old CMakeVMMaker that Igor did(and I just learned about Ian CMake work by finding in the mailing list). I just adapted the plain CMake scripts that I normally use for building my own projects to build the VM.
Are you on board? Would you like to discuss this on skype or somewhere?
Sure we can discuss it, I am on chile so we are not far apart in time zones.
As long as building the VM can be done automatically in Unixes by running a short number of commands, that work most of the time, I am OK with it. As for Windows, it has to be possible to use Visual Studio in addition to MinGW or Cygwin. Visual Studio is required to be able to build for the Universal Windows Platform, and getting the VM working in a retail XBox One ( https://msdn.microsoft.com/en-us/windows/uwp/xbox-apps/index ).
Best regards, Ronie
2016-11-26 14:09 GMT-03:00 Eliot Miranda eliot.miranda@gmail.com:
Hi Roni,
On Nov 23, 2016, at 11:12 PM, Ronie Salgado roniesalg@gmail.com wrote:
Hello,
I am working on removing most of windowing code from the VM, and in trying to unify the platform specific code of the VM. In the MinimalistHeadless branch of https://github.com/ronsaldo/op ensmalltalk-vm I made the following changes:
Fantastic, except:
- Unified standard CMake building scripts for Unixes. I hate autoconf. I
want to use them in Windows too.
We have agreed that we will use makefiles and restrict make to generating platform config.h files that define platform facilities. I have very good reasons for doing this. I have discussed this at length this year. Please don't make me argue this all over again. Are you on board? Would you like to discuss this on skype or somewhere? ... [goodness deleted]
Does it make sense, to always trying to create a display in startup time?
Good question. The answer should be "no" but that only makes sense if the system is written to not depend on a display when in headless mode.
Best regards, Ronie
On Mon, Nov 28, 2016 at 5:01 AM, phil@highoctane.be phil@highoctane.be wrote:
Ronie,
Is all of this in a branch somewhere so that I can have a look?
For cairo, I think that I can build it with another compiler than MinGW, it doesn't really matter as it just makes a dll (or well, I can just skip that and reuse an already built one).
-ltro is the flag that makes the whole thing trips up in GCC5 and requires additional flags, but adding them just kicked the can down the road a bit further as they were still unresolved symbols. Need to see why cairo build gives that but seems a cairo issue that others are facing as well. May end up building cairo with the old compiler but this defeats the purpose.
For VS2015, I've got it too and yes, the debugging facilities are much better, not to mention that the compilation times are much nicer. GCC is just dog slow on Windows compared to Linux. I read that it was because of sub-bar fork support in Windows.
Interesting to note Microsoft is moving to Clang [1] to replace their C1 front-end, which should(?) make cross-platform with OSX and Linux a bit easier. Anyone know if that means MVSC might be better to use on its own than using MinGW/Cygwin?
[1] http://www.theregister.co.uk/2015/10/21/microsoft_promises_clang_for_windows...
cheers -ben
Now I found the FFI problem with MinGW , I was missing some old compilation flags on my CMake scripts. I still have to fix this same bug with MSVC, but at least normal Pharo image can eval "1+1" without crashing, and stdout is working properly now in Windows. During the rest of the week I will make this VM variant working completely on Linux, Windows and OS X, with SDL2 and hopefully we will get OSWindow working alot better in Windows and Mac. (In Linux it was already working pretty well, probably because each X connection has its own event queue).
Interesting to note Microsoft is moving to Clang [1] to replace their
C1 front-end, which should(?) make cross-platform with OSX and Linux a bit easier. Anyone know if that means MVSC might be better to use on its own than using MinGW/Cygwin?
[1] http://www.theregister.co.uk/2015/10/21/microsoft_promises_ clang_for_windows_in_november_visual_c_update/
cheers -ben
In Windows the standard compiler is MSVC, despite it has a poorer quality than gcc/clang. The movement to the clang front-end is something where it is not possible to rely until it becomes stable. It is quite clear that the VM is not meant to be used with MSVC, because it is not being built and tested with it all of the time. However, in some platforms, the only alternative is MSVC.
Best regards, Ronie
2016-11-28 0:46 GMT-03:00 Ben Coman btc@openinworld.com:
On Mon, Nov 28, 2016 at 5:01 AM, phil@highoctane.be phil@highoctane.be wrote:
Ronie,
Is all of this in a branch somewhere so that I can have a look?
For cairo, I think that I can build it with another compiler than MinGW,
it doesn't really matter as it just makes a dll (or well, I can just skip that and reuse an already built one).
-ltro is the flag that makes the whole thing trips up in GCC5 and
requires additional flags, but adding them just kicked the can down the road a bit further as they were still unresolved symbols. Need to see why cairo build gives that but seems a cairo issue that others are facing as well. May end up building cairo with the old compiler but this defeats the purpose.
For VS2015, I've got it too and yes, the debugging facilities are much
better, not to mention that the compilation times are much nicer. GCC is just dog slow on Windows compared to Linux. I read that it was because of sub-bar fork support in Windows.
Interesting to note Microsoft is moving to Clang [1] to replace their C1 front-end, which should(?) make cross-platform with OSX and Linux a bit easier. Anyone know if that means MVSC might be better to use on its own than using MinGW/Cygwin?
[1] http://www.theregister.co.uk/2015/10/21/microsoft_promises_ clang_for_windows_in_november_visual_c_update/
cheers -ben
vm-dev@lists.squeakfoundation.org