[Vm-dev] OSProcess fork issue with Debian built VM

[Eliot Miranda]

Hi Holger,

    I have no disagreement with your time lines, but I do disagree with the black magic accusation, and you introduce your own piece of magic (see below).

> On Jun 2, 2017, at 4:20 AM, Holger Freyther <holger at freyther.de> wrote:
> 
> 
> 
>> On 2. Jun 2017, at 18:43, K K Subbu <kksubbu.ml at gmail.com> wrote:
>> 
> 
> Hey!
> 
> 
>> +1. I would go further and propose that plugins, other than those which are integral to VM[1], be moved into separate projects and built in their own tree with no references to vm source paths. They should be free to organize their platform-specific and platform-independent code and makefiles.
> 
> why? How many plugins are used that don't ship with the pharo-vm (or squeakvm)? I have to say I like the pharo-vm approach a lot where the VMMaker* packages and generated sources sit next to each other.
> 
> Let's compare two options with the OSProcess timeline at hand
> 
> 
> What happened:
> 
> * I compile with a new compiler, look at the compiler warning and agree with it
> * David Lewis agrees with my analysis and fixes it
> ...
> * Max runs into a crash
> * Max debugs a crash.. hours? days?
> * Max involves others
> ...
> * I have to debug the crash myself..
> * I notice that the fix is not there yet
> * I have to wait for someone privileged to regenerate the source (black magic?)

One builds a VMMaker image, which will pull in the new source, and therein evaluates 

        VMMaker generateVMPlugins

and then checks in the new sources.  David should have been able to do this.  The process is not documented.

The souce generation process is also set up to generate new source for plugins for those classes that have been changed since last generation, on the basis of a time stamp (it has ever been so).  This is a problem; if one builds a VMMaker image just to regenerate source for plugins then all plugins will be generated.  In this case there's no harm done because the check in script (scripts/gitci) reverts any plugins whose code hasn't changed.  But if some new vm source needed to be generated (say one method in one JIT backend was changed) then all source would be regenerated by

    [VMMaker generateAllConfigurationsUnderVersionControl] valueAnswering: false

(The false says don't generate new source for header files and classes that haven't changed).  The time stamp scheme works for me because I keep a VMMaker image (it's my life) and generate when required.  It doesn't work for someone wanting to build an image and regenerate.

So the time stamp stuff needs more thought.  If, on building a VMMaker image, class timestamps were derived from method timestamps then the classes would be accurate.  Then if one applied the scripts/gitrevert, which uses touch -t to set last modification times to check in times, to the generated source files in the source tree their time stamps would be accurate.  The trouble is that I believe at least the Smalltalk method timestamps are in local time, not UTC.

> 
> 
> My preferred option
> 
> * I compile with a new compiler, look at the compiler warning and agree with it
> * David Lewis agrees with my analysis and commits/pushes the fix to mc/ directory

You're glossing over some steps here.  Note that the fix is in the Smalltalk source, upstream of the C source.

The missing step is that the plugin source is regenerated .  As a Smalltalk-side-of-the-business person David knows how to regenerate source.  David, why didn't you regenerate? [honest Q, not an accusation]

One *mustn't* manually fix the generated source because doing so means
- the fix is lost the next time source is regenerated from the Smalltalk
- the C source has a misleading version stamp so is effectively untracked, masquerading as something else

One *must* regenerate from an unmodified checked in Monticello package because doing so means
- the C source is version stamped as originating from that package and so one can locate the correct version of the Smalltalk from which it derived 

> * New VM will be build and regenerates sources on the fly

How does this work? Seems like magic to me.  Source has to be generated from a VMMaker image.  We need to regenerate source only for that which has changed, not for the entire generated source.  But right now that's problematic because of the time issues I mention above.

> * Max never runs into the problem
> 
> Let's do it the Smalltalk way? Make simple things simple and complex (allow externally maintained plugins) possible?

Agreed.  No need to complicate source generation.  But we still need people who fix the Smalltalk to be able to regenerate source.  Several of us, Ronie, Clément, Nicolas, myself.  David, you should feel free to do so also.

> 
> have a nice weekend
> 
>    holger