[Vm-dev] VM Maker: VMMaker.oscog-eem.1881.mcz

[Eliot Miranda]

Hi Ben,


> On Jun 6, 2016, at 5:30 PM, Ben Coman <btc at openinworld.com> wrote:
> 
> 
>> On Tue, Jun 7, 2016 at 3:56 AM,  <commits at source.squeak.org> wrote:
>> 
>> Eliot Miranda uploaded a new version of VMMaker to project VM Maker:
>> http://source.squeak.org/VMMaker/VMMaker.oscog-eem.1881.mcz
>> 
>> ==================== Summary ====================
>> 
>> Name: VMMaker.oscog-eem.1881
>> Author: eem
>> Time: 6 June 2016, 12:55:12.14917 pm
>> UUID: c51b1764-ddf8-49fc-92bd-d587b3d65101
>> Ancestors: VMMaker.oscog-eem.1880
>> 
>> On Spur, handle external primitive failures answering PrimErrNoMemory by doing a scavenge and retrying and then on subsequent PrimErrNoMemory failure a full GC and then retrying.  Rename maybeRetryFailureDueToForwarding to maybeRetryPrimitiveOnFailure.
>> 
>> In the Cogit, change the retry mechanism so that the primitive call is made in the run-time, instead of the run-time answering whether the primitive should be retried and returning to machine code that then retries.
>> 
>> In the simulator, go through the contortions to map[ back whatever primitiveFunctionPointer is to that which dispatchFunctionPointer: accepts.
>> 
>> Discussion:
>> First, this mechanism applies only to external (plugin) primitives.  Spur's core primtives are written in the expectation that image-level primitive failure code will deal with the failure and retry.  This corresponds to the VisualWorks style and allows the image to implement GC policy (e.g. to favour reclamation over growth).  So with the current Spur code, heavily influenced by the author's experience with VisualWorks, it would be inapproprate to retry corte primitives on being out-of-memory.  The pathology is that attempts at very large allocations which will inevitably fail will cause a GC in the VM, very possibly followed by a GC from the image level primitive failure code.  Whereas leaving it to the image, the image code may be intellkigent enough to spot an allocation too large to ever succeed and avoid the GC altogether.
>> 
>> I'm not sure if I like this auto-retry or not.  Perhaps it is a good idea and the image-level primitive failure code should be changed; there would certainly be fewer image-level methods.  But that means fewer hooks for the image to implement GC policy.  retryPrimitiveOnFailure which manages the retry isn't exactly simple (mapping back the primitiveFunctionPointer to something dispatchFunctionPointer: can manage isn't simple either, but that's simulation only, and so its complications carry much less weight).
> 
> 
> Is there something that can be called from the image side primitive
> failure code? Like...
>    self primitiveRetry.

The primitive itself.  Look at the pattern in new handleFailingNew and handleFailingFailingNew for example.

> 
> At the moment I only guess that retrying from the primitive failure
> code requires an explicit call to the primitive again, but that sort
> of recursion would seem to grow the stack unnecessarily.

It was a sequence including a conditional (eg self doPrimitive. (self failed and: [self scanAndFixForwarders]) ifTrue: [self doPrimitive].

Now it's a loop, not recursion.  The loop checks for failure due to memory exhaustion, or infers it may be due to forwarding.  It guarantees to call the scan for forwarders, the scavenger and the full GC once only as appropriate after a specific initial primitive invocation has failed.  Read #retryPrimitiveOnFailure.


>> + maybeMapPrimitiveFunctionPointerBackToSomethingEvaluable
>> +       "In the real VM primitiveFunctionPointer is either an index (for quick primitives)
>> +        or a proper function pointer to a primitive.  In the simulator it may be a small
>> +        index (corresponding to a quick primitive index), a symbol (corresponding to
>> +        a function pointer) or an index into the externalPrimitiveTable, or an invalid
>> +        address that references an evaluable in the simulatedTrampolines dictionary
>> +        of the Cogit.  The simulator expects dispatchFunctionPointer to be called with
>> +        primitiveFunctionPointer being a symbol only for internal primitives.  External
>> +        primitives must have their funciton pointer mapped back to an index.  This
>> +        method does the mapping back from fake addresses."
> 
> Thanks for that description.  It nicely answers my question in another
> mail thread.  As a newbie it was easy to assume that what the
> simulator sees is *exactly* what happens in-real-life, or perhaps a
> matter of knowing there may be simulator shortcuts but not knowing
> what they are.

yes.  The main source of these is in modeling C's modulo integer arithmetic with Smalltalk's infinite precision arithmetic.  But there are other hacks.  I've tried to reduce them as they're the source of infidelities which make the simulator less useful.  In the Cogit simulator I use no shortcuts.  Instead I use illegal addresses that cause exceptions that are caught to get at Smalltalk objects such as the CoInterpreter's instance variables.  Hence machine code in the simulator is identical to machine code in the JIT apart from address.  I believe this has been essential in making the JIT debuggable.

> 
> cheers -ben