[Vm-dev] Materializing BlockClosure's outerContext?

[Eliot Miranda]

Hi Ben,

On Fri, Jan 4, 2019 at 5:30 PM Ben Coman <btc at openinworld.com> wrote:

>
> On Sat, 5 Jan 2019 at 03:08, Eliot Miranda <eliot.miranda at gmail.com>
> wrote:
>
>>
>> Hi Fabio,  Hi All,
>>
>> On Fri, Jan 4, 2019 at 9:11 AM Fabio Niephaus <lists at fniephaus.com>
>> wrote:
>>
>>>
>>> Hi all,
>>>
>>> I'm trying to understand when the outerContext of a BlockClosure is
>>> materialized in Cog. I'm assuming Cog does much better than materializing
>>> the context at allocation time of the BlockClosure, but I wasn't able to
>>> find that out by browsing VMMaker code.
>>>
>>
>> Unless Sista is being used then the outer context is always materialized
>> whenever a closure is created.  In the StackInterpreter the two methods
>> called by the actual bytecode routines
>> are pushClosureNumArgs:copiedValues:blockSize:
>> and pushFullClosureNumArgs:copiedValues:compiledBlock:receiverIsOnStack:ignoreContext:.
>> You'll see that the first thing pushClosureNumArgs:copiedValues:blockSize:
>> does is ensure the current frame is married, i.e. it materializes the
>> context if it isn't already.  In
>> pushFullClosureNumArgs:copiedValues:compiledBlock:receiverIsOnStack:ignoreContext:
>> the first thing that happens is the same unless the ignoreContext flag has
>> been set, in which case the outerContext will be nil.
>>
>> In Cog the JIT generates code to materialize as quickly as possible.
>> See CogObjectRepresentationForSpur>>#genGetActiveContextLarge:inBlock:
>> which is sent four times to create four trampolines ceSmallMethodContext,
>> ceSmallBlockContext, ceSmallFullBlockContext, and ceLargeMethodContext.
>> These are used in turn by
>> CogObjectRepresentationForSpur>>#genGetActiveContextNumArgs:large:inBlock:,
>> which is used by
>> CogObjectRepresentationForSpur>>#genNoPopCreateClosureAt:numArgs:numCopied:contextNumArgs:large:inBlock:
>> &
>> CogObjectRepresentationForSpur>>#genCreateFullClosure:numArgs:numCopied:ignoreContext:contextNumArgs:large:inBlock:,
>> which generates the JIT code that parallels the StackInterpreter bytecodes.
>>
>> Some rationale:
>> In VW there is no adaptive optimization and so no closure4 inlining.
>> Instead there are three different kinds of blocks:
>>
>> clean: created at compile time, no copied values, only arguments, and
>> hence no need for an outer context.  If in a clean closure in a debugger
>> there is no information as to where the closure was activated; only its
>> static method name is known.
>> copying: created at run-time, but since there is no up-arrow return there
>> is no need to create an outer context.  If in a copying closure in a
>> debugger there is no information as to where the closure was activated;
>> only its static method name is known.
>> full: created at run-time; requires the outerContext to be materialized
>>
>> When I came to add closures to Squeak (good in itself but also essential
>> in implementing a sane context-to-stack mapping scheme) there were only 8
>> unused bytecodes, and two of these were already being used in a Newspeak
>> implementation I had affection for.  Qwaq wanted to keep things as simple
>> as possible, as did I.  Therefore the logical thing to do was to only
>> provide closures that always had an outerContext; KISS.  Implementing
>> closures used 5 bytecodes (create closure, create indirection vector,
>> push/store/pop indirect), leaving three to spare (at this time I had no
>> idea about a Sista bytecode set or about multiple bytecode sets). Knowing
>> that at some stage adaptive optimization would be a much better approach
>> than simply micro-optimizing closure creation, with all the extra
>> complexity and infidelity in the debugger, and that I could optimize
>> materialization aggressively, I think I made the right decision.  We can
>> still implement the equivalent of clean blocks in the compiler (and indeed
>> someone has done this in Pharo, and with full blocks it would be easy to do
>> in Squeak).  But much more interesting is to finish Sista/Scorch.  To this
>> end I'm currently frustrated by not being able to load Clément's tonel
>> Scorch repository into Squeak.
>>
>> Why do I want too be able to load Sista in Squeak right now? (Forgive me
>> if I've already told you this).  Clément has been developing Scorch (Sista
>> is the name for the overall architecture, Scorch is the name of the
>> image-level optimizing compiler) in a live Pharo image.  That means any
>> bugs crash his system and progress is slow.  But the simulator can be
>> modified to intercept the counter callback and instead of delivering it
>> into the image being simulated can deliver it to Scorch in the current
>> image.  The simulator can provide "facade" proxy context objects for
>> contexts in the simulation.  These appear to be contexts, but are actually
>> proxies for objects in the simulation (we do the same for methods so we can
>> use InstructionPrinter, StackDepthFinder et al on methods in the
>> simulation).  And we can rewrite the back end of Scorch, the part that
>> installs methods into method dictionaries, to use a mirror object.  Then we
>> can substitute a mirror that materializes a method in the simulation and
>> invokes code in the simulation to install the method there-in.
>>
>> So this allows Scorch to be developed as intended, in the current image,
>> but have it affect only the simulation, and therefore be immune from
>> crashing the current system.  This should speed up productization a lot,
>> and will allow Sophie and I to work on register allocation in the Sista
>> JIT.
>>
>
>
>
>> So any energy anyone can put into getting Scorch to load into Squeak
>> would be much appreciated.
>>
>
> What is the work breakdown required for this?
>

Basically get the following to work in 64-bit Squeak trunk.  For me none of
the Scorch load attempts work.

Installer ensureRecentMetacello.
(Smalltalk classNamed: #Metacello) new
  repository: 'github://j4yk/tonel:squeak'; "Loads Jakob's squeak branch"
  baseline: 'Tonel';
  load.

Metacello new
repository: 'github://clementbera/Scorch:master/repository';
baseline: 'Scorch';
onWarningLog;
load.

"or maybe..."
Metacello new
repository: 'github://clementbera/Scorch:repository';
baseline: 'Scorch';
onWarningLog;
load.
"or maybe..."
Metacello new
githubUser: 'clementbera' project: 'Scorch' commitish: 'master' path:
'repository';
baseline: 'Scorch';
onWarningLog;
load.

_,,,^..^,,,_
best, Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20190104/143a74c0/attachment.html>