[squeak-dev] Switching to Sista bytecodes in trunk (was: The Inbox: Kernel-dtl.1310.mcz)

Thiede, Christoph Christoph.Thiede at student.hpi.uni-potsdam.de
Tue Mar 17 23:30:26 UTC 2020

Hi Eliot,

(better answer late than never :-))

Thanks a lot for the detailed answer! It's really interesting to hear about these changes, and it would be fantastic to benefit from Scorch in Squeak someday.

> It does not change semantics at all.

Actually, I found a small change indeed:

[] method now returns a CompiledBlock rather than a CompiledMethod. This makes sense, but the term #method is not very accurate any longer, is it? ;) See also Tools-ct.958. Should we maybe rename this thing?

> Why is no one in the community joining me and Clément in trying to finish this work?

I can only speak for myself: I'm still doing my basic studies and did not yet have an opportunity to learn the fundaments of VM programming at all. I love to contribute to the inbox as you can effect a lot by making small steps. Compared to this, acquiring the conditions for co-developing at a VM structure appears to consume quite more time than I could spend besides studying ... In a few years, when I have learned more about this stuff, I would be happy to address projects like Scorch. But probably you will have completed your thesis until then :-)



Von: Squeak-dev <squeak-dev-bounces at lists.squeakfoundation.org> im Auftrag von Eliot Miranda <eliot.miranda at gmail.com>
Gesendet: Montag, 9. März 2020 17:24:18
An: The general-purpose Squeak developers list
Betreff: Re: [squeak-dev] Switching to Sista bytecodes in trunk (was: The Inbox: Kernel-dtl.1310.mcz)

Hi Christoph, Hi All,

On Sat, Mar 7, 2020 at 10:12 AM Thiede, Christoph <Christoph.Thiede at student.hpi.uni-potsdam.de<mailto:Christoph.Thiede at student.hpi.uni-potsdam.de>> wrote:

Hi Dave, hi all,

this sounds like a really interesting changeover. I have heard and read some rumors about Sista in the past, but I could not find any brief but complete summary of all the changes this will make to our existing Smalltalk system. So it would be great if you could point us to some kind of changelog.

The class comment for EncoderForSistaV1 gives a good introduction.  Further information is available in the papers and presentations mentioned on squeak.org/research<http://squeak.org/research>  But the key property is that, as a bytecode set, it does not affect semantics.  It is an instruction set, not a new virtual machine.

  1.  Clément Béra, Eliot Miranda, Tim Felgentreff, Marcus Denker, and Stéphane Ducasse. “Sista: Saving Optimized Code in Snapshots for Fast Start-Up.” In Proceedings of the 14th International Conference on Managed Languages and Runtimes, 1–11. ManLang 2017. New York, NY, USA: ACM, 2017.http://doi.acm.org/10.1145/3132190.3132201.
  2.  Clément Béra, Eliot Miranda, Marcus Denker, and Stéphane Ducasse. “Practical Validation of Bytecode to Bytecode JIT Compiler Dynamic Deoptimization.” Journal of Object Technology 15, no. 2 (2016): 1:1–26. https://doi.org/10.5381/jot.2016.15.2.a1.

In what way will this affect your typical Squeak experience?

You will be able to compile larger methods.

What about performance?

In an interpreter one may be able to find examples that are faster, but by very small percentages.  Sista as a bytecode set on its own does not affect performance.  However, once used by an adaptive optimizer (Clément's Scorch) performance should be in the 3x to 4x faster range.  Of course, there is zero support and zero funding for this work.  Clément is now at Google.  I am attempting to self-fund this work for my PhD.  I feel extremely let down by the entire community.  This is excellent work but there is *zero*controbution from outside Clément and myself.  Inria continue to crow about the PhartoVM (intellectual theft) but contribute nothing materially to improving the VM.  The Potsdam folks continue to work on Graal, leaving this more interesting and more concentrative, all in Smalltalk work alone.

I feel very negative about the various communities' inability to support, and especially, join in this work.  I find the position of Inria downright unethical; intellectual theft, deeply selfish.  I find the lack of participation by the Potsdam folks very dispiriting.

What about debugging/simulating code?

No difference.  It is a bytecode set.

As someone who did not yet deal with any VM side effects, I would be interested to hear about the practical effects you can see when playing around with Context & Compiler stuff.

Again there should be no effects.  There may be bugs lurking in the bytecode set specific code (see the class side of BytecodeEncoder, EncoderForv3, EncoderForV3PlusClosures and EncoderForSistaV1).  The main source of bugs for the SistaV1 bytecode set is in its use of prefix bytecodes to extend the ranges of operands (see the class comment).  The scheme is more complex than a non-prefix bytecode set, but I've been using the set for a year now and have fixed a few bugs over that time, the last known bug having been fixed in September of last year.

I read about FullBlockClosure which appears to be kind of detached of its defining method. How will this change accessing thisContext home etc. from a FullBlockClosure, for example?

It does not change semantics at all.  It only changes where the bytecodes and literals for a block reside, not what their execution effects.

I read Sista stores an optimization cache between several runs. Where will this information be saved, at the image side or at the VM side?

Scorch, the optimizer that exploits the unsafe primitives in the Sista bytecode set, optimizes by inlining unoptimized byte coded methods into byte coded methods which can contain unsafe primitives.  Execution is faster due to the elimination of overheads (such as block creation) through inlining and through the use of unsafe primitives, proved safe by the Scorch optimizer, to perform arithmetic, object access, and instantiation.  The optimized methods are stored in normal method dictionaries, shadowing their unoptimized source methods. So they persist in method dictionaries.

Lots of questions, maybe someone can answer a few of them :-)

I have a question.
Why is no one in the community joining me and Clément in trying to finish this work?



Von: Squeak-dev <squeak-dev-bounces at lists.squeakfoundation.org<mailto:squeak-dev-bounces at lists.squeakfoundation.org>> im Auftrag von David T. Lewis <lewis at mail.msen.com<mailto:lewis at mail.msen.com>>
Gesendet: Samstag, 7. März 2020 18:29:40
An: squeak-dev at lists.squeakfoundation.org<mailto:squeak-dev at lists.squeakfoundation.org>
Betreff: Re: [squeak-dev] Switching to Sista bytecodes in trunk (was: The Inbox: Kernel-dtl.1310.mcz)

I will leave this in the inbox for a couple of days, and if there are
no objections I will move it to trunk.


On Fri, Mar 06, 2020 at 07:43:29PM -0500, David T. Lewis wrote:
> The Sista bytecode set enables important reasearch and development work.
> Eliot and Clemont can explain better than me (apologies to Clemont Bera
> for using my seven-bit character set on your name).
> Squeak 5.3 is released and we are starting the new release cycle, so it
> is time to make Sista the default in trunk. Kernel-dtl.1310 (in the inbox)
> activates Sista in the package postscipt.
> Dave
> On Sat, Mar 07, 2020 at 12:29:13AM +0000, commits at source.squeak.org<mailto:commits at source.squeak.org> wrote:
> > David T. Lewis uploaded a new version of Kernel to project The Inbox:
> > http://source.squeak.org/inbox/Kernel-dtl.1310.mcz
> >
> > ==================== Summary ====================
> >
> > Name: Kernel-dtl.1310
> > Author: dtl
> > Time: 6 March 2020, 7:29:11.316779 pm
> > UUID: 683e4e14-fc18-4d55-a776-eece91f579f5
> > Ancestors: Kernel-mt.1309
> >
> > Change the Squeak default bytecode set to Sista in trunk.
> >
> > Add CompiledCode>>multipleBytecodeSetsActive: to optionally inform the VM that Sista is in use, see VMMaker.oscog-dtl.2711 and http://lists.squeakfoundation.org/pipermail/vm-dev/2020-January/032441.html.
> >
> > Add CompiledCode>>useSista: convenience method for switching to and from Sista bytecodes.
> >
> > Package postscript activates the change to Sista bytecodes, which can be reversed by evaluating CompiledCode useSista: false.
> >
> > =============== Diff against Kernel-mt.1309 ===============
> >
> > Item was added:
> > + ----- Method: CompiledCode class>>multipleBytecodeSetsActive: (in category 'method encoding') -----
> > + multipleBytecodeSetsActive: aBoolean
> > +    "Inform the VM when multiple bytecode sets, typically the Sista bytecodes
> > +    in addition to the traditional V3 bytecode set, are now in use is this image.
> > +    The VM may use this information to update the image format number when
> > +    saving the image to the file system."
> > +
> > +    <primitive: 'primitiveMultipleBytecodeSetsActive'>
> > + !
> >
> > Item was added:
> > + ----- Method: CompiledCode class>>useSista: (in category 'method encoding') -----
> > + useSista: useSistaEncoder
> > +    "Switch to or from the Sista bytecode encoder, and recompile the system
> > +    using that encoder. Assumes that Compiler recompileAll is working for the
> > +    existing system. Assumes that the currently available primary and secondary
> > +    bytecode encoders are EncoderForV3PlusClosures and EncoderForSistaV1.
> > +    This is a convenience method that must be updated as the available encoders
> > +    are changed."
> > +
> > +    "CompiledCode useSista: true"
> > +    "CompiledCode useSista: false"
> > +
> > +    | standardEncoder sistaEncoder activeEncoder |
> > +    standardEncoder := Smalltalk classNamed: #EncoderForV3PlusClosures.
> > +    sistaEncoder := Smalltalk classNamed: #EncoderForSistaV1.
> > +    activeEncoder := self preferredBytecodeSetEncoderClass.
> > +    useSistaEncoder
> > +            ifTrue: [sistaEncoder ifNil: [self error: 'EncoderForSistaV1 not present in this image'].
> > +                    self preferredBytecodeSetEncoderClass: sistaEncoder.
> > +                    activeEncoder ~= sistaEncoder
> > +                            ifTrue: [(Smalltalk classNamed: #Compiler) recompileAll.
> > +                    self multipleBytecodeSetsActive: true "VM should support Sista plus V3" ]]
> > +            ifFalse: [standardEncoder ifNil: [self error: 'EncoderForV3PlusClosures not present in this image'].
> > +                    self preferredBytecodeSetEncoderClass: standardEncoder.
> > +                    activeEncoder ~= standardEncoder
> > +                            ifTrue: [(Smalltalk classNamed: #Compiler) recompileAll.
> > +                    self multipleBytecodeSetsActive: false "VM needs to support V3 only" ]].
> > +
> > + !
> >
> > Item was changed:
> > + (PackageInfo named: 'Kernel') postscript: '"Activate Sista bytecodes in the image"
> > +
> > + CompiledCode useSista: true.
> > + '!
> > - (PackageInfo named: 'Kernel') postscript: '"below, add code to be run after the loading of this package"
> > - "Since Kernel-eem.1198 redefines LargePositiveInteger hash,
> > -  rehash all hashed collections that contain hashed large integers."
> > - HashedCollection allSubclassesDo:
> > -    [:c| | f |
> > -    f := (c includesBehavior: Set)
> > -                    ifTrue: [[:i| i]]
> > -                    ifFalse: [[:i| i keys]].
> > -    c allInstancesDo:
> > -            [:h|
> > -             ((f value: h) detect: [:e| e isInteger and: [e class ~~ SmallInteger]] ifNone: nil) ifNotNil:
> > -                    [h rehash]]]'!
> >
> >

best, Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20200317/e5d77851/attachment.html>

More information about the Squeak-dev mailing list