Exupery November 2006

exupery@lists.squeakfoundation.org

7 participants
7 discussions

Thinking about the next release.
by bryce＠kampjes.demon.co.uk 26 Nov '06

26 Nov '06

At the moment I'd like to get the following into the next release of Exupery: * A new primitive. * A new context class for Exupery's method contexts. * Return true and false primitives. * Get the stress test running in 3.9 The new primitive should mean that compiling the large explorers benchmark will always be beneficial. At the moment if it tries to inline point creation then there's a 5-10% speed loss. New is also a useful building block because it's used in many other primitives. The new context class is to interact with the debugger and other tools. It will also provide an easy place to put decompilation hooks if a context is being manipulated via reflection. The goal is to have the debugger and profilers work nicely with Exupery's compiled methods again. Creating a new context class to do this should make it easier to add full method inlining in 2.0. Return true and false primitives are to enable these simple primitives to be compiled and inlined as they do almost nothing. At the moment compiled code does a full primitive lookup to execute these primitives while the interpreter inlines them into commonSend. The stress test crashes again. This looks like it's because there are more tests in 3.9 and some of those tests cause Exupery problems. I also think there may be a bug or two due to upgrading VMMaker to 3.8b6. That just leaves implementing interrupt checking in compiled code before Exupery is ready to begin running in the background. I suspect that the register allocator will need a bit of tuning to speed up compile times and more bugs will need to be fixed before background compilation is practical. Bryce

1 0

Common Smalltalk VM Summit
by David Griswold 20 Nov '06

20 Nov '06

Hi everybody, Dan Ingalls and I have been talking, trying to figure out what to do about the major opportunity offered by the recent release of the Strongtalk virtual machine as open source. Rather than keep this discussion to ourselves, our thinking was that this would be the perfect time to call a kind of summit, with representatives of all the major Smalltalk implementations, both open-source and commercial. The topic: what if we could build a shared high-performance open-source platform suitable for hosting a number of different Smalltalk systems, one that we can all share and work on together? While the details of the type-feedback techniques used in the Strongtalk VM are arcane, the benefits are not: *much* higher performance for general Smalltalk code. Dan, myself, and many others who know about type-feedback and the pioneering Self system, have been dreaming for many years about the possibility that someday this technology might make it into mainstream Smalltalk VMs. It would take Smalltalk performance to a whole new level. That someday is here now, if the different factions within the Smalltalk community can pull together a little bit so that we don't miss this opportunity. There may be debate within the community about some aspects of the Strongtalk project, for example the type system, but we should all be able to agree on the simple idea that a whole lot more performance would be a Good Thing. Now a huge performance gift has suddenly shown up on our doorstep. The last thing Smalltalk needs is another incompatible implementation. The splintering of Smalltalk implementations has dispersed the huge amount of talent and effort needed to build, port, maintain, and extend a really good virtual-machine. Alone, this is a problem for each of us. Together, a really good, super-fast type-feedback VM is for the first time within reach. I would like to invite the smart people out there who know and care most about the various Smalltalk virtual machines, to join Dan and I in a fairly focused discussion about this starting tomorrow (Thursday, PST) on the Strongtalk discussion group, at http://groups.google.com/group/strongtalk-general. I will be out of the country for 6 weeks starting Wed the 11th, so I would like to propose that we try to go back and forth about this a few times by the end of Friday, so we can think about this over the weekend, and maybe come up with a proposed general course of action by the middle of next week, so we all have something to think about until my return. Let's not lose this opportunity. Cheers, Dave

3 2

exupery vm
by Chris Muller 17 Nov '06

17 Nov '06

One more question. How safe is it to stick with the exupery vm for my non-exupery-loaded images?

4 4

testing with new 0.1
by Chris Muller 16 Nov '06

16 Nov '06

Hi Bryce, thanks for the new release. I have been playing around with it and have some questions. First, I was surprised to see the suggestion to compile Array>>at: and Array>>at:put:. These methods are already inlined, right? They inherit the implementation in Object which are already both primitives. Primitives are already "compiled" right? So how does Exupery speed them up? Overall, does it ever make sense to compile a method with just a single send in it? Does Exupery dive into the called method and compile that too? If not, I don't see how it could help unless some inlining was done.. This allso led to my other question. These Swiki instructions tell Exupery to compile an inherited method, not an actual method that exists on Array. Since we don't pass the actual CompiledMethod object to Exupery (e.g., Exupery compile: Array>>#at:), instead a selector and a class, what interpretations about these does Exupery make with respect to inheritance. Forging onward, I decided to piggyback on your easy approach provided by ExuperyBenchmarks to experiment so I can easily see results of compiling some of my own methods. I have these two methods which set or get an unsigned integer into a ByteArray. Because of the lesser results I get, I want to post their implementation right in the email so you may have an idea.. ByteArray>>maUint: bits at: anInteger | answer bytes | bits == 64 ifTrue: [ ^ self maUnsigned64At: anInteger + 1 ]. bits == 56 ifTrue: [ ^ self maUnsigned56At: anInteger + 1 ]. bits == 48 ifTrue: [ ^ self maUnsigned48At: anInteger + 1 ]. bits == 40 ifTrue: [ ^ self maUnsigned40At: anInteger + 1 ]. bits == 32 ifTrue: [ ^self unsignedLongAt: anInteger + 1 bigEndian: false ]. bits == 16 ifTrue: [ ^self unsignedShortAt: anInteger + 1 bigEndian: false ]. bits == 8 ifTrue: [ ^self byteAt: anInteger + 1 ]. bytes _ bits // 8. answer _ LargePositiveInteger new: bytes. 1 to: bytes do: [ :digitPosition | answer digitAt: digitPosition put: (self at: digitPosition + anInteger) ]. ^answer normalize and ByteArray>>#maUint: bits at: position put: anInteger position + 1 to: position + (bits // 8) do: [ :pos | self at: pos put: (anInteger digitAt: pos-position) ]. ^anInteger Now, when I just compiled these two methods alone, I get: maUintAtPutBenchmark 5625 compiled 7912 ratio: 0.711 maUintAtBenchmark 2610 compiled 3474 ratio: 0.751 Cumulative Time 27.124 compiled 30.748 ratio 0.882 So, this tells me that compiling *can* actually make things worse. Is there any way for Exupery to detect and prevent this or is this strictly the user/developers responsibility to profile everything for comparison? Next I tried adding compilation of some of the lower-level methods called by these methods. I first just added Exupery compileMethod: #at: class: ByteArray Normally I wouldn't do this but since the Swiki suggested compiling Array>>#at: I thought it worth a try. But it caused the stack-trace at the end of this e-mail. I tried just a few more experiments such as SmallInteger>>#digitAt: and Integer>>#bitShift: but generally couldn't get results above 1.0. BTW, on a completely separate experiment, one method I tried to compile it said "Unknow bytecode" (it was bytecode 136). Ok, so obviously I'm a novice at this! This project is exciting and I'm at least I know Exupery is doing something (compared to my last attempt where I didn't even know to compile methods), but hope that I can figure out how to actually make it go *faster*. :) Can you help me with any advice / guidelines for *what* to compile? Thanks, Chris 15 November 2006 10:53:18 pm VM: Win32 - a SmalltalkImage Image: Squeak3.8 [latest update: #6665] SecurityManager state: Restricted: false FileAccess: true SocketAccess: true Working Dir C:\Development\Chris\Development\Squeak Trusted Dir C:\Development\Chris\Development\Squeak\Chris Untrusted Dir C:\My Squeak\Chris IntermediateSimplifier>>primitiveAt: Receiver: an IntermediateSimplifier Arguments and temporary variables: aMedPrimitive: #(#primitive 60 'block6' #(#mem #(#add #(#mem #activeContext) #(...etc... methodBytecodes: nil index: nil resultAddress: nil receiver: nil Receiver's instance variables: source: a MedMethod result: a MedMethod emitter: an IntermediateEmitter currentBlock: (block1 (primitiveReturn #(#primitive 60 'block6' #(#mem #(#add ...etc... stack: an OrderedCollection() simplifier: an IntermediateSimplifier stacksForBlocks: a Dictionary() IntermediateSimplifier>>visitPrimitive: Receiver: an IntermediateSimplifier Arguments and temporary variables: aMedPrimitive: #(#primitive 60 'block6' #(#mem #(#add #(#mem #activeContext) #(...etc... Receiver's instance variables: source: a MedMethod result: a MedMethod emitter: an IntermediateEmitter currentBlock: (block1 (primitiveReturn #(#primitive 60 'block6' #(#mem #(#add ...etc... stack: an OrderedCollection() simplifier: an IntermediateSimplifier stacksForBlocks: a Dictionary() MedPrimitive>>visitWith: Receiver: #(#primitive 60 'block6' #(#mem #(#add #(#mem #activeContext) #(#add #(#sal #(#mem #(#add ...etc... Arguments and temporary variables: aTreeOptimiser: an IntermediateSimplifier Receiver's instance variables: in: nil out: nil primitiveName: primitive 60 arguments: #(#(#mem #(#add #(#mem #activeContext) #(#add #(#sal #(#mem #(#add #...etc... failureBlock: (block6 (createContext) (deconvertBoolean falseObj #(#send #(#m...etc... receiver: ByteArray IntermediateSimplifier>>visitPrimitiveReturn: Receiver: an IntermediateSimplifier Arguments and temporary variables: aMedPrimitiveReturn: #(#primitiveReturn 2 #(#primitive 60 'block6' #(#mem #(#ad...etc... Receiver's instance variables: source: a MedMethod result: a MedMethod emitter: an IntermediateEmitter currentBlock: (block1 (primitiveReturn #(#primitive 60 'block6' #(#mem #(#add ...etc... stack: an OrderedCollection() simplifier: an IntermediateSimplifier stacksForBlocks: a Dictionary() --- The full stack --- IntermediateSimplifier>>primitiveAt: IntermediateSimplifier>>visitPrimitive: MedPrimitive>>visitWith: IntermediateSimplifier>>visitPrimitiveReturn: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - MedPrimitiveReturn>>visitWith: [] in IntermediateSimplifier(IntermediateCopier)>>simplifyBlock: {[:each | emitter addExpression: (each visitWith: self)]} OrderedCollection>>do: MedBlock>>instructionsDo: IntermediateSimplifier(IntermediateCopier)>>simplifyBlock: IntermediateSimplifier>>simplifyBlock: [] in IntermediateSimplifier>>visitMethod: {[:each | self prepareBlockFrom: each. self simplifyBlock: each]} OrderedCollection>>do: IntermediateSimplifier>>visitMethod: MedMethod>>visitWith: IntermediateSimplifier(IntermediateCopier)>>run Exupery>>convertIntermediate Exupery>>run Exupery class>>compileMethod:inlining:receiver: Exupery class>>compileMethod:into:forClass:inlining: Exupery class>>compileMethod:class:inlining: Exupery class>>compileMethod:class: ExuperyBenchmarks>>compilemaUintAt ExuperyBenchmarks>>runBenchmark:compilingWith: [] in ExuperyBenchmarks>>run {[:each | self runBenchmark: (each at: 1) compilingWith: (each at: 2)]} Array(SequenceableCollection)>>do: ExuperyBenchmarks>>run UndefinedObject>>DoIt Compiler>>evaluate:in:to:notifying:ifFail:logged: [] in TextMorphEditor(ParagraphEditor)>>evaluateSelection {[rcvr class evaluatorClass new evaluate: self selectionAsStream in: ctxt...]} BlockContext>>on:do: TextMorphEditor(ParagraphEditor)>>evaluateSelection TextMorphEditor(ParagraphEditor)>>doIt [] in TextMorphEditor(ParagraphEditor)>>doIt: {[self doIt]} TextMorphEditor(Controller)>>terminateAndInitializeAround: TextMorphEditor(ParagraphEditor)>>doIt: TextMorphEditor(ParagraphEditor)>>dispatchOnCharacter:with: TextMorphEditor>>dispatchOnCharacter:with: TextMorphEditor(ParagraphEditor)>>readKeyboard ...etc...

2 1

[ANN] Exupery 0.10 released, now on Windows and Linux
by bryce＠kampjes.demon.co.uk 16 Nov '06

16 Nov '06

Exupery 0.10 is now released. There are prebuilt VM's available for both Windows and Linux. This release now provides a measurable speed improvement for the compilerBenchmark macro benchmark due to work on dynamic primitive inlining. Instructions for installation and a link to a pre-built image is here: http://minnow.cc.gatech.edu/squeak/3945 Benchmarks on my Athlon 64 3500+ ========================================================= arithmaticLoopBenchmark 1398 compiled 92 ratio: 15.196 bytecodeBenchmark 2134 compiled 469 ratio: 4.550 sendBenchmark 1580 compiled 697 ratio: 2.267 doLoopsBenchmark 1090 compiled 840 ratio: 1.298 largeExplorers 334 compiled 358 ratio: 0.933 compilerBenchmark 733 compiled 705 ratio: 1.040 Cumulative Time 4167 compiled 1448 ratio 2.878 1,067,222,511 bytecodes/sec; 16,716,421 sends/sec Benchmarks on Andy's Mobile Pentium 3 ========================================================= arithmaticLoopBenchmark 2487 compiled 285 ratio: 8.726 bytecodeBenchmark 4271 compiled 1255 ratio: 3.403 sendBenchmark 3482 compiled 1772 ratio: 1.965 doLoopsBenchmark 2078 compiled 1663 ratio: 1.250 largeExplorers 2224 compiled 1683 ratio: 1.321 compilerBenchmark 2093 compiled 1712 ratio: 1.223 Cumulative Time 12903 compiled 4971 ratio 2.596 Benchmarks from my Pentium-M laptop ========================================================= arithmaticLoopBenchmark 1003 compiled 191 ratio: 5.251 bytecodeBenchmark 1773 compiled 683 ratio: 2.596 sendBenchmark 1446 compiled 922 ratio: 1.568 doLoopsBenchmark 991 compiled 918 ratio: 1.080 largeExplorers 418 compiled 441 ratio: 0.948 compilerBenchmark 718 compiled 683 ratio: 1.051 Cumulative Time 3773 compiled 2015 ratio 1.872 It's interesting that on Andy's machine Exupery is providing a nice performance improvement for largeExplorers while on my machine there is a 7% performance loss. The loss is due to the interpreter inlining Point>>@ into the main interpreter loop while Exupery executes it as a normal primitive. Andy's benchmarks are promising enough for a 1.0, pity relative performance isn't so high on the other two machines. There is a mailing list for those interested in the project here: http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/exupery Many thanks to Andy Tween for doing the Windows port and building the official Windows VM. Thanks also to Patrick Mauritz for doing a Solaris x86 port which was the first OS port. Bryce

2 2

Is it worth delaying the release?
by bryce＠kampjes.demon.co.uk 14 Nov '06

14 Nov '06

Is it worth delaying the release until after the 3.9 upgrade and the VMMaker upgrade? I've just started moving to a 3.9 image. There are 24 failing tests, most are because 3.9 produces different bytecodes than 3.8 for some things. This seems to be due to block processing but I haven't fully investigated. There are a few other failures including one due to a bug in 3.9. Is it more useful for me to release as is. A solid release on 3.8 before moving to 3.9 or to upgrade everything now. I'm going to upgrade next anyway. I'm not going to maintain different tests for both releases so once I fix them for 3.9 they will fail in 3.8. Bryce

4 13

Exupery 0.10 is nearly ready.
by bryce＠kampjes.demon.co.uk 12 Nov '06

12 Nov '06

It passes all it's tests and it also passes the stress test without any compiler errors. This is a first, until now the stress test only passed without crashing, a few methods failed to compile due to caught compiler bugs. Here's some performance figures: =============================== arithmaticLoopBenchmark 1401 compiled 92 ratio: 15.228 bytecodeBenchmark 2139 compiled 459 ratio: 4.660 sendBenchmark 1580 compiled 683 ratio: 2.313 doLoopsBenchmark 1093 compiled 843 ratio: 1.297 largeExplorers 344 compiled 373 ratio: 0.922 compilerBenchmark 739 compiled 691 ratio: 1.069 Cumulative Time 4205 compiled 1443 ratio 2.914 I'm now trying to build a 3.9 Exupery image on top of the squeak-dev images. If it works, I'll add it to the release. The release is on SqueakMap but not yet published. Bryce

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Exupery November 2006