Hi All,
I've released a new version of Cog that has a substantially improved code generator along the lines of Peter Deutsch's HPS (VisualWorks) and various of Ian Piumarta's VMs. These all use a simple tecnique to identify constant references in bytecode and to support a register-based calling convention. While this does produce faster code it tends to accelerate low-level code much more than high-level code as you can see by the following benchmarks:
SimpleStackBasedCogit: [1 to: 100000000 do: [:i|]] timeToRun 691 StackToRegisterMappingCogit: [1 to: 100000000 do: [:i|]] timeToRun 192 192 - 691 / 6.91 -72%
SimpleStackBasedCogit: 0 tinyBenchmarks '753495217 bytecodes/sec; 64769127 sends/sec' StackToRegisterMappingCogit: 0 tinyBenchmarks '931756141 bytecodes/sec; 128157989 sends/sec' 931756141 - 753495217 / 7534952.17 -24% 128157989 - 64769127 / 647691.27 -98%
SimpleStackBasedCogit: [Compiler recompileAll] timeToRun 47013 (no transcript StackToRegisterMappingCogit: [Compiler recompileAll] timeToRun 43406 (no transcript) 43406 - 47013 / 470.13 -7.67234594686576
The status of this code is essentially beta. The test suite runs the same on the new code generator as on the old, but I think there are still bugs because I get the occasional transient error. I am therefore very interested in any reproducible errors you can find.
The VMs (http://www.mirandabanda.org/files/Cog/VM/VM.r2334/) contain a few other important changes:
- a bug fix to bytecode<->native pc mappng that produced incorrect results for methods containing blocks with ^-returns in them. One symptom is incorrect highlighting of the pc in the debugger, althoguh symptoms could be much serious.
- jitting interpreted methods on backward branches. Currently any interpreted method that performs more than 20 backward branches will be considered for JIT compilation and if it is suitable (default, <= 60 literals) will be compiled to native code.
- new callback support. I need to commit some changes to the Alien package to provide access to this but essentially the VM's callback support is now able to be ported to architectures with register-based calling conventions (ARM, PowerPC, SPARC etc). I'll try and get the Alien code released soon, and to back-port the changes to the standard VM before the end of the holiday.
One thing that is still /not/ fixed is the lack of a SoundPlugin on win32. Apologies. I'll try and get a fix for this before the end of the holidays too, but time might be too tight. There are other priorities such as harmonising the standard and Cog VMs for the 4.2 release.
best Eliot
Hi Eliot, Thanks for the new version!
And here's the first bug report, reproducible :)
http://dl.dropbox.com/u/17707726/image.tar.xz Using this image (pharo 1.1.1 from Lukas' hudson with omnibrowser & friends & seaside loaded, pharov10.sources) - opening the settings from the world menu crashes the VM. The output is named 'out1' in the same archive.
Regards, Stanislav Paskalev
On Fri, Dec 31, 2010 at 4:02 AM, Eliot Miranda eliot.miranda@gmail.com wrote:
Hi All, I've released a new version of Cog that has a substantially improved code generator along the lines of Peter Deutsch's HPS (VisualWorks) and various of Ian Piumarta's VMs. These all use a simple tecnique to identify constant references in bytecode and to support a register-based calling convention. While this does produce faster code it tends to accelerate low-level code much more than high-level code as you can see by the following benchmarks: SimpleStackBasedCogit: [1 to: 100000000 do: [:i|]] timeToRun 691 StackToRegisterMappingCogit: [1 to: 100000000 do: [:i|]] timeToRun 192 192 - 691 / 6.91 -72% SimpleStackBasedCogit: 0 tinyBenchmarks '753495217 bytecodes/sec; 64769127 sends/sec' StackToRegisterMappingCogit: 0 tinyBenchmarks '931756141 bytecodes/sec; 128157989 sends/sec' 931756141 - 753495217 / 7534952.17 -24% 128157989 - 64769127 / 647691.27 -98% SimpleStackBasedCogit: [Compiler recompileAll] timeToRun 47013 (no transcript StackToRegisterMappingCogit: [Compiler recompileAll] timeToRun 43406 (no transcript) 43406 - 47013 / 470.13 -7.67234594686576 The status of this code is essentially beta. The test suite runs the same on the new code generator as on the old, but I think there are still bugs because I get the occasional transient error. I am therefore very interested in any reproducible errors you can find. The VMs (http://www.mirandabanda.org/files/Cog/VM/VM.r2334/) contain a few other important changes:
- a bug fix to bytecode<->native pc mappng that produced incorrect results for methods containing blocks with ^-returns in them. One symptom is incorrect highlighting of the pc in the debugger, althoguh symptoms could be much serious.
- jitting interpreted methods on backward branches. Currently any interpreted method that performs more than 20 backward branches will be considered for JIT compilation and if it is suitable (default, <= 60 literals) will be compiled to native code.
- new callback support. I need to commit some changes to the Alien package to provide access to this but essentially the VM's callback support is now able to be ported to architectures with register-based calling conventions (ARM, PowerPC, SPARC etc). I'll try and get the Alien code released soon, and to back-port the changes to the standard VM before the end of the holiday.
One thing that is still /not/ fixed is the lack of a SoundPlugin on win32. Apologies. I'll try and get a fix for this before the end of the holidays too, but time might be too tight. There are other priorities such as harmonising the standard and Cog VMs for the 4.2 release. best Eliot
Hi Stanislav,
what OS do you run the test on?
TIA Eliot
On Fri, Dec 31, 2010 at 12:35 AM, Stanislav Paskalev kshorg@gmail.comwrote:
Hi Eliot, Thanks for the new version!
And here's the first bug report, reproducible :)
http://dl.dropbox.com/u/17707726/image.tar.xz Using this image (pharo 1.1.1 from Lukas' hudson with omnibrowser & friends & seaside loaded, pharov10.sources) - opening the settings from the world menu crashes the VM. The output is named 'out1' in the same archive.
Regards, Stanislav Paskalev
On Fri, Dec 31, 2010 at 4:02 AM, Eliot Miranda eliot.miranda@gmail.com wrote:
Hi All, I've released a new version of Cog that has a substantially improved
code generator along the lines of Peter Deutsch's HPS (VisualWorks) and various of Ian Piumarta's VMs. These all use a simple tecnique to identify constant references in bytecode and to support a register-based calling convention. While this does produce faster code it tends to accelerate low-level code much more than high-level code as you can see by the following benchmarks:
SimpleStackBasedCogit: [1 to: 100000000 do: [:i|]] timeToRun 691 StackToRegisterMappingCogit: [1 to: 100000000 do: [:i|]] timeToRun 192 192 - 691 / 6.91 -72% SimpleStackBasedCogit: 0 tinyBenchmarks '753495217 bytecodes/sec;
64769127 sends/sec'
StackToRegisterMappingCogit: 0 tinyBenchmarks '931756141 bytecodes/sec;
128157989 sends/sec'
931756141 - 753495217 / 7534952.17 -24% 128157989 - 64769127 / 647691.27 -98% SimpleStackBasedCogit: [Compiler recompileAll] timeToRun 47013 (no
transcript
StackToRegisterMappingCogit: [Compiler recompileAll] timeToRun 43406 (no
transcript)
43406 - 47013 / 470.13 -7.67234594686576 The status of this code is essentially beta. The test suite runs the
same on the new code generator as on the old, but I think there are still bugs because I get the occasional transient error. I am therefore very interested in any reproducible errors you can find.
The VMs (http://www.mirandabanda.org/files/Cog/VM/VM.r2334/) contain a
few other important changes:
- a bug fix to bytecode<->native pc mappng that produced incorrect
results for methods containing blocks with ^-returns in them. One symptom is incorrect highlighting of the pc in the debugger, althoguh symptoms could be much serious.
- jitting interpreted methods on backward branches. Currently any
interpreted method that performs more than 20 backward branches will be considered for JIT compilation and if it is suitable (default, <= 60 literals) will be compiled to native code.
- new callback support. I need to commit some changes to the Alien
package to provide access to this but essentially the VM's callback support is now able to be ported to architectures with register-based calling conventions (ARM, PowerPC, SPARC etc). I'll try and get the Alien code released soon, and to back-port the changes to the standard VM before the end of the holiday.
One thing that is still /not/ fixed is the lack of a SoundPlugin on
win32. Apologies. I'll try and get a fix for this before the end of the holidays too, but time might be too tight. There are other priorities such as harmonising the standard and Cog VMs for the 4.2 release.
best Eliot
I totally forgot about it :) Linux i386, Debian Squeeze
Stanislav Paskalev
On Fri, Dec 31, 2010 at 9:32 PM, Eliot Miranda eliot.miranda@gmail.com wrote:
Hi Stanislav, what OS do you run the test on? TIA Eliot
On Thu, Dec 30, 2010 at 06:02:36PM -0800, Eliot Miranda wrote:
Hi All,
I've released a new version of Cog that has a substantially improvedcode generator along the lines of Peter Deutsch's HPS (VisualWorks) and various of Ian Piumarta's VMs.
This is very impressive - thank you!
Dave
On Thu, 30 Dec 2010, Eliot Miranda wrote:
(pine can't quote your mail, sorry)
Thanks for the update, the numbers are impressive. I found an issue which is easily reproducible on windows. Just evaluate [Smalltalk garbageCollect] and the VM will crash. Since the crash is during GC the VM will also crash during the log writing, probably because the stack is corrupted. Here's the relevant part of crash.dmp:
Fri Dec 31 20:46:37 2010
Exception code: C0000005 Exception addr: 00407880 Access violation (read access) at B4650090 EAX:B4650090 EBX:B4650090 ECX:00000000 EDX:00000090 ESI:100353D7 EDI:00000000 EBP:008FDE48 ESP:008FDE1C EIP:00407880 EFL:00010246 FP Control: FFFF027F FP Status: FFFF4022 FP Tag: FFFFFFFF
Crashed in the VM thread Stack backtrace: [00407880] _remap + 8 in Croquet.exe [004280F9] _mapForperformUntilarg + 88 in Croquet.exe [004339F0] _mapObjectReferencesInMachineCode + 109 in Croquet.exe [00407A35] _mapPointersInObjectsFromto + 420 in Croquet.exe [00408CD1] _incCompBody + 261 in Croquet.exe [0041A922] _fullGC + 686 in Croquet.exe [00426584] _primitiveFullGC + 65 in Croquet.exe [0041E5A9] _interpret + 513 in Croquet.exe [00423147] _enterSmalltalkExecutiveImplementation + 97 in Croquet.exe [00423625] _initStackPagesAndInterpret + 517 in Croquet.exe [0043D1F5] _sqMain + 610 in Croquet.exe [0043D503] _WinMain@16 + 170 in Croquet.exe [00517C18] _main + 264 in Croquet.exe [0040124B] ___mingw_CRTStartup + 251 in Croquet.exe [004012B8] _WinMainCRTStartup + 24 in Croquet.exe [768DD0E9] AcquireSRWLockExclusive + 18 in kernel32.dll [77BE19BB] RtlInitializeNtUserPfn + 99 in ntdll.dll [77BE198E] RtlInitializeNtUserPfn + 54 in ntdll.dll
...
VM Version: Teleplace VM 1.0.15 (release) from Dec 30 2010 Compiler: gcc 3.4.4 (cygming special, gdc 0.12, using dmd 0.125) Interpreter Build: CoInterpreter VMMaker-oscog.40 uuid: 637db40c-33c6-4263-816e-1b8cc19e3c99 Dec 30 2010 Cogit Build: StackToRegisterMappingCogit VMMaker-oscog.40 uuid: 637db40c-33c6-4263-816e-1b8cc19e3c99 Dec 30 2010
Current byte code: 16 Primitive index: 0
Another way to reliably crash the VM is to open a file list and wait a few seconds. I'm not sure if it's related to GC or not, but the smalltalk stack trace is incomplete, just like in the previous case. In this case the dump is:
Exception code: C0000005 Exception addr: 00433864 Access violation (read access) at 1D1B6659 EAX:0000003B EBX:FFFF2308 ECX:1D1B6654 EDX:1D1B668F ESI:00549E28 EDI:004337FB EBP:008FDFB8 ESP:008FDFB0 EIP:00433864 EFL:00010202 FP Control: FFFF027F FP Status: FFFF4022 FP Tag: FFFFFFFF
Crashed in the VM thread Stack backtrace: [00433864] _relocateIfCallOrMethodReferencemcpcdelta + 105 in Croquet.exe [004280F9] _mapForperformUntilarg + 88 in Croquet.exe [004334CA] _compactCogCompiledCode + 914 in Croquet.exe [00402BFE] _commenceCogCompiledCodeCompaction + 146 in Croquet.exe [0041B3E2] _checkForEventsMayContextSwitch + 742 in Croquet.exe [0041B4F1] _handleStackOverflowOrEventAllowContextSwitch + 32 in Croquet.exe [0041C3F8] _activateNewMethod + 607 in Croquet.exe [0041D16B] _activateInterpreterMethodFromMachineCode + 194 in Croquet.exe [1002005C] ceSend0Args + 44 in CogCode [00423625] _initStackPagesAndInterpret + 517 in Croquet.exe [0043D1F5] _sqMain + 610 in Croquet.exe [0043D503] _WinMain@16 + 170 in Croquet.exe [00517C18] _main + 264 in Croquet.exe [0040124B] ___mingw_CRTStartup + 251 in Croquet.exe [004012B8] _WinMainCRTStartup + 24 in Croquet.exe [768DD0E9] AcquireSRWLockExclusive + 18 in kernel32.dll [77BE19BB] RtlInitializeNtUserPfn + 99 in ntdll.dll [77BE198E] RtlInitializeNtUserPfn + 54 in ntdll.dll
Levente
Hi All,
find new VMs in VM.r2344/http://www.mirandabanda.org/files/Cog/VM/VM.r2344/ (SimpleStackBasedCogit) and VM.r2345/ http://www.mirandabanda.org/files/Cog/VM/VM.r2345/ (StackToRegisterMappingCogit) and VMMaker-oscog-45. These fix a bug that could cause crashes when converting an interpreted method to a machine-code method on backward branch (i.e. when the Vm detects an interpreted method is looping).
I hope there are no new issues with VM.r2344/http://www.mirandabanda.org/files/Cog/VM/VM.r2344/ (SimpleStackBasedCogit). However, Matthew Fulmer has found that on linux the VM.r2343/http://www.mirandabanda.org/files/Cog/VM/VM.r2343/ VM.r2345/ http://www.mirandabanda.org/files/Cog/VM/VM.r2345/ (StackToRegisterMappingCogit) VMs crash immediately loading Squeak4.2-10856-beta.image. I can reproduce this but haven't had time to investigate. The StackToRegisterMappingCogit VMs do start up this image on Mac and Windows.
Agan thanks for pounding on these. I may not be able to attend to the linux/StackToRegisterMappingCogit/Squeak4.2-10856-beta.image until next weekend. Apologies.
best Eliot
On Thu, Dec 30, 2010 at 6:02 PM, Eliot Miranda eliot.miranda@gmail.comwrote:
Hi All,
I've released a new version of Cog that has a substantially improvedcode generator along the lines of Peter Deutsch's HPS (VisualWorks) and various of Ian Piumarta's VMs. These all use a simple tecnique to identify constant references in bytecode and to support a register-based calling convention. While this does produce faster code it tends to accelerate low-level code much more than high-level code as you can see by the following benchmarks:
SimpleStackBasedCogit: [1 to: 100000000 do: [:i|]] timeToRun 691 StackToRegisterMappingCogit: [1 to: 100000000 do: [:i|]] timeToRun 192 192 - 691 / 6.91 -72%
SimpleStackBasedCogit: 0 tinyBenchmarks '753495217 bytecodes/sec; 64769127 sends/sec' StackToRegisterMappingCogit: 0 tinyBenchmarks '931756141 bytecodes/sec; 128157989 sends/sec' 931756141 - 753495217 / 7534952.17 -24% 128157989 - 64769127 / 647691.27 -98%
SimpleStackBasedCogit: [Compiler recompileAll] timeToRun 47013 (no transcript StackToRegisterMappingCogit: [Compiler recompileAll] timeToRun 43406 (no transcript) 43406 - 47013 / 470.13 -7.67234594686576
The status of this code is essentially beta. The test suite runs the same on the new code generator as on the old, but I think there are still bugs because I get the occasional transient error. I am therefore very interested in any reproducible errors you can find.
The VMs (http://www.mirandabanda.org/files/Cog/VM/VM.r2334/) contain a few other important changes:
- a bug fix to bytecode<->native pc mappng that produced incorrect results
for methods containing blocks with ^-returns in them. One symptom is incorrect highlighting of the pc in the debugger, althoguh symptoms could be much serious.
- jitting interpreted methods on backward branches. Currently any
interpreted method that performs more than 20 backward branches will be considered for JIT compilation and if it is suitable (default, <= 60 literals) will be compiled to native code.
- new callback support. I need to commit some changes to the Alien package
to provide access to this but essentially the VM's callback support is now able to be ported to architectures with register-based calling conventions (ARM, PowerPC, SPARC etc). I'll try and get the Alien code released soon, and to back-port the changes to the standard VM before the end of the holiday.
One thing that is still /not/ fixed is the lack of a SoundPlugin on win32. Apologies. I'll try and get a fix for this before the end of the holidays too, but time might be too tight. There are other priorities such as harmonising the standard and Cog VMs for the 4.2 release.
best Eliot
Hi All,
I've uploaded new VMs to VM.r2348/http://www.mirandabanda.org/files/Cog/VM/VM.r2348/ (SimpleStackBasedCogit) and VM.r2349/ http://www.mirandabanda.org/files/Cog/VM/VM.r2349/ (StackToRegisterMappingCogit). The former are simply to be up-to-date with svn & Monticello. The latter also fix a bad bug with the StackToRegisterMappingCogit where e.g. 16r10 bitOr: 16r20 would evaluate to 1. I've smoke-tested the 2349 VMs on the Squeak4.2 beta test suite and things look good.
best Eliot
vm-dev@lists.squeakfoundation.org
-
David T. Lewis -
Eliot Miranda -
Levente Uzonyi -
Stanislav Paskalev