[Vm-dev] Simulator & Pharo [was: Second coming of Java...]

Wed Nov 27 22:12:20 UTC 2013

Hi Phil,

answers in-line.

On Wed, Nov 27, 2013 at 1:34 PM, phil at highoctane.be <phil at highoctane.be>wrote:

>
> Hello,
>
> I gave it a shot based on the generator.image that PharoVM uses to
> generate the VM.
>
> But none of :
>
> (| vm |
> vm := StackInterpreterSimulator newWithOptions: #().
> vm openOn: '/Users/eliot/Cog/startreader.image'.
> vm openAsMorph; run)
>
> and
>
> (| vm |
> vm := CogVMSimulator newWithOptions: #(ObjectMemory
> Spur32BitCoMemoryManager
>  Cogit StackToRegisterMappingCogit).
> vm desiredNumStackPages: 8.
> vm objectMemory setCheckForLeaks: 1.
> vm openOn: '/Users/eliot/Cog/spurreader.image'.
> vm openAsMorph; toggleTranscript; halt; run)
>
> is going to work right away as newWithOptions: isn't there in that image.
>

If you try and load e.g. VMMaker.oscog-eem.527 into a Squeak trunk image do
you get further?  i.e. if you follow the instructions on
http://www.mirandabanda.org/cogblog/build-image/ do you get further?

> So, there is a initializeWithOptions: #().
>
> When trying out that one, I do get another error.
>
> In fact, the initializeWithOptions: method is wrong:
>
> initializeWithOptions: optionsDictionary
>  "ObjectMemory initializeWithOptions: Dictionary new"
>
> self initBytesPerWord: (optionsDictionary at: #BytesPerWord ifAbsent: [4]).
>
> ...
>
> the thing is doing #BytesPerWork ifAbsent: ... :-(
>
> Of course, as #() is an Array an not a Dictionary.
>
> So, I tried:
>
> StackInterpreterSimulator initializeWithOptions: Dictionary new.
> vm := StackInterpreterSimulator new.
>
> Which got the ball rolling.
>
> But then, of course, same story as here:
>
>
> http://forum.world.st/Got-quot-Error-basicNew-failed-quot-when-running-InterpreterSimulator-td3835723.html
>
>
> Well, based on Clement's comments, I fiddled a bit with the
> TranscriptStream, which I turned into a simple Transcript and let go of the
> local image name for the display window.
>
> Also, did
>
> transcript := Transcript.
> displayForm := (ImageMorph fromString: 'Display has not yet been
> installed') form.
>
> in StackInterpreterSimulator initialize.
>
>
> So, here is the current stance:
>
> | vm |
> StackInterpreterSimulator initializeWithOptions: Dictionary new.
> vm := StackInterpreterSimulator new.
> vm openOn: 'C:\MinGW\msys\1.0\home\User\pharo\pharo.image'.
> vm openAsMorph.
> vm run.
>
>
> the morph opens.
>
> Now, when I do run, I do get a problem in:
>
> InterpreterStackPage>>headFP: pointer
>
>
> There is an self assert: (pointer = 0 or: [pointer < baseAddress and:
> [realStackLimit - (LargeContextBytes / 2) <= pointer]]).
>
> And LargeContextBytes is nil.
>
> I looked around for initialization for that one.
>
> It is only in the InterpreterStackPage class >> initialize
>

Hmmm, that's an initialization order problem.  The code assumes
LargeContextSize is initialized first.  It might not be.  I've just changed
the code so that it refers to LargeContextSize directly and I've removed
LargeContextBytes.  I recommend you do the same.

So I did;
>
> | vm |
> StackInterpreterSimulator initializeWithOptions: Dictionary new.
> vm := StackInterpreterSimulator new.
> InterpreterStackPage initialize.
> vm openOn: 'C:\MinGW\msys\1.0\home\User\pharo\pharo.image'.
> vm openAsMorph.
> vm run.
>
> Then I got a blinking square running in the top corner.
>
> No amount of stopping processes would interrupt it.
>

Use the process browser.  That is a good sign.  It is being produced by
either StackInterpreterSimulator>>ioRelinquishProcessorForMicroseconds: or
CogVMSimulator>>ioRelinquishProcessorForMicroseconds: and means the VM's
idle process is running.  Looks like the system is working to some degree.

You'll probably find that a few plugin primitives are failing on start-up,
and that the start-up sequence has tried to open a notifier before the GUi
is initialized.  In this case I've tried to either a) make the relevant
primitives simulate correctly, or b) used an egregious hack to not perform
certain initializations.  e.g. in this launch script the initializations of
ObjectiveCAlien, ObjectiveCClass et al are not executed in the launch of a
Newspeak image:

| cos |
cos := CogVMSimulator newWithOptions: #(NewspeakVM true
MULTIPLEBYTECODESETS true Cogit StackToRegisterMappingCogit).
cos desiredNumStackPages: 8.
cos openOn: '/Users/eliot/Newspeak/newsqueakV4/absentSendBug.image'.
cos setBreakSelector: #testNewCompiler.
cos
openAsMorph;
"toggleTranscript;"
forShortPrintString: 'class ObjectiveCAlien' filterPerformMessages:
#(startUp:);
forShortPrintString: 'class ObjectiveCClass' filterPerformMessages:
#(startUp:);
forShortPrintString: 'class BlackMarket' filterPerformMessages: #(startUp:);
forShortPrintString: 'class ThisOSProcess' filterPerformMessages:
#(startUp:);
halt;
run

This works by hacking perform in CogVMSimulator.  You'll have to port the
code to StackInterpreterSimulator to get it to work there.

So the simulator is running the run loop:
>
> self fetchNextBytecode.
> [true] whileTrue:
>  [self assertValidExecutionPointers.
>  atEachStepBlock value. "N.B. may be nil"
>  self dispatchOn: currentBytecode in: BytecodeTable.
>   self incrementByteCount].
>
> and relinquishing processor every once in a while and this square is shown
> (Display reverse below] ...
>
> (which is where I end up when interrupting)
>
> ioRelinquishProcessorForMicroseconds: microseconds
> "In the simulator give an indication that we're idling and check for
> input."
>  Display reverse: (0 at 0 extent: 16 at 16).
> Sensor peekEvent ifNotNil:
> [self forceInterruptCheck].
>  Processor activeProcess == UIManager default uiProcess ifTrue:
> [World doOneCycle].
> microseconds >= 1000
>  ifTrue: [(Delay forMilliseconds: microseconds + 999 // 1000) wait]
> ifFalse: [Processor yield]
>
> If one could tell me how to get the display to show, it would be welcome.
>
> The display instVar contains stuff in the debugger but isn't ok to open
> with something like:
>
> In the transcript, I do see things related to initialization:
>
> (56) Looking for primitiveSetGCBiasToGrowGCLimit in vm
> Looking for module  ... not found
> (3751) Looking for primitiveFileWrite in FilePlugin
> Looking for module FilePlugin
> (3751) Looking for secCanCreatePathOfSize in SecurityPlugin
> Looking for module SecurityPlugin ... loaded
> (3751) Looking for secCanDeletePathOfSize in SecurityPlugin
> (3751) Looking for secCanGetFileTypeOfSize in SecurityPlugin
> (3751) Looking for secCanListPathOfSize in SecurityPlugin
> (3751) Looking for secCanSetFileTypeOfSize in SecurityPlugin
> (3751) Looking for secDisableFileAccess in SecurityPlugin
> (3751) Looking for secCanDeleteFileOfSize in SecurityPlugin
> (3751) Looking for secCanOpenFileOfSizeWritable in SecurityPlugin
> (3751) Looking for secCanRenameFileOfSize in SecurityPlugin
> (3751) Looking for secHasFileAccess in SecurityPlugin ... loaded
> (3774) Looking for primitiveFileSize in FilePlugin
> (4390) Looking for primitiveCompareString in MiscPrimitivePlugin
> Looking for module MiscPrimitivePlugin ... loaded
> (4623) Looking for primitiveDirectoryDelimitor in FilePlugin
> (7806) Looking for primitiveIndexOfAsciiInString in MiscPrimitivePlugin
> (57216) Looking for primitiveStringHash in MiscPrimitivePlugin
> (177977) Looking for primitiveFindFirstInString in MiscPrimitivePlugin
> (179055) Looking for primitiveFileOpen in FilePlugin
> (179815) Looking for primitiveFileGetPosition in FilePlugin
> (179832) Looking for primitiveFileAtEnd in FilePlugin
> (179873) Looking for primitiveFileRead in FilePlugin
> (182837) Looking for primitiveFileSize in FilePlugin
> (182868) Looking for primitiveFileSetPosition in FilePlugin
> (186627) Looking for primDigitCompare in LargeIntegers
> Looking for module LargeIntegers ... loaded
> (186688) Looking for primDigitMultiplyNegative in LargeIntegers
> (186884) Looking for primNormalizeNegative in LargeIntegers
> (186985) Looking for primDigitAdd in LargeIntegers
> (186986) Looking for primNormalizePositive in LargeIntegers
> (187038) Looking for primDigitSubtract in LargeIntegers
>

This is great.  The system's working.

> Is there a way to stop the interpreter in order to debug things?
>

Lots of ways.
1. put a halt before "run", and interact with the VM via the bottom right
simulator window's utilities menu to
     a) set a bytecode breakpoint
     b) set a selector breakpoint

2. add or modify methods in the interpreter simulators
(StackInterpreterSimulator, CogVMSimulator) or memory simulators
(NewObjectMemorySimualtor, NewCoObjectMemorySimulator) to add conditional
breakpoints.  e.g. comment-out and modify

longAt: byteAddress put: a32BitValue
"Note: Adjusted for Smalltalk's 1-based array indexing."
"(byteAddress = 16r183FB00 and: [a32BitValue = 16r3FFFFC]) ifTrue:
[self halt]."
"(byteAddress between: 16r33FBB8 and: 16r33FBCF) ifTrue:
[self halt]."
byteAddress \\ 4 ~= 0 ifTrue: [self unalignedAccessError].
^memory at: byteAddress // 4 + 1 put: a32BitValue

>
> I saw: runAtEachStep: aBlock
> self initStackPages.
> self loadInitialContext.
> self internalizeIPandSP.
>  self fetchNextBytecode.
> [true] whileTrue:
> [self assertValidExecutionPointers.
>   aBlock value: currentBytecode.
>  self dispatchOn: currentBytecode in: BytecodeTable.
>  self incrementByteCount].
>  localIP := localIP - 1.
> "undo the pre-increment of IP before returning"
> self externalizeIPandSP
>
> How to use that?
>

You can use the block to do what you want, e.g. collect bytecode stats,
compare the stream of bytecodes in a buggy VM with those from a correct VM
to find out where they diverge, etc.  But "run" uses the atEachStepBlock,
and that is used to implement breakpoints at specific bytecodes or bytecode
counts, etc.  See utlitiesMenu: in the interpreter simulators.

Why do I not see the display in the sim?
>

Dunno.  That you have to fix.  It works in Squeak.

> I saw a "quitBlock". What is this about?
>

It connects the exit routines such as ioExit, ioExitWithErrorCode: to
closing of the simulator window.  Browse inst var refs.

> I am not far from getting this working...
>

Indeed.  In fact it is already working to a large extent.  Stick with it,
you;re nearly there.  Good luck!

>
>
> Thanks for some clues!
>
> Phil
> ---
> Philippe Back
> Dramatic Performance Improvements
> Mob: +32(0) 478 650 140 | Fax: +32 (0) 70 408 027
> Mail:phil at highoctane.be | Web: http://philippeback.eu
> Blog: http://philippeback.be | Twitter: @philippeback
> Youtube: http://www.youtube.com/user/philippeback/videos
>
> High Octane SPRL
> rue cour Boisacq 101 | 1301 Bierges | Belgium
>
> Pharo Consortium Member - http://consortium.pharo.org/
> Featured on the Software Process and Measurement Cast -
> http://spamcast.libsyn.com
> Sparx Systems Enterprise Architect and Ability Engineering EADocX Value
> Added Reseller
>
>
>
>

-- 
best,
Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20131127/7b17a86c/attachment-0001.htm