[Vm-dev] Indexed vs named primitives (was: Help/advice tracking down a squeak-vm regression)

Mon May 14 12:01:31 UTC 2012

On 13.05.2012, at 15:37, David T. Lewis wrote:

> Apologies, I see that I failed to read Bert's note carefully. The
> Scratch image is *not* using any numbered primitives in the range
> 200 - 229, which is the area of possible conflict with the newer Cog
> and closures usages. So my question below concerning backward compatibility
> is moot.

Err, it does. Using Eliot's list:

(CompiledMethod allInstances select: [:m | #(161 169 175 176 185 213 214 215 218  240 241 242 243) includes: m primitive]) collect: [:m | m primitive -> m who] 

(169->(FileDirectory #primSetMacFileNamed:type:creator:)
218->(Socket #primSocket:listenOn:)
215->(Socket #primSocketRemoteAddress:)
213->(Socket #primSocketLocalAddress:)
218->(Socket #primSocket:listenOn:backlogSize:)
214->(Socket #primSocketLocalPort:)
218->(Socket #primSocket:setPort:)
241->(SerialPort #primReadPort:into:startingAt:count:)
240->(SerialPort #primWritePort:from:startingAt:count:)
243->(String class #translate:from:to:table:)
161->(FileDirectory class #primPathNameDelimiter)
185->(SampledSound #mixSampleCount:into:startingAt:leftVol:rightVol:))

> Regarding Bert's question:
> 
>> 
>> Do we have a list of old primitive numbers and their corresponding
>> new named equivalents?
>> 
> 
> As far as I know we do not have such a list. There are comments in
> the initializePrimitiveTable method indicating prior assignments, but
> aside from that, it's necessary to refer to the older versions in
> Monticello.
> 
> Dave

Maybe I can make one now :) 

I executed this in the old Scratch image:

((CompiledMethod allInstances select: [:m | #((96 96) (104 104) (146 147) (150 159) (161 164) (169 185) (190 194) (207 209) (213 220) (223 229) (234 246) (520 569)) anySatisfy: [:i | m primitive between: i first and: i second]]) collect: [:m | m primitive -> m who]) sort

Hopefully this covers all formerly indexed prims. And then with a little massaging in an Etoys image I get this:

#(
#(96 '<primitive: ''primitiveCopyBits'' module: ''BitBltPlugin''>' 'BitBlt>>copyBits')
#(96 '<primitive: ''primitiveCopyBits'' module: ''BitBltPlugin''>' 'BitBlt>>copyBitsTranslucent:')
#(96 '<primitive: ''primitiveCopyBits'' module: ''BitBltPlugin''>' 'BitBlt>>copyBitsAgain')
#(104 '<primitive: ''primitiveDrawLoop'' module: ''BitBltPlugin''>' 'BitBlt>>drawLoopX:Y:')
#(147 '<primitive: ''primitiveWarpBits'' module: ''BitBltPlugin''>' 'WarpBlt>>warpBitsSmoothing:sourceMap:')
#(150 '<primitive: ''primitiveFileAtEnd'' module: ''FilePlugin''>' 'StandardFileStream>>primAtEnd:')
#(151 '<primitive: ''primitiveFileClose'' module: ''FilePlugin''>' 'StandardFileStream>>primClose:')
#(151 '<primitive: ''primitiveFileClose'' module: ''FilePlugin''>' 'StandardFileStream>>primCloseNoError:')
#(152 '<primitive: ''primitiveFileGetPosition'' module: ''FilePlugin''>' 'StandardFileStream>>primGetPosition:')
#(153 '<primitive: ''primitiveFileOpen'' module: ''FilePlugin''>' 'StandardFileStream>>primOpen:writable:')
#(154 '<primitive: ''primitiveFileRead'' module: ''FilePlugin''>' 'StandardFileStream>>primRead:into:startingAt:count:')
#(155 '<primitive: ''primitiveFileSetPosition'' module: ''FilePlugin''>' 'StandardFileStream>>primSetPosition:to:')
#(156 '<primitive: ''primitiveFileDelete'' module: ''FilePlugin''>' 'FileDirectory>>primDeleteFileNamed:')
#(157 '<primitive: ''primitiveFileSize'' module: ''FilePlugin''>' 'StandardFileStream>>primSizeNoError:')
#(157 '<primitive: ''primitiveFileSize'' module: ''FilePlugin''>' 'StandardFileStream>>primSize:')
#(158 '<primitive: ''primitiveFileWrite'' module: ''FilePlugin''>' 'StandardFileStream>>primWrite:from:startingAt:count:')
#(159 '<primitive: ''primitiveFileRename'' module: ''FilePlugin''>' 'FileDirectory>>primRename:to:')
#(161 '<primitive: ''primitiveDirectoryDelimitor'' module: ''FilePlugin''>' 'FileDirectory class>>primPathNameDelimiter')
#(162 '<primitive: ''primitiveDirectoryLookup'' module: ''FilePlugin''>' 'FileDirectory>>primLookupEntryIn:index:')
#(163 '<primitive: ''primitiveDirectoryDelete'' module: ''FilePlugin''>' 'FileDirectory>>primDeleteDirectory:')
#(169 '<primitive: ''primitiveDirectorySetMacTypeAndCreator'' module: ''FilePlugin''>' 'FileDirectory>>primSetMacFileNamed:type:creator:')
#(170 '<primitive: ''primitiveSoundStart'' module: ''SoundPlugin''>' 'SoundPlayer class>>primSoundStartBufferSize:rate:stereo:')
#(171 '<primitive: ''primitiveSoundStartWithSemaphore'' module: ''SoundPlugin''>' 'SoundPlayer class>>primSoundStartBufferSize:rate:stereo:semaIndex:')
#(172 '<primitive: ''primitiveSoundStop'' module: ''SoundPlugin''>' 'SoundPlayer class>>primSoundStop')
#(173 '<primitive: ''primitiveSoundAvailableSpace'' module: ''SoundPlugin''>' 'SoundPlayer class>>primSoundAvailableBytes')
#(174 '<primitive: ''primitiveSoundPlaySamples'' module: ''SoundPlugin''>' 'SoundPlayer class>>primSoundPlaySamples:from:startingAt:')
#(180 '<primitive: ''primitiveMixFMSound'' module: ''SoundGenerationPlugin''>' 'FMSound>>mixSampleCount:into:startingAt:leftVol:rightVol:')
#(181 '<primitive: ''primitiveMixPluckedSound'' module: ''SoundGenerationPlugin''>' 'PluckedSound>>mixSampleCount:into:startingAt:leftVol:rightVol:')
#(183 '<primitive: ''primitiveApplyReverb'' module: ''SoundGenerationPlugin''>' 'ReverbSound>>applyReverbTo:startingAt:count:')
#(184 '<primitive: ''primitiveMixLoopedSampledSound'' module: ''SoundGenerationPlugin''>' 'LoopedSampledSound>>mixSampleCount:into:startingAt:leftVol:rightVol:')
#(185 '<primitive: ''primitiveMixSampledSound'' module: ''SoundGenerationPlugin''>' 'SampledSound>>mixSampleCount:into:startingAt:leftVol:rightVol:')
#(190 '<primitive: ''primitiveSoundStartRecording'' module: ''SoundPlugin''>' 'SoundRecorder>>primStartRecordingDesiredSampleRate:stereo:semaIndex:')
#(191 '<primitive: ''primitiveSoundStopRecording'' module: ''SoundPlugin''>' 'SoundRecorder>>primStopRecording')
#(192 '<primitive: ''primitiveSoundGetRecordingSampleRate'' module: ''SoundPlugin''>' 'SoundRecorder>>primGetActualRecordingSampleRate')
#(193 '<primitive: ''primitiveSoundRecordSamples'' module: ''SoundPlugin''>' 'SoundRecorder>>primRecordSamplesInto:startingAt:')
#(194 '<primitive: ''primitiveSoundSetRecordLevel'' module: ''SoundPlugin''>' 'SoundRecorder>>primSetRecordLevel:')
#(207 '<primitive: ''primitiveResolverStatus'' module: ''SocketPlugin''>' 'NetNameResolver class>>primNameResolverStatus')
#(208 '<primitive: ''primitiveResolverError'' module: ''SocketPlugin''>' 'NetNameResolver class>>primNameResolverError')
#(209 '<primitive: ''primitiveSocketCreate'' module: ''SocketPlugin''>' 'Socket>>primSocketCreateNetwork:type:receiveBufferSize:sendBufSize:semaIndex:')
#(213 '<primitive: ''primitiveSocketLocalAddress'' module: ''SocketPlugin''>' 'Socket>>primSocketLocalAddress:')
#(214 '<primitive: ''primitiveSocketLocalPort'' module: ''SocketPlugin''>' 'Socket>>primSocketLocalPort:')
#(215 '<primitive: ''primitiveSocketRemoteAddress'' module: ''SocketPlugin''>' 'Socket>>primSocketRemoteAddress:')
#(216 '<primitive: ''primitiveSocketRemotePort'' module: ''SocketPlugin''>' 'Socket>>primSocketRemotePort:')
#(217 '<primitive: ''primitiveSocketConnectToPort'' module: ''SocketPlugin''>' 'Socket>>primSocket:connectTo:port:')
#(218 '<primitive: ''primitiveSocketListenWithOrWithoutBacklog'' module: ''SocketPlugin''>' 'Socket>>primSocket:listenOn:')
#(218 '<primitive: ''primitiveSocketListenWithOrWithoutBacklog'' module: ''SocketPlugin''>' 'Socket>>primSocket:listenOn:backlogSize:')
#(218 '<primitive: ''primitiveSocketListenWithOrWithoutBacklog'' module: ''SocketPlugin''>' 'Socket>>primSocket:setPort:')
#(219 '<primitive: ''primitiveSocketCloseConnection'' module: ''SocketPlugin''>' 'Socket>>primSocketCloseConnection:')
#(220 '<primitive: ''primitiveSocketAbortConnection'' module: ''SocketPlugin''>' 'Socket>>primSocketAbortConnection:')
#(223 '<primitive: ''primitiveSocketSendDataBufCount'' module: ''SocketPlugin''>' 'Socket>>primSocket:sendData:startIndex:count:')
#(224 '<primitive: ''primitiveSocketSendDone'' module: ''SocketPlugin''>' 'Socket>>primSocketSendDone:')
#(225 '<primitive: ''primitiveSocketAccept'' module: ''SocketPlugin''>' 'Socket>>primAcceptFrom:receiveBufferSize:sendBufSize:semaIndex:')
#(234 '<primitive: ''primitiveDecompressFromByteArray'' module: ''MiscPrimitivePlugin''>' 'Bitmap>>decompress:fromByteArray:at:')
#(235 '-' 'String>>compare:with:collated:')
#(236 '<primitive: ''primitiveConvert8BitSigned'' module: ''MiscPrimitivePlugin''>' 'SampledSound class>>convert8bitSignedFrom:to16Bit:')
#(237 '<primitive: ''primitiveCompressToByteArray'' module: ''MiscPrimitivePlugin''>' 'Bitmap>>compress:toByteArray:')
#(238 '<primitive: ''primitiveSerialPortOpen'' module: ''SerialPlugin''>' 'SerialPort>>primOpenPort:baudRate:stopBitsType:parityType:dataBits:inFlowControlType:outFlowControlType:xOnByte:xOffByte:')
#(239 '<primitive: ''primitiveSerialPortClose'' module: ''SerialPlugin''>' 'SerialPort>>primClosePort:')
#(240 '<primitive: ''primitiveSerialPortWrite'' module: ''SerialPlugin''>' 'SerialPort>>primWritePort:from:startingAt:count:')
#(241 '<primitive: ''primitiveSerialPortRead'' module: ''SerialPlugin''>' 'SerialPort>>primReadPort:into:startingAt:count:')
#(243 '-' 'String class>>translate:from:to:table:')
#(244 '-' 'String class>>findFirstInString:inSet:startingAt:')
#(245 '-' 'String class>>indexOfAscii:inString:startingAt:')
#(246 '-' 'String>>findSubstring:in:startingAt:matchTable:')
#(521 '<primitive: ''primitiveMIDIClosePort'' module: ''MIDIPlugin''>' 'SimpleMIDIPort>>primMIDIClosePort:')
#(523 '<primitive: ''primitiveMIDIGetPortCount'' module: ''MIDIPlugin''>' 'SimpleMIDIPort class>>primPortCount')
#(524 '<primitive: ''primitiveMIDIGetPortDirectionality'' module: ''MIDIPlugin''>' 'SimpleMIDIPort class>>primPortDirectionalityOf:')
#(525 '<primitive: ''primitiveMIDIGetPortName'' module: ''MIDIPlugin''>' 'SimpleMIDIPort class>>primPortNameOf:')
#(526 '<primitive: ''primitiveMIDIOpenPort'' module: ''MIDIPlugin''>' 'SimpleMIDIPort>>primMIDIOpenPort:readSemaIndex:interfaceClockRate:')
#(528 '<primitive: ''primitiveMIDIRead'' module: ''MIDIPlugin''>' 'SimpleMIDIPort>>primMIDIReadPort:into:')
#(529 '<primitive: ''primitiveMIDIWrite'' module: ''MIDIPlugin''>' 'SimpleMIDIPort>>primMIDIWritePort:from:at:')
#(550 '<primitive: ''primitiveDecodeMono'' module: ''ADPCMCodecPlugin''>' 'ADPCMCodec>>privateDecodeMono:')
#(551 '<primitive: ''primitiveDecodeStereo'' module: ''ADPCMCodecPlugin''>' 'ADPCMCodec>>privateDecodeStereo:')
#(552 '<primitive: ''primitiveEncodeMono'' module: ''ADPCMCodecPlugin''>' 'ADPCMCodec>>privateEncodeMono:')
#(553 '<primitive: ''primitiveEncodeStereo'' module: ''ADPCMCodecPlugin''>' 'ADPCMCodec>>privateEncodeStereo:'))

So except for the String prims there is a named equivalent now.

- Bert -

> On Sat, May 12, 2012 at 05:34:20PM -0400, David T. Lewis wrote:
>> 
>> On Fri, May 11, 2012 at 01:40:15PM -0700, Eliot Miranda wrote:
>>> 
>>> On Fri, May 11, 2012 at 11:33 AM, Bert Freudenberg <bert at freudenbergs.de>wrote:
>>> 
>>>> 
>>>> 
>>>> On 11.05.2012, at 19:43, Eliot Miranda wrote:
>>>>> 
>>>>> 
>>>>> 
>>>>> On Fri, May 11, 2012 at 6:46 AM, Bert Freudenberg <bert at freudenbergs.de>wrote:
>>>>> 
>>>>>> 
>>>>>> I just checked. Most, if not all of the problems, come from Scratch's use
>>>>>> of numbered primitives. They were deprecated in favor of named primitives.
>>>>>> 
>>>>>> IMHO it would be reasonably simple to re-instate these primitives in the
>>>>>> VM. Right now they are simply mapped to primitiveFail. This would get us
>>>>>> back the ability to run many old images, unmodified.
>>>>>> 
>>>>> 
>>>>> Alas I've used a few of these for Cog:
>>>>> 
>>>>> #(#(161 #primitiveSetIdentityHash)
>>>>>    #(169 #primitiveNotIdentical)
>>>>>    #(175 #primitiveBehaviorHash)
>>>>>    #(176 #primitiveMaxIdentityHash)
>>>>>    #(185 #primitiveExitCriticalSection)
>>>>>    #(213 #primitiveContextXray)
>>>>>    #(214 #primitiveVoidVMState)
>>>>>    #(215 #primitiveVoidVMStateForMethod)
>>>>>    #(218 #primitiveDoNamedPrimitiveWithArgs)
>>>>>    #(240 #primitiveUTCMicrosecondClock)
>>>>>    #(241 #primitiveLocalMicrosecondClock)
>>>>>    #(242 #primitiveSignalAtUTCMicroseconds)
>>>>>    #(243 #primitiveUpdateTimezone))
>>>>> 
>>>>> I think your change set is the way to go.
>>>> 
>>>> 
>>>> Well, as David suggested, we could switch them based on the image magic
>>>> number.
>>>> 
>>> 
>>> Good point.  Excuse me for missing that David.
>>> 
>> 
>> This seems like a workable enough approach (although I won't have a
>> chance to try it until next week). But I'm worried about one possible
>> side effect:
>> 
>> If we arrange for the interpreter to use an old primitive table when
>> the image format is either 6502 or 68000, and use the current primitive
>> table with closure support for all others, then we presumably will get
>> the backward compatibility to support older images.  But what happens
>> if someone wants to add closure support to Scratch? They would need
>> to go through the bootstrapping process that Eliot provides and that
>> Juan has successfully done for Cuis. If our VM just decides to turn
>> off the closure primitives when it detects a 6502 image, I expect
>> that this would be a problem for the closure bootstrapping.
>> 
>> Eliot, my assumption would be that in order to add closure support
>> to an older 6502 image, it is first necessary to have a VM with the
>> closure primitives in place. Is that correct? If so, I think that
>> we could still provide the backward compatibility strategy (which
>> Miriam Ruiz pointed out would be useful for some images other than
>> Scratch), but we would then want to *also* provide a way to tell
>> the VM that we really do want closure support in place. I am thinking
>> of e.g. an "-imageformat 6504" command line parameter to tell the
>> interpreter that the image expects closure support even though the
>> file header says 6502.
>> 
>> Dave