[Vm-dev] [squeak-dev] Primitive 105 issues (was: Re: how to see primitive source these days?)

Eliot Miranda eliot.miranda at gmail.com
Wed Jan 11 16:05:39 UTC 2017


Hi Levente,


> On Jan 10, 2017, at 4:00 PM, Levente Uzonyi <leves at caesar.elte.hu> wrote:
> 
> The error code is #'inappropriate operation', which is probably PrimErrInappropriate.
> But based on #primitiveStringReplace that shouldn't happen, because both arrays have the same format, and the error doesn't happen when the sizes match.
> So, I think it's either a compiler issue, or this primitive also has a jitted version, which is buggy.

It only appears that way :-).  In fact, the formats are not sufficiently the same :-).

The format field includes the number of unused elements in a non-pointer array.  On 32-bits the least significant two bits encode how many unused bytes there are in the last 32-bit word (formats 16 through 19), and the least significant bit encodes how many unused shorts there are in the last word (formats 12 & 13). On 64-bits the least significant three bits encode how many unused bytes there are in the last 64-bit word (formats 16 through 23), the least significant 2 bits encode how many unused shorts there are in the last word (formats 12 through 15), and the least significant bit encodes how many unused 32-bit words there are in the last word (formats 10 & 11).  So the primitive must mask off the relevant number of bits to compare formats.

In V3 there are only 32-bit and 8-but non-pointer collections and so the primitive was written to only mask the formats of byte collections.  The result is that for 64-bit Spur the primitive succeeds for 32-bit collections only if receiver and replacement are both even in length or both odd in length.

The fix involved masking off the right number of bits and extending the primitive to handle 16- and 64-bit collections.

HTH

> 
> Levente
> 
>> On Tue, 10 Jan 2017, David T. Lewis wrote:
>> 
>>> On Tue, Jan 10, 2017 at 10:39:46PM +0100, Levente Uzonyi wrote:
>>> The same issue is present on 64-bit Linux. When the target is smaller than the source, the primitive will just fail for 4-byte collections. Here's an example with WordArray:
>>> source := #(1 2 3) asWordArray.
>>> target := WordArray new: 2. "change to 3 to make it use the primitive"
>>> target
>>>    replaceFrom: 1
>>>    to: target size
>>>    with: source
>>>    startingAt: 1
>> 
>> I'm not sure this is a useful data point, but it does work as expected on a
>> 64-bit V3 image (format 68002). Comparing the sources for #primitiveStringReplace
>> in oscog versus interpreter VM, they seem to have identical logic for array
>> size check, and I do not see any other meaningful difference, so I do not see
>> an obvious reason for the problem.
>> 
>>> The same problem is present for 2-byte (DoubleByteArray) and 8-byte (DoubleWordArray) collections, but, to my surprise, it works for ByteArray. (Haven't checked the C code yet.)
>> 
>> DoubleByteArray and DoubleWordArray are not supported on V3 so I can't
>> check those.
>> 
>> Probably someone will need to look at it in the simulator or gdb debugger.
>> 
>> Dave
>> 
>>> Levente
>>> On Tue, 10 Jan 2017, Bob Arning wrote:
>>> >
>>> >I got the 64-bit macos vm and the problem is still there.
>>> >
>>> >
>>> >/Users/bob/squeak/Squeak5.1-16548-64bit-All-in-One/Squeak5.1-16548-64bit-All-in-One.app/Contents/Resources/CocoaFast.app/Contents/MacOS/Squeak
>>> >Croquet Closure Cog[Spur] VM [CoInterpreterPrimitives >VMMaker.oscog-cb.1919]
>>> >Mac OS X built on Sep 25 2016 16:07:11 UTC Compiler: 4.2.1 Compatible >Apple LLVM 6.0 (clang-600.0.54)
>>> >platform sources revision VM: 201608171728 >https://github.com/OpenSmalltalk/opensmalltalk-vm.git $ Date: Wed Aug 17 >10:28:01 2016 -0700 $ Plugins: 201608171728
>>> >https://github.com/OpenSmalltalk/opensmalltalk-vm.git $
>>> >CoInterpreter VMMaker.oscog-cb.1919 uuid: >00a8dd2a-bc8d-4552-b400-be781c8aabec Sep 25 2016
>>> >StackToRegisterMappingCogit VMMaker.oscog-cb.1919 uuid: >00a8dd2a-bc8d-4552-b400-be781c8aabec Sep 25 2016
>>> >
>>> >
>>> >On 1/10/17 1:00 PM, David T. Lewis wrote:
>>> >
>>> >It sounds like a VM bug. I am away and cannot check anything further right
>>> >now, but one thing I would mention is that there has been a good deal of
>>> >recent VM development that may be related to this, so if you don't mind
>>> >experimenting you may want to try one of the latest VMs here:
>>> >
>>> >https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/201608171728
>>> >
>>> >That will at least let you see if it is a bug that someone has already
>>> >fixed (which I think is quite likely).
>>> >
>>> >Dave
>>> >
>>> >
>>> >Thanks, Dave,
>>> >
>>> >There are a number of errors possible in that primitive and I don't know
>>> >if one can see the particular code after a failure. I was able to narrow
>>> >it down, though. Adding this to FloatArray:
>>> >
>>> >testreplaceFrom: start to: stop with: replacement startingAt: repStart
>>> >"
>>> >(FloatArray new: 8) testreplaceFrom: 1 to: 3 with: (FloatArray new: 3)
>>> >startingAt: 1
>>> >"
>>> >     <primitive: 105>
>>> >     self halt.
>>> >
>>> >Runs fine in 32-bit squeak and halts in 64-bit.
>>> >
>>> >-----------succeeds on-----------
>>> >
>>> >Virtual Machine
>>> >---------------
>>> >/Users/bob/squeak/old squeak5.1/ast 5.1.app/Contents/MacOS/Squeak
>>> >Croquet Closure Cog[Spur] VM [CoInterpreterPrimitives
>>> >VMMaker.oscog-cb.1919] 32 bit
>>> >Mac OS X built on Aug 17 2016 18:59:49 UTC Compiler: 4.2.1 Compatible
>>> >Apple LLVM 6.0 (clang-600.0.54)
>>> >platform sources revision VM: 201608171728
>>> >https://github.com/OpenSmalltalk/opensmalltalk-vm.git $ Date: Wed Aug 17
>>> >10:28:01 2016 -0700 $ Plugins: 201608171728
>>> >https://github.com/OpenSmalltalk/opensmalltalk-vm.git $
>>> >CoInterpreter VMMaker.oscog-cb.1919 uuid:
>>> >00a8dd2a-bc8d-4552-b400-be781c8aabec Aug 17 2016
>>> >StackToRegisterMappingCogit VMMaker.oscog-cb.1919 uuid:
>>> >00a8dd2a-bc8d-4552-b400-be781c8aabec Aug 17 2016
>>> >
>>> >--------fails on--------
>>> >
>>> >Virtual Machine
>>> >---------------
>>> >/Users/bob/squeak/Squeak5.1-16548-64bit-All-in-One/Squeak5.1-16548-64bit-All-in-One.app/Contents/MacOS/Squeak
>>> >Croquet Closure Cog[Spur] VM [CoInterpreterPrimitives
>>> >VMMaker.oscog-cb.1919] 64 bit
>>> >Mac OS X built on Aug 17 2016 18:51:56 UTC Compiler: 4.2.1 Compatible
>>> >Apple LLVM 6.0 (clang-600.0.54)
>>> >platform sources revision VM: 201608171728
>>> >https://github.com/OpenSmalltalk/opensmalltalk-vm.git $ Date: Wed Aug 17
>>> >10:28:01 2016 -0700 $ Plugins: 201608171728
>>> >https://github.com/OpenSmalltalk/opensmalltalk-vm.git $
>>> >CoInterpreter VMMaker.oscog-cb.1919 uuid:
>>> >00a8dd2a-bc8d-4552-b400-be781c8aabec Aug 17 2016
>>> >StackToRegisterMappingCogit VMMaker.oscog-cb.1919 uuid:
>>> >00a8dd2a-bc8d-4552-b400-be781c8aabec Aug 17 2016
>>> >
>>> >
>>> >
>>> >
>>> >On 1/10/17 9:06 AM, David T. Lewis wrote:
>>> >
>>> >How to see primitive source these days:
>>> >
>>> >The code is in the VMMaker repository
>>> >'http://source.squeak.org/VMMaker'.
>>> >
>>> >For the Spur VM, the package is 'VMMaker.oscog'. For purposes of just
>>> >taking a quick look at the code, you can load the latest version
>>> >VMMaker.oscog-eem.2079 (this will not exactly corresponding to the
>>> >version
>>> >you are running but here I assume you just want a quick look at what
>>> >primitive 105 is doing).
>>> >
>>> >To find the numbered primitive, see CoInterpreter
>>> >class>>initializePrimitiveTable,
>>> >which calls StackInterpreter class>>initializePrimitiveTable, which
>>> >shows
>>> >primitive 105 as #primitiveStringReplace.
>>> >
>>> >The Smalltalk code for this primitive is
>>> >InterpreterPrimitives>>primitiveStringReplace.
>>> >The generated C code for this can be found in the GitHub repository
>>> >https://github.com/OpenSmalltalk/opensmalltalk-vm, look for the
>>> >primitiveStringReplace function in
>>> >opensmalltalk-vm/src/vm/gcc3x-cointerpmt.c.
>>> >
>>> >Dave
>>> >
>>> >
>>> >
>>> >On Tue, Jan 10, 2017 at 07:57:31AM -0500, Bob Arning wrote:
>>> >
>>> >I'm was copying a FloatArray and it seems not to be using the primitive
>>> >to move the data over to the copy.
>>> >Float>>replaceFrom:to:with:startingAt: says it uses a primitive, but it
>>> >seem to fallback to the code in SequenceableCollection. How can I see
>>> >the priitive code to see what the issue may be? This is what it says it
>>> >does:
>>> >
>>> >replaceFrom: start to: stop with: replacement startingAt: repStart
>>> >"Primitive. This destructively replaces elements from start to stop in
>>> >the receiver starting at index, repStart, in the collection,
>>> >replacement. Answer the receiver. Range checks are performed in the
>>> >primitive only. Optional. See Object documentation whatIsAPrimitive."
>>> >     <primitive: 105>     super replaceFrom: start to: stop with:
>>> >replacement startingAt: repStart
>>> >
>>> >This is what I did to see if the prim failed:
>>> >
>>> >replaceFrom: start to: stop with: replacement startingAt: repStart
>>> >"Primitive. This destructively replaces elements from start to stop in
>>> >the receiver starting at index, repStart, in the collection,
>>> >replacement. Answer the receiver. Range checks are performed in the
>>> >primitive only. Optional. See Object documentation whatIsAPrimitive."
>>> >     <primitive: 105>     WOOPS ifNil: [         WOOPS _ #WOOPS.
>>> >self halt     ].     super replaceFrom: start to: stop with:
>>> >replacement
>>> >startingAt: repStart
>>> >
>>> >And it did fail and I'm puzzled as to why:
>>> >
>>> >Halt:
>>> >10 January 2017 7:46:09.382669 am
>>> >
>>> >VM: Mac OS - Smalltalk
>>> >Image: Squeak5.1 [latest update: #16548]
>>> >
>>> >SecurityManager state:
>>> >Restricted: false
>>> >FileAccess: true
>>> >SocketAccess: true
>>> >Working Dir
>>> >/Users/bob/squeak/Squeak5.1-16548-64bit-All-in-One/Squeak5.1-16548-64bit-All-in-One.app/Contents/Resources
>>> >Trusted Dir /Users/bob/Library/Application Support/Squeak/
>>> >Untrusted Dir /Users/bob/Documents/Squeak/
>>> >
>>> >FloatArray(Object)>>halt
>>> >     Receiver: a FloatArray(0.0 0.0 0.0)
>>> >     Arguments and temporary variables:
>>> >
>>> >     Receiver's instance variables:
>>> >a FloatArray(0.0 0.0 0.0)
>>> >FloatArray>>replaceFrom:to:with:startingAt:
>>> >     Receiver: a FloatArray(0.0 0.0 0.0)
>>> >     Arguments and temporary variables:
>>> >         start:     1
>>> >         stop:     3
>>> >         replacement:     a FloatArray(1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
>>> >1.0 1.0 1.0 1.0 1.0 1...etc...
>>> >         repStart:     1
>>> >     Receiver's instance variables:
>>> >a FloatArray(0.0 0.0 0.0)
>>> >FloatArray(SequenceableCollection)>>copyFrom:to:
>>> >     Receiver: a FloatArray(1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
>>> >1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1...etc...
>>> >     Arguments and temporary variables:
>>> >         start:     1
>>> >         stop:     3
>>> >         newSize:     3
>>> >     Receiver's instance variables:
>>> >a FloatArray(1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
>>> >1.0
>>> >1.0 1.0 1.0 1.0 1...etc...
>>> >ConvolutionalLayer>>forward:isTraining:
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> 


More information about the Vm-dev mailing list