[Vm-dev] Reed Solomon plugins & performance slow down

[Robert Withers]

Extremely strange. I unloaded the CryptographyRSErasure and CryptographyRSErasureTests packages. I still get a crash dump referencing RSErasureGaloisWithPlugin>galoisMultiply:by:. I am totally confused! This class isn't even loaded in my image! WTH is going on!?

> Smalltalk stack dump:
> 0x7ffde5e6a1b0 I RSErasureGaloisWithPlugin>galoisMultiply:by: 0xc7ed9d8: a(n) RSErasureGaloisWithPlugin
> 0x7ffde5e6a220 I RSErasureGaloisWithPlugin>generateMultiplicationTable 0xc7ed9d8: a(n) RSErasureGaloisWithPlugin
> 0x7ffde5e59df0 I RSErasureGaloisWithPlugin>initialize 0xc7ed9d8: a(n) RSErasureGaloisWithPlugin
> 0x7ffde5e59e20 M RSErasureGaloisWithPlugin class(Behavior)>new 0x7fdeec8: a(n) RSErasureGaloisWithPlugin class
> 0x7ffde5e59e60 I RSErasureGalois class>newGalois 0x6a89510: a(n) RSErasureGalois class
> 0x7ffde5e59e90 M UndefinedObject>DoIt 0x2de78e0: a(n) UndefinedObject

---
Kindly,
Robert

On 5/31/21 4:00 PM, Robert Withers wrote:

> The segFault occurs with a call to a RSFECPlugin primitive>>#primMultiplyPolySelfCoefficients:otherCoefficients:product:fieldSize:. But the crash.dmp shows a RSErasureGaloisWithPlugin as the error site. I am very confused as the call I am making in the FEC test is to the RSFECPlugin, but the RSErasurePlugin is the one cited as causing the segFault. I even removed the RSErasurePlugin from the vm directory and this is still occurring.
>
>> RSFECGenericGFPolyWithPlugin>>#multiplyPoly: other
>>
>> | product |
>> (field = other field)
>> ifFalse: [RSErasureIllegalArgumentError signal: 'GenericGFPolys do not have same GenericGF field'].
>> (self isZero)
>> ifTrue: [^ field zero]
>> ifFalse: [other isZero ifTrue: [^ field zero]].
>> product := Array new: (self coefficients size + other coefficients size - 1) withAll: 0.
>>
>> product := self primMultiplyPolySelfCoefficients: self coefficients otherCoefficients: other coefficients product: product fieldSize: field size.
>>
>> ^ RSFECGenericGFPoly newField: field coefficients: product.
>
> Any assistance most welcome. This has me stumped.
>
> ---
> Kindly,
> Robert
>
> On 5/31/21 3:43 PM, Robert Withers wrote:
>
>> My previous email is awaiting approval as it is too big. I attached a crash.dmp. Here is the email with a link to the crash dump I placed on dropbox.
>>
>> I implemented a set of 5 primitives for Polynomial math in the RSFECPlugin class and added a RSFECGenericGFPolyWithPlugin to call them. In testing FEC I get a segFault, but looking at the Smalltalk stack dump it is happening in the RSErasurePlugin which is not called by the RSFEC code. It looks like a DoIt and I cannot find the offending code. Where can I look for a DoIt that is calling RSErasure code when running RSFEC code? Here is the smalltalk stack dump and I am attaching the crash.dmp file [1].
>>
>>> 0x7ffde5e6a1b0 I RSErasureGaloisWithPlugin>galoisMultiply:by: 0xc7ed9d8: a(n) RSErasureGaloisWithPlugin
>>> 0x7ffde5e6a220 I RSErasureGaloisWithPlugin>generateMultiplicationTable 0xc7ed9d8: a(n) RSErasureGaloisWithPlugin
>>> 0x7ffde5e59df0 I RSErasureGaloisWithPlugin>initialize 0xc7ed9d8: a(n) RSErasureGaloisWithPlugin
>>> 0x7ffde5e59e20 M RSErasureGaloisWithPlugin class(Behavior)>new 0x7fdeec8: a(n) RSErasureGaloisWithPlugin class
>>> 0x7ffde5e59e60 I RSErasureGalois class>newGalois 0x6a89510: a(n) RSErasureGalois class
>>> 0x7ffde5e59e90 M UndefinedObject>DoIt 0x2de78e0: a(n) UndefinedObject
>>
>> []1 crash.zip - https://www.dropbox.com/s/qyq735yksmc4dbr/crash.zip?dl=0
>>
>>>
>>
>> ---
>> Kindly,
>> Robert
>> ---
>> Kindly,
>> Robert
>>
>> On 5/30/21 8:50 PM, Robert Withers wrote:
>>
>>> Hi Levente,
>>>
>>> Thank you for having a look-see and letting me know your thoughts! It
>>> makes sense what you are saying. so I think the only way to speed it up
>>> is to embed the loops into primitives. I have 5 of 6 methods from Poly
>>> that I am stuffing into the plugin. The #dividePoly: is a bit more
>>> complicated, updating the remainder poly and so forth. I do not think a
>>> Poly can be within a primitive. Hopefully, {product := Array new: size
>>> withAll: 0} will work in translation. I fixed #initializeExpTable per
>>> your suggestion.
>>>
>>> Thanks!
>>>
>>> ---
>>> Kindly,
>>> Robert
>>>
>>> On 5/30/21 8:18 PM, Levente Uzonyi wrote:
>>>
>>>> Hi Robert,
>>>>
>>>> I had a quick look at the plugin code. The operations you implemented in
>>>> the plugin are way too simple. The VM's JIT makes the smalltalk code run
>>>> faster, because it has the same operations implemented, and it can avoid
>>>> calling the overhead of calling the primitive many times. Hence the
>>>> slowdown.
>>>>
>>>> For example, RSFECPlugin >> #primitiveAddOrSubtractby seems to implement
>>>> bitwise xor on two 32-bit unsigned values, but the arguments are always
>>>> 0-255. The VM has primitive 16, which does bitwise xor and covers that
>>>> input range, so the JIT, combined with other operations can generate
>>>> machine code directly with that without jumping back and forth between
>>>> native code and smalltalk.
>>>>
>>>> If you want a plugin to provide any noticable speedup, you need to do
>>>> more computation in a single primtiive call.
>>>>
>>>> Levente
>>>>
>>>> P.S.: I think, in RSFECGenericGC >> #initializeExpTable, the lines
>>>>
>>>>					x := x bitXor: primitive.
>>>>					x bitAnd: (size - 1)]].
>>>>
>>>> should read
>>>>
>>>>					x := (x bitXor: primitive) bitAnd: size - 1 ]].
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210531/d0c84e2d/attachment.html>