[Cryptography Team] [Vm-dev] Reed Solomon plugins & performance slow down

[Robert Withers]

CryptographyRSPluginExtending-rww.7.mcz now has all 7 missing primitives, for the FECDecoder, the GFPoly and the GaloisCodingLoop. Here is a list.

-  GaloisCodingLoopOutputByteInputExpCodingLoopWithPlugin>>#primCheckSomeShardsMatrixRows: matrixRows
inputs: inputs
toCheck: toCheck
offset: offset
byteCount: byteCount
- GaloisCodingLoopOutputByteInputExpCodingLoopWithPlugin>>#primCodeSomeShardsMatrixRows: matrixRows
inputs: inputs
outputs: outputs
offset: offset
byteCount: byteCount
- GaloisCodingLoopOutputByteInputExpCodingLoopWithPlugin>>#primComputeValueMatrixRow: matrixRow
inputs: inputs
inputIndex: inputIndex
byteIndex: byteIndex
value: value
- FECGFPolyWithPlugin>>#primInitializePolyFieldSize: fieldSize
coefficients: localCoefficients
- FECGFPolyWithPlugin>>#primDividePolySelfCoefficients: coefficients
otherCoefficients: otherCoefficients
fieldSize: fieldSize
- FECDecoderWithPlugin>>#primDecode: decoded
twoS: twoS
generatorBase: generatorBase
- FECDecoderWithPlugin>>#primRunEuclideanAlgorithmPolyA: polyA
polyB: polyB
degrees: fieldSize

---
Kindly,
Robert

On 6/5/21 9:50 AM, Robert Withers wrote:

> A bit more detail on my refactoring. I combined the Plugins to 1 RSPlugin.
>
> I updated the package: CryptographyRSPluginExtending to include all problematic primitives and their send sites. This includes GFPolyWithPlugin>>#initializeField:coefficients: and the ErasureOutputByteInputCodingLoopWithPlugin methods. To load this extension (which blows up the image by the way) run this compound script:
>
>> Installer ss
>> project: 'Cryptography';
>> install: 'ProCrypto-1-1-1';
>> install: 'ProCryptoTests-1-1-1';
>> install: 'CryptographyRSPlugin'.
>>
>> Installer ss
>> project: 'Cryptography';
>> install: 'CryptographyRSPluginExtending'.
>
> To unload and restore prior functionality, run this compound script:
>
>> Installer ss
>> project: 'Cryptography';
>> unload: 'CryptographyRSPluginExtending'.
>>
>> Installer ss
>> project: 'Cryptography';
>> install: 'CryptographyRSErasure';
>> install: 'CryptographyRSFEC';
>> install: 'CryptographyRSPlugin'.
>
> That is all.
>
> ---
> Kindly,
> Robert
>
> On 6/5/21 8:34 AM, Robert Withers wrote:
>
>> I consolidated the RS Plugins into 1 RSPlugin class. Here is the new load script:
>>
>>> Installer ss
>>> project: 'Cryptography';
>>> install: 'ProCrypto-1-1-1';
>>> install: 'ProCryptoTests-1-1-1';
>>> install: 'CryptographyRSPlugin'.
>>
>> ---
>> Kindly,
>> Robert
>>
>> On 6/4/21 2:51 PM, Robert Withers wrote:
>>
>>> Hi Levente,
>>>
>>> I hope you are having a great day! Are you in Budapest? That must be awesome! A very old city.
>>>
>>> I thought I would post the Squeak code for this instantiatePoly primitive. My image still blows up on startUp:. I am grateful you have guided me through this!
>>>
>>>> primitiveInitializePolyFieldSizeCoefficients
>>>>
>>>> <export: true>
>>>> <var: 'coefficients' type: 'unsigned char*' >
>>>>
>>>> | coefficients count coefficientsOop fieldSize result |
>>>> interpreterProxy methodArgumentCount = 2
>>>> ifFalse: [ ^interpreterProxy primitiveFailFor: PrimErrBadNumArgs ].
>>>> fieldSize := interpreterProxy stackIntegerValue: 1.
>>>> coefficientsOop := interpreterProxy stackObjectValue: 0.
>>>>
>>>> (interpreterProxy isIntegerValue: fieldSize)
>>>> ifFalse: [ ^interpreterProxy primitiveFailFor: PrimErrBadArgument ].
>>>> (interpreterProxy isBytes: coefficientsOop)
>>>> ifFalse: [ ^interpreterProxy primitiveFailFor: PrimErrBadArgument ].
>>>> count := interpreterProxy stSizeOf: coefficientsOop.
>>>> coefficients := interpreterProxy firstIndexableField: coefficientsOop.
>>>>
>>>> (count = 0)
>>>> ifTrue: [ ^interpreterProxy primitiveFailFor: PrimErrBadArgument ].
>>>>
>>>> result := self
>>>> initializePolyFieldSize: fieldSize
>>>> coefficients: coefficients
>>>> coefficientsCount: count.
>>>>
>>>> ^ interpreterProxy failed
>>>> ifTrue: [interpreterProxy primitiveFail]
>>>> ifFalse: [interpreterProxy methodReturnValue: result].
>>>
>>> Which is calling:
>>>
>>>> initializePolyFieldSize: fieldSize coefficients: coefficients coefficientsCount: coefficientsCount
>>>>
>>>> <var: 'coefficients' type: #'unsigned char*'>
>>>>
>>>> | coefficientsLength firstNonZero mutableCoefficients |
>>>> mutableCoefficients := coefficients.
>>>> coefficientsLength := coefficientsCount.
>>>> ((coefficientsLength > 1) & ((coefficients at: 1) = 0))
>>>> ifTrue: [
>>>> firstNonZero := 2.
>>>> [(firstNonZero <= coefficientsLength) and: [(mutableCoefficients at: firstNonZero) = 0]]
>>>> whileTrue: [firstNonZero := firstNonZero + 1].
>>>> (firstNonZero > coefficientsLength)
>>>> ifTrue: [
>>>> mutableCoefficients := interpreterProxy
>>>> instantiateClass: interpreterProxy classByteArray
>>>> indexableSize: 1]
>>>> ifFalse: [
>>>> mutableCoefficients := interpreterProxy
>>>> instantiateClass: interpreterProxy classByteArray
>>>> indexableSize: (coefficientsLength - firstNonZero + 1).
>>>> 0 to: (coefficientsLength - 1)
>>>> do: [:index |
>>>> coefficients at: index put: (mutableCoefficients at: index)]]].
>>>> ^ mutableCoefficients.
>>>
>>> ---
>>> Köszönöm,
>>> Robert
>>>
>>> On 6/4/21 1:38 PM, Robert Withers wrote:
>>>
>>>> I am getting this warning when cross compiling the Squeak plugin to C code.
>>>>
>>>>> warning, signature of InterpreterProxy>>instantiateClass:indexableSize: does not match reference implementation.
>>>>
>>>> ---
>>>> Kindly,
>>>> Robert
>>>>
>>>> On 6/4/21 1:21 PM, Robert Withers wrote:
>>>>
>>>>> Nope, this wasn't it.
>>>>>
>>>>> ---
>>>>> Kindly,
>>>>> Robert
>>>>>
>>>>> On 6/4/21 12:36 PM, Robert wrote:
>>>>>
>>>>>> Oh! Heading for a coffee with my nurse. I realized I may be passing the field into the primitive instead of the field size. I’ll check it when I get back home!
>>>>>>
>>>>>> Kindly,
>>>>>> Robert
>>>>>> . .. ... ‘...^,^
>>>>>>
>>>>>> On Fri, Jun 4, 2021 at 12:12, Robert Withers <robert.withers at pm.me> wrote:
>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> Oh! I forgot to relocate leaves that have already been plugganized. This leaves (heh) 3 possible plugganizations that all instantiate ByteArrays. Here, I fixed it.
>>>>>>>
>>>>>>>> WITH GF & GFPOLY PRIMITIVES AND DECODER
>>>>>>>>
>>>>>>>> PRIMITIVES
>>>>>>>>
>>>>>>>> (
>>>>>>>>
>>>>>>>> 3
>>>>>>>>
>>>>>>>> asterix
>>>>>>>>
>>>>>>>> for
>>>>>>>>
>>>>>>>> in-progress plugganization)
>>>>>>>>
>>>>>>>> - 22194 tallies, 22648 msec.
>>>>>>>>
>>>>>>>> **Leaves**
>>>>>>>> 29.1%
>>>>>>>>
>>>>>>>> {
>>>>>>>>
>>>>>>>> 6586
>>>>>>>>
>>>>>>>> ms} RSFECDecoderWithPlugin>>decode:twoS:
>>>>>>>>
>>>>>>>> 14.7
>>>>>>>>
>>>>>>>> % {
>>>>>>>>
>>>>>>>> 3329
>>>>>>>>
>>>>>>>> ms} RSFECGenericGFPoly
>>>>>>>>
>>>>>>>> class
>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>>newField:coefficients:
>>>>>>>>
>>>>>>>> 1.0
>>>>>>>>
>>>>>>>> % {
>>>>>>>>
>>>>>>>> 237
>>>>>>>>
>>>>>>>> ms} RSFECDecoderWithPlugin>>runEuclideanAlgorithmPoly:poly:rDegrees:
>>>>>>>
>>>>>>> Calls to plugganized GF/GFPoly methods, so I think these are as optimized as possible:
>>>>>>>
>>>>>>>> 7.3
>>>>>>>>
>>>>>>>> % {
>>>>>>>>
>>>>>>>> 1646
>>>>>>>>
>>>>>>>> ms} RSFECDecoderWithPlugin>>primFindErrorLocationsDegree:coefficients:result:fieldSize:
>>>>>>>>
>>>>>>>> 2.9
>>>>>>>>
>>>>>>>> % {
>>>>>>>>
>>>>>>>> 654
>>>>>>>>
>>>>>>>> ms} RSFECDecoderWithPlugin>>findErrorMagnitudes:errorLocations:
>>>>>>>>
>>>>>>>> 1.4
>>>>>>>>
>>>>>>>> % {
>>>>>>>>
>>>>>>>> 317
>>>>>>>>
>>>>>>>> ms} RSFECGenericGFWithPlugin>>
>>>>>>>>
>>>>>>>> log
>>>>>>>>
>>>>>>>> :
>>>>>>>
>>>>>>> ---
>>>>>>> Kindly,
>>>>>>> Robert
>>>>>>>
>>>>>>> On 6/4/21 12:02 PM, Robert Withers wrote:
>>>>>>>
>>>>>>>>> WITH GF & GFPOLY PRIMITIVES AND DECODER
>>>>>>>>>
>>>>>>>>> PRIMITIVES
>>>>>>>>>
>>>>>>>>> (
>>>>>>>>>
>>>>>>>>> 3
>>>>>>>>>
>>>>>>>>> asterix
>>>>>>>>>
>>>>>>>>> for
>>>>>>>>>
>>>>>>>>> in-progress plugganization)
>>>>>>>>>
>>>>>>>>> - 22194 tallies, 22648 msec.
>>>>>>>>>
>>>>>>>>> **Leaves**
>>>>>>>>> 29.1%
>>>>>>>>>
>>>>>>>>> {
>>>>>>>>>
>>>>>>>>> 6586
>>>>>>>>>
>>>>>>>>> ms} RSFECDecoderWithPlugin>>decode:twoS:
>>>>>>>>>
>>>>>>>>> 14.7
>>>>>>>>>
>>>>>>>>> % {
>>>>>>>>>
>>>>>>>>> 3329
>>>>>>>>>
>>>>>>>>> ms} RSFECGenericGFPoly
>>>>>>>>>
>>>>>>>>> class
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>newField:coefficients:
>>>>>>>>>
>>>>>>>>> 7.3
>>>>>>>>>
>>>>>>>>> % {
>>>>>>>>>
>>>>>>>>> 1646
>>>>>>>>>
>>>>>>>>> ms} RSFECDecoderWithPlugin>>primFindErrorLocationsDegree:coefficients:result:fieldSize:
>>>>>>>>>
>>>>>>>>> 2.9
>>>>>>>>>
>>>>>>>>> % {
>>>>>>>>>
>>>>>>>>> 654
>>>>>>>>>
>>>>>>>>> ms} RSFECDecoderWithPlugin>>findErrorMagnitudes:errorLocations:
>>>>>>>>>
>>>>>>>>> 1.0
>>>>>>>>>
>>>>>>>>> % {
>>>>>>>>>
>>>>>>>>> 237
>>>>>>>>>
>>>>>>>>> ms} RSFECDecoderWithPlugin>>runEuclideanAlgorithmPoly:poly:rDegrees:
>>>>>>>>
>>>>>>>> Calls to plugganized GF/GFPoly methods:
>>>>>>>>
>>>>>>>>> 1.4% {317ms} RSFECGenericGFWithPlugin>>log:
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/cryptography/attachments/20210605/2d6594e8/attachment-0001.html>