[Vm-dev] Reed Solomon plugins & performance slow down

Thu Jun 3 11:29:28 UTC 2021

Oops, email size too big. I zipped everything up. The SqueakDebug.log has some good info in it. Here is the evidence:

> https://www.dropbox.com/s/fcinfptyvv4xb42/Decoder-primitiveFailed.zip?dl=0

Levente,

Some more info. I saved another version of the segfault package. I was able to run the test resulting in a Squeak debugger. I am listing two snapshots, one is on the #decode:twoS: stack frame and the other is the next frame #findErrorLocations: As you can see the sigma method variable is a GFPolynomial, while the errorLocator in the subsequent frame is nil, even though the sigma is passed as that argument. How did *that* happen? Here is the DoIt to load everything needed:

> Installer ss
> project: 'Cryptography';
> install: 'ProCrypto-1-1-1';
> install: 'ProCryptoTests-1-1-1';
> install: 'CryptographyRSFECPlugins';
> install: 'TraceMonitor';
> install: 'CryptographyRSFECPluginsDecoder-segfault'.

I am also attaching the etrace.log with some debug info I logged. I am confused. When I have gotten a crash.dmp, one of the lines in the c stack is a call to isBytes.

> /usr/local/bin/../lib/squeak/5.0-202010010729/squeak(isBytes+0x9)[0x44d289]

Grazie!
Robert
---
Kindly,
Robert

On 6/2/21 2:49 PM, Robert Withers wrote:

> Hey Levente,
>
> I am trying to primitize the #findErrorLocations: & #findErrorMagnitudes:errorLocations: methods. I thought I had it done, but I am getting a segfault when running the RSFECReedSolomonTest>>#testDataMatrix test. I tried to dig into it and find out the issue. I think one of the params to the primitive is nil. The strange thing I observed is a debug stack in Squeak, where the argument to the prior stack frame is an instance, but the argument in the top frame is nil. I do not understand how it can go from type -> nil.
>
> If you wish to help me figure this out, the current RSFEC code is loaded with:
>
>> Installer ss
>> project: 'Cryptography';
>> install: 'ProCrypto-1-1-1';
>> install: 'ProCryptoTests-1-1-1'.
>
> Then the Decoder primitives to RSFECPlugin is loaded from:
>
>> CryptographyRSFECPluginsDecoder-segfault-rww.2
>
> ---
> Kindly,
> Robert
>
> On 6/2/21 10:40 AM, Levente Uzonyi wrote:
>
>> Hi Robert,
>>
>> On Tue, 1 Jun 2021, Robert Withers wrote:
>>
>>> Hey ya, Levente,
>>>
>>> Woot!!! I was able to get all the primitives working with ByteArrays. Everything is passing green! (except for several GF fields: Aztec and Maxicode and QRCode) I think I will primitize one of the codingLoops on the RSErasure
>>> side. Here are the numbers.  Take note of the tallies and time: 50 seconds versus 114 seconds. I updated the squeak wiki Cryptography page (
>>> http://wiki.squeak.org/squeak/5776
>>> ) with d/l links to the plugins. Here are those
>>
>> Brilliant. ByteArray is indeed the optimal choice there.
>>
>> Let me answer your question in your previous email here:
>> AFAIK, on all the currently supported platforms and C compilers, unsigned
>> int is 32-bits, so it matches WordArray in Squeak.
>> DoubleWordArray is 64-bits, and in C that matches the type unsigned long long (on the currently supported platforms and compilers).
>>
>> I think you can shave off a few more seconds there by adding those
>> <inline: #always> pragmas to the methods the primitive methods send
>> either directly or indirectly.
>>
>> And, I think the logTable and expTable variables can be declared as
>> constant as well as unsigned char arrays instead of unsigned int arrays.
>> That might give more opportunities to the C compiler to optimize the code.
>>
>> I prefer to use #var:type:array: instead of #var:declareC: when possible.
>> E.g.
>>
>>	cg
>>		var: #logTable declareC: 'unsigned int logTable[256] = {
>> 0, 0, 1, 240, 2, 225, 241, 53, 3, 38, 226, 133, 242, 43, 54, 210, 4, 195,
>> 39, 114, 227, 106, 134, 28, 243, 140, 44, 23, 55, 118, 211, 234, 5, 219,
>> 196, 96, 40, 222, 115, 103, 228, 78, 107, 125, 135, 8, 29, 162, 244, 186,
>> 141, 180, 45, 99, 24, 49, 56, 13, 119, 153, 212, 199, 235, 91, 6, 76, 220,
>> 217, 197, 11, 97, 184, 41, 36, 223, 253, 116, 138, 104, 193, 229, 86, 79,
>> 171, 108, 165, 126, 145, 136, 34, 9, 74, 30, 32, 163, 84, 245, 173, 187,
>> 204, 142, 81, 181, 190, 46, 88, 100, 159, 25, 231, 50, 207, 57, 147, 14,
>> 67, 120, 128, 154, 248, 213, 167, 200, 63, 236, 110, 92, 176, 7, 161, 77,
>> 124, 221, 102, 218, 95, 198, 90, 12, 152, 98, 48, 185, 179, 42, 209, 37,
>> 132, 224, 52, 254, 239, 117, 233, 139, 22, 105, 27, 194, 113, 230, 206,
>> 87, 158, 80, 189, 172, 203, 109, 175, 166, 62, 127, 247, 146, 66, 137,
>> 192, 35, 252, 10, 183, 75, 216, 31, 83, 33, 73, 164, 144, 85, 170, 246,
>> 65, 174, 61, 188, 202, 205, 157, 143, 169, 82, 72, 182, 215, 191, 251, 47,
>> 178, 89, 151, 101, 94, 160, 123, 26, 112, 232, 21, 51, 238, 208, 131, 58,
>> 69, 148, 18, 15, 16, 68, 17, 121, 149, 129, 19, 155, 59, 249, 70, 214,
>> 250, 168, 71, 201, 156, 64, 60, 237, 130, 111, 20, 93, 122, 177, 150 }';
>>
>> could be written as
>>
>>	cg
>>		var: #logTable
>>		type: #'const unsigned char'
>>		array:  #[0 0 1 240 2 225 241 53 3 38 226 133 242 43 54
>> 210 4 195 39 114 227 106 134 28 243 140 44 23 55 118 211 234 5 219 196 96
>> 40 222 115 103 228 78 107 125 135 8 29 162 244 186 141 180 45 99 24 49 56
>> 13 119 153 212 199 235 91 6 76 220 217 197 11 97 184 41 36 223 253 116 138
>> 104 193 229 86 79 171 108 165 126 145 136 34 9 74 30 32 163 84 245 173 187
>> 204 142 81 181 190 46 88 100 159 25 231 50 207 57 147 14 67 120 128 154
>> 248 213 167 200 63 236 110 92 176 7 161 77 124 221 102 218 95 198 90 12
>> 152 98 48 185 179 42 209 37 132 224 52 254 239 117 233 139 22 105 27 194
>> 113 230 206 87 158 80 189 172 203 109 175 166 62 127 247 146 66 137 192 35
>> 252 10 183 75 216 31 83 33 73 164 144 85 170 246 65 174 61 188 202 205 157
>> 143 169 82 72 182 215 191 251 47 178 89 151 101 94 160 123 26 112 232 21
>> 51 238 208 131 58 69 148 18 15 16 68 17 121 149 129 19 155 59 249 70 214
>> 250 168 71 201 156 64 60 237 130 111 20 93 122 177 150]
>>
>> Levente
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210603/f92a451e/attachment.html>