Hi Levente,
Thank you for having a look-see and letting me know your thoughts! It makes sense what you are saying. so I think the only way to speed it up is to embed the loops into primitives. I have 5 of 6 methods from Poly that I am stuffing into the plugin. The #dividePoly: is a bit more complicated, updating the remainder poly and so forth. I do not think a Poly can be within a primitive. Hopefully, {product := Array new: size withAll: 0} will work in translation. I fixed #initializeExpTable per your suggestion.
Thanks!
--- Kindly, Robert
On 5/30/21 8:18 PM, Levente Uzonyi wrote:
Hi Robert,
I had a quick look at the plugin code. The operations you implemented in the plugin are way too simple. The VM's JIT makes the smalltalk code run faster, because it has the same operations implemented, and it can avoid calling the overhead of calling the primitive many times. Hence the slowdown.
For example, RSFECPlugin >> #primitiveAddOrSubtractby seems to implement bitwise xor on two 32-bit unsigned values, but the arguments are always 0-255. The VM has primitive 16, which does bitwise xor and covers that input range, so the JIT, combined with other operations can generate machine code directly with that without jumping back and forth between native code and smalltalk.
If you want a plugin to provide any noticable speedup, you need to do more computation in a single primtiive call.
Levente
P.S.: I think, in RSFECGenericGC >> #initializeExpTable, the lines
x := x bitXor: primitive. x bitAnd: (size - 1)]].should read
x := (x bitXor: primitive) bitAnd: size - 1 ]].
I implemented a set of 5 primitives for Polynomial math in the RSFECPlugin class and added a RSFECGenericGFPolyWithPlugin to call them. In testing FEC I get a segFault, but looking at the Smalltalk stack dump it is happening in the RSErasurePlugin which is not called by the RSFEC code. It looks like a DoIt and I cannot find the offending code. Where can I look for a DoIt that is calling RSErasure code when running RSFEC code? Here is the smalltalk stack dump and I am attaching the crash.dmp file.
0x7ffde5e6a1b0 I RSErasureGaloisWithPlugin>galoisMultiply:by: 0xc7ed9d8: a(n) RSErasureGaloisWithPlugin 0x7ffde5e6a220 I RSErasureGaloisWithPlugin>generateMultiplicationTable 0xc7ed9d8: a(n) RSErasureGaloisWithPlugin 0x7ffde5e59df0 I RSErasureGaloisWithPlugin>initialize 0xc7ed9d8: a(n) RSErasureGaloisWithPlugin 0x7ffde5e59e20 M RSErasureGaloisWithPlugin class(Behavior)>new 0x7fdeec8: a(n) RSErasureGaloisWithPlugin class 0x7ffde5e59e60 I RSErasureGalois class>newGalois 0x6a89510: a(n) RSErasureGalois class 0x7ffde5e59e90 M UndefinedObject>DoIt 0x2de78e0: a(n) UndefinedObject
--- Kindly, Robert
On 5/30/21 8:50 PM, Robert Withers wrote:
Hi Levente,
Thank you for having a look-see and letting me know your thoughts! It makes sense what you are saying. so I think the only way to speed it up is to embed the loops into primitives. I have 5 of 6 methods from Poly that I am stuffing into the plugin. The #dividePoly: is a bit more complicated, updating the remainder poly and so forth. I do not think a Poly can be within a primitive. Hopefully, {product := Array new: size withAll: 0} will work in translation. I fixed #initializeExpTable per your suggestion.
Thanks!
Kindly, Robert
On 5/30/21 8:18 PM, Levente Uzonyi wrote:
Hi Robert,
I had a quick look at the plugin code. The operations you implemented in the plugin are way too simple. The VM's JIT makes the smalltalk code run faster, because it has the same operations implemented, and it can avoid calling the overhead of calling the primitive many times. Hence the slowdown.
For example, RSFECPlugin >> #primitiveAddOrSubtractby seems to implement bitwise xor on two 32-bit unsigned values, but the arguments are always 0-255. The VM has primitive 16, which does bitwise xor and covers that input range, so the JIT, combined with other operations can generate machine code directly with that without jumping back and forth between native code and smalltalk.
If you want a plugin to provide any noticable speedup, you need to do more computation in a single primtiive call.
Levente
P.S.: I think, in RSFECGenericGC >> #initializeExpTable, the lines
x := x bitXor: primitive. x bitAnd: (size - 1)]].should read
x := (x bitXor: primitive) bitAnd: size - 1 ]].
My previous email is awaiting approval as it is too big. I attached a crash.dmp. Here is the email with a link to the crash dump I placed on dropbox.
I implemented a set of 5 primitives for Polynomial math in the RSFECPlugin class and added a RSFECGenericGFPolyWithPlugin to call them. In testing FEC I get a segFault, but looking at the Smalltalk stack dump it is happening in the RSErasurePlugin which is not called by the RSFEC code. It looks like a DoIt and I cannot find the offending code. Where can I look for a DoIt that is calling RSErasure code when running RSFEC code? Here is the smalltalk stack dump and I am attaching the crash.dmp file [1].
0x7ffde5e6a1b0 I RSErasureGaloisWithPlugin>galoisMultiply:by: 0xc7ed9d8: a(n) RSErasureGaloisWithPlugin 0x7ffde5e6a220 I RSErasureGaloisWithPlugin>generateMultiplicationTable 0xc7ed9d8: a(n) RSErasureGaloisWithPlugin 0x7ffde5e59df0 I RSErasureGaloisWithPlugin>initialize 0xc7ed9d8: a(n) RSErasureGaloisWithPlugin 0x7ffde5e59e20 M RSErasureGaloisWithPlugin class(Behavior)>new 0x7fdeec8: a(n) RSErasureGaloisWithPlugin class 0x7ffde5e59e60 I RSErasureGalois class>newGalois 0x6a89510: a(n) RSErasureGalois class 0x7ffde5e59e90 M UndefinedObject>DoIt 0x2de78e0: a(n) UndefinedObject
[]1 crash.zip - https://www.dropbox.com/s/qyq735yksmc4dbr/crash.zip?dl=0
--- Kindly, Robert --- Kindly, Robert
On 5/30/21 8:50 PM, Robert Withers wrote:
Hi Levente,
Thank you for having a look-see and letting me know your thoughts! It makes sense what you are saying. so I think the only way to speed it up is to embed the loops into primitives. I have 5 of 6 methods from Poly that I am stuffing into the plugin. The #dividePoly: is a bit more complicated, updating the remainder poly and so forth. I do not think a Poly can be within a primitive. Hopefully, {product := Array new: size withAll: 0} will work in translation. I fixed #initializeExpTable per your suggestion.
Thanks!
Kindly, Robert
On 5/30/21 8:18 PM, Levente Uzonyi wrote:
Hi Robert,
I had a quick look at the plugin code. The operations you implemented in the plugin are way too simple. The VM's JIT makes the smalltalk code run faster, because it has the same operations implemented, and it can avoid calling the overhead of calling the primitive many times. Hence the slowdown.
For example, RSFECPlugin >> #primitiveAddOrSubtractby seems to implement bitwise xor on two 32-bit unsigned values, but the arguments are always 0-255. The VM has primitive 16, which does bitwise xor and covers that input range, so the JIT, combined with other operations can generate machine code directly with that without jumping back and forth between native code and smalltalk.
If you want a plugin to provide any noticable speedup, you need to do more computation in a single primtiive call.
Levente
P.S.: I think, in RSFECGenericGC >> #initializeExpTable, the lines
x := x bitXor: primitive. x bitAnd: (size - 1)]].should read
x := (x bitXor: primitive) bitAnd: size - 1 ]].
The segFault occurs with a call to a RSFECPlugin primitive>>#primMultiplyPolySelfCoefficients:otherCoefficients:product:fieldSize:. But the crash.dmp shows a RSErasureGaloisWithPlugin as the error site. I am very confused as the call I am making in the FEC test is to the RSFECPlugin, but the RSErasurePlugin is the one cited as causing the segFault. I even removed the RSErasurePlugin from the vm directory and this is still occurring.
RSFECGenericGFPolyWithPlugin>>#multiplyPoly: other
| product | (field = other field) ifFalse: [RSErasureIllegalArgumentError signal: 'GenericGFPolys do not have same GenericGF field']. (self isZero) ifTrue: [^ field zero] ifFalse: [other isZero ifTrue: [^ field zero]]. product := Array new: (self coefficients size + other coefficients size - 1) withAll: 0.
product := self primMultiplyPolySelfCoefficients: self coefficients otherCoefficients: other coefficients product: product fieldSize: field size.
^ RSFECGenericGFPoly newField: field coefficients: product.
Any assistance most welcome. This has me stumped.
--- Kindly, Robert
On 5/31/21 3:43 PM, Robert Withers wrote:
My previous email is awaiting approval as it is too big. I attached a crash.dmp. Here is the email with a link to the crash dump I placed on dropbox.
I implemented a set of 5 primitives for Polynomial math in the RSFECPlugin class and added a RSFECGenericGFPolyWithPlugin to call them. In testing FEC I get a segFault, but looking at the Smalltalk stack dump it is happening in the RSErasurePlugin which is not called by the RSFEC code. It looks like a DoIt and I cannot find the offending code. Where can I look for a DoIt that is calling RSErasure code when running RSFEC code? Here is the smalltalk stack dump and I am attaching the crash.dmp file [1].
0x7ffde5e6a1b0 I RSErasureGaloisWithPlugin>galoisMultiply:by: 0xc7ed9d8: a(n) RSErasureGaloisWithPlugin 0x7ffde5e6a220 I RSErasureGaloisWithPlugin>generateMultiplicationTable 0xc7ed9d8: a(n) RSErasureGaloisWithPlugin 0x7ffde5e59df0 I RSErasureGaloisWithPlugin>initialize 0xc7ed9d8: a(n) RSErasureGaloisWithPlugin 0x7ffde5e59e20 M RSErasureGaloisWithPlugin class(Behavior)>new 0x7fdeec8: a(n) RSErasureGaloisWithPlugin class 0x7ffde5e59e60 I RSErasureGalois class>newGalois 0x6a89510: a(n) RSErasureGalois class 0x7ffde5e59e90 M UndefinedObject>DoIt 0x2de78e0: a(n) UndefinedObject
[]1 crash.zip - https://www.dropbox.com/s/qyq735yksmc4dbr/crash.zip?dl=0
Kindly, Robert
Kindly, Robert
On 5/30/21 8:50 PM, Robert Withers wrote:
Hi Levente,
Thank you for having a look-see and letting me know your thoughts! It makes sense what you are saying. so I think the only way to speed it up is to embed the loops into primitives. I have 5 of 6 methods from Poly that I am stuffing into the plugin. The #dividePoly: is a bit more complicated, updating the remainder poly and so forth. I do not think a Poly can be within a primitive. Hopefully, {product := Array new: size withAll: 0} will work in translation. I fixed #initializeExpTable per your suggestion.
Thanks!
Kindly, Robert
On 5/30/21 8:18 PM, Levente Uzonyi wrote:
Hi Robert,
I had a quick look at the plugin code. The operations you implemented in the plugin are way too simple. The VM's JIT makes the smalltalk code run faster, because it has the same operations implemented, and it can avoid calling the overhead of calling the primitive many times. Hence the slowdown.
For example, RSFECPlugin >> #primitiveAddOrSubtractby seems to implement bitwise xor on two 32-bit unsigned values, but the arguments are always 0-255. The VM has primitive 16, which does bitwise xor and covers that input range, so the JIT, combined with other operations can generate machine code directly with that without jumping back and forth between native code and smalltalk.
If you want a plugin to provide any noticable speedup, you need to do more computation in a single primtiive call.
Levente
P.S.: I think, in RSFECGenericGC >> #initializeExpTable, the lines
x := x bitXor: primitive. x bitAnd: (size - 1)]].should read
x := (x bitXor: primitive) bitAnd: size - 1 ]].
Extremely strange. I unloaded the CryptographyRSErasure and CryptographyRSErasureTests packages. I still get a crash dump referencing RSErasureGaloisWithPlugin>galoisMultiply:by:. I am totally confused! This class isn't even loaded in my image! WTH is going on!?
Smalltalk stack dump: 0x7ffde5e6a1b0 I RSErasureGaloisWithPlugin>galoisMultiply:by: 0xc7ed9d8: a(n) RSErasureGaloisWithPlugin 0x7ffde5e6a220 I RSErasureGaloisWithPlugin>generateMultiplicationTable 0xc7ed9d8: a(n) RSErasureGaloisWithPlugin 0x7ffde5e59df0 I RSErasureGaloisWithPlugin>initialize 0xc7ed9d8: a(n) RSErasureGaloisWithPlugin 0x7ffde5e59e20 M RSErasureGaloisWithPlugin class(Behavior)>new 0x7fdeec8: a(n) RSErasureGaloisWithPlugin class 0x7ffde5e59e60 I RSErasureGalois class>newGalois 0x6a89510: a(n) RSErasureGalois class 0x7ffde5e59e90 M UndefinedObject>DoIt 0x2de78e0: a(n) UndefinedObject
--- Kindly, Robert
On 5/31/21 4:00 PM, Robert Withers wrote:
The segFault occurs with a call to a RSFECPlugin primitive>>#primMultiplyPolySelfCoefficients:otherCoefficients:product:fieldSize:. But the crash.dmp shows a RSErasureGaloisWithPlugin as the error site. I am very confused as the call I am making in the FEC test is to the RSFECPlugin, but the RSErasurePlugin is the one cited as causing the segFault. I even removed the RSErasurePlugin from the vm directory and this is still occurring.
RSFECGenericGFPolyWithPlugin>>#multiplyPoly: other
| product | (field = other field) ifFalse: [RSErasureIllegalArgumentError signal: 'GenericGFPolys do not have same GenericGF field']. (self isZero) ifTrue: [^ field zero] ifFalse: [other isZero ifTrue: [^ field zero]]. product := Array new: (self coefficients size + other coefficients size - 1) withAll: 0.
product := self primMultiplyPolySelfCoefficients: self coefficients otherCoefficients: other coefficients product: product fieldSize: field size.
^ RSFECGenericGFPoly newField: field coefficients: product.
Any assistance most welcome. This has me stumped.
Kindly, Robert
On 5/31/21 3:43 PM, Robert Withers wrote:
My previous email is awaiting approval as it is too big. I attached a crash.dmp. Here is the email with a link to the crash dump I placed on dropbox.
I implemented a set of 5 primitives for Polynomial math in the RSFECPlugin class and added a RSFECGenericGFPolyWithPlugin to call them. In testing FEC I get a segFault, but looking at the Smalltalk stack dump it is happening in the RSErasurePlugin which is not called by the RSFEC code. It looks like a DoIt and I cannot find the offending code. Where can I look for a DoIt that is calling RSErasure code when running RSFEC code? Here is the smalltalk stack dump and I am attaching the crash.dmp file [1].
0x7ffde5e6a1b0 I RSErasureGaloisWithPlugin>galoisMultiply:by: 0xc7ed9d8: a(n) RSErasureGaloisWithPlugin 0x7ffde5e6a220 I RSErasureGaloisWithPlugin>generateMultiplicationTable 0xc7ed9d8: a(n) RSErasureGaloisWithPlugin 0x7ffde5e59df0 I RSErasureGaloisWithPlugin>initialize 0xc7ed9d8: a(n) RSErasureGaloisWithPlugin 0x7ffde5e59e20 M RSErasureGaloisWithPlugin class(Behavior)>new 0x7fdeec8: a(n) RSErasureGaloisWithPlugin class 0x7ffde5e59e60 I RSErasureGalois class>newGalois 0x6a89510: a(n) RSErasureGalois class 0x7ffde5e59e90 M UndefinedObject>DoIt 0x2de78e0: a(n) UndefinedObject
[]1 crash.zip - https://www.dropbox.com/s/qyq735yksmc4dbr/crash.zip?dl=0
Kindly, Robert
Kindly, Robert
On 5/30/21 8:50 PM, Robert Withers wrote:
Hi Levente,
Thank you for having a look-see and letting me know your thoughts! It makes sense what you are saying. so I think the only way to speed it up is to embed the loops into primitives. I have 5 of 6 methods from Poly that I am stuffing into the plugin. The #dividePoly: is a bit more complicated, updating the remainder poly and so forth. I do not think a Poly can be within a primitive. Hopefully, {product := Array new: size withAll: 0} will work in translation. I fixed #initializeExpTable per your suggestion.
Thanks!
Kindly, Robert
On 5/30/21 8:18 PM, Levente Uzonyi wrote:
Hi Robert,
I had a quick look at the plugin code. The operations you implemented in the plugin are way too simple. The VM's JIT makes the smalltalk code run faster, because it has the same operations implemented, and it can avoid calling the overhead of calling the primitive many times. Hence the slowdown.
For example, RSFECPlugin >> #primitiveAddOrSubtractby seems to implement bitwise xor on two 32-bit unsigned values, but the arguments are always 0-255. The VM has primitive 16, which does bitwise xor and covers that input range, so the JIT, combined with other operations can generate machine code directly with that without jumping back and forth between native code and smalltalk.
If you want a plugin to provide any noticable speedup, you need to do more computation in a single primtiive call.
Levente
P.S.: I think, in RSFECGenericGC >> #initializeExpTable, the lines
x := x bitXor: primitive. x bitAnd: (size - 1)]].should read
x := (x bitXor: primitive) bitAnd: size - 1 ]].
All right some more info. I still don't see what this has to do with RSErasureGaloisWithPlugin. This class in not loaded and I found out some info on the call into primitiveMultiplyByPolySelfCoefficientsOtherCoefficientsProductFieldSize. I used kdbg and set a breakpoint at the beginning of the function. Listed below with the BOOM site bold/italic. The last line in the listing is where it sigterms. The issue is that the 3 Oops, accessed just previously as stack values 3. 2 & 1 are all 0, when stepped over in the kdbg. I tried to recompile the build.debug and installed it. Same exact problem. I recompiled the #primMultiply calling site to this function, still no change. I hope an angel will fix it!
EXPORT(sqInt)
primitiveMultiplyPolySelfCoefficientsOtherCoefficientsProductFieldSize(void)
{
unsigned int aCoeff;
unsigned int *aCoefficients;
sqInt aIndex;
sqInt aLength;
unsigned int *bCoefficients;
sqInt bIndex;
sqInt bLength;
unsigned int fieldSize;
unsigned int *otherCoefficients;
sqInt otherCoefficientsOop;
unsigned int otherCount;
unsigned int *product;
sqInt productOop;
unsigned int *result;
unsigned int *selfCoefficients;
sqInt selfCoefficientsOop;
unsigned int selfCount;
if (!((methodArgumentCount()) == 4)) {
return primitiveFailFor(PrimErrBadNumArgs);
}
selfCoefficientsOop = stackIntegerValue(3);
otherCoefficientsOop = stackIntegerValue(2);
productOop = stackIntegerValue(1);
fieldSize = stackIntegerValue(0);
selfCount = stSizeOf(selfCoefficientsOop);
--- Kindly, Robert
On 5/31/21 4:22 PM, Robert Withers wrote:
Extremely strange. I unloaded the CryptographyRSErasure and CryptographyRSErasureTests packages. I still get a crash dump referencing RSErasureGaloisWithPlugin>galoisMultiply:by:. I am totally confused! This class isn't even loaded in my image! WTH is going on!?
Smalltalk stack dump: 0x7ffde5e6a1b0 I RSErasureGaloisWithPlugin>galoisMultiply:by: 0xc7ed9d8: a(n) RSErasureGaloisWithPlugin 0x7ffde5e6a220 I RSErasureGaloisWithPlugin>generateMultiplicationTable 0xc7ed9d8: a(n) RSErasureGaloisWithPlugin 0x7ffde5e59df0 I RSErasureGaloisWithPlugin>initialize 0xc7ed9d8: a(n) RSErasureGaloisWithPlugin 0x7ffde5e59e20 M RSErasureGaloisWithPlugin class(Behavior)>new 0x7fdeec8: a(n) RSErasureGaloisWithPlugin class 0x7ffde5e59e60 I RSErasureGalois class>newGalois 0x6a89510: a(n) RSErasureGalois class 0x7ffde5e59e90 M UndefinedObject>DoIt 0x2de78e0: a(n) UndefinedObject
Kindly, Robert
On 5/31/21 4:00 PM, Robert Withers wrote:
The segFault occurs with a call to a RSFECPlugin primitive>>#primMultiplyPolySelfCoefficients:otherCoefficients:product:fieldSize:. But the crash.dmp shows a RSErasureGaloisWithPlugin as the error site. I am very confused as the call I am making in the FEC test is to the RSFECPlugin, but the RSErasurePlugin is the one cited as causing the segFault. I even removed the RSErasurePlugin from the vm directory and this is still occurring.
RSFECGenericGFPolyWithPlugin>>#multiplyPoly: other | product | (field = other field) ifFalse: [RSErasureIllegalArgumentError signal: 'GenericGFPolys do not have same GenericGF field']. (self isZero) ifTrue: [^ field zero] ifFalse: [other isZero ifTrue: [^ field zero]]. product := Array new: (self coefficients size + other coefficients size - 1) withAll: 0. product := self primMultiplyPolySelfCoefficients: self coefficients otherCoefficients: other coefficients product: product fieldSize: field size. ^ RSFECGenericGFPoly newField: field coefficients: product.
Any assistance most welcome. This has me stumped.
Kindly, Robert
On 5/31/21 3:43 PM, Robert Withers wrote:
My previous email is awaiting approval as it is too big. I attached a crash.dmp. Here is the email with a link to the crash dump I placed on dropbox.
I implemented a set of 5 primitives for Polynomial math in the RSFECPlugin class and added a RSFECGenericGFPolyWithPlugin to call them. In testing FEC I get a segFault, but looking at the Smalltalk stack dump it is happening in the RSErasurePlugin which is not called by the RSFEC code. It looks like a DoIt and I cannot find the offending code. Where can I look for a DoIt that is calling RSErasure code when running RSFEC code? Here is the smalltalk stack dump and I am attaching the crash.dmp file [1].
0x7ffde5e6a1b0 I RSErasureGaloisWithPlugin>galoisMultiply:by: 0xc7ed9d8: a(n) RSErasureGaloisWithPlugin 0x7ffde5e6a220 I RSErasureGaloisWithPlugin>generateMultiplicationTable 0xc7ed9d8: a(n) RSErasureGaloisWithPlugin 0x7ffde5e59df0 I RSErasureGaloisWithPlugin>initialize 0xc7ed9d8: a(n) RSErasureGaloisWithPlugin 0x7ffde5e59e20 M RSErasureGaloisWithPlugin class(Behavior)>new 0x7fdeec8: a(n) RSErasureGaloisWithPlugin class 0x7ffde5e59e60 I RSErasureGalois class>newGalois 0x6a89510: a(n) RSErasureGalois class 0x7ffde5e59e90 M UndefinedObject>DoIt 0x2de78e0: a(n) UndefinedObject
[]1 crash.zip - https://www.dropbox.com/s/qyq735yksmc4dbr/crash.zip?dl=0
Kindly, Robert
Kindly, Robert
On 5/30/21 8:50 PM, Robert Withers wrote:
Hi Levente,
Thank you for having a look-see and letting me know your thoughts! It makes sense what you are saying. so I think the only way to speed it up is to embed the loops into primitives. I have 5 of 6 methods from Poly that I am stuffing into the plugin. The #dividePoly: is a bit more complicated, updating the remainder poly and so forth. I do not think a Poly can be within a primitive. Hopefully, {product := Array new: size withAll: 0} will work in translation. I fixed #initializeExpTable per your suggestion.
Thanks!
Kindly, Robert
On 5/30/21 8:18 PM, Levente Uzonyi wrote:
Hi Robert,
I had a quick look at the plugin code. The operations you implemented in the plugin are way too simple. The VM's JIT makes the smalltalk code run faster, because it has the same operations implemented, and it can avoid calling the overhead of calling the primitive many times. Hence the slowdown.
For example, RSFECPlugin >> #primitiveAddOrSubtractby seems to implement bitwise xor on two 32-bit unsigned values, but the arguments are always 0-255. The VM has primitive 16, which does bitwise xor and covers that input range, so the JIT, combined with other operations can generate machine code directly with that without jumping back and forth between native code and smalltalk.
If you want a plugin to provide any noticable speedup, you need to do more computation in a single primtiive call.
Levente
P.S.: I think, in RSFECGenericGC >> #initializeExpTable, the lines
x := x bitXor: primitive. x bitAnd: (size - 1)]].should read
x := (x bitXor: primitive) bitAnd: size - 1 ]].
cryptography@lists.squeakfoundation.org
-
Robert Withers