I thought I would post to these lists. I have a nascent implementation of Reed-Solomon encoding using Galois Fields. Computation within the field is governed by polynomial representation and operations (add, subtract, multiply, divide, inverse, exp, log). These operations on polynomials is where the profile says most time is being used. The code can be seen in the SecureSession package in squeaksource's Cryptography repository.
The point of this email is to see if the folks who would know how to represent polynomials and operations in Slang would be interested in working towards this performance effort, whether through discussion of implementation. How would you Slanginize a polynomial and it's operations?
manythanks,
On 18-12-2015, at 11:25 AM, Robert Withers robert.w.withers@gmail.com wrote: The point of this email is to see if the folks who would know how to represent polynomials and operations in Slang would be interested in working towards this performance effort, whether through discussion of implementation. How would you Slanginize a polynomial and it's operations?
No need. Just write that part in a plain old C or whatever file. Call the relevant functions from a stub generated by slang. Look at, for example, DropPlugin; there is a generic part created from the slang that calls eg dropRequestFileName which is implemented in platforms/??/plugins/DropPlugin/sq??DragDrop.c etc
tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Fractured Idiom:- RESPONDEZ S'IL VOUS PLAID - Honk if you're Scots
I think I see your suggestion. Is it to find a c implementation of RS/Galois polynomials and wrap it in a facade model in squeak?
This has me thinking whether that sort of thing is stateful or not. I would suppose that context state (lookup tables) are there, but not operational state (specific polynomials that get operated upon: polyA divide: polyB and such). Well, that will depend on the c library.
Here are 3 I found with a quick search. I will research further.
http://rscode.sourceforge.net/ "probably this one, with open-source support" http://web.eecs.utk.edu/~plank/plank/gflib/
In my implementation, I can recover the original message, though it may be damaged. If it is damaged, my euclidean algorithm and others are failing to repair. Here is a section on RS decoding: https://en.wikiversity.org/wiki/Reed%E2%80%93Solomon_codes_for_coders#RS_dec.... This would be another advantage of a third-party library.
robert
On 12/18/2015 02:45 PM, tim Rowledge wrote:
On 18-12-2015, at 11:25 AM, Robert Withers robert.w.withers@gmail.com wrote: The point of this email is to see if the folks who would know how to represent polynomials and operations in Slang would be interested in working towards this performance effort, whether through discussion of implementation. How would you Slanginize a polynomial and it's operations?
No need. Just write that part in a plain old C or whatever file. Call the relevant functions from a stub generated by slang. Look at, for example, DropPlugin; there is a generic part created from the slang that calls eg dropRequestFileName which is implemented in platforms/??/plugins/DropPlugin/sq??DragDrop.c etc
tim
tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Fractured Idiom:- RESPONDEZ S'IL VOUS PLAID - Honk if you're Scots
On 18-12-2015, at 11:55 AM, Robert Withers robert.w.withers@gmail.com wrote:
I think I see your suggestion. Is it to find a c implementation of RS/Galois polynomials and wrap it in a facade model in squeak?
No, not particularly - I know damn all about galois. Isn’t it a brand of stinky French cigarettes?
Just reminding you that you do not need to do everything in slang for plugins. If you can write your stuff in C (or even C++ with some ugly wrapper bits) you can include it in a plugin.
tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Strange OpCodes: CDC: Clear Disks and Crash
On 12/18/2015 03:09 PM, tim Rowledge wrote:
On 18-12-2015, at 11:55 AM, Robert Withers robert.w.withers@gmail.com wrote:
I think I see your suggestion. Is it to find a c implementation of RS/Galois polynomials and wrap it in a facade model in squeak?
No, not particularly - I know damn all about galois. Isn’t it a brand of stinky French cigarettes?
IIRC, Gauloise were 3 dollars a carton in Beirut, after the currency exchange. Cheap french cigarettes in the old colonies, I love it!
Just reminding you that you do not need to do everything in slang for plugins. If you can write your stuff in C (or even C++ with some ugly wrapper bits) you can include it in a plugin.
Ahh, yes I see. Well, that does ensure & preserve ownership, which has benefit. I will research & correct the repair code.
I would dearly love to not have platform code, come to think of it. That is a great advantage to crypto plugins, currently, I feel. I would like to go back to me original query, as such, about slanginizing a polynomial model. If we could slang GenericGF, GenericGFPoly and GaloisField, that would actually be preferable for the cross-platform case. Is that possible?
robert
tim
tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Strange OpCodes: CDC: Clear Disks and Crash
On 18-12-2015, at 12:15 PM, Robert Withers robert.w.withers@gmail.com wrote: I would dearly love to not have platform code, come to think of it.
Then don’t have any. It’s all capable of doing that. See Squeak3D for example. The slang code in B3DEnginePlugin and its subclasses implements a lot of stuff and all the C code is in platforms/Cross/plugins/Squeak3D No platform specific code.
tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Oxymorons: Clearly misunderstood
That's the perfect resource to look into. Thank you!
On 12/18/2015 03:46 PM, tim Rowledge wrote:
On 18-12-2015, at 12:15 PM, Robert Withers robert.w.withers@gmail.com wrote: I would dearly love to not have platform code, come to think of it.
Then don’t have any. It’s all capable of doing that. See Squeak3D for example. The slang code in B3DEnginePlugin and its subclasses implements a lot of stuff and all the C code is in platforms/Cross/plugins/Squeak3D No platform specific code.
tim
tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Oxymorons: Clearly misunderstood
It is a strange place I live in. Without changing any code, the below test passed:
testFECDamagedEncodeAndDecodeRS_15_9
| chunk decodedBytes proxy decoder encoder encodedByteFrame | chunk := self buildChunk: 8.
decoder := FECDecoder new. encoder := FECEncoder new. proxy := decoder, encoder. proxy asProtocolStack.
encodedByteFrame := encoder encodeFrame: (Frame onPayload: chunk). 1 timesRepeat: [encodedByteFrame payload at: encodedByteFrame payload size atRandom put: 255 atRandom]. decodedBytes := (decoder decodeFrame: encodedByteFrame) payload.
self assert: chunk equals: decodedBytes.
bless, robert
On 12/18/2015 03:46 PM, tim Rowledge wrote:
On 18-12-2015, at 12:15 PM, Robert Withers robert.w.withers@gmail.com wrote: I would dearly love to not have platform code, come to think of it.
Then don’t have any. It’s all capable of doing that. See Squeak3D for example. The slang code in B3DEnginePlugin and its subclasses implements a lot of stuff and all the C code is in platforms/Cross/plugins/Squeak3D No platform specific code.
tim
tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Oxymorons: Clearly misunderstood
I hope you don't mind if I share my thoughts about this. This is not the first time I saw this mutation succeed...I believe it is the third time, now. Clearly a byte is damaged or is it? There is the chance of the identity mutation, whereby the same value is set. It has happened 3 time out of 30 bytes, not unreasonable. Sorry to take away the magic!
However, due to 18 bytes being turned into 30 bytes, then back again, there may be capacity. After the back again, only then will Euclid get his shot to fix outright errors, but due to the increased information in the 30 bytes, I am thinking that some degree of flexibility to correct may exist. I suppose that's up to the mathematics of GFs.
If not, I wonder if there's a Vatrix, super-imposition overlay mathematics that combines this approach with an inside approach.
best, robert
On 12/18/2015 04:59 PM, Robert Withers wrote:
It is a strange place I live in. Without changing any code, the below test passed:
testFECDamagedEncodeAndDecodeRS_15_9 | chunk decodedBytes proxy decoder encoder encodedByteFrame | chunk := self buildChunk: 8. decoder := FECDecoder new. encoder := FECEncoder new. proxy := decoder, encoder. proxy asProtocolStack. encodedByteFrame := encoder encodeFrame: (Frame onPayload: chunk). 1 timesRepeat: [encodedByteFrame payload at: encodedByteFrame payload size atRandom put: 255 atRandom]. decodedBytes := (decoder decodeFrame: encodedByteFrame) payload. self assert: chunk equals: decodedBytes.bless, robert
On 12/18/2015 03:46 PM, tim Rowledge wrote:
On 18-12-2015, at 12:15 PM, Robert Withersrobert.w.withers@gmail.com wrote: I would dearly love to not have platform code, come to think of it.
Then don’t have any. It’s all capable of doing that. See Squeak3D for example. The slang code in B3DEnginePlugin and its subclasses implements a lot of stuff and all the C code is in platforms/Cross/plugins/Squeak3D No platform specific code.
tim
tim Rowledge;tim@rowledge.org;http://www.rowledge.org/tim Oxymorons: Clearly misunderstood
-- . .. .. ^,^ robert
Vatrix was the term I came up with to describe a volumetric matrix construct, with a root plane matrix and non root plan matrices. This idea came from the meta model of smalltalk, so, through observation. Heh. eperimental physics it turns out, Sheridan. :)
The idea is that off-root plane elements are inverse determinant-like projections, or dimensional inflations. 1 dimension inflates to 3 dimensions and so on. I came up with a name for the undefined operations to project and collapse dimensions: closure-bindings. :) 3 dimensions get bound, then closed into the higher plane.
We are actually living in 3 spacetimes.
Peace, robert
On 12/18/2015 05:17 PM, Robert Withers wrote:
I hope you don't mind if I share my thoughts about this. This is not the first time I saw this mutation succeed...I believe it is the third time, now. Clearly a byte is damaged or is it? There is the chance of the identity mutation, whereby the same value is set. It has happened 3 time out of 30 bytes, not unreasonable. Sorry to take away the magic!
However, due to 18 bytes being turned into 30 bytes, then back again, there may be capacity. After the back again, only then will Euclid get his shot to fix outright errors, but due to the increased information in the 30 bytes, I am thinking that some degree of flexibility to correct may exist. I suppose that's up to the mathematics of GFs.
If not, I wonder if there's a Vatrix, super-imposition overlay mathematics that combines this approach with an inside approach.
best, robert
On 12/18/2015 04:59 PM, Robert Withers wrote:
It is a strange place I live in. Without changing any code, the below test passed:
testFECDamagedEncodeAndDecodeRS_15_9 | chunk decodedBytes proxy decoder encoder encodedByteFrame | chunk := self buildChunk: 8. decoder := FECDecoder new. encoder := FECEncoder new. proxy := decoder, encoder. proxy asProtocolStack. encodedByteFrame := encoder encodeFrame: (Frame onPayload: chunk). 1 timesRepeat: [encodedByteFrame payload at: encodedByteFrame payload size atRandom put: 255 atRandom]. decodedBytes := (decoder decodeFrame: encodedByteFrame) payload. self assert: chunk equals: decodedBytes.bless, robert
On 12/18/2015 03:46 PM, tim Rowledge wrote:
On 18-12-2015, at 12:15 PM, Robert Withersrobert.w.withers@gmail.com wrote: I would dearly love to not have platform code, come to think of it.
Then don’t have any. It’s all capable of doing that. See Squeak3D for example. The slang code in B3DEnginePlugin and its subclasses implements a lot of stuff and all the C code is in platforms/Cross/plugins/Squeak3D No platform specific code.
tim
tim Rowledge;tim@rowledge.org;http://www.rowledge.org/tim Oxymorons: Clearly misunderstood
-- . .. .. ^,^ robert
-- . .. .. ^,^ robert
I see, you mentioned this plugin to me some days back, relating to consolidating the crypto plugins. You meant me to make them subclasses? Such that they are inside the scope of the outer built plugin? That would be nice. It is on the list of things to do.
robert . .. ...
On 12/18/2015 03:46 PM, tim Rowledge wrote:
On 18-12-2015, at 12:15 PM, Robert Withers robert.w.withers@gmail.com wrote: I would dearly love to not have platform code, come to think of it.
Then don’t have any. It’s all capable of doing that. See Squeak3D for example. The slang code in B3DEnginePlugin and its subclasses implements a lot of stuff and all the C code is in platforms/Cross/plugins/Squeak3D No platform specific code.
tim
tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Oxymorons: Clearly misunderstood
On Fri, Dec 18, 2015 at 11:45:26AM -0800, tim Rowledge wrote:
On 18-12-2015, at 11:25 AM, Robert Withers robert.w.withers@gmail.com wrote: The point of this email is to see if the folks who would know how to represent polynomials and operations in Slang would be interested in working towards this performance effort, whether through discussion of implementation. How would you Slanginize a polynomial and it's operations?
No need. Just write that part in a plain old C or whatever file. Call the relevant functions from a stub generated by slang. Look at, for example, DropPlugin; there is a generic part created from the slang that calls eg dropRequestFileName which is implemented in platforms/??/plugins/DropPlugin/sq??DragDrop.c etc
Actually either way is fine. If it's easy to implement in C, then do that exactly as Tim suggests. If you prefer implementing in Smalltalk, you can do that too, just stick to simple Smalltalk that is likely to be translatable.
Robert, I'm not offering to implement this, but if you go to the trouble of producing a Smalltalk implementation then I'm sure that either Tim or I or someone else will be happy to help you make it "slangified" such that it can be translated to C in a plugin, like the other Cryptography plugins.
If you are implementing your algorithm from scratch, then it would be nice if it could be done in Smalltalk, because that means we can all read it, play with it in the image, and write unit tests to validate and document its behavior. We can figure out the translation to C afterwords, once you have a good implementation in Smalltalk (and yes I will help with that).
On the other hand, if you are trying to adapt some existing C code that already works, then you are better off following Tim's original suggestion above.
Dave
On 12/18/2015 08:07 PM, David T. Lewis wrote:
On Fri, Dec 18, 2015 at 11:45:26AM -0800, tim Rowledge wrote:
On 18-12-2015, at 11:25 AM, Robert Withers robert.w.withers@gmail.com wrote: The point of this email is to see if the folks who would know how to represent polynomials and operations in Slang would be interested in working towards this performance effort, whether through discussion of implementation. How would you Slanginize a polynomial and it's operations?
No need. Just write that part in a plain old C or whatever file. Call the relevant functions from a stub generated by slang. Look at, for example, DropPlugin; there is a generic part created from the slang that calls eg dropRequestFileName which is implemented in platforms/??/plugins/DropPlugin/sq??DragDrop.c etc
Actually either way is fine. If it's easy to implement in C, then do that exactly as Tim suggests. If you prefer implementing in Smalltalk, you can do that too, just stick to simple Smalltalk that is likely to be translatable.
Robert, I'm not offering to implement this, but if you go to the trouble of producing a Smalltalk implementation then I'm sure that either Tim or I or someone else will be happy to help you make it "slangified" such that it can be translated to C in a plugin, like the other Cryptography plugins.
If you are implementing your algorithm from scratch, then it would be nice if it could be done in Smalltalk, because that means we can all read it, play with it in the image, and write unit tests to validate and document its behavior. We can figure out the translation to C afterwords, once you have a good implementation in Smalltalk (and yes I will help with that).
I have a smalltalk implmentation that came from Google Java code and is fully implemented in squeak/pharo. I am intermittently fixing byte mutations in the payload, so it is working to a degree. Check out the classes GenericGF, GenericGFPoly and GaloisField to see the insides. FECEncoder and FECDecoder are calling the galoisField which builds a ReedSolomon Decoder. That last is where the error detection occurs.
Thank you all very much for looking out!
On the other hand, if you are trying to adapt some existing C code that already works, then you are better off following Tim's original suggestion above.
Dave
On Fri, Dec 18, 2015 at 08:13:00PM -0500, Robert Withers wrote:
On 12/18/2015 08:07 PM, David T. Lewis wrote:
If you are implementing your algorithm from scratch, then it would be nice if it could be done in Smalltalk, because that means we can all read it, play with it in the image, and write unit tests to validate and document its behavior. We can figure out the translation to C afterwords, once you have a good implementation in Smalltalk (and yes I will help with that).
I have a smalltalk implmentation that came from Google Java code and is fully implemented in squeak/pharo. I am intermittently fixing byte mutations in the payload, so it is working to a degree. Check out the classes GenericGF, GenericGFPoly and GaloisField to see the insides. FECEncoder and FECDecoder are calling the galoisField which builds a ReedSolomon Decoder. That last is where the error detection occurs.
Thank you all very much for looking out!
I guess my suggestion would be that once you have a fairly good working implementation that you are comfortable with from a functional point of view, then let's look at which methods in that implementation would benefit from being translated to primitives in C. It would be especially good if you can run a profiler on your Smalltalk implementation and see where the time is being spent. Those would be the things we would want to turn into primitives. But first things first, let's get it working 100% (not "to a degree") and then we can do the translation to primitives.
Dave
Here is a profile. The broken part uses GenericGF and GenericGFPoly, so those can be optimized without the broken bits affecting it.
Note that 70% of the time is inside of GenericGFPoly>>#divide:, but addOrSubtract: is really the one.
robert
**Leaves** 21.4% {1084ms} OrderedCollection>>at: 11.7% {592ms} GenericGFPoly>>addOrSubtract: 11.7% {590ms} ProcessorScheduler class>>idleProcess 10.8% {548ms} GenericGF class>>addOrSubtract:coefficient: 5.7% {287ms} Array(SequenceableCollection)>>from:to:put: 5.5% {277ms} OrderedCollection>>at:put: 3.8% {194ms} GenericGF>>logarithm: 3.3% {167ms} [] DelayWaitTimeout>>wait 2.8% {144ms} GenericGF>>multiply:scalar: 2.6% {129ms} GenericGF>>exponent: 2.1% {107ms} Array(SequenceableCollection)>>atAllPut: 2.1% {104ms} SmallInteger(Integer)>><< 1.6% {81ms} GenericGF>>checkInit 1.5% {78ms} LargePositiveInteger>>bitAt: 1.2% {60ms} OrderedCollection>>size 1.1% {57ms} SmallInteger(Magnitude)>>min:
- 5056 tallies, 5056 msec.
**Tree** -------------------------------- Process: other processes -------------------------------- 11.7% {590ms} [] ProcessorScheduler class>>startUp |11.7% {590ms} ProcessorScheduler class>>idleProcess 6.1% {310ms} [] TransportEndpoint(ProtocolEndpoint)>>run |6.1% {310ms} TransportEndpoint(ProtocolEndpoint)>>serverLoop | 6.1% {310ms} BlockClosure>>ensure: | 6.1% {310ms} [] TransportEndpoint(ProtocolEndpoint)>>serverLoop | 6.1% {310ms} BlockClosure>>on:do: | 6.1% {310ms} [[]] TransportEndpoint(ProtocolEndpoint)>>serverLoop | 6.1% {310ms} BlockClosure>>on:do: | 6.1% {308ms} [[[]]] TransportEndpoint(ProtocolEndpoint)>>serverLoop | 5.9% {296ms} TransportEndpoint>>upcallFrame: | 5.9% {296ms} TransportEndpoint(ProtocolLayer)>>upcallFrame: | 5.9% {296ms} BinaryChunkProtocol>>upcallFrame: | 5.9% {296ms} BinaryChunkProtocol>>processChunk | 5.9% {296ms} OperationProtocol>>upcallFrame: | 5.9% {296ms} OperationProtocol(StatefulProtocol)>>transitionEvent:with: | 5.9% {296ms} ProtocolStateTransition>>transitionFor:with: | 5.9% {296ms} OperationProtocol>>processStartupMsg: | 5.9% {296ms} StartupProtocol(StatefulProtocol)>>transitionEvent:with: | 5.9% {296ms} ProtocolStateTransition>>transitionFor:with: | 2.3% {116ms} StartupProtocol>>processGo: | 2.3% {115ms} StartupProtocol>>processReplyInfo: | |2.2% {113ms} StartupProtocol>>dhParm | | 2.2% {113ms} EncryptionSecrets>>dhParm | | 2.2% {113ms} DiffieHellman>>sendMessage | | 1.3% {66ms} SecureRandom class(RandomGenerator class)>>picker | | 1.3% {66ms} SecureRandom class(RandomGenerator class)>>withGeneratedKey | | 1.3% {66ms} SecureRandom class(RandomGenerator class)>>generateKey | | 1.3% {64ms} ByteArray class(SequenceableCollection class)>>streamContents: | | 1.3% {64ms} ByteArray class(SequenceableCollection class)>>new:streamContents: | | 1.3% {64ms} [] SecureRandom class(RandomGenerator class)>>generateKey | | 1.3% {64ms} SecureRandom class(RandomGenerator class)>>unpredictableStringsDo: | | 1.3% {64ms} Time class>>millisecondsToRun: | | 1.3% {64ms} [] SecureRandom class(RandomGenerator class)>>unpredictableStringsDo: | 1.3% {65ms} StartupProtocol>>processGoToo: 3.3% {165ms} [] UnixOSProcessAccessor>>grimReaperProcess 3.1% {155ms} Semaphore>>waitTimeoutMSecs: 3.1% {155ms} DelayWaitTimeout>>wait 3.1% {155ms} BlockClosure>>ensure: 3.1% {155ms} [] DelayWaitTimeout>>wait -------------------------------- Process: (40s) 50028: nil -------------------------------- 78.5% {3967ms} MessageTally class>>spyOn:reportOtherProcesses: 78.5% {3967ms} MessageTally>>spyEvery:on: 78.5% {3967ms} BlockClosure>>ensure: 78.5% {3967ms} [] TestRunner>>runProfiled 78.5% {3967ms} TestRunner>>runAll 78.5% {3967ms} TestRunner>>runSuite: 78.5% {3967ms} TestRunner>>basicRunSuite:do: 78.5% {3967ms} BlockClosure>>ensure: 78.5% {3967ms} [] TestRunner>>basicRunSuite:do: 78.5% {3967ms} OrderedCollection(Collection)>>do:displayingProgress:every: 78.5% {3967ms} ByteString(String)>>displayProgressFrom:to:during: 78.5% {3967ms} ByteString(String)>>displayProgressAt:from:to:during: 78.5% {3967ms} ProgressInitiationException class>>display:at:from:to:during: 78.5% {3967ms} ProgressInitiationException>>display:at:from:to:during: 78.5% {3967ms} ProgressInitiationException(Exception)>>signal 78.5% {3967ms} MethodContext(ContextPart)>>handleSignal: 78.5% {3967ms} UndefinedObject>>handleSignal: 78.5% {3967ms} ProgressInitiationException>>defaultAction 78.5% {3967ms} ProgressInitiationException(Exception)>>resume 78.5% {3967ms} ProgressInitiationException>>defaultResumeValue 78.5% {3967ms} MorphicUIManager>>displayProgress:at:from:to:during: 78.4% {3965ms} BlockClosure>>ensure: 78.4% {3965ms} [] MorphicUIManager>>displayProgress:at:from:to:during: 78.4% {3965ms} BlockClosure>>on:do: 78.4% {3965ms} [[]] MorphicUIManager>>displayProgress:at:from:to:during: 78.4% {3965ms} [] OrderedCollection(Collection)>>do:displayingProgress:every: 78.4% {3965ms} OrderedCollection>>do: 78.4% {3965ms} [[]] OrderedCollection(Collection)>>do:displayingProgress:every: 78.1% {3947ms} [] TestRunner>>runSuite: 78.1% {3947ms} TestRunner>>runTest: 78.0% {3945ms} SecureSessionPerformanceTestCase(TestCase)>>run: 78.0% {3945ms} TestResult>>runCase: 78.0% {3945ms} BlockClosure>>on:do: 78.0% {3945ms} [] TestResult>>runCase: 78.0% {3945ms} BlockClosure>>on:do: 78.0% {3945ms} [[]] TestResult>>runCase: 78.0% {3945ms} SecureSessionPerformanceTestCase(TestCase)>>runCase 78.0% {3945ms} BlockClosure>>ensure: 77.4% {3912ms} [] SecureSessionPerformanceTestCase(TestCase)>>runCase 77.4% {3912ms} SecureSessionPerformanceTestCase(TestCase)>>timeout:after: 77.4% {3912ms} BlockClosure>>ensure: 77.4% {3912ms} [] SecureSessionPerformanceTestCase(TestCase)>>timeout:after: 77.4% {3912ms} BlockClosure>>on:do: 77.4% {3912ms} [[]] SecureSessionPerformanceTestCase(TestCase)>>runCase 77.4% {3912ms} SecureSessionPerformanceTestCase(TestCase)>>performTest 77.4% {3912ms} SecureSessionPerformanceTestCase>>testPerformance1 77.2% {3902ms} SmallInteger(Integer)>>timesRepeat: 77.2% {3902ms} [] SecureSessionPerformanceTestCase>>testPerformance1 77.2% {3902ms} SecureSessionTerminal>>sendBytes: 77.2% {3902ms} SecureSessionTerminal>>downcallFrame: 77.2% {3902ms} SecureSessionTerminal(ProtocolLayer)>>downcallFrame: 77.2% {3902ms} FullDuplexProxy>>downcallFrame: 77.1% {3900ms} MACComputeAndLengthWriter>>writeFrame:handler: 77.1% {3898ms} [] FullDuplexProxy>>downcallFrame: 77.1% {3898ms} FullDuplexProxy(ProtocolLayer)>>downcallFrame: 77.1% {3898ms} FullDuplexProxy>>downcallFrame: 77.1% {3898ms} Encryptor>>writeFrame:handler: 76.4% {3864ms} [] FullDuplexProxy>>downcallFrame: 76.4% {3864ms} FullDuplexProxy(ProtocolLayer)>>downcallFrame: 76.4% {3864ms} FullDuplexProxy>>downcallFrame: 76.4% {3864ms} MACMessageWriter>>writeFrame:handler: 76.4% {3864ms} [] FullDuplexProxy>>downcallFrame: 76.4% {3864ms} FullDuplexProxy(ProtocolLayer)>>downcallFrame: 76.4% {3864ms} FullDuplexProxy>>downcallFrame: 76.4% {3864ms} FECEncoder>>writeFrame:handler: 76.4% {3862ms} FECEncoder>>encodeFrame: 76.4% {3862ms} GaloisField>>encode: 76.3% {3860ms} GaloisField>>encodeBlock: 76.2% {3851ms} ReedSolomonEncoder>>encode: 72.9% {3685ms} GenericGFPoly>>divide: |52.0% {2628ms} GenericGFPoly>>addOrSubtract: | |17.4% {881ms} OrderedCollection>>at: | |11.7% {592ms} primitives | |10.7% {542ms} GenericGF class>>addOrSubtract:coefficient: | |4.5% {225ms} OrderedCollection class>>new:withAll: | | |4.4% {223ms} Array class(ArrayedCollection class)>>new:withAll: | | | 4.3% {219ms} Array(SequenceableCollection)>>atAllPut: | | | 3.3% {166ms} Array(SequenceableCollection)>>from:to:put: | | | |2.6% {133ms} primitives | | | 1.0% {53ms} primitives | |4.3% {216ms} OrderedCollection>>at:put: | |2.1% {105ms} GenericGFPoly class>>newOnGF:coefficients: | | 1.7% {86ms} GenericGFPoly>>initializeOnGF:coefficients: | | 1.3% {64ms} OrderedCollection>>copyFrom:to: | | 1.2% {60ms} OrderedCollection>>postCopyFrom:to: |16.9% {852ms} GenericGFPoly>>multiply:monomial: | |11.6% {589ms} GenericGF>>multiply:scalar: | | |7.0% {352ms} GenericGF>>logarithm: | | | |3.6% {180ms} primitives | | | |2.5% {127ms} OrderedCollection>>at: | | |2.5% {127ms} GenericGF>>exponent: | | | |1.8% {93ms} primitives | | |1.9% {98ms} primitives | |2.8% {142ms} OrderedCollection class>>new:withAll: | | |2.7% {136ms} Array class(ArrayedCollection class)>>new:withAll: | | | 2.7% {134ms} Array(SequenceableCollection)>>atAllPut: | | | 2.0% {102ms} Array(SequenceableCollection)>>from:to:put: | | | 1.8% {92ms} primitives | |1.1% {55ms} OrderedCollection>>at:put: |2.1% {108ms} GenericGF>>buildMonomial:coefficient: | 1.9% {96ms} Array class(ArrayedCollection class)>>new:withAll: | 1.9% {96ms} Array(SequenceableCollection)>>atAllPut: | 1.5% {76ms} Array(SequenceableCollection)>>from:to:put: | 1.2% {62ms} primitives 2.8% {142ms} ReedSolomonEncoder>>buildGenerator: 2.8% {140ms} GenericGFPoly>>multiply: 2.2% {110ms} GenericGF>>multiply:scalar: **Leaves** 21.4% {1084ms} OrderedCollection>>at: 11.7% {592ms} GenericGFPoly>>addOrSubtract: 11.7% {590ms} ProcessorScheduler class>>idleProcess 10.8% {548ms} GenericGF class>>addOrSubtract:coefficient: 5.7% {287ms} Array(SequenceableCollection)>>from:to:put: 5.5% {277ms} OrderedCollection>>at:put: 3.8% {194ms} GenericGF>>logarithm: 3.3% {167ms} [] DelayWaitTimeout>>wait 2.8% {144ms} GenericGF>>multiply:scalar: 2.6% {129ms} GenericGF>>exponent: 2.1% {107ms} Array(SequenceableCollection)>>atAllPut: 2.1% {104ms} SmallInteger(Integer)>><< 1.6% {81ms} GenericGF>>checkInit 1.5% {78ms} LargePositiveInteger>>bitAt: 1.2% {60ms} OrderedCollection>>size 1.1% {57ms} SmallInteger(Magnitude)>>min:
**Memory** old +0 bytes young +1,005,072 bytes used +1,005,072 bytes free -1,005,072 bytes
**GCs** full 1 totalling 146 ms (2.89% uptime), avg 146 ms incr 259 totalling 41 ms (0.8% uptime), avg 0.2 ms tenures 3,681 (avg 0 GCs/tenure) root table 0 overflows
On 12/18/2015 08:59 PM, David T. Lewis wrote:
On Fri, Dec 18, 2015 at 08:13:00PM -0500, Robert Withers wrote:
On 12/18/2015 08:07 PM, David T. Lewis wrote:
If you are implementing your algorithm from scratch, then it would be nice if it could be done in Smalltalk, because that means we can all read it, play with it in the image, and write unit tests to validate and document its behavior. We can figure out the translation to C afterwords, once you have a good implementation in Smalltalk (and yes I will help with that).
I have a smalltalk implmentation that came from Google Java code and is fully implemented in squeak/pharo. I am intermittently fixing byte mutations in the payload, so it is working to a degree. Check out the classes GenericGF, GenericGFPoly and GaloisField to see the insides. FECEncoder and FECDecoder are calling the galoisField which builds a ReedSolomon Decoder. That last is where the error detection occurs.
Thank you all very much for looking out!
I guess my suggestion would be that once you have a fairly good working implementation that you are comfortable with from a functional point of view, then let's look at which methods in that implementation would benefit from being translated to primitives in C. It would be especially good if you can run a profiler on your Smalltalk implementation and see where the time is being spent. Those would be the things we would want to turn into primitives. But first things first, let's get it working 100% (not "to a degree") and then we can do the translation to primitives.
Dave
A little further thinking and looking. The ReedSolomon Decoder is absolutely using the euclidean algorithm to rebuild the data from the code. It is partially working, as sometimes it repairs errors and othertimes not. Keep mashing the run selected tests button to see this. I have seen all 3 GFs rebuild mutations in both the inner tests: the RS tests and the outer tests: FEC.
On 12/18/2015 09:20 PM, Robert Withers wrote:
Here is a profile. The broken part uses GenericGF and GenericGFPoly, so those can be optimized without the broken bits affecting it.
Note that 70% of the time is inside of GenericGFPoly>>#divide:, but addOrSubtract: is really the one.
robert
**Leaves** 21.4% {1084ms} OrderedCollection>>at: 11.7% {592ms} GenericGFPoly>>addOrSubtract: 11.7% {590ms} ProcessorScheduler class>>idleProcess 10.8% {548ms} GenericGF class>>addOrSubtract:coefficient: 5.7% {287ms} Array(SequenceableCollection)>>from:to:put: 5.5% {277ms} OrderedCollection>>at:put: 3.8% {194ms} GenericGF>>logarithm: 3.3% {167ms} [] DelayWaitTimeout>>wait 2.8% {144ms} GenericGF>>multiply:scalar: 2.6% {129ms} GenericGF>>exponent: 2.1% {107ms} Array(SequenceableCollection)>>atAllPut: 2.1% {104ms} SmallInteger(Integer)>><< 1.6% {81ms} GenericGF>>checkInit 1.5% {78ms} LargePositiveInteger>>bitAt: 1.2% {60ms} OrderedCollection>>size 1.1% {57ms} SmallInteger(Magnitude)>>min:
- 5056 tallies, 5056 msec.
**Tree**
Process: other processes
11.7% {590ms} [] ProcessorScheduler class>>startUp |11.7% {590ms} ProcessorScheduler class>>idleProcess 6.1% {310ms} [] TransportEndpoint(ProtocolEndpoint)>>run |6.1% {310ms} TransportEndpoint(ProtocolEndpoint)>>serverLoop | 6.1% {310ms} BlockClosure>>ensure: | 6.1% {310ms} [] TransportEndpoint(ProtocolEndpoint)>>serverLoop | 6.1% {310ms} BlockClosure>>on:do: | 6.1% {310ms} [[]] TransportEndpoint(ProtocolEndpoint)>>serverLoop | 6.1% {310ms} BlockClosure>>on:do: | 6.1% {308ms} [[[]]] TransportEndpoint(ProtocolEndpoint)>>serverLoop | 5.9% {296ms} TransportEndpoint>>upcallFrame: | 5.9% {296ms} TransportEndpoint(ProtocolLayer)>>upcallFrame: | 5.9% {296ms} BinaryChunkProtocol>>upcallFrame: | 5.9% {296ms} BinaryChunkProtocol>>processChunk | 5.9% {296ms} OperationProtocol>>upcallFrame: | 5.9% {296ms} OperationProtocol(StatefulProtocol)>>transitionEvent:with: | 5.9% {296ms} ProtocolStateTransition>>transitionFor:with: | 5.9% {296ms} OperationProtocol>>processStartupMsg: | 5.9% {296ms} StartupProtocol(StatefulProtocol)>>transitionEvent:with: | 5.9% {296ms} ProtocolStateTransition>>transitionFor:with: | 2.3% {116ms} StartupProtocol>>processGo: | 2.3% {115ms} StartupProtocol>>processReplyInfo: | |2.2% {113ms} StartupProtocol>>dhParm | | 2.2% {113ms} EncryptionSecrets>>dhParm | | 2.2% {113ms} DiffieHellman>>sendMessage | | 1.3% {66ms} SecureRandom class(RandomGenerator class)>>picker | | 1.3% {66ms} SecureRandom class(RandomGenerator class)>>withGeneratedKey | | 1.3% {66ms} SecureRandom class(RandomGenerator class)>>generateKey | | 1.3% {64ms} ByteArray class(SequenceableCollection class)>>streamContents: | | 1.3% {64ms} ByteArray class(SequenceableCollection class)>>new:streamContents: | | 1.3% {64ms} [] SecureRandom class(RandomGenerator class)>>generateKey | | 1.3% {64ms} SecureRandom class(RandomGenerator class)>>unpredictableStringsDo: | | 1.3% {64ms} Time class>>millisecondsToRun: | | 1.3% {64ms} [] SecureRandom class(RandomGenerator class)>>unpredictableStringsDo: | 1.3% {65ms} StartupProtocol>>processGoToo: 3.3% {165ms} [] UnixOSProcessAccessor>>grimReaperProcess 3.1% {155ms} Semaphore>>waitTimeoutMSecs: 3.1% {155ms} DelayWaitTimeout>>wait 3.1% {155ms} BlockClosure>>ensure: 3.1% {155ms} [] DelayWaitTimeout>>wait
Process: (40s) 50028: nil
78.5% {3967ms} MessageTally class>>spyOn:reportOtherProcesses: 78.5% {3967ms} MessageTally>>spyEvery:on: 78.5% {3967ms} BlockClosure>>ensure: 78.5% {3967ms} [] TestRunner>>runProfiled 78.5% {3967ms} TestRunner>>runAll 78.5% {3967ms} TestRunner>>runSuite: 78.5% {3967ms} TestRunner>>basicRunSuite:do: 78.5% {3967ms} BlockClosure>>ensure: 78.5% {3967ms} [] TestRunner>>basicRunSuite:do: 78.5% {3967ms} OrderedCollection(Collection)>>do:displayingProgress:every: 78.5% {3967ms} ByteString(String)>>displayProgressFrom:to:during: 78.5% {3967ms} ByteString(String)>>displayProgressAt:from:to:during: 78.5% {3967ms} ProgressInitiationException class>>display:at:from:to:during: 78.5% {3967ms} ProgressInitiationException>>display:at:from:to:during: 78.5% {3967ms} ProgressInitiationException(Exception)>>signal 78.5% {3967ms} MethodContext(ContextPart)>>handleSignal: 78.5% {3967ms} UndefinedObject>>handleSignal: 78.5% {3967ms} ProgressInitiationException>>defaultAction 78.5% {3967ms} ProgressInitiationException(Exception)>>resume 78.5% {3967ms} ProgressInitiationException>>defaultResumeValue 78.5% {3967ms} MorphicUIManager>>displayProgress:at:from:to:during: 78.4% {3965ms} BlockClosure>>ensure: 78.4% {3965ms} [] MorphicUIManager>>displayProgress:at:from:to:during: 78.4% {3965ms} BlockClosure>>on:do: 78.4% {3965ms} [[]] MorphicUIManager>>displayProgress:at:from:to:during: 78.4% {3965ms} [] OrderedCollection(Collection)>>do:displayingProgress:every: 78.4% {3965ms} OrderedCollection>>do: 78.4% {3965ms} [[]] OrderedCollection(Collection)>>do:displayingProgress:every: 78.1% {3947ms} [] TestRunner>>runSuite: 78.1% {3947ms} TestRunner>>runTest: 78.0% {3945ms} SecureSessionPerformanceTestCase(TestCase)>>run: 78.0% {3945ms} TestResult>>runCase: 78.0% {3945ms} BlockClosure>>on:do: 78.0% {3945ms} [] TestResult>>runCase: 78.0% {3945ms} BlockClosure>>on:do: 78.0% {3945ms} [[]] TestResult>>runCase: 78.0% {3945ms} SecureSessionPerformanceTestCase(TestCase)>>runCase 78.0% {3945ms} BlockClosure>>ensure: 77.4% {3912ms} [] SecureSessionPerformanceTestCase(TestCase)>>runCase 77.4% {3912ms} SecureSessionPerformanceTestCase(TestCase)>>timeout:after: 77.4% {3912ms} BlockClosure>>ensure: 77.4% {3912ms} [] SecureSessionPerformanceTestCase(TestCase)>>timeout:after: 77.4% {3912ms} BlockClosure>>on:do: 77.4% {3912ms} [[]] SecureSessionPerformanceTestCase(TestCase)>>runCase 77.4% {3912ms} SecureSessionPerformanceTestCase(TestCase)>>performTest 77.4% {3912ms} SecureSessionPerformanceTestCase>>testPerformance1 77.2% {3902ms} SmallInteger(Integer)>>timesRepeat: 77.2% {3902ms} [] SecureSessionPerformanceTestCase>>testPerformance1 77.2% {3902ms} SecureSessionTerminal>>sendBytes: 77.2% {3902ms} SecureSessionTerminal>>downcallFrame: 77.2% {3902ms} SecureSessionTerminal(ProtocolLayer)>>downcallFrame: 77.2% {3902ms} FullDuplexProxy>>downcallFrame: 77.1% {3900ms} MACComputeAndLengthWriter>>writeFrame:handler: 77.1% {3898ms} [] FullDuplexProxy>>downcallFrame: 77.1% {3898ms} FullDuplexProxy(ProtocolLayer)>>downcallFrame: 77.1% {3898ms} FullDuplexProxy>>downcallFrame: 77.1% {3898ms} Encryptor>>writeFrame:handler: 76.4% {3864ms} [] FullDuplexProxy>>downcallFrame: 76.4% {3864ms} FullDuplexProxy(ProtocolLayer)>>downcallFrame: 76.4% {3864ms} FullDuplexProxy>>downcallFrame: 76.4% {3864ms} MACMessageWriter>>writeFrame:handler: 76.4% {3864ms} [] FullDuplexProxy>>downcallFrame: 76.4% {3864ms} FullDuplexProxy(ProtocolLayer)>>downcallFrame: 76.4% {3864ms} FullDuplexProxy>>downcallFrame: 76.4% {3864ms} FECEncoder>>writeFrame:handler: 76.4% {3862ms} FECEncoder>>encodeFrame: 76.4% {3862ms} GaloisField>>encode: 76.3% {3860ms} GaloisField>>encodeBlock: 76.2% {3851ms} ReedSolomonEncoder>>encode: 72.9% {3685ms} GenericGFPoly>>divide: |52.0% {2628ms} GenericGFPoly>>addOrSubtract: | |17.4% {881ms} OrderedCollection>>at: | |11.7% {592ms} primitives | |10.7% {542ms} GenericGF class>>addOrSubtract:coefficient: | |4.5% {225ms} OrderedCollection class>>new:withAll: | | |4.4% {223ms} Array class(ArrayedCollection class)>>new:withAll: | | | 4.3% {219ms} Array(SequenceableCollection)>>atAllPut: | | | 3.3% {166ms} Array(SequenceableCollection)>>from:to:put: | | | |2.6% {133ms} primitives | | | 1.0% {53ms} primitives | |4.3% {216ms} OrderedCollection>>at:put: | |2.1% {105ms} GenericGFPoly class>>newOnGF:coefficients: | | 1.7% {86ms} GenericGFPoly>>initializeOnGF:coefficients: | | 1.3% {64ms} OrderedCollection>>copyFrom:to: | | 1.2% {60ms} OrderedCollection>>postCopyFrom:to: |16.9% {852ms} GenericGFPoly>>multiply:monomial: | |11.6% {589ms} GenericGF>>multiply:scalar: | | |7.0% {352ms} GenericGF>>logarithm: | | | |3.6% {180ms} primitives | | | |2.5% {127ms} OrderedCollection>>at: | | |2.5% {127ms} GenericGF>>exponent: | | | |1.8% {93ms} primitives | | |1.9% {98ms} primitives | |2.8% {142ms} OrderedCollection class>>new:withAll: | | |2.7% {136ms} Array class(ArrayedCollection class)>>new:withAll: | | | 2.7% {134ms} Array(SequenceableCollection)>>atAllPut: | | | 2.0% {102ms} Array(SequenceableCollection)>>from:to:put: | | | 1.8% {92ms} primitives | |1.1% {55ms} OrderedCollection>>at:put: |2.1% {108ms} GenericGF>>buildMonomial:coefficient: | 1.9% {96ms} Array class(ArrayedCollection class)>>new:withAll: | 1.9% {96ms} Array(SequenceableCollection)>>atAllPut: | 1.5% {76ms} Array(SequenceableCollection)>>from:to:put: | 1.2% {62ms} primitives 2.8% {142ms} ReedSolomonEncoder>>buildGenerator: 2.8% {140ms} GenericGFPoly>>multiply: 2.2% {110ms} GenericGF>>multiply:scalar: **Leaves** 21.4% {1084ms} OrderedCollection>>at: 11.7% {592ms} GenericGFPoly>>addOrSubtract: 11.7% {590ms} ProcessorScheduler class>>idleProcess 10.8% {548ms} GenericGF class>>addOrSubtract:coefficient: 5.7% {287ms} Array(SequenceableCollection)>>from:to:put: 5.5% {277ms} OrderedCollection>>at:put: 3.8% {194ms} GenericGF>>logarithm: 3.3% {167ms} [] DelayWaitTimeout>>wait 2.8% {144ms} GenericGF>>multiply:scalar: 2.6% {129ms} GenericGF>>exponent: 2.1% {107ms} Array(SequenceableCollection)>>atAllPut: 2.1% {104ms} SmallInteger(Integer)>><< 1.6% {81ms} GenericGF>>checkInit 1.5% {78ms} LargePositiveInteger>>bitAt: 1.2% {60ms} OrderedCollection>>size 1.1% {57ms} SmallInteger(Magnitude)>>min:
**Memory** old +0 bytes young +1,005,072 bytes used +1,005,072 bytes free -1,005,072 bytes
**GCs** full 1 totalling 146 ms (2.89% uptime), avg 146 ms incr 259 totalling 41 ms (0.8% uptime), avg 0.2 ms tenures 3,681 (avg 0 GCs/tenure) root table 0 overflows
On 12/18/2015 08:59 PM, David T. Lewis wrote:
On Fri, Dec 18, 2015 at 08:13:00PM -0500, Robert Withers wrote:
On 12/18/2015 08:07 PM, David T. Lewis wrote:
If you are implementing your algorithm from scratch, then it would be nice if it could be done in Smalltalk, because that means we can all read it, play with it in the image, and write unit tests to validate and document its behavior. We can figure out the translation to C afterwords, once you have a good implementation in Smalltalk (and yes I will help with that).
I have a smalltalk implmentation that came from Google Java code and is fully implemented in squeak/pharo. I am intermittently fixing byte mutations in the payload, so it is working to a degree. Check out the classes GenericGF, GenericGFPoly and GaloisField to see the insides. FECEncoder and FECDecoder are calling the galoisField which builds a ReedSolomon Decoder. That last is where the error detection occurs.
Thank you all very much for looking out!
I guess my suggestion would be that once you have a fairly good working implementation that you are comfortable with from a functional point of view, then let's look at which methods in that implementation would benefit from being translated to primitives in C. It would be especially good if you can run a profiler on your Smalltalk implementation and see where the time is being spent. Those would be the things we would want to turn into primitives. But first things first, let's get it working 100% (not "to a degree") and then we can do the translation to primitives.
Dave
Further, please keep the original Smalltalk non-primitive implementation around do that when we deliver Sista we can see how well we stack up against C.
_,,,^..^,,,_ (phone)
On Dec 18, 2015, at 5:59 PM, David T. Lewis lewis@mail.msen.com wrote:
On Fri, Dec 18, 2015 at 08:13:00PM -0500, Robert Withers wrote:
On 12/18/2015 08:07 PM, David T. Lewis wrote:
If you are implementing your algorithm from scratch, then it would be nice if it could be done in Smalltalk, because that means we can all read it, play with it in the image, and write unit tests to validate and document its behavior. We can figure out the translation to C afterwords, once you have a good implementation in Smalltalk (and yes I will help with that).
I have a smalltalk implmentation that came from Google Java code and is fully implemented in squeak/pharo. I am intermittently fixing byte mutations in the payload, so it is working to a degree. Check out the classes GenericGF, GenericGFPoly and GaloisField to see the insides. FECEncoder and FECDecoder are calling the galoisField which builds a ReedSolomon Decoder. That last is where the error detection occurs.
Thank you all very much for looking out!
I guess my suggestion would be that once you have a fairly good working implementation that you are comfortable with from a functional point of view, then let's look at which methods in that implementation would benefit from being translated to primitives in C. It would be especially good if you can run a profiler on your Smalltalk implementation and see where the time is being spent. Those would be the things we would want to turn into primitives. But first things first, let's get it working 100% (not "to a degree") and then we can do the translation to primitives.
Dave
Eliot, what does Sista do and how is that accomplished? It sounds like a next level kind of system. Does it compare to what other JITs are doing in various VMs?
thank you, robert
. .. ... ^,^
On 12/18/2015 11:11 PM, Eliot Miranda wrote:
Further, please keep the original Smalltalk non-primitive implementation around do that when we deliver Sista we can see how well we stack up against C.
_,,,^..^,,,_ (phone)
On Dec 18, 2015, at 5:59 PM, David T. Lewis lewis@mail.msen.com wrote:
On Fri, Dec 18, 2015 at 08:13:00PM -0500, Robert Withers wrote:
On 12/18/2015 08:07 PM, David T. Lewis wrote:
If you are implementing your algorithm from scratch, then it would be nice if it could be done in Smalltalk, because that means we can all read it, play with it in the image, and write unit tests to validate and document its behavior. We can figure out the translation to C afterwords, once you have a good implementation in Smalltalk (and yes I will help with that).
I have a smalltalk implmentation that came from Google Java code and is fully implemented in squeak/pharo. I am intermittently fixing byte mutations in the payload, so it is working to a degree. Check out the classes GenericGF, GenericGFPoly and GaloisField to see the insides. FECEncoder and FECDecoder are calling the galoisField which builds a ReedSolomon Decoder. That last is where the error detection occurs.
Thank you all very much for looking out!
I guess my suggestion would be that once you have a fairly good working implementation that you are comfortable with from a functional point of view, then let's look at which methods in that implementation would benefit from being translated to primitives in C. It would be especially good if you can run a profiler on your Smalltalk implementation and see where the time is being spent. Those would be the things we would want to turn into primitives. But first things first, let's get it working 100% (not "to a degree") and then we can do the translation to primitives.
Dave
Hi Robert,
On Dec 18, 2015, at 9:51 PM, Robert Withers robert.w.withers@gmail.com wrote:
Eliot, what does Sista do and how is that accomplished?
Run-time adaptive optimisation via an SSA-based image-level bytecode-to-bytecode inlining compiler. See
http://www.mirandabanda.org/cogblog/on-line-papers-and-presentations/
It sounds like a next level kind of system. Does it compare to what other JITs are doing in various VMs?
Yes it does. Clément recently attended a workshop at Google and found that Sista is very similar in the kinds of optimisations it performs to V8 and Dart. The key research funding will be how close the performance of targeting stack-oriented bytecode that the Cog JIT converts to register-based code can be made to JITs that directly target register-based machine code.
_,,,^..^,,,_ (phone)
thank you, robert
. .. ... ^,^
On 12/18/2015 11:11 PM, Eliot Miranda wrote: Further, please keep the original Smalltalk non-primitive implementation around do that when we deliver Sista we can see how well we stack up against C.
_,,,^..^,,,_ (phone)
On Dec 18, 2015, at 5:59 PM, David T. Lewis lewis@mail.msen.com wrote:
On Fri, Dec 18, 2015 at 08:13:00PM -0500, Robert Withers wrote:
On 12/18/2015 08:07 PM, David T. Lewis wrote:
If you are implementing your algorithm from scratch, then it would be nice if it could be done in Smalltalk, because that means we can all read it, play with it in the image, and write unit tests to validate and document its behavior. We can figure out the translation to C afterwords, once you have a good implementation in Smalltalk (and yes I will help with that).
I have a smalltalk implmentation that came from Google Java code and is fully implemented in squeak/pharo. I am intermittently fixing byte mutations in the payload, so it is working to a degree. Check out the classes GenericGF, GenericGFPoly and GaloisField to see the insides. FECEncoder and FECDecoder are calling the galoisField which builds a ReedSolomon Decoder. That last is where the error detection occurs.
Thank you all very much for looking out!
I guess my suggestion would be that once you have a fairly good working implementation that you are comfortable with from a functional point of view, then let's look at which methods in that implementation would benefit from being translated to primitives in C. It would be especially good if you can run a profiler on your Smalltalk implementation and see where the time is being spent. Those would be the things we would want to turn into primitives. But first things first, let's get it working 100% (not "to a degree") and then we can do the translation to primitives.
Dave
-- . .. .. ^,^ robert
On 12/19/2015 02:09 AM, Eliot Miranda wrote:
Hi Robert,
On Dec 18, 2015, at 9:51 PM, Robert Withers <robert.w.withers@gmail.com mailto:robert.w.withers@gmail.com> wrote:
Eliot, what does Sista do and how is that accomplished?
Run-time adaptive optimisation via an SSA-based image-level bytecode-to-bytecode inlining compiler. See
http://www.mirandabanda.org/cogblog/on-line-papers-and-presentations/
It sounds like a next level kind of system. Does it compare to what other JITs are doing in various VMs?
Yes it does. Clément recently attended a workshop at Google and found that Sista is very similar in the kinds of optimisations it performs to V8 and Dart. The key research funding will be how close the performance of targeting stack-oriented bytecode that the Cog JIT converts to register-based code can be made to JITs that directly target register-based machine code.
This sounds like you are saying that Sista performs better or more naturally with stack-oriented bytecode. If you wouldn't mind taking the time to explore it with us, what's the difference between stack-oriented and register-based code, such that Sista would be different between them? I'm off to read some.
_,,,^..^,,,_ (phone)
thank you, robert
. .. ... ^,^
On 12/18/2015 11:11 PM, Eliot Miranda wrote:
Further, please keep the original Smalltalk non-primitive implementation around do that when we deliver Sista we can see how well we stack up against C.
_,,,^..^,,,_ (phone)
On Dec 18, 2015, at 5:59 PM, David T. Lewis <lewis@mail.msen.com mailto:lewis@mail.msen.com> wrote:
On Fri, Dec 18, 2015 at 08:13:00PM -0500, Robert Withers wrote:
On 12/18/2015 08:07 PM, David T. Lewis wrote:
If you are implementing your algorithm from scratch, then it would be nice if it could be done in Smalltalk, because that means we can all read it, play with it in the image, and write unit tests to validate and document its behavior. We can figure out the translation to C afterwords, once you have a good implementation in Smalltalk (and yes I will help with that).
I have a smalltalk implmentation that came from Google Java code and is fully implemented in squeak/pharo. I am intermittently fixing byte mutations in the payload, so it is working to a degree. Check out the classes GenericGF, GenericGFPoly and GaloisField to see the insides. FECEncoder and FECDecoder are calling the galoisField which builds a ReedSolomon Decoder. That last is where the error detection occurs.
Thank you all very much for looking out!
I guess my suggestion would be that once you have a fairly good working implementation that you are comfortable with from a functional point of view, then let's look at which methods in that implementation would benefit from being translated to primitives in C. It would be especially good if you can run a profiler on your Smalltalk implementation and see where the time is being spent. Those would be the things we would want to turn into primitives. But first things first, let's get it working 100% (not "to a degree") and then we can do the translation to primitives.
Dave
-- . .. .. ^,^ robert
On Fri, Dec 18, 2015 at 11:25 PM, Robert Withers <robert.w.withers@gmail.com
wrote:
On 12/19/2015 02:09 AM, Eliot Miranda wrote:
Hi Robert,
On Dec 18, 2015, at 9:51 PM, Robert Withers < robert.w.withers@gmail.com robert.w.withers@gmail.com> wrote:
Eliot, what does Sista do and how is that accomplished?
Run-time adaptive optimisation via an SSA-based image-level bytecode-to-bytecode inlining compiler. See
http://www.mirandabanda.org/cogblog/on-line-papers-and-presentations/
It sounds like a next level kind of system. Does it compare to what other JITs are doing in various VMs?
Yes it does. Clément recently attended a workshop at Google and found that Sista is very similar in the kinds of optimisations it performs to V8 and Dart. The key research funding will be how close the performance of targeting stack-oriented bytecode that the Cog JIT converts to register-based code can be made to JITs that directly target register-based machine code.
This sounds like you are saying that Sista performs better or more naturally with stack-oriented bytecode. If you wouldn't mind taking the time to explore it with us, what's the difference between stack-oriented and register-based code, such that Sista would be different between them? I'm off to read some.
I'd rather not. That's something there's lots of papers on, and I'm not sure it's really important that I explain. Yoiu can inform yourself, and the Sista talks I pointed to at least mention the topic. I will say that our thesis is that we can generate efficient register-based machine code from stack-oriented bytecode, and that's one of the things we're trying to demonstrate with Sista.
_,,,^..^,,,_ (phone)
thank you, robert
. .. ... ^,^
On 12/18/2015 11:11 PM, Eliot Miranda wrote:
Further, please keep the original Smalltalk non-primitive implementation around do that when we deliver Sista we can see how well we stack up against C.
_,,,^..^,,,_ (phone)
On Dec 18, 2015, at 5:59 PM, David T. Lewis lewis@mail.msen.com wrote:
On Fri, Dec 18, 2015 at 08:13:00PM -0500, Robert Withers wrote:
On 12/18/2015 08:07 PM, David T. Lewis wrote:
If you are implementing your algorithm from scratch, then it would be
nice if it could be done in Smalltalk, because that means we can all
read it, play with it in the image, and write unit tests to validate and
document its behavior. We can figure out the translation to C afterwords,
once you have a good implementation in Smalltalk (and yes I will help
with that).
I have a smalltalk implmentation that came from Google Java code and is
fully implemented in squeak/pharo. I am intermittently fixing byte
mutations in the payload, so it is working to a degree. Check out the
classes GenericGF, GenericGFPoly and GaloisField to see the insides.
FECEncoder and FECDecoder are calling the galoisField which builds a
ReedSolomon Decoder. That last is where the error detection occurs.
Thank you all very much for looking out!
I guess my suggestion would be that once you have a fairly good working
implementation that you are comfortable with from a functional point of
view, then let's look at which methods in that implementation would benefit
from being translated to primitives in C. It would be especially good if
you can run a profiler on your Smalltalk implementation and see where the
time is being spent. Those would be the things we would want to turn into
primitives. But first things first, let's get it working 100% (not "to a
degree") and then we can do the translation to primitives.
Dave
-- . .. .. ^,^ robert
-- . .. .. ^,^ robert
On 12/19/2015 04:23 PM, Eliot Miranda wrote:
On Fri, Dec 18, 2015 at 11:25 PM, Robert Withers <robert.w.withers@gmail.com mailto:robert.w.withers@gmail.com> wrote:
On 12/19/2015 02:09 AM, Eliot Miranda wrote:Hi Robert, On Dec 18, 2015, at 9:51 PM, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> wrote:Eliot, what does Sista do and how is that accomplished?Run-time adaptive optimisation via an SSA-based image-level bytecode-to-bytecode inlining compiler. See http://www.mirandabanda.org/cogblog/on-line-papers-and-presentations/It sounds like a next level kind of system. Does it compare to what other JITs are doing in various VMs?Yes it does. Clément recently attended a workshop at Google and found that Sista is very similar in the kinds of optimisations it performs to V8 and Dart. The key research funding will be how close the performance of targeting stack-oriented bytecode that the Cog JIT converts to register-based code can be made to JITs that directly target register-based machine code.This sounds like you are saying that Sista performs better or more naturally with stack-oriented bytecode. If you wouldn't mind taking the time to explore it with us, what's the difference between stack-oriented and register-based code, such that Sista would be different between them? I'm off to read some.I'd rather not. That's something there's lots of papers on, and I'm not sure it's really important that I explain. Yoiu can inform yourself, and the Sista talks I pointed to at least mention the topic. I will say that our thesis is that we can generate efficient register-based machine code from stack-oriented bytecode, and that's one of the things we're trying to demonstrate with Sista.
That's all very interesting to me, surely,just to know if not to play. It is the physics of this world and I studied that and am oriented that way. Clement's talk touched on it (was it constructing and deconstucting the machine code and rewriting the stack to point to registers...I think).
I was actually much more curious about this CPIC. Is there a write-up about that so I minimize my distracting you?
robert
_,,,^..^,,,_ (phone)thank you, robert . .. ... ^,^ On 12/18/2015 11:11 PM, Eliot Miranda wrote:Further, please keep the original Smalltalk non-primitive implementation around do that when we deliver Sista we can see how well we stack up against C. _,,,^..^,,,_ (phone)On Dec 18, 2015, at 5:59 PM, David T. Lewis <lewis@mail.msen.com <mailto:lewis@mail.msen.com>> wrote:On Fri, Dec 18, 2015 at 08:13:00PM -0500, Robert Withers wrote:> On 12/18/2015 08:07 PM, David T. Lewis wrote: > > If you are implementing your algorithm from scratch, then it > would be > nice if it could be done in Smalltalk, because that means we > can all > read it, play with it in the image, and write unit tests to > validate and > document its behavior. We can figure out the translation to > C afterwords, > once you have a good implementation in Smalltalk (and yes I > will help > with that). I have a smalltalk implmentation that came from Google Java code and is fully implemented in squeak/pharo. I am intermittently fixing byte mutations in the payload, so it is working to a degree. Check out the classes GenericGF, GenericGFPoly and GaloisField to see the insides. FECEncoder and FECDecoder are calling the galoisField which builds a ReedSolomon Decoder. That last is where the error detection occurs.
Thank you all very much for looking out!I guess my suggestion would be that once you have a fairly good working implementation that you are comfortable with from a functional point of view, then let's look at which methods in that implementation would benefit from being translated to primitives in C. It would be especially good if you can run a profiler on your Smalltalk implementation and see where the time is being spent. Those would be the things we would want to turn into primitives. But first things first, let's get it working 100% (not "to a degree") and then we can do the translation to primitives. Dave-- . .. .. ^,^ robert-- . .. .. ^,^ robert-- _,,,^..^,,,_ best, Eliot
Well, please excuse me from asking for time with no clear benefit when we both have so much going on, Eliot. I was looking for distraction as I am completely frustrated by my code riight now. The nexus of nested frames & headers, FEC encoding, ASN.1 encoding and pipeline processing order is totally challenging myself.
So, yes, it was a distraction request, my apologies.
robert . .. ... ^,^
On 12/19/2015 04:40 PM, Robert Withers wrote:
On 12/19/2015 04:23 PM, Eliot Miranda wrote:
On Fri, Dec 18, 2015 at 11:25 PM, Robert Withers robert.w.withers@gmail.com wrote:
On 12/19/2015 02:09 AM, Eliot Miranda wrote:Hi Robert, On Dec 18, 2015, at 9:51 PM, Robert Withers <robert.w.withers@gmail.com> wrote:Eliot, what does Sista do and how is that accomplished?Run-time adaptive optimisation via an SSA-based image-level bytecode-to-bytecode inlining compiler. See http://www.mirandabanda.org/cogblog/on-line-papers-and-presentations/It sounds like a next level kind of system. Does it compare to what other JITs are doing in various VMs?Yes it does. Clément recently attended a workshop at Google and found that Sista is very similar in the kinds of optimisations it performs to V8 and Dart. The key research funding will be how close the performance of targeting stack-oriented bytecode that the Cog JIT converts to register-based code can be made to JITs that directly target register-based machine code.This sounds like you are saying that Sista performs better or more naturally with stack-oriented bytecode. If you wouldn't mind taking the time to explore it with us, what's the difference between stack-oriented and register-based code, such that Sista would be different between them? I'm off to read some.I'd rather not. That's something there's lots of papers on, and I'm not sure it's really important that I explain. Yoiu can inform yourself, and the Sista talks I pointed to at least mention the topic. I will say that our thesis is that we can generate efficient register-based machine code from stack-oriented bytecode, and that's one of the things we're trying to demonstrate with Sista.
That's all very interesting to me, surely,just to know if not to play. It is the physics of this world and I studied that and am oriented that way. Clement's talk touched on it (was it constructing and deconstucting the machine code and rewriting the stack to point to registers...I think).
I was actually much more curious about this CPIC. Is there a write-up about that so I minimize my distracting you?
robert
_,,,^..^,,,_ (phone)thank you, robert . .. ... ^,^ On 12/18/2015 11:11 PM, Eliot Miranda wrote:Further, please keep the original Smalltalk non-primitive implementation around do that when we deliver Sista we can see how well we stack up against C. _,,,^..^,,,_ (phone)On Dec 18, 2015, at 5:59 PM, David T. Lewis <lewis@mail.msen.com> wrote:> On Fri, Dec 18, 2015 at 08:13:00PM -0500, Robert Withers wrote: > >> On 12/18/2015 08:07 PM, David T. Lewis wrote: >> >> If you are implementing your algorithm from scratch, then >> it would be >> nice if it could be done in Smalltalk, because that means >> we can all >> read it, play with it in the image, and write unit tests to >> validate and >> document its behavior. We can figure out the translation to >> C afterwords, >> once you have a good implementation in Smalltalk (and yes I >> will help >> with that). > I have a smalltalk implmentation that came from Google Java > code and is > fully implemented in squeak/pharo. I am intermittently > fixing byte > mutations in the payload, so it is working to a degree. > Check out the > classes GenericGF, GenericGFPoly and GaloisField to see the > insides. > FECEncoder and FECDecoder are calling the galoisField which > builds a > ReedSolomon Decoder. That last is where the error detection > occurs. > > Thank you all very much for looking out! I guess my suggestion would be that once you have a fairly good working implementation that you are comfortable with from a functional point of view, then let's look at which methods in that implementation would benefit from being translated to primitives in C. It would be especially good if you can run a profiler on your Smalltalk implementation and see where the time is being spent. Those would be the things we would want to turn into primitives. But first things first, let's get it working 100% (not "to a degree") and then we can do the translation to primitives.
Dave-- . .. .. ^,^ robert-- . .. .. ^,^ robert-- _,,,^..^,,,_ best, Eliot
-- . .. .. ^,^ robert
Eliot, I read what you wrote about a partial read barrier (to own memory for scans?) to support fast become which allows for forwarders in the stack execution. Is this right? If so it is a read barrier into ObjectMemory. And so I think the forwarders access the CPIC somehow.
I was under a different conceptualization...that if you had a read-only mode, where all non-local/temp variables were immutable (you could set local vars), then more than one OS thread could be in the image, if only one had the write lock. Perhaps this is a write barrier.
The idea being that the lookup could occur separately from the invocation and could be threaded. Just a thought and I am not sure where it may fit. Please carry on.
best, robert
On 12/19/2015 04:23 PM, Eliot Miranda wrote:
On Fri, Dec 18, 2015 at 11:25 PM, Robert Withers <robert.w.withers@gmail.com mailto:robert.w.withers@gmail.com> wrote:
On 12/19/2015 02:09 AM, Eliot Miranda wrote:Hi Robert, On Dec 18, 2015, at 9:51 PM, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> wrote:Eliot, what does Sista do and how is that accomplished?Run-time adaptive optimisation via an SSA-based image-level bytecode-to-bytecode inlining compiler. See http://www.mirandabanda.org/cogblog/on-line-papers-and-presentations/It sounds like a next level kind of system. Does it compare to what other JITs are doing in various VMs?Yes it does. Clément recently attended a workshop at Google and found that Sista is very similar in the kinds of optimisations it performs to V8 and Dart. The key research funding will be how close the performance of targeting stack-oriented bytecode that the Cog JIT converts to register-based code can be made to JITs that directly target register-based machine code.This sounds like you are saying that Sista performs better or more naturally with stack-oriented bytecode. If you wouldn't mind taking the time to explore it with us, what's the difference between stack-oriented and register-based code, such that Sista would be different between them? I'm off to read some.I'd rather not. That's something there's lots of papers on, and I'm not sure it's really important that I explain. Yoiu can inform yourself, and the Sista talks I pointed to at least mention the topic. I will say that our thesis is that we can generate efficient register-based machine code from stack-oriented bytecode, and that's one of the things we're trying to demonstrate with Sista.
_,,,^..^,,,_ (phone)thank you, robert . .. ... ^,^ On 12/18/2015 11:11 PM, Eliot Miranda wrote:Further, please keep the original Smalltalk non-primitive implementation around do that when we deliver Sista we can see how well we stack up against C. _,,,^..^,,,_ (phone)On Dec 18, 2015, at 5:59 PM, David T. Lewis <lewis@mail.msen.com <mailto:lewis@mail.msen.com>> wrote:On Fri, Dec 18, 2015 at 08:13:00PM -0500, Robert Withers wrote:> On 12/18/2015 08:07 PM, David T. Lewis wrote: > > If you are implementing your algorithm from scratch, then it > would be > nice if it could be done in Smalltalk, because that means we > can all > read it, play with it in the image, and write unit tests to > validate and > document its behavior. We can figure out the translation to > C afterwords, > once you have a good implementation in Smalltalk (and yes I > will help > with that). I have a smalltalk implmentation that came from Google Java code and is fully implemented in squeak/pharo. I am intermittently fixing byte mutations in the payload, so it is working to a degree. Check out the classes GenericGF, GenericGFPoly and GaloisField to see the insides. FECEncoder and FECDecoder are calling the galoisField which builds a ReedSolomon Decoder. That last is where the error detection occurs.
Thank you all very much for looking out!I guess my suggestion would be that once you have a fairly good working implementation that you are comfortable with from a functional point of view, then let's look at which methods in that implementation would benefit from being translated to primitives in C. It would be especially good if you can run a profiler on your Smalltalk implementation and see where the time is being spent. Those would be the things we would want to turn into primitives. But first things first, let's get it working 100% (not "to a degree") and then we can do the translation to primitives. Dave-- . .. .. ^,^ robert-- . .. .. ^,^ robert-- _,,,^..^,,,_ best, Eliot
On 12/19/2015 04:23 PM, Eliot Miranda wrote:
On Fri, Dec 18, 2015 at 11:25 PM, Robert Withers <robert.w.withers@gmail.com mailto:robert.w.withers@gmail.com> wrote:
On 12/19/2015 02:09 AM, Eliot Miranda wrote:Hi Robert, On Dec 18, 2015, at 9:51 PM, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> wrote:Eliot, what does Sista do and how is that accomplished?Run-time adaptive optimisation via an SSA-based image-level bytecode-to-bytecode inlining compiler. See http://www.mirandabanda.org/cogblog/on-line-papers-and-presentations/It sounds like a next level kind of system. Does it compare to what other JITs are doing in various VMs?Yes it does. Clément recently attended a workshop at Google and found that Sista is very similar in the kinds of optimisations it performs to V8 and Dart. The key research funding will be how close the performance of targeting stack-oriented bytecode that the Cog JIT converts to register-based code can be made to JITs that directly target register-based machine code.This sounds like you are saying that Sista performs better or more naturally with stack-oriented bytecode. If you wouldn't mind taking the time to explore it with us, what's the difference between stack-oriented and register-based code, such that Sista would be different between them? I'm off to read some.I'd rather not. That's something there's lots of papers on, and I'm not sure it's really important that I explain. Yoiu can inform yourself, and the Sista talks I pointed to at least mention the topic. I will say that our thesis is that we can generate efficient register-based machine code from stack-oriented bytecode, and that's one of the things we're trying to demonstrate with Sista.
Having had a little bit of sleep, upon waking I just have to say I am disappointed in this response. As a seeker of knwoledge, I'm certainly willing to research deeply, yet that is a time investment when I also have a full plate. Nonetheless, the negative response to a search of knowledge is indicative of an attitude I often find and it is a real shame, a real shame for everyone. I reread what I wrote and yes, in the scope of an exploration, as I said, denotes a big time investment to dive deep. You could have said so, not a big time investment, but go towards more of a paragraph then a sentence in helping to feed me when I was hungry. A more paranoid personality that lives within me thinks it may have been something I personally did to garnish a more negative response, but I'm convinced it was the way I broached.
You see how I talk myself out of being frustrated? I try to stand in your shoes, for a moment, without ego nor assuming the worst. It works.
alright then, best, robert
_,,,^..^,,,_ (phone)thank you, robert . .. ... ^,^ On 12/18/2015 11:11 PM, Eliot Miranda wrote:Further, please keep the original Smalltalk non-primitive implementation around do that when we deliver Sista we can see how well we stack up against C. _,,,^..^,,,_ (phone)On Dec 18, 2015, at 5:59 PM, David T. Lewis <lewis@mail.msen.com <mailto:lewis@mail.msen.com>> wrote:On Fri, Dec 18, 2015 at 08:13:00PM -0500, Robert Withers wrote:> On 12/18/2015 08:07 PM, David T. Lewis wrote: > > If you are implementing your algorithm from scratch, then it > would be > nice if it could be done in Smalltalk, because that means we > can all > read it, play with it in the image, and write unit tests to > validate and > document its behavior. We can figure out the translation to > C afterwords, > once you have a good implementation in Smalltalk (and yes I > will help > with that). I have a smalltalk implmentation that came from Google Java code and is fully implemented in squeak/pharo. I am intermittently fixing byte mutations in the payload, so it is working to a degree. Check out the classes GenericGF, GenericGFPoly and GaloisField to see the insides. FECEncoder and FECDecoder are calling the galoisField which builds a ReedSolomon Decoder. That last is where the error detection occurs.
Thank you all very much for looking out!I guess my suggestion would be that once you have a fairly good working implementation that you are comfortable with from a functional point of view, then let's look at which methods in that implementation would benefit from being translated to primitives in C. It would be especially good if you can run a profiler on your Smalltalk implementation and see where the time is being spent. Those would be the things we would want to turn into primitives. But first things first, let's get it working 100% (not "to a degree") and then we can do the translation to primitives. Dave-- . .. .. ^,^ robert-- . .. .. ^,^ robert-- _,,,^..^,,,_ best, Eliot
On Sun, Dec 20, 2015 at 01:55:32AM -0500, Robert Withers wrote:
On 12/19/2015 04:23 PM, Eliot Miranda wrote:
On Fri, Dec 18, 2015 at 11:25 PM, Robert Withers <robert.w.withers@gmail.com mailto:robert.w.withers@gmail.com> wrote:
This sounds like you are saying that Sista performs better or more naturally with stack-oriented bytecode. If you wouldn't mind taking the time to explore it with us, what's the difference between stack-oriented and register-based code, such that Sista would be different between them? I'm off to read some.
I'd rather not. That's something there's lots of papers on, and I'm not sure it's really important that I explain. Yoiu can inform yourself, and the Sista talks I pointed to at least mention the topic. I will say that our thesis is that we can generate efficient register-based machine code from stack-oriented bytecode, and that's one of the things we're trying to demonstrate with Sista.
Having had a little bit of sleep, upon waking I just have to say I am disappointed in this response. As a seeker of knwoledge, I'm certainly willing to research deeply, yet that is a time investment when I also have a full plate. Nonetheless, the negative response to a search of knowledge is indicative of an attitude I often find and it is a real shame, a real shame for everyone. I reread what I wrote and yes, in the scope of an exploration, as I said, denotes a big time investment to dive deep. You could have said so, not a big time investment, but go towards more of a paragraph then a sentence in helping to feed me when I was hungry. A more paranoid personality that lives within me thinks it may have been something I personally did to garnish a more negative response, but I'm convinced it was the way I broached.
You see how I talk myself out of being frustrated? I try to stand in your shoes, for a moment, without ego nor assuming the worst. It works.
Robert,
Eliot is extremely generous in his sharing of knowledge. He has written extensively on the work that he has been doing, and he makes those writings freely available on his blog and in these mailing lists. He is also a busy person who is making an effort to take time to keep us all informed and involved. Please be considerate in your requests for additional explanations.
Dave
On 12/20/2015 02:33 AM, David T. Lewis wrote:
On Sun, Dec 20, 2015 at 01:55:32AM -0500, Robert Withers wrote:
On 12/19/2015 04:23 PM, Eliot Miranda wrote:
On Fri, Dec 18, 2015 at 11:25 PM, Robert Withers <robert.w.withers@gmail.com mailto:robert.w.withers@gmail.com> wrote:
This sounds like you are saying that Sista performs better or more naturally with stack-oriented bytecode. If you wouldn't mind taking the time to explore it with us, what's the difference between stack-oriented and register-based code, such that Sista would be different between them? I'm off to read some.I'd rather not. That's something there's lots of papers on, and I'm not sure it's really important that I explain. Yoiu can inform yourself, and the Sista talks I pointed to at least mention the topic. I will say that our thesis is that we can generate efficient register-based machine code from stack-oriented bytecode, and that's one of the things we're trying to demonstrate with Sista.
Having had a little bit of sleep, upon waking I just have to say I am disappointed in this response. As a seeker of knwoledge, I'm certainly willing to research deeply, yet that is a time investment when I also have a full plate. Nonetheless, the negative response to a search of knowledge is indicative of an attitude I often find and it is a real shame, a real shame for everyone. I reread what I wrote and yes, in the scope of an exploration, as I said, denotes a big time investment to dive deep. You could have said so, not a big time investment, but go towards more of a paragraph then a sentence in helping to feed me when I was hungry. A more paranoid personality that lives within me thinks it may have been something I personally did to garnish a more negative response, but I'm convinced it was the way I broached.
You see how I talk myself out of being frustrated? I try to stand in your shoes, for a moment, without ego nor assuming the worst. It works.
Robert,
Eliot is extremely generous in his sharing of knowledge. He has written extensively on the work that he has been doing, and he makes those writings freely available on his blog and in these mailing lists. He is also a busy person who is making an effort to take time to keep us all informed and involved. Please be considerate in your requests for additional explanations.
You're right, I know this and I apologize if I wasn't considerate. I wanted ice cream. Sorry.
robert
Dave
On Sun, Dec 20, 2015 at 5:55 PM, Robert Withers robert.w.withers@gmail.com wrote:
On 12/19/2015 04:23 PM, Eliot Miranda wrote: On Fri, Dec 18, 2015 at 11:25 PM, Robert Withers robert.w.withers@gmail.com wrote:
On 12/19/2015 02:09 AM, Eliot Miranda wrote: Hi Robert,
On Dec 18, 2015, at 9:51 PM, Robert Withers robert.w.withers@gmail.com wrote:
Eliot, what does Sista do and how is that accomplished?
Run-time adaptive optimisation via an SSA-based image-level bytecode-to-bytecode inlining compiler. See
http://www.mirandabanda.org/cogblog/on-line-papers-and-presentations/
It sounds like a next level kind of system. Does it compare to what other JITs are doing in various VMs?
Yes it does. Clément recently attended a workshop at Google and found that Sista is very similar in the kinds of optimisations it performs to V8 and Dart. The key research funding will be how close the performance of targeting stack-oriented bytecode that the Cog JIT converts to register-based code can be made to JITs that directly target register-based machine code.
This sounds like you are saying that Sista performs better or more naturally with stack-oriented bytecode. If you wouldn't mind taking the time to explore it with us, what's the difference between stack-oriented and register-based code, such that Sista would be different between them? I'm off to read some.
I'd rather not. That's something there's lots of papers on, and I'm not sure it's really important that I explain. You can inform yourself, and the Sista talks I pointed to at least mention the topic. I will say that our thesis is that we can generate efficient register-based machine code from stack-oriented bytecode, and that's one of the things we're trying to demonstrate with Sista.
Having had a little bit of sleep, upon waking I just have to say I am disappointed in this response. As a seeker of knowledge, I'm certainly willing to research deeply, yet that is a time investment when I also have a full plate. Nonetheless, the negative response to a search of knowledge is indicative of an attitude I often find and it is a real shame, a real shame for everyone. I reread what I wrote and yes, in the scope of an exploration, as I said, denotes a big time investment to dive deep. You could have said so, not a big time investment, but go towards more of a paragraph then a sentence in helping to feed me when I was hungry. A more paranoid personality that lives within me thinks it may have been something I personally did to garnish a more negative response, but I'm convinced it was the way I broached.
You're over thinking it. Please read... http://www.catb.org/esr/faqs/smart-questions.html sections... + Be explicit about your question + How To Interpret Answers
cheers -ben
Thank you, Eliot, for this work. I was wondering what OSR entry and exit stood for.
Also, I was very curious how lazy forwarders and this CPIC, I have heard mention, interact?
best, robert
On 12/19/2015 02:09 AM, Eliot Miranda wrote:
Hi Robert,
On Dec 18, 2015, at 9:51 PM, Robert Withers <robert.w.withers@gmail.com mailto:robert.w.withers@gmail.com> wrote:
Eliot, what does Sista do and how is that accomplished?
Run-time adaptive optimisation via an SSA-based image-level bytecode-to-bytecode inlining compiler. See
http://www.mirandabanda.org/cogblog/on-line-papers-and-presentations/
It sounds like a next level kind of system. Does it compare to what other JITs are doing in various VMs?
Yes it does. Clément recently attended a workshop at Google and found that Sista is very similar in the kinds of optimisations it performs to V8 and Dart. The key research funding will be how close the performance of targeting stack-oriented bytecode that the Cog JIT converts to register-based code can be made to JITs that directly target register-based machine code.
_,,,^..^,,,_ (phone)
thank you, robert
. .. ... ^,^
On 12/18/2015 11:11 PM, Eliot Miranda wrote:
Further, please keep the original Smalltalk non-primitive implementation around do that when we deliver Sista we can see how well we stack up against C.
_,,,^..^,,,_ (phone)
On Dec 18, 2015, at 5:59 PM, David T. Lewis <lewis@mail.msen.com mailto:lewis@mail.msen.com> wrote:
On Fri, Dec 18, 2015 at 08:13:00PM -0500, Robert Withers wrote:
On 12/18/2015 08:07 PM, David T. Lewis wrote:
If you are implementing your algorithm from scratch, then it would be nice if it could be done in Smalltalk, because that means we can all read it, play with it in the image, and write unit tests to validate and document its behavior. We can figure out the translation to C afterwords, once you have a good implementation in Smalltalk (and yes I will help with that).
I have a smalltalk implmentation that came from Google Java code and is fully implemented in squeak/pharo. I am intermittently fixing byte mutations in the payload, so it is working to a degree. Check out the classes GenericGF, GenericGFPoly and GaloisField to see the insides. FECEncoder and FECDecoder are calling the galoisField which builds a ReedSolomon Decoder. That last is where the error detection occurs.
Thank you all very much for looking out!
I guess my suggestion would be that once you have a fairly good working implementation that you are comfortable with from a functional point of view, then let's look at which methods in that implementation would benefit from being translated to primitives in C. It would be especially good if you can run a profiler on your Smalltalk implementation and see where the time is being spent. Those would be the things we would want to turn into primitives. But first things first, let's get it working 100% (not "to a degree") and then we can do the translation to primitives.
Dave
-- . .. .. ^,^ robert
vm-dev@lists.squeakfoundation.org
-
Ben Coman -
David T. Lewis -
Eliot Miranda -
Robert Withers -
tim Rowledge