<body><div id="__MailbirdStyleContent" style="font-size: 10pt;font-family: Arial;color: #000000">
Hi Levente,<div><br></div><div>thanks for taking a look at it. :-)</div><div><br></div><div>Best,</div><div>Marcel</div><div class="mb_sig"></div><blockquote class="history_container" type="cite" style="border-left-style:solid;border-width:1px; margin-top:20px; margin-left:0px;padding-left:10px;">
<p style="color: #AAAAAA; margin-top: 10px;">Am 14.06.2020 00:13:05 schrieb Levente Uzonyi <leves@caesar.elte.hu>:</p><div style="font-family:Arial,Helvetica,sans-serif">Hi Marcel,
<br>
<br>The numbers look good to me and the code too.
<br>
<br>
<br>Levente
<br>
<br>On Sat, 13 Jun 2020, Marcel Taeumel wrote:
<br>
<br>> Hi all!
<br>>
<br>> I made some tweaks to further speed-up the manual method lookup. The extra operations are now as follows:
<br>>
<br>> currentToken last == $:
<br>> Symbol lookup: currentToken allButLast
<br>> self class methodDict at: ifPresent:
<br>> method pragmas anySatisfy ...
<br>> self executeMethod:
<br>>
<br>> As it is now, it is a fairly generic way to re-direct method dispatch based on two information: (1) string token that is already interned as symbol and (2) a pragma that is in a method with that symbol as selector. I suppose
<br>> there are other places in the system that could be modularized with such a mechanism. Not sure whether it is worthwhile outside a string-parsing context.
<br>>
<br>> Anyway, here are the results for the Shout parser:
<br>>
<br>> === AFTER (optmized) ===
<br>>
<br>> {
<br>> "FFITestLibrary>>ffiPrintString:":[
<br>> "73,500 per second. 13.6 microseconds per run. 2.78 % GC time.",
<br>> "73,800 per second. 13.6 microseconds per run. 2.46 % GC time.",
<br>> "74,000 per second. 13.5 microseconds per run. 2.43951 % GC time."
<br>> ],
<br>> "BitBlt>>copyBits":[
<br>> "8,050 per second. 124 microseconds per run. 2.42 % GC time.",
<br>> "7,990 per second. 125 microseconds per run. 2.73945 % GC time.",
<br>> "8,040 per second. 124 microseconds per run. 2.65947 % GC time."
<br>> ],
<br>> "ExternalPoolReadWriter>>fetchFromFile":[
<br>> "39,700 per second. 25.2 microseconds per run. 2.18 % GC time.",
<br>> "39,600 per second. 25.3 microseconds per run. 2.52 % GC time.",
<br>> "39,400 per second. 25.4 microseconds per run. 2.08 % GC time."
<br>> ],
<br>> "Win32Pool class>>winver":[
<br>> "21,600 per second. 46.3 microseconds per run. 2.29954 % GC time.",
<br>> "21,700 per second. 46.2 microseconds per run. 2.3 % GC time.",
<br>> "21,800 per second. 45.9 microseconds per run. 2.28 % GC time."
<br>> ]
<br>> }
<br>>
<br>> Best,
<br>> Marcel
<br>>
<br>> Am 13.06.2020 10:03:41 schrieb Marcel Taeumel <marcel.taeumel@hpi.de>:
<br>>
<br>> Hi all!
<br>>
<br>> I made some benchmarks. While I haven't got the slowest machine here, it gives a rough impression of the performance impact of this extension hook I proposed.
<br>>
<br>> What is currently a simple message send, will be replaced by a lookup mechanism that dispatches the method dictionary manually. Here are the extra operations in a nutshell from SHParserST80 >> #parsePragmaStatement
<br>>
<br>> currentToken last == $:
<br>> currentToken asSimpleGetter
<br>> self class includesSelector:
<br>> self class compiledMethodAt:
<br>> CompiledMethod >> #pragmas ... anySatisfy:
<br>> self executeMethod:
<br>>
<br>> So, there is one linear search for the pragma <pragmaparser> to avoid calling an arbitrary method in Parser. :-) I suppose that #asSimpleGetter could be reduced to #allButLast and maybe Symbol class >> #lookup: can
<br>> further speed things up.
<br>>
<br>> Here is the benchmark code:
<br>>
<br>> ({
<br>> FFITestLibrary >> #ffiPrintString:. "<cdecl...>"
<br>> BitBlt >> #copyBits. "<primitive: ...="">"
<br>> ExternalPoolReadWriter >> #fetchFromFile. "1 simple pragma"
<br>> Win32Pool class >> #winver. "9 simple pragmas"
<br>> } collect: [:method |
<br>> | source styler |
<br>> source := method getSource.
<br>> styler := SHTextStylerST80 new
<br>> classOrMetaClass: method methodClass.
<br>> (method methodClass name, '>>', method selector)
<br>> -> ((1 to: 3) collect: [:e | [styler styledTextFor: source] bench])]
<br>> as: OrderedDictionary) asJsonString.
<br>>
<br>> Here are the results:
<br>>
<br>> === BEFORE ===
<br>>
<br>> {
<br>> "FFITestLibrary>>ffiPrintString:":[
<br>> "77,700 per second. 12.9 microseconds per run. 3.0194 % GC time.",
<br>> "77,100 per second. 13 microseconds per run. 3.11938 % GC time.",
<br>> "77,300 per second. 12.9 microseconds per run. 3.35933 % GC time."
<br>> ],
<br>> "BitBlt>>copyBits":[
<br>> "8,020 per second. 125 microseconds per run. 3.15937 % GC time.",
<br>> "7,990 per second. 125 microseconds per run. 3.32 % GC time.",
<br>> "8,030 per second. 125 microseconds per run. 3.24 % GC time."
<br>> ],
<br>> "ExternalPoolReadWriter>>fetchFromFile":[
<br>> "39,200 per second. 25.5 microseconds per run. 2.63947 % GC time.",
<br>> "38,600 per second. 25.9 microseconds per run. 3.28 % GC time.",
<br>> "38,200 per second. 26.2 microseconds per run. 3.04 % GC time."
<br>> ],
<br>> "Win32Pool class>>winver":[
<br>> "23,400 per second. 42.7 microseconds per run. 2.96 % GC time.",
<br>> "23,500 per second. 42.5 microseconds per run. 2.95941 % GC time.",
<br>> "23,600 per second. 42.5 microseconds per run. 2.73945 % GC time."
<br>> ]
<br>> }
<br>>
<br>> === AFTER ===
<br>>
<br>> {
<br>> "FFITestLibrary>>ffiPrintString:":[
<br>> "66,400 per second. 15.1 microseconds per run. 2.85943 % GC time.",
<br>> "66,900 per second. 14.9 microseconds per run. 2.66 % GC time.",
<br>> "69,100 per second. 14.5 microseconds per run. 2.44 % GC time."
<br>> ],
<br>> "BitBlt>>copyBits":[
<br>> "7,610 per second. 131 microseconds per run. 2.81944 % GC time.",
<br>> "7,520 per second. 133 microseconds per run. 2.77944 % GC time.",
<br>> "7,640 per second. 131 microseconds per run. 2.62 % GC time."
<br>> ],
<br>> "ExternalPoolReadWriter>>fetchFromFile":[
<br>> "37,100 per second. 26.9 microseconds per run. 2.03959 % GC time.",
<br>> "36,600 per second. 27.3 microseconds per run. 2.44049 % GC time.",
<br>> "36,800 per second. 27.2 microseconds per run. 2.45951 % GC time."
<br>> ],
<br>> "Win32Pool class>>winver":[
<br>> "19,200 per second. 52.1 microseconds per run. 2.05959 % GC time.",
<br>> "18,900 per second. 52.9 microseconds per run. 1.9796 % GC time.",
<br>> "19,000 per second. 52.6 microseconds per run. 2.2 % GC time."
<br>> ]
<br>> }
<br>>
<br>> Best,
<br>> Marcel
<br>>
<br>> Am 12.06.2020 17:31:18 schrieb Marcel Taeumel <marcel.taeumel@hpi.de>:
<br>>
<br>> @Levente: Would this be okay-ish from a performance-perspective? :-)
<br>> Best,
<br>> Marcel
<br>>
<br>> Am 12.06.2020 17:28:25 schrieb commits@source.squeak.org <commits@source.squeak.org>:
<br>>
<br>> A new version of ShoutCore was added to project The Inbox:
<br>> http://source.squeak.org/inbox/ShoutCore-mt.78.mcz
<br>>
<br>> ==================== Summary ====================
<br>>
<br>> Name: ShoutCore-mt.78
<br>> Author: mt
<br>> Time: 12 June 2020, 5:28:15.527851 pm
<br>> UUID: 7decada5-8996-304c-ba9c-a93b00f0cc33
<br>> Ancestors: ShoutCore-mt.77
<br>>
<br>> Complements Compiler-mt.436 (inbox).
<br>>
<br>> Adds a (extension) method-based hook to install custom pragma-parsing methods. Use it to move FFI-specific pragma-parsing, i.e. and , into FFI packages.
<br>>
<br>> =============== Diff against ShoutCore-mt.77 ===============
<br>>
<br>> Item was removed:
<br>> - ----- Method: SHParserST80>>isTokenExternalFunctionCallingConvention (in category 'token testing') -----
<br>> - isTokenExternalFunctionCallingConvention
<br>> -
<br>> - currentToken ifNil: [ ^false ].
<br>> - ^(Smalltalk classNamed: #ExternalFunction)
<br>> - ifNil: [ false ]
<br>> - ifNotNil: [ :descriptorClass |
<br>> - (descriptorClass callingConventionFor: currentToken) notNil ]!
<br>>
<br>> Item was removed:
<br>> - ----- Method: SHParserST80>>parseExternalCall (in category 'parse') -----
<br>> - parseExternalCall
<br>> - [self scanNext.
<br>> - ((Smalltalk at: #ExternalFunction) callingConventionModifierFor: currentToken) notNil]
<br>> - whileTrue.
<br>> - self failUnless: currentToken notNil.
<br>> - self scanPast: #externalCallType.
<br>> - currentToken = '*'
<br>> - ifTrue: [self scanPast: #externalCallTypePointerIndicator].
<br>> - currentTokenFirst isDigit
<br>> - ifTrue: [self scanPast: #integer]
<br>> - ifFalse: [
<br>> - self failUnless: currentTokenFirst == $'.
<br>> - self parseString].
<br>> - self failUnless: currentTokenFirst == $(.
<br>> - self scanPast: #leftParenthesis.
<br>> - [currentTokenFirst ~= $)]
<br>> - whileTrue: [
<br>> - self failUnless: currentToken notNil.
<br>> - self scanPast: #externalCallType.
<br>> - currentToken = '*'
<br>> - ifTrue: [self scanPast: #externalCallTypePointerIndicator]].
<br>> - self scanPast: #rightParenthesis.
<br>> - currentToken = 'module:'
<br>> - ifTrue: [
<br>> - self scanPast: #module.
<br>> - self parseStringOrSymbol ].
<br>> - currentToken = 'error:' ifTrue: [
<br>> - self scanPast: #primitive. "there's no rangeType for error"
<br>> - self currentTokenType == #name
<br>> - ifTrue: [ self parseTemporary: #patternTempVar ]
<br>> - ifFalse: [ self parseStringOrSymbol ] ].
<br>> - self failUnless: currentToken = '>'.
<br>> - self scanPast: #primitiveOrExternalCallEnd!
<br>>
<br>> Item was changed:
<br>> + ----- Method: SHParserST80>>parsePragmaBinary (in category 'parse pragma') -----
<br>> - ----- Method: SHParserST80>>parsePragmaBinary (in category 'parse') -----
<br>> parsePragmaBinary
<br>>
<br>> self scanPast: #pragmaBinary.
<br>> self currentTokenType == #name
<br>> ifTrue:[self scanPast: (self resolvePragmaArgument: currentToken)]
<br>> ifFalse:[ self parseLiteral: false].
<br>> self failUnless: currentToken = '>'.
<br>> self scanPast: #primitiveOrExternalCallEnd!
<br>>
<br>> Item was changed:
<br>> + ----- Method: SHParserST80>>parsePragmaKeyword (in category 'parse pragma') -----
<br>> - ----- Method: SHParserST80>>parsePragmaKeyword (in category 'parse') -----
<br>> parsePragmaKeyword
<br>>
<br>> [self currentTokenType == #keyword]
<br>> whileTrue:[
<br>> self scanPast: #pragmaKeyword.
<br>> self currentTokenType == #name
<br>> ifTrue:[self scanPast: (self resolvePragmaArgument: currentToken)]
<br>> ifFalse:[ self parseLiteral: false]].
<br>> self failUnless: currentToken = '>'.
<br>> self scanPast: #primitiveOrExternalCallEnd!
<br>>
<br>> Item was changed:
<br>> + ----- Method: SHParserST80>>parsePragmaSequence (in category 'parse pragma') -----
<br>> - ----- Method: SHParserST80>>parsePragmaSequence (in category 'parse') -----
<br>> parsePragmaSequence
<br>> +
<br>> [currentToken = '<'><'>
<br>> + whileTrue: [
<br>> - whileTrue:[
<br>> self scanPast: #primitiveOrExternalCallStart.
<br>> + self parsePragmaStatement].!
<br>> - currentToken = 'primitive:'
<br>> - ifTrue: [
<br>> - self addRangeType: #primitive.
<br>> - self parsePrimitive]
<br>> - ifFalse:[
<br>> - self isTokenExternalFunctionCallingConvention
<br>> - ifTrue: [
<br>> - self addRangeType: #externalFunctionCallingConvention.
<br>> - self parseExternalCall]
<br>> - ifFalse:[
<br>> - self currentTokenType
<br>> - caseOf: {
<br>> - [ #name ] -> [
<br>> - self scanPast: #pragmaUnary.
<br>> - self failUnless: currentToken = '>'.
<br>> - self scanPast: #primitiveOrExternalCallEnd ].
<br>> - [ #binary ] -> [ self parsePragmaBinary ].
<br>> - [ #keyword ] -> [ self parsePragmaKeyword ] }
<br>> - otherwise: [ self fail ": 'Invalid External Function Calling convention'" ] ] ] ]!
<br>>
<br>> Item was added:
<br>> + ----- Method: SHParserST80>>parsePragmaStatement (in category 'parse pragma') -----
<br>> + parsePragmaStatement
<br>> +
<br>> + | parserSelector parserMethod |
<br>> + currentToken last == $: ifFalse: [
<br>> + "Quick exit to not break one-word pragmas such as or ."
<br>> + ^ self parsePragmaStatementKeywords].
<br>> +
<br>> + (self class includesSelector: (parserSelector := currentToken asSimpleGetter)) ifTrue: [
<br>> + ((parserMethod := self class compiledMethodAt: parserSelector) pragmas
<br>> + anySatisfy: [:pragma | pragma keyword == #pragmaParser])
<br>> + ifTrue: [^ self executeMethod: parserMethod]].
<br>> +
<br>> + ^ self parsePragmaStatementKeywords!
<br>>
<br>> Item was added:
<br>> + ----- Method: SHParserST80>>parsePragmaStatementKeywords (in category 'parse pragma') -----
<br>> + parsePragmaStatementKeywords
<br>> +
<br>> + self currentTokenType
<br>> + caseOf: {
<br>> + [ #name ] -> [
<br>> + self scanPast: #pragmaUnary.
<br>> + self failUnless: currentToken = '>'.
<br>> + self scanPast: #primitiveOrExternalCallEnd ].
<br>> + [ #binary ] -> [ self parsePragmaBinary ].
<br>> + [ #keyword ] -> [ self parsePragmaKeyword ] }
<br>> + otherwise: [ self fail ": 'Invalid External Function Calling convention'" ]. !
<br>>
<br>> Item was removed:
<br>> - ----- Method: SHParserST80>>parsePrimitive (in category 'parse') -----
<br>> - parsePrimitive
<br>> -
<br>> - self scanNext.
<br>> - currentTokenFirst isDigit
<br>> - ifTrue: [ self scanPast: #integer ]
<br>> - ifFalse: [
<br>> - self parseStringOrSymbol.
<br>> - currentToken = 'module:' ifTrue: [
<br>> - self scanPast: #module.
<br>> - self parseStringOrSymbol ] ].
<br>> - currentToken = 'error:' ifTrue: [
<br>> - self scanPast: #primitive. "there's no rangeType for error"
<br>> - self currentTokenType == #name
<br>> - ifTrue: [ self parseTemporary: #patternTempVar ]
<br>> - ifFalse: [ self parseStringOrSymbol ] ].
<br>> - self failUnless: currentToken = '>'.
<br>> - self scanPast: #primitiveOrExternalCallEnd!
<br>>
<br>> Item was added:
<br>> + ----- Method: SHParserST80>>primitive (in category 'parse pragma') -----
<br>> + primitive
<br>> + "Parse keywords and literals of primitive pragmas differently to emit different range tokens for different UI styling. This hook exists so that packages do not break primitive-pragma parsing by
<br>> accident. Instead, this method needs to be replaced intentionally."
<br>> +
<br>> +
<br>> + self addRangeType: #primitive.
<br>> +
<br>> + self scanNext.
<br>> + currentTokenFirst isDigit
<br>> + ifTrue: [ self scanPast: #integer ]
<br>> + ifFalse: [
<br>> + self parseStringOrSymbol.
<br>> + currentToken = 'module:' ifTrue: [
<br>> + self scanPast: #module.
<br>> + self parseStringOrSymbol ] ].
<br>> + currentToken = 'error:' ifTrue: [
<br>> + self scanPast: #primitive. "there's no rangeType for error"
<br>> + self currentTokenType == #name
<br>> + ifTrue: [ self parseTemporary: #patternTempVar ]
<br>> + ifFalse: [ self parseStringOrSymbol ] ].
<br>> + self failUnless: currentToken = '>'.
<br>> + self scanPast: #primitiveOrExternalCallEnd!
<br>>
<br>>
<br>>
<br>><br></'></'></commits@source.squeak.org></marcel.taeumel@hpi.de></primitive:></cdecl...></pragmaparser></marcel.taeumel@hpi.de></div></blockquote>
</div></body>