<body><div id="__MailbirdStyleContent" style="font-size: 10pt;font-family: Arial;color: #000000">
<div>Hi all!</div><div><br></div><div>I made some tweaks to further speed-up the manual method lookup. The extra operations are now as follows:</div><div><br></div><div>currentToken last == $:</div><div>Symbol lookup: currentToken allButLast</div><div>self class methodDict at: ifPresent:</div><div>method pragmas anySatisfy ...</div><div>self executeMethod:</div><div><br></div><div>As it is now, it is a fairly generic way to re-direct method dispatch based on two information: (1) string token that is already interned as symbol and (2) a pragma that is in a method with that symbol as selector. I suppose there are other places in the system that could be modularized with such a mechanism. Not sure whether it is worthwhile outside a string-parsing context.</div><div><br></div><div>Anyway, here are the results for the Shout parser:</div><div><br></div><div>=== AFTER (optmized) ===</div><div><br></div><div>{</div><div> "FFITestLibrary>>ffiPrintString:":[</div><div> "73,500 per second. 13.6 microseconds per run. 2.78 % GC time.",</div><div> "73,800 per second. 13.6 microseconds per run. 2.46 % GC time.",</div><div> "74,000 per second. 13.5 microseconds per run. 2.43951 % GC time."</div><div> ],</div><div> "BitBlt>>copyBits":[</div><div> "8,050 per second. 124 microseconds per run. 2.42 % GC time.",</div><div> "7,990 per second. 125 microseconds per run. 2.73945 % GC time.",</div><div> "8,040 per second. 124 microseconds per run. 2.65947 % GC time."</div><div> ],</div><div> "ExternalPoolReadWriter>>fetchFromFile":[</div><div> "39,700 per second. 25.2 microseconds per run. 2.18 % GC time.",</div><div> "39,600 per second. 25.3 microseconds per run. 2.52 % GC time.",</div><div> "39,400 per second. 25.4 microseconds per run. 2.08 % GC time."</div><div> ],</div><div> "Win32Pool class>>winver":[</div><div> "21,600 per second. 46.3 microseconds per run. 2.29954 % GC time.",</div><div> "21,700 per second. 46.2 microseconds per run. 2.3 % GC time.",</div><div> "21,800 per second. 45.9 microseconds per run. 2.28 % GC time."</div><div> ]</div><div>}</div><div><br></div><div>Best,</div><div>Marcel</div><div class="mb_sig"></div><blockquote class="history_container" type="cite" style="border-left-style:solid;border-width:1px; margin-top:20px; margin-left:0px;padding-left:10px;">
<p style="color: #AAAAAA; margin-top: 10px;">Am 13.06.2020 10:03:41 schrieb Marcel Taeumel <marcel.taeumel@hpi.de>:</p><div style="font-family:Arial,Helvetica,sans-serif"><div id="__MailbirdStyleContent" style="font-size: 10pt;font-family: Arial;color: #000000">
<div>Hi all!</div><div><br></div><div>I made some benchmarks. While I haven't got the slowest machine here, it gives a rough impression of the performance impact of this extension hook I proposed.</div><div><br></div><div>What is currently a simple message send, will be replaced by a lookup mechanism that dispatches the method dictionary manually. Here are the extra operations in a nutshell from SHParserST80 >> #parsePragmaStatement</div><div><br></div><div>currentToken last == $:</div><div>currentToken asSimpleGetter</div><div>self class includesSelector:</div><div>self class compiledMethodAt:</div><div>CompiledMethod >> #pragmas ... anySatisfy:</div><div>self executeMethod:</div><div><br></div><div>So, there is one linear search for the pragma <pragmaParser> to avoid calling an arbitrary method in Parser. :-) I suppose that #asSimpleGetter could be reduced to #allButLast and maybe Symbol class >> #lookup: can further speed things up.</div><div><br></div><div>Here is the benchmark code:</div><div><br></div><div>({</div><div> FFITestLibrary >> #ffiPrintString:. "<cdecl...>"</div><div> BitBlt >> #copyBits. "<primitive: ...>"</div><div> ExternalPoolReadWriter >> #fetchFromFile. "1 simple pragma"</div><div> Win32Pool class >> #winver. "9 simple pragmas"</div><div>} collect: [:method |</div><div> | source styler |</div><div> source := method getSource.</div><div> styler := SHTextStylerST80 new</div><div> classOrMetaClass: method methodClass.</div><div> (method methodClass name, '>>', method selector)</div><div> -> ((1 to: 3) collect: [:e | [styler styledTextFor: source] bench])]</div><div> as: OrderedDictionary) asJsonString.</div><div><br></div><div>Here are the results:</div><div><br></div><div>=== BEFORE ===</div><div><br></div><div>{</div><div> "FFITestLibrary>>ffiPrintString:":[</div><div> "77,700 per second. 12.9 microseconds per run. 3.0194 % GC time.",</div><div> "77,100 per second. 13 microseconds per run. 3.11938 % GC time.",</div><div> "77,300 per second. 12.9 microseconds per run. 3.35933 % GC time."</div><div> ],</div><div> "BitBlt>>copyBits":[</div><div> "8,020 per second. 125 microseconds per run. 3.15937 % GC time.",</div><div> "7,990 per second. 125 microseconds per run. 3.32 % GC time.",</div><div> "8,030 per second. 125 microseconds per run. 3.24 % GC time."</div><div> ],</div><div> "ExternalPoolReadWriter>>fetchFromFile":[</div><div> "39,200 per second. 25.5 microseconds per run. 2.63947 % GC time.",</div><div> "38,600 per second. 25.9 microseconds per run. 3.28 % GC time.",</div><div> "38,200 per second. 26.2 microseconds per run. 3.04 % GC time."</div><div> ],</div><div> "Win32Pool class>>winver":[</div><div> "23,400 per second. 42.7 microseconds per run. 2.96 % GC time.",</div><div> "23,500 per second. 42.5 microseconds per run. 2.95941 % GC time.",</div><div> "23,600 per second. 42.5 microseconds per run. 2.73945 % GC time."</div><div> ]</div><div>}</div><div><br></div><div>=== AFTER ===</div><div><br></div><div>{</div><div> "FFITestLibrary>>ffiPrintString:":[</div><div> "66,400 per second. 15.1 microseconds per run. 2.85943 % GC time.",</div><div> "66,900 per second. 14.9 microseconds per run. 2.66 % GC time.",</div><div> "69,100 per second. 14.5 microseconds per run. 2.44 % GC time."</div><div> ],</div><div> "BitBlt>>copyBits":[</div><div> "7,610 per second. 131 microseconds per run. 2.81944 % GC time.",</div><div> "7,520 per second. 133 microseconds per run. 2.77944 % GC time.",</div><div> "7,640 per second. 131 microseconds per run. 2.62 % GC time."</div><div> ],</div><div> "ExternalPoolReadWriter>>fetchFromFile":[</div><div> "37,100 per second. 26.9 microseconds per run. 2.03959 % GC time.",</div><div> "36,600 per second. 27.3 microseconds per run. 2.44049 % GC time.",</div><div> "36,800 per second. 27.2 microseconds per run. 2.45951 % GC time."</div><div> ],</div><div> "Win32Pool class>>winver":[</div><div> "19,200 per second. 52.1 microseconds per run. 2.05959 % GC time.",</div><div> "18,900 per second. 52.9 microseconds per run. 1.9796 % GC time.",</div><div> "19,000 per second. 52.6 microseconds per run. 2.2 % GC time."</div><div> ]</div><div>}</div><div><br></div><div>Best,</div><div>Marcel</div><div><br></div><div class="mb_sig"></div><blockquote class="history_container" type="cite" style="border-left-style:solid;border-width:1px; margin-top:20px; margin-left:0px;padding-left:10px;">
<p style="color: #AAAAAA; margin-top: 10px;">Am 12.06.2020 17:31:18 schrieb Marcel Taeumel <marcel.taeumel@hpi.de>:</p><div style="font-family:Arial,Helvetica,sans-serif"><div id="__MailbirdStyleContent" style="font-size: 10pt;font-family: Arial;color: #000000">
@Levente: Would this be okay-ish from a performance-perspective? :-)<div><br></div><div>Best,</div><div>Marcel</div><div class="mb_sig"></div><blockquote class="history_container" type="cite" style="border-left-style:solid;border-width:1px; margin-top:20px; margin-left:0px;padding-left:10px;">
<p style="color: #AAAAAA; margin-top: 10px;">Am 12.06.2020 17:28:25 schrieb commits@source.squeak.org <commits@source.squeak.org>:</p><div style="font-family:Arial,Helvetica,sans-serif">A new version of ShoutCore was added to project The Inbox:<br>http://source.squeak.org/inbox/ShoutCore-mt.78.mcz<br><br>==================== Summary ====================<br><br>Name: ShoutCore-mt.78<br>Author: mt<br>Time: 12 June 2020, 5:28:15.527851 pm<br>UUID: 7decada5-8996-304c-ba9c-a93b00f0cc33<br>Ancestors: ShoutCore-mt.77<br><br>Complements Compiler-mt.436 (inbox).<br><br>Adds a (extension) method-based hook to install custom pragma-parsing methods. Use it to move FFI-specific pragma-parsing, i.e. <apicall: ...=""> and <cdecl: ...="">, into FFI packages.<br><br>=============== Diff against ShoutCore-mt.77 ===============<br><br>Item was removed:<br>- ----- Method: SHParserST80>>isTokenExternalFunctionCallingConvention (in category 'token testing') -----<br>- isTokenExternalFunctionCallingConvention<br>- <br>- currentToken ifNil: [ ^false ].<br>- ^(Smalltalk classNamed: #ExternalFunction)<br>- ifNil: [ false ]<br>- ifNotNil: [ :descriptorClass |<br>- (descriptorClass callingConventionFor: currentToken) notNil ]!<br><br>Item was removed:<br>- ----- Method: SHParserST80>>parseExternalCall (in category 'parse') -----<br>- parseExternalCall<br>- [self scanNext.<br>- ((Smalltalk at: #ExternalFunction) callingConventionModifierFor: currentToken) notNil]<br>- whileTrue.<br>- self failUnless: currentToken notNil.<br>- self scanPast: #externalCallType.<br>- currentToken = '*' <br>- ifTrue: [self scanPast: #externalCallTypePointerIndicator].<br>- currentTokenFirst isDigit<br>- ifTrue: [self scanPast: #integer]<br>- ifFalse: [<br>- self failUnless: currentTokenFirst == $'.<br>- self parseString].<br>- self failUnless: currentTokenFirst == $(.<br>- self scanPast: #leftParenthesis.<br>- [currentTokenFirst ~= $)] <br>- whileTrue: [<br>- self failUnless: currentToken notNil.<br>- self scanPast: #externalCallType.<br>- currentToken = '*' <br>- ifTrue: [self scanPast: #externalCallTypePointerIndicator]].<br>- self scanPast: #rightParenthesis.<br>- currentToken = 'module:' <br>- ifTrue: [<br>- self scanPast: #module.<br>- self parseStringOrSymbol ].<br>- currentToken = 'error:' ifTrue: [<br>- self scanPast: #primitive. "there's no rangeType for error"<br>- self currentTokenType == #name<br>- ifTrue: [ self parseTemporary: #patternTempVar ]<br>- ifFalse: [ self parseStringOrSymbol ] ].<br>- self failUnless: currentToken = '>'.<br>- self scanPast: #primitiveOrExternalCallEnd!<br><br>Item was changed:<br>+ ----- Method: SHParserST80>>parsePragmaBinary (in category 'parse pragma') -----<br>- ----- Method: SHParserST80>>parsePragmaBinary (in category 'parse') -----<br> parsePragmaBinary<br> <br> self scanPast: #pragmaBinary.<br> self currentTokenType == #name<br> ifTrue:[self scanPast: (self resolvePragmaArgument: currentToken)] <br> ifFalse:[ self parseLiteral: false].<br> self failUnless: currentToken = '>'.<br> self scanPast: #primitiveOrExternalCallEnd!<br><br>Item was changed:<br>+ ----- Method: SHParserST80>>parsePragmaKeyword (in category 'parse pragma') -----<br>- ----- Method: SHParserST80>>parsePragmaKeyword (in category 'parse') -----<br> parsePragmaKeyword<br> <br> [self currentTokenType == #keyword]<br> whileTrue:[<br> self scanPast: #pragmaKeyword.<br> self currentTokenType == #name<br> ifTrue:[self scanPast: (self resolvePragmaArgument: currentToken)] <br> ifFalse:[ self parseLiteral: false]].<br> self failUnless: currentToken = '>'.<br> self scanPast: #primitiveOrExternalCallEnd!<br><br>Item was changed:<br>+ ----- Method: SHParserST80>>parsePragmaSequence (in category 'parse pragma') -----<br>- ----- Method: SHParserST80>>parsePragmaSequence (in category 'parse') -----<br> parsePragmaSequence<br>+ <br> [currentToken = '<'></'><br>+ whileTrue: [<br>- whileTrue:[<br> self scanPast: #primitiveOrExternalCallStart.<br>+ self parsePragmaStatement].!<br>- currentToken = 'primitive:' <br>- ifTrue: [<br>- self addRangeType: #primitive.<br>- self parsePrimitive]<br>- ifFalse:[<br>- self isTokenExternalFunctionCallingConvention <br>- ifTrue: [<br>- self addRangeType: #externalFunctionCallingConvention.<br>- self parseExternalCall]<br>- ifFalse:[<br>- self currentTokenType<br>- caseOf: {<br>- [ #name ] -> [ <br>- self scanPast: #pragmaUnary.<br>- self failUnless: currentToken = '>'.<br>- self scanPast: #primitiveOrExternalCallEnd ].<br>- [ #binary ] -> [ self parsePragmaBinary ].<br>- [ #keyword ] -> [ self parsePragmaKeyword ] }<br>- otherwise: [ self fail ": 'Invalid External Function Calling convention'" ] ] ] ]!<br><br>Item was added:<br>+ ----- Method: SHParserST80>>parsePragmaStatement (in category 'parse pragma') -----<br>+ parsePragmaStatement<br>+ <br>+ | parserSelector parserMethod |<br>+ currentToken last == $: ifFalse: [<br>+ "Quick exit to not break one-word pragmas such as <primitive> or <foobar>."<br>+ ^ self parsePragmaStatementKeywords].<br>+ <br>+ (self class includesSelector: (parserSelector := currentToken asSimpleGetter)) ifTrue: [<br>+ ((parserMethod := self class compiledMethodAt: parserSelector) pragmas<br>+ anySatisfy: [:pragma | pragma keyword == #pragmaParser])<br>+ ifTrue: [^ self executeMethod: parserMethod]].<br>+ <br>+ ^ self parsePragmaStatementKeywords!<br><br>Item was added:<br>+ ----- Method: SHParserST80>>parsePragmaStatementKeywords (in category 'parse pragma') -----<br>+ parsePragmaStatementKeywords<br>+ <br>+ self currentTokenType<br>+ caseOf: {<br>+ [ #name ] -> [ <br>+ self scanPast: #pragmaUnary.<br>+ self failUnless: currentToken = '>'.<br>+ self scanPast: #primitiveOrExternalCallEnd ].<br>+ [ #binary ] -> [ self parsePragmaBinary ].<br>+ [ #keyword ] -> [ self parsePragmaKeyword ] }<br>+ otherwise: [ self fail ": 'Invalid External Function Calling convention'" ]. !<br><br>Item was removed:<br>- ----- Method: SHParserST80>>parsePrimitive (in category 'parse') -----<br>- parsePrimitive<br>- <br>- self scanNext.<br>- currentTokenFirst isDigit<br>- ifTrue: [ self scanPast: #integer ]<br>- ifFalse: [<br>- self parseStringOrSymbol.<br>- currentToken = 'module:' ifTrue: [<br>- self scanPast: #module.<br>- self parseStringOrSymbol ] ].<br>- currentToken = 'error:' ifTrue: [<br>- self scanPast: #primitive. "there's no rangeType for error"<br>- self currentTokenType == #name<br>- ifTrue: [ self parseTemporary: #patternTempVar ]<br>- ifFalse: [ self parseStringOrSymbol ] ].<br>- self failUnless: currentToken = '>'.<br>- self scanPast: #primitiveOrExternalCallEnd!<br><br>Item was added:<br>+ ----- Method: SHParserST80>>primitive (in category 'parse pragma') -----<br>+ primitive<br>+ "Parse keywords and literals of primitive pragmas differently to emit different range tokens for different UI styling. This hook exists so that packages do not break primitive-pragma parsing by accident. Instead, this method needs to be replaced intentionally."<br>+ <pragmaparser><br>+ <br>+ self addRangeType: #primitive.<br>+ <br>+ self scanNext.<br>+ currentTokenFirst isDigit<br>+ ifTrue: [ self scanPast: #integer ]<br>+ ifFalse: [<br>+ self parseStringOrSymbol.<br>+ currentToken = 'module:' ifTrue: [<br>+ self scanPast: #module.<br>+ self parseStringOrSymbol ] ].<br>+ currentToken = 'error:' ifTrue: [<br>+ self scanPast: #primitive. "there's no rangeType for error"<br>+ self currentTokenType == #name<br>+ ifTrue: [ self parseTemporary: #patternTempVar ]<br>+ ifFalse: [ self parseStringOrSymbol ] ].<br>+ self failUnless: currentToken = '>'.<br>+ self scanPast: #primitiveOrExternalCallEnd!<br><br><br></pragmaparser></foobar></primitive></cdecl:></apicall:></div></blockquote>
</div></div></blockquote>
</div></div></blockquote>
</div></body>