[squeak-dev] The Inbox: ShoutCore-ul.80.mczcommits at source.squeak.org commits at source.squeak.orgSat Jun 13 22:20:46 UTC 2020
Levente Uzonyi uploaded a new version of ShoutCore to project The Inbox: http://source.squeak.org/inbox/ShoutCore-ul.80.mcz ==================== Summary ==================== Name: ShoutCore-ul.80 Author: ul Time: 14 June 2020, 12:12:00.544627 am UUID: e6ef45f1-c973-4d64-9d91-8c4a80257173 Ancestors: ShoutCore-mt.79 SHParserST80 performance tweaks: - cache size of source in a new instance variable: sourceSize - #currentChar, #nextChar and #peekChar send #at: instead of #at:ifAbsent: to source. - use sourceSize instead of source size =============== Diff against ShoutCore-mt.79 =============== Item was changed: Object subclass: #SHParserST80 + instanceVariableNames: 'classOrMetaClass source sourceSize workspace arguments sourcePosition currentToken currentTokenFirst temporaries instanceVariables errorBlock currentTokenSourcePosition bracketDepth ranges environment allowUnderscoreAssignments allowUnderscoreSelectors allowBlockArgumentAssignment parseAMethod currentTokenType context' - instanceVariableNames: 'classOrMetaClass source workspace arguments sourcePosition currentToken currentTokenFirst temporaries instanceVariables errorBlock currentTokenSourcePosition bracketDepth ranges environment allowUnderscoreAssignments allowUnderscoreSelectors allowBlockArgumentAssignment parseAMethod currentTokenType context' classVariableNames: '' poolDictionaries: '' category: 'ShoutCore-Parsing'! + !SHParserST80 commentStamp: 'ul 6/14/2020 00:05' prior: 0! - !SHParserST80 commentStamp: 'ul 7/30/2019 00:31' prior: 0! I am a Smalltalk method / expression parser. Rather than creating an Abstract Syntax Tree, I create a sequence of SHRanges (in my 'ranges' instance variable), which represent the tokens within the String I am parsing. I am used by a SHTextStylerST80 to parse method source strings. I am able to parse incomplete / incorrect methods, and so can be used to parse methods that are being edited. Instance Variables allowBlockArgumentAssignment: <Boolean> allowUnderscoreAssignments: <Boolean> allowUnderscoreSelectors: <Boolean> arguments: <OrderedCollection<OrderedCollection<String>|nil> bracketDepth: <Integer> classOrMetaClass: <Class|nil> currentToken: <String|nil> currentTokenFirst: <Character> currentTokenSourcePosition: <Integer|nil> currentTokenType: <Symbol|nil> environment: <Environment> errorBlock: <Block> instanceVariables: <Array> parseAMethod: <Boolean> ranges: <OrderedCollection<SHRange>> source: <String> sourcePosition: <Integer> + sourceSIze: <Integer> temporaries: <OrderedCollection<OrderedCollection<String>|nil> workspace: <Workspace|nil> context: <Context|nil> allowBlockArgumentAssignment The value cached at the beginning of parsing of Scanner allowBlockArgumentAssignment. allowUnderscoreAssignments The value cached at the beginning of parsing of Scanner allowUnderscoreAsAssignment. allowUnderscoreSelectors The value cached at the beginning of parsing of Scanner prefAllowUnderscoreSelectors. arguments This OrderedCollection has an element for each scope encapsulating the current scope. The current scope's arguments are stored in the last element. The first element holds the outermost scope's arguments. Each element is nil when the corresponding scope doesn't have any arguments, and the element is an OrderedCollection with the names of the arguments declared at the given scope when there's at least one. The size of this variable is the same as the size of temporaries. bracketDepth Stores the number of unclosed brackets "(" and parentheses "[" before the current sourcePosition. classOrMetaClass The Class or MetaClass instance, class and pool variables should be looked up during parsing or nil when not parsing code in the context of a class (e.g. when parsing code written in a Workspace). Having this set doesn't mean a method is being parsed. currentToken The token being analyzed for which the next range should be created for. currentTokenFirst The first character of currentToken cached for quick access or a space character when there are no more tokens to parse. Being always a Character helps avoiding extra checks. currentTokenSourcePosition The position of source the current token starts at or nil when there are no more tokens to process. currentTokenType The type of the current token calculated lazily by #currentTokenType. When it has been calculated, Its value is one of #keyword, #assignment, #ansiAssignment, #binary, #name, #other and occasionally #invalid. environment The Environment globals and classes should be looked up at during parsing when classOrMetaClass is nil. Its value is Smalltalk globals by default. errorBlock A block used to quickly stop parsing in case of an unrecoverable parse error. instanceVariables An Array with the instance variable names of classOrMetaClass or an empty Array when classOrMetaClass is nil. parseAMethod A way to tell the parser to parse source as a code snippet instead of a method. Mainly used by inspectors. ranges The SHRanges parsed by the parser. source The source code as a String to be parsed. sourcePosition souce is treated as a stream by the parser. This variable stores the stream position. + + sourceSize + The size of source cached when source is assigned. temporaries This OrderedCollection has an element for each scope encapsulating the current scope. The current scope's temporaries are stored in the last element. The first element holds the outermost scope's temporaries. Each element is nil when the corresponding scope doesn't have any temporary variables, and the element is an OrderedCollection with the names of the temporaries declared at the given scope when there's at least one. The size of this variable is the same as the size of arguments. workspace The Workspace in whose context variables should be looked up during parsing or nil when not parsing code in a workspace. context The Context in which variables should be looked up during parsing or nil when not parsing within a context. Example (explore it): ranges := SHParserST80 new classOrMetaClass: Object; source: 'testMethod ^self'; parse; ranges Benchmark (print it): SHParserST80 benchmark! Item was changed: ----- Method: SHParserST80>>currentChar (in category 'scan') ----- currentChar + + ^sourcePosition <= sourceSize ifTrue: [ source at: sourcePosition ]! - ^source at: sourcePosition ifAbsent: nil! Item was changed: ----- Method: SHParserST80>>fail (in category 'error handling') ----- fail | start | start := (ranges isEmpty ifTrue: [ 1 ] ifFalse: [ ranges last end + 1 ]). + start <= sourceSize ifTrue: [ - start <= source size ifTrue: [ self addRangeType: #excessCode start: start + end: sourceSize ]. - end: source size ]. errorBlock value! Item was changed: ----- Method: SHParserST80>>initializeVariablesFromContext (in category 'parse support') ----- initializeVariablesFromContext | contextSourcePcIndex contextSourceParser | contextSourcePcIndex := (context debuggerMap rangeForPC: (context isDead ifTrue: [context endPC] ifFalse: [context pc]) in: context method contextIsActiveContext: true "... to really use the context's pc.") start. contextSourceParser := self class new classOrMetaClass: context method methodClass; environment: self environment; + source: (context method getSource asString first: contextSourcePcIndex); - source: (context method getSource first: contextSourcePcIndex); yourself. contextSourceParser parse. arguments := contextSourceParser activeArguments. temporaries := contextSourceParser activeTemporaries.! Item was changed: ----- Method: SHParserST80>>nextChar (in category 'scan') ----- nextChar + ^(sourcePosition := sourcePosition + 1) <= sourceSize + ifTrue: [ source at: sourcePosition ] + ifFalse: [ $ ]! - ^source at: (sourcePosition := sourcePosition + 1) ifAbsent: $ ! Item was changed: ----- Method: SHParserST80>>parseString (in category 'parse') ----- parseString | stringStart | stringStart := sourcePosition - 1. [ (sourcePosition := source indexOf: $' startingAt: sourcePosition) = 0 ifTrue: [ self + addRangeType: #unfinishedString start: stringStart end: sourceSize; - addRangeType: #unfinishedString start: stringStart end: source size; fail ": 'unfinished string'"]. self peekChar == $' ifTrue: [ sourcePosition := sourcePosition + 1. true ] ifFalse: [ false ] ] whileTrue: [ sourcePosition := sourcePosition + 1 ]. sourcePosition := sourcePosition + 1. self scanPast: #string start: stringStart end: sourcePosition - 1! Item was changed: ----- Method: SHParserST80>>parseSymbolString (in category 'parse') ----- parseSymbolString | first c last | first := sourcePosition. self nextChar. [(c := self currentChar) ifNil: [ + self addRangeType: #unfinishedString start: first end: sourceSize. - self addRangeType: #unfinishedString start: first end: source size. self fail ": 'unfinished string'"]. c ~~ $' or: [ self peekChar == $' ifTrue: [sourcePosition := sourcePosition + 1.true] ifFalse: [false]] ] whileTrue: [sourcePosition := sourcePosition + 1]. last := sourcePosition. self nextChar; scanPast: #stringSymbol start: first - 1 end: last! Item was changed: ----- Method: SHParserST80>>peekChar (in category 'scan') ----- peekChar + ^sourcePosition < sourceSize + ifTrue: [ source at: sourcePosition + 1 ] + ifFalse: [ $ ]! - ^source at: sourcePosition + 1 ifAbsent: $ ! Item was changed: ----- Method: SHParserST80>>scanComment (in category 'scan') ----- scanComment | start | start := sourcePosition. (sourcePosition := source indexOf: $" startingAt: start + 1) = 0 ifTrue: [ + sourcePosition := sourceSize + 1. - sourcePosition := source size + 1. ^self + addRangeType: #unfinishedComment start: start end: sourceSize; - addRangeType: #unfinishedComment start: start end: source size; fail ]. self addRangeType: #comment start: start end: sourcePosition; nextChar; scanWhitespace! Item was changed: ----- Method: SHParserST80>>scanWhitespace (in category 'scan') ----- scanWhitespace (self currentChar ifNil: [ ^self ]) isSeparator ifTrue: [ sourcePosition := source indexOfAnyOf: CharacterSet nonSeparators startingAt: sourcePosition + 1. sourcePosition = 0 ifTrue: [ "Not found" + sourcePosition := sourceSize + 1 ] ]. - sourcePosition := source size + 1 ] ]. self currentChar == $" ifTrue: [ self scanComment ]! Item was changed: ----- Method: SHParserST80>>source: (in category 'accessing') ----- source: aStringOrText + source := aStringOrText asString. + sourceSize := source size! - source := aStringOrText asString!
More information about the Squeak-dev mailing list |