[squeak-dev] The Trunk: Compiler-nice.232.mcz

commits at source.squeak.org commits at source.squeak.org
Mon Aug 13 21:53:38 UTC 2012


Nicolas Cellier uploaded a new version of Compiler to project The Trunk:
http://source.squeak.org/trunk/Compiler-nice.232.mcz

==================== Summary ====================

Name: Compiler-nice.232
Author: nice
Time: 13 August 2012, 11:53:00.403 pm
UUID: 8ab216d0-f621-4dc4-b50a-3b7697817cb2
Ancestors: Compiler-eem.231

Patch scanAllTokenPositionsInto: to correctly scan end of method...
This will result in an empty #doIt as the last token type.

=============== Diff against Compiler-eem.231 ===============

Item was changed:
  ----- Method: Scanner>>scanAllTokenPositionsInto: (in category 'expression types') -----
  scanAllTokenPositionsInto: aBlock
  	"Evaluate aBlock with the start and end positions of all separate non-white-space tokens, including comments."
  
  	| lastMark |
  	lastMark := 1.
  	[currentComment ifNotNil:
  		[currentComment do:
  			[:cmnt| | idx |
  			 idx := source originalContents indexOfSubCollection: cmnt startingAt: lastMark.
  			 (idx > 0 and: [idx < mark]) ifTrue:
  				[aBlock value: idx - 1 value: (lastMark := idx + cmnt size)]].
  		 currentComment := nil].
  	mark ifNotNil:
  		[(token == #- 
  		  and: [(self typeTableAt: hereChar) == #xDigit]) ifTrue:
  			[| savedMark |
  			 savedMark := mark.
  			 self scanToken.
  			 token := token negated.
  			 mark := savedMark].
  		"Compensate for the fact that the parser uses two character lookahead.  Normally we must
  		  remove the extra two characters.  But this mustn't happen for the last token at the end of stream."
  		 aBlock
  			value: mark
+ 			value: source position - (aheadChar == DoItCharacter
+ 				ifTrue: [hereChar == DoItCharacter
+ 					ifTrue: [0]
+ 					ifFalse: [1]]
+ 				ifFalse: [2])].
- 			value: (source atEnd
- 					ifTrue: [tokenType := #doIt. "to cause an immediate ^self" source position]
- 					ifFalse: [source position - 2])].
  	 (tokenType == #rightParenthesis
  	  or: [tokenType == #doIt]) ifTrue:
  		[^self].
  	tokenType == #leftParenthesis
  		ifTrue: 
  			[self scanToken; scanAllTokenPositionsInto: aBlock]
  		ifFalse: 
  			[(tokenType == #word or: [tokenType == #keyword or: [tokenType == #colon]])
  				ifTrue: 
  					[self scanLitWord.
  					 token == #true ifTrue: [token := true].
  					 token == #false ifTrue: [token := false].
  					 token == #nil ifTrue: [token := nil]]
  				ifFalse:
  					[(token == #- 
  					  and: [(self typeTableAt: hereChar) == #xDigit])
  						ifTrue: 
  							[self scanToken.
  							 token := token negated]]].
  		self scanToken ] repeat!



More information about the Squeak-dev mailing list