[squeak-dev] The Trunk: Compiler-nice.218.mcz

commits at source.squeak.org commits at source.squeak.org
Tue Oct 4 19:16:37 UTC 2011


Nicolas Cellier uploaded a new version of Compiler to project The Trunk:
http://source.squeak.org/trunk/Compiler-nice.218.mcz

==================== Summary ====================

Name: Compiler-nice.218
Author: nice
Time: 4 October 2011, 9:16:03.441 pm
UUID: 32663b2d-f5b4-4d87-bd75-22732f1821c0
Ancestors: Compiler-eem.217

Add a backward compatibility preference for allowing compilation of #, #. and other uni-character symbols.
This is necessary for loading some outdated packages.

=============== Diff against Compiler-eem.217 ===============

Item was changed:
  Object subclass: #Scanner
  	instanceVariableNames: 'source mark hereChar aheadChar token tokenType currentComment buffer typeTable'
+ 	classVariableNames: 'AllowBlockArgumentAssignment AllowUnderscoreAssignments AllowUnderscoreSelectors AllowUnicharSymbol DoItCharacter TypeTable'
- 	classVariableNames: 'AllowBlockArgumentAssignment AllowUnderscoreAssignments AllowUnderscoreSelectors DoItCharacter TypeTable'
  	poolDictionaries: ''
  	category: 'Compiler-Kernel'!
  
  !Scanner commentStamp: 'ul 4/3/2011 02:04' prior: 0!
  I scan a string or text, picking out Smalltalk syntactic tokens. I look one character ahead. I put each token found into the instance variable, token, and its type (a Symbol) into the variable, tokenType. At the end of the input stream, I pretend to see an endless sequence of special characters called doIts.
  
  Instance Variables
  	aheadChar:		<Character>
  	buffer:		<WriteStream>
  	currentComment:		<OrderedCollection>
  	hereChar:		<Character>
  	mark:		<Integer>
  	source:		<ReadStream>
  	token:		<Symbol|String|NumberCharacter|Boolean|nil>
  	tokenType:		<Symbol>
  	typeTable:		<Array>
  
  aheadChar
  	- the next character in the input stream
  
  buffer
  	- a reusable WriteStream on a String which is used for building strings. Shouldn't be used from multiple methods without resetting.
  
  currentComment
  	- an OrderedCollection of strings which contain all comments between the current token and the previous token or the beginning of the source.
  
  hereChar
  	- the current character
  
  mark
  	- the position of the current token in the source stream
  
  source
  	- the input stream of characters
  
  token
  	- the current token
  
  tokenType
  	- the type of the current token. The possible token types are: #binary, #character, #colon, #doIt, #keyword, #leftArrow, #leftBrace, #leftBracket, #leftParenthesis, #literal, #period, #rightBrace, #rightBracket, #rightParenthesis, #semicolon, #string, #upArrow, #verticalBar, #word, #xBinary, #xColon, #xDelimiter, #xDigit, #xDollar, #xDoubleQuote, #xLetter, #xLitQuote, #xSingleQuote, #xUnderscore
  
  typeTable
  	- an array that maps each an evaluable tokenType to each character with asciiValue between 0 and 255!

Item was added:
+ ----- Method: Scanner class>>prefAllowUnicharSymbol (in category 'preferences') -----
+ prefAllowUnicharSymbol
+ 	"Accessor for the system-wide preference"
+ 	<preference: 'Allow symbols with unique character like #,'
+ 		category: 'Compiler'
+ 		description: 'When true, the historical syntax #, #; or #. is allowed.'
+ 		type: #Boolean>
+ 	^AllowUnicharSymbol ifNil: [false]!

Item was added:
+ ----- Method: Scanner class>>prefAllowUnicharSymbol: (in category 'preferences') -----
+ prefAllowUnicharSymbol: aBoolean
+ 	"Accessor for the system-wide preference"
+ 	AllowUnicharSymbol := aBoolean!

Item was changed:
  ----- Method: Scanner>>xLitQuote (in category 'multi-character scans') -----
  xLitQuote
  	"Symbols and vectors: #(1 (4 5) 2 3) #ifTrue:ifFalse: #'abc'."
  	| start |
  	start := mark.
  	self step. "litQuote"
  	self scanToken.
  	tokenType == #leftParenthesis
  		ifTrue: [self scanToken; scanLitVec.
  			mark := start + 1.
  			tokenType == #doIt
  				ifTrue: [self offEnd: 'Unmatched parenthesis']]
  		ifFalse: [tokenType == #leftBracket
  				ifTrue: [self scanToken; scanLitByteVec.
  					mark := start + 1.
  					tokenType == #doIt
  						ifTrue: [self offEnd: 'Unmatched bracket']]
  				ifFalse: [(tokenType == #word or: [tokenType == #keyword or: [tokenType == #colon]])
  						ifTrue: [self scanLitWord]
  						ifFalse: [tokenType == #string
  							ifTrue: [token := token asSymbol]
  							ifFalse: [(tokenType == #binary or: [ tokenType == #verticalBar ]) 
+ 								ifFalse: [(token isCharacter and: [self class prefAllowUnicharSymbol])
+ 									ifTrue:
+ 										[tokenType := Symbol.
+ 										token := Symbol with: token]
+ 									ifFalse: [self notify: 'Invalid literal character' at: start + 1]]]]]].
- 								ifFalse: [self notify: 'Invalid literal character' at: start + 1]]]]].
  	mark := start.
  	tokenType := #literal
  
  	"#(Pen)
  	#Pen
  	#'Pen'
  	##Pen
  	###Pen
  	"!




More information about the Squeak-dev mailing list