[squeak-dev] The Trunk: Compiler-nice.218.mcz
commits at source.squeak.org
commits at source.squeak.org
Tue Oct 4 19:16:37 UTC 2011
Nicolas Cellier uploaded a new version of Compiler to project The Trunk:
http://source.squeak.org/trunk/Compiler-nice.218.mcz
==================== Summary ====================
Name: Compiler-nice.218
Author: nice
Time: 4 October 2011, 9:16:03.441 pm
UUID: 32663b2d-f5b4-4d87-bd75-22732f1821c0
Ancestors: Compiler-eem.217
Add a backward compatibility preference for allowing compilation of #, #. and other uni-character symbols.
This is necessary for loading some outdated packages.
=============== Diff against Compiler-eem.217 ===============
Item was changed:
Object subclass: #Scanner
instanceVariableNames: 'source mark hereChar aheadChar token tokenType currentComment buffer typeTable'
+ classVariableNames: 'AllowBlockArgumentAssignment AllowUnderscoreAssignments AllowUnderscoreSelectors AllowUnicharSymbol DoItCharacter TypeTable'
- classVariableNames: 'AllowBlockArgumentAssignment AllowUnderscoreAssignments AllowUnderscoreSelectors DoItCharacter TypeTable'
poolDictionaries: ''
category: 'Compiler-Kernel'!
!Scanner commentStamp: 'ul 4/3/2011 02:04' prior: 0!
I scan a string or text, picking out Smalltalk syntactic tokens. I look one character ahead. I put each token found into the instance variable, token, and its type (a Symbol) into the variable, tokenType. At the end of the input stream, I pretend to see an endless sequence of special characters called doIts.
Instance Variables
aheadChar: <Character>
buffer: <WriteStream>
currentComment: <OrderedCollection>
hereChar: <Character>
mark: <Integer>
source: <ReadStream>
token: <Symbol|String|NumberCharacter|Boolean|nil>
tokenType: <Symbol>
typeTable: <Array>
aheadChar
- the next character in the input stream
buffer
- a reusable WriteStream on a String which is used for building strings. Shouldn't be used from multiple methods without resetting.
currentComment
- an OrderedCollection of strings which contain all comments between the current token and the previous token or the beginning of the source.
hereChar
- the current character
mark
- the position of the current token in the source stream
source
- the input stream of characters
token
- the current token
tokenType
- the type of the current token. The possible token types are: #binary, #character, #colon, #doIt, #keyword, #leftArrow, #leftBrace, #leftBracket, #leftParenthesis, #literal, #period, #rightBrace, #rightBracket, #rightParenthesis, #semicolon, #string, #upArrow, #verticalBar, #word, #xBinary, #xColon, #xDelimiter, #xDigit, #xDollar, #xDoubleQuote, #xLetter, #xLitQuote, #xSingleQuote, #xUnderscore
typeTable
- an array that maps each an evaluable tokenType to each character with asciiValue between 0 and 255!
Item was added:
+ ----- Method: Scanner class>>prefAllowUnicharSymbol (in category 'preferences') -----
+ prefAllowUnicharSymbol
+ "Accessor for the system-wide preference"
+ <preference: 'Allow symbols with unique character like #,'
+ category: 'Compiler'
+ description: 'When true, the historical syntax #, #; or #. is allowed.'
+ type: #Boolean>
+ ^AllowUnicharSymbol ifNil: [false]!
Item was added:
+ ----- Method: Scanner class>>prefAllowUnicharSymbol: (in category 'preferences') -----
+ prefAllowUnicharSymbol: aBoolean
+ "Accessor for the system-wide preference"
+ AllowUnicharSymbol := aBoolean!
Item was changed:
----- Method: Scanner>>xLitQuote (in category 'multi-character scans') -----
xLitQuote
"Symbols and vectors: #(1 (4 5) 2 3) #ifTrue:ifFalse: #'abc'."
| start |
start := mark.
self step. "litQuote"
self scanToken.
tokenType == #leftParenthesis
ifTrue: [self scanToken; scanLitVec.
mark := start + 1.
tokenType == #doIt
ifTrue: [self offEnd: 'Unmatched parenthesis']]
ifFalse: [tokenType == #leftBracket
ifTrue: [self scanToken; scanLitByteVec.
mark := start + 1.
tokenType == #doIt
ifTrue: [self offEnd: 'Unmatched bracket']]
ifFalse: [(tokenType == #word or: [tokenType == #keyword or: [tokenType == #colon]])
ifTrue: [self scanLitWord]
ifFalse: [tokenType == #string
ifTrue: [token := token asSymbol]
ifFalse: [(tokenType == #binary or: [ tokenType == #verticalBar ])
+ ifFalse: [(token isCharacter and: [self class prefAllowUnicharSymbol])
+ ifTrue:
+ [tokenType := Symbol.
+ token := Symbol with: token]
+ ifFalse: [self notify: 'Invalid literal character' at: start + 1]]]]]].
- ifFalse: [self notify: 'Invalid literal character' at: start + 1]]]]].
mark := start.
tokenType := #literal
"#(Pen)
#Pen
#'Pen'
##Pen
###Pen
"!
More information about the Squeak-dev
mailing list
|