lists.squeakfoundation.org
Sign In
Sign Up
Sign In
Sign Up
Manage this list
×
Keyboard Shortcuts
Thread View
j
: Next unread message
k
: Previous unread message
j a
: Jump to all threads
j l
: Jump to MailingList overview
2024
May
April
March
February
January
2023
December
November
October
September
August
July
June
May
April
March
February
January
2022
December
November
October
September
August
July
June
May
April
March
February
January
2021
December
November
October
September
August
July
June
May
April
March
February
January
2020
December
November
October
September
August
July
June
May
April
March
February
January
2019
December
November
October
September
August
July
June
May
April
March
February
January
2018
December
November
October
September
August
July
June
May
April
March
February
January
2017
December
November
October
September
August
July
June
May
April
March
February
January
2016
December
November
October
September
August
July
June
May
April
March
February
January
2015
December
November
October
September
August
July
June
May
April
March
February
January
2014
December
November
October
September
August
July
June
May
April
March
February
January
2013
December
November
October
September
August
July
June
May
April
March
February
January
2012
December
November
October
September
August
July
June
May
April
March
February
January
2011
December
November
October
September
August
July
June
May
April
March
February
January
2010
December
November
October
September
August
July
June
May
April
March
February
January
2009
December
November
October
September
August
July
June
May
April
March
February
January
2008
December
November
October
September
August
July
June
May
April
March
February
January
2007
December
November
October
September
August
July
June
May
April
March
February
January
2006
December
November
October
September
August
July
June
May
April
March
February
January
2005
December
November
October
September
August
July
June
May
April
March
February
List overview
Download
Packages
April 2011
----- 2024 -----
May 2024
April 2024
March 2024
February 2024
January 2024
----- 2023 -----
December 2023
November 2023
October 2023
September 2023
August 2023
July 2023
June 2023
May 2023
April 2023
March 2023
February 2023
January 2023
----- 2022 -----
December 2022
November 2022
October 2022
September 2022
August 2022
July 2022
June 2022
May 2022
April 2022
March 2022
February 2022
January 2022
----- 2021 -----
December 2021
November 2021
October 2021
September 2021
August 2021
July 2021
June 2021
May 2021
April 2021
March 2021
February 2021
January 2021
----- 2020 -----
December 2020
November 2020
October 2020
September 2020
August 2020
July 2020
June 2020
May 2020
April 2020
March 2020
February 2020
January 2020
----- 2019 -----
December 2019
November 2019
October 2019
September 2019
August 2019
July 2019
June 2019
May 2019
April 2019
March 2019
February 2019
January 2019
----- 2018 -----
December 2018
November 2018
October 2018
September 2018
August 2018
July 2018
June 2018
May 2018
April 2018
March 2018
February 2018
January 2018
----- 2017 -----
December 2017
November 2017
October 2017
September 2017
August 2017
July 2017
June 2017
May 2017
April 2017
March 2017
February 2017
January 2017
----- 2016 -----
December 2016
November 2016
October 2016
September 2016
August 2016
July 2016
June 2016
May 2016
April 2016
March 2016
February 2016
January 2016
----- 2015 -----
December 2015
November 2015
October 2015
September 2015
August 2015
July 2015
June 2015
May 2015
April 2015
March 2015
February 2015
January 2015
----- 2014 -----
December 2014
November 2014
October 2014
September 2014
August 2014
July 2014
June 2014
May 2014
April 2014
March 2014
February 2014
January 2014
----- 2013 -----
December 2013
November 2013
October 2013
September 2013
August 2013
July 2013
June 2013
May 2013
April 2013
March 2013
February 2013
January 2013
----- 2012 -----
December 2012
November 2012
October 2012
September 2012
August 2012
July 2012
June 2012
May 2012
April 2012
March 2012
February 2012
January 2012
----- 2011 -----
December 2011
November 2011
October 2011
September 2011
August 2011
July 2011
June 2011
May 2011
April 2011
March 2011
February 2011
January 2011
----- 2010 -----
December 2010
November 2010
October 2010
September 2010
August 2010
July 2010
June 2010
May 2010
April 2010
March 2010
February 2010
January 2010
----- 2009 -----
December 2009
November 2009
October 2009
September 2009
August 2009
July 2009
June 2009
May 2009
April 2009
March 2009
February 2009
January 2009
----- 2008 -----
December 2008
November 2008
October 2008
September 2008
August 2008
July 2008
June 2008
May 2008
April 2008
March 2008
February 2008
January 2008
----- 2007 -----
December 2007
November 2007
October 2007
September 2007
August 2007
July 2007
June 2007
May 2007
April 2007
March 2007
February 2007
January 2007
----- 2006 -----
December 2006
November 2006
October 2006
September 2006
August 2006
July 2006
June 2006
May 2006
April 2006
March 2006
February 2006
January 2006
----- 2005 -----
December 2005
November 2005
October 2005
September 2005
August 2005
July 2005
June 2005
May 2005
April 2005
March 2005
February 2005
packages@lists.squeakfoundation.org
1 participants
507 discussions
Start a n
N
ew thread
The Trunk: Morphic-ul.527.mcz
by commits@source.squeak.org
02 Apr '11
02 Apr '11
Levente Uzonyi uploaded a new version of Morphic to project The Trunk:
http://source.squeak.org/trunk/Morphic-ul.527.mcz
==================== Summary ==================== Name: Morphic-ul.527 Author: ul Time: 2 April 2011, 1:36:51.526 am UUID: d571ac0f-9ba7-3042-8746-7222f60f5cb5 Ancestors: Morphic-nice.526, Morphic-ul.526 merged =============== Diff against Morphic-nice.526 =============== Item was added: + ----- Method: TextEditor>>model (in category 'accessing') ----- + model + + ^model! Item was changed: ----- Method: TextEditor>>referencesToIt (in category 'menu messages') ----- referencesToIt "Open a references browser on the selected symbol" + | symbol environment binding | - | aSymbol | self selectLine. + symbol := self selectedSymbol ifNil: [ ^morph flash ]. + environment := (model respondsTo: #selectedClassOrMetaClass) + ifTrue: [ model selectedClassOrMetaClass ifNil: [ Smalltalk globals ] ] + ifFalse: [ Smalltalk globals ]. + binding := (environment bindingOf: symbol) ifNil: [ ^morph flash ]. + self systemNavigation browseAllCallsOn: binding! - ((aSymbol := self selectedSymbol) == nil or: - [(Smalltalk globals includesKey: aSymbol) not]) - ifTrue: [^ morph flash]. - - self systemNavigation browseAllCallsOn: (Smalltalk globals associationAt: self selectedSymbol)!
1
0
0
0
The Trunk: Morphic-ul.527.mcz
by commits@source.squeak.org
02 Apr '11
02 Apr '11
Levente Uzonyi uploaded a new version of Morphic to project The Trunk:
http://source.squeak.org/trunk/Morphic-ul.527.mcz
==================== Summary ==================== Name: Morphic-ul.527 Author: ul Time: 2 April 2011, 1:36:51.526 am UUID: d571ac0f-9ba7-3042-8746-7222f60f5cb5 Ancestors: Morphic-nice.526, Morphic-ul.526 merged =============== Diff against Morphic-nice.526 =============== Item was added: + ----- Method: TextEditor>>model (in category 'accessing') ----- + model + + ^model! Item was changed: ----- Method: TextEditor>>referencesToIt (in category 'menu messages') ----- referencesToIt "Open a references browser on the selected symbol" + | symbol environment binding | - | aSymbol | self selectLine. + symbol := self selectedSymbol ifNil: [ ^morph flash ]. + environment := (model respondsTo: #selectedClassOrMetaClass) + ifTrue: [ model selectedClassOrMetaClass ifNil: [ Smalltalk globals ] ] + ifFalse: [ Smalltalk globals ]. + binding := (environment bindingOf: symbol) ifNil: [ ^morph flash ]. + self systemNavigation browseAllCallsOn: binding! - ((aSymbol := self selectedSymbol) == nil or: - [(Smalltalk globals includesKey: aSymbol) not]) - ifTrue: [^ morph flash]. - - self systemNavigation browseAllCallsOn: (Smalltalk globals associationAt: self selectedSymbol)!
1
0
0
0
The Trunk: Compiler-ul.204.mcz
by commits@source.squeak.org
02 Apr '11
02 Apr '11
Levente Uzonyi uploaded a new version of Compiler to project The Trunk:
http://source.squeak.org/trunk/Compiler-ul.204.mcz
==================== Summary ==================== Name: Compiler-ul.204 Author: ul Time: 2 April 2011, 12:53:54.869 am UUID: 9a4461b2-a9c8-1a4e-82bc-40d15ff7ac7c Ancestors: Compiler-ul.203 Scanner changes: - added a new class variable DoItCharacter. All checks for doIt characters (30) should use this. (part 2) - reuse buffer in #xDoubleQuote instead of a custom stream =============== Diff against Compiler-ul.203 =============== Item was changed: ----- Method: Scanner>>scanToken (in category 'expression types') ----- scanToken [(tokenType := self typeTableAt: hereChar) == #xDelimiter] whileTrue: [self step]. "Skip delimiters fast, there almost always is one." + mark := (aheadChar == DoItCharacter and: [source atEnd]) - mark := (aheadChar asciiValue = 30 and: [source atEnd]) ifTrue: [source position] ifFalse: [source position - 1]. (tokenType at: 1) == $x "x as first letter" ifTrue: [self perform: tokenType "means perform to compute token & type"] ifFalse: [token := self step asSymbol "else just unique the first char"]. ^token! Item was changed: ----- Method: Scanner>>step (in category 'expression types') ----- step | c | c := hereChar. hereChar := aheadChar. source atEnd + ifTrue: [aheadChar := DoItCharacter] - ifTrue: [aheadChar := 30 asCharacter "doit"] ifFalse: [aheadChar := source next]. ^c! Item was changed: ----- Method: Scanner>>xDigit (in category 'multi-character scans') ----- xDigit "Form a number." tokenType := #number. + (aheadChar == DoItCharacter and: [source atEnd + and: [source skip: -1. source next ~~ DoItCharacter]]) - (aheadChar asciiValue = 30 and: [source atEnd - and: [source skip: -1. source next asciiValue ~= 30]]) ifTrue: [source skip: -1 "Read off the end last time"] ifFalse: [source skip: -2]. token := (SqNumberParser on: source) failBlock: [:errorString :position | self notify: errorString at:position]; nextNumber. self step; step! Item was changed: ----- Method: Scanner>>xDoubleQuote (in category 'multi-character scans') ----- xDoubleQuote "Collect a comment." "wod 1/10/98: Allow 'empty' comments by testing the first character for $"" rather than blindly adding it to the comment being collected." + + buffer reset. + self step. + [ hereChar == $" ] whileFalse: [ + (hereChar == DoItCharacter and: [ source atEnd ]) ifTrue: [ + ^self offEnd: 'Unmatched comment quote' ]. + buffer nextPut: self step ]. + self step. + (currentComment ifNil: [ + currentComment := OrderedCollection new ]) + add: buffer contents. + self scanToken! - | aStream stopChar | - stopChar := 30 asCharacter. - aStream := WriteStream on: (String new: 200). - self step. - [hereChar == $"] - whileFalse: - [(hereChar == stopChar and: [source atEnd]) - ifTrue: [^self offEnd: 'Unmatched comment quote']. - aStream nextPut: self step.]. - self step. - currentComment == nil - ifTrue: [currentComment := OrderedCollection with: aStream - contents] - ifFalse: [currentComment add: aStream contents]. - self scanToken! Item was changed: ----- Method: Scanner>>xSingleQuote (in category 'multi-character scans') ----- xSingleQuote "String." self step. buffer reset. [hereChar == $' and: [aheadChar == $' ifTrue: [self step. false] ifFalse: [true]]] whileFalse: [buffer nextPut: self step. + (hereChar == DoItCharacter and: [source atEnd]) - (hereChar asciiValue = 30 and: [source atEnd]) ifTrue: [^self offEnd: 'Unmatched string quote']]. self step. token := buffer contents. tokenType := #string!
1
0
0
0
The Trunk: Compiler-ul.204.mcz
by commits@source.squeak.org
02 Apr '11
02 Apr '11
Levente Uzonyi uploaded a new version of Compiler to project The Trunk:
http://source.squeak.org/trunk/Compiler-ul.204.mcz
==================== Summary ==================== Name: Compiler-ul.204 Author: ul Time: 2 April 2011, 12:53:54.869 am UUID: 9a4461b2-a9c8-1a4e-82bc-40d15ff7ac7c Ancestors: Compiler-ul.203 Scanner changes: - added a new class variable DoItCharacter. All checks for doIt characters (30) should use this. (part 2) - reuse buffer in #xDoubleQuote instead of a custom stream =============== Diff against Compiler-ul.203 =============== Item was changed: ----- Method: Scanner>>scanToken (in category 'expression types') ----- scanToken [(tokenType := self typeTableAt: hereChar) == #xDelimiter] whileTrue: [self step]. "Skip delimiters fast, there almost always is one." + mark := (aheadChar == DoItCharacter and: [source atEnd]) - mark := (aheadChar asciiValue = 30 and: [source atEnd]) ifTrue: [source position] ifFalse: [source position - 1]. (tokenType at: 1) == $x "x as first letter" ifTrue: [self perform: tokenType "means perform to compute token & type"] ifFalse: [token := self step asSymbol "else just unique the first char"]. ^token! Item was changed: ----- Method: Scanner>>step (in category 'expression types') ----- step | c | c := hereChar. hereChar := aheadChar. source atEnd + ifTrue: [aheadChar := DoItCharacter] - ifTrue: [aheadChar := 30 asCharacter "doit"] ifFalse: [aheadChar := source next]. ^c! Item was changed: ----- Method: Scanner>>xDigit (in category 'multi-character scans') ----- xDigit "Form a number." tokenType := #number. + (aheadChar == DoItCharacter and: [source atEnd + and: [source skip: -1. source next ~~ DoItCharacter]]) - (aheadChar asciiValue = 30 and: [source atEnd - and: [source skip: -1. source next asciiValue ~= 30]]) ifTrue: [source skip: -1 "Read off the end last time"] ifFalse: [source skip: -2]. token := (SqNumberParser on: source) failBlock: [:errorString :position | self notify: errorString at:position]; nextNumber. self step; step! Item was changed: ----- Method: Scanner>>xDoubleQuote (in category 'multi-character scans') ----- xDoubleQuote "Collect a comment." "wod 1/10/98: Allow 'empty' comments by testing the first character for $"" rather than blindly adding it to the comment being collected." + + buffer reset. + self step. + [ hereChar == $" ] whileFalse: [ + (hereChar == DoItCharacter and: [ source atEnd ]) ifTrue: [ + ^self offEnd: 'Unmatched comment quote' ]. + buffer nextPut: self step ]. + self step. + (currentComment ifNil: [ + currentComment := OrderedCollection new ]) + add: buffer contents. + self scanToken! - | aStream stopChar | - stopChar := 30 asCharacter. - aStream := WriteStream on: (String new: 200). - self step. - [hereChar == $"] - whileFalse: - [(hereChar == stopChar and: [source atEnd]) - ifTrue: [^self offEnd: 'Unmatched comment quote']. - aStream nextPut: self step.]. - self step. - currentComment == nil - ifTrue: [currentComment := OrderedCollection with: aStream - contents] - ifFalse: [currentComment add: aStream contents]. - self scanToken! Item was changed: ----- Method: Scanner>>xSingleQuote (in category 'multi-character scans') ----- xSingleQuote "String." self step. buffer reset. [hereChar == $' and: [aheadChar == $' ifTrue: [self step. false] ifFalse: [true]]] whileFalse: [buffer nextPut: self step. + (hereChar == DoItCharacter and: [source atEnd]) - (hereChar asciiValue = 30 and: [source atEnd]) ifTrue: [^self offEnd: 'Unmatched string quote']]. self step. token := buffer contents. tokenType := #string!
1
0
0
0
The Trunk: Compiler-ul.204.mcz
by commits@source.squeak.org
02 Apr '11
02 Apr '11
Levente Uzonyi uploaded a new version of Compiler to project The Trunk:
http://source.squeak.org/trunk/Compiler-ul.204.mcz
==================== Summary ==================== Name: Compiler-ul.204 Author: ul Time: 2 April 2011, 12:53:54.869 am UUID: 9a4461b2-a9c8-1a4e-82bc-40d15ff7ac7c Ancestors: Compiler-ul.203 Scanner changes: - added a new class variable DoItCharacter. All checks for doIt characters (30) should use this. (part 2) - reuse buffer in #xDoubleQuote instead of a custom stream =============== Diff against Compiler-ul.203 =============== Item was changed: ----- Method: Scanner>>scanToken (in category 'expression types') ----- scanToken [(tokenType := self typeTableAt: hereChar) == #xDelimiter] whileTrue: [self step]. "Skip delimiters fast, there almost always is one." + mark := (aheadChar == DoItCharacter and: [source atEnd]) - mark := (aheadChar asciiValue = 30 and: [source atEnd]) ifTrue: [source position] ifFalse: [source position - 1]. (tokenType at: 1) == $x "x as first letter" ifTrue: [self perform: tokenType "means perform to compute token & type"] ifFalse: [token := self step asSymbol "else just unique the first char"]. ^token! Item was changed: ----- Method: Scanner>>step (in category 'expression types') ----- step | c | c := hereChar. hereChar := aheadChar. source atEnd + ifTrue: [aheadChar := DoItCharacter] - ifTrue: [aheadChar := 30 asCharacter "doit"] ifFalse: [aheadChar := source next]. ^c! Item was changed: ----- Method: Scanner>>xDigit (in category 'multi-character scans') ----- xDigit "Form a number." tokenType := #number. + (aheadChar == DoItCharacter and: [source atEnd + and: [source skip: -1. source next ~~ DoItCharacter]]) - (aheadChar asciiValue = 30 and: [source atEnd - and: [source skip: -1. source next asciiValue ~= 30]]) ifTrue: [source skip: -1 "Read off the end last time"] ifFalse: [source skip: -2]. token := (SqNumberParser on: source) failBlock: [:errorString :position | self notify: errorString at:position]; nextNumber. self step; step! Item was changed: ----- Method: Scanner>>xDoubleQuote (in category 'multi-character scans') ----- xDoubleQuote "Collect a comment." "wod 1/10/98: Allow 'empty' comments by testing the first character for $"" rather than blindly adding it to the comment being collected." + + buffer reset. + self step. + [ hereChar == $" ] whileFalse: [ + (hereChar == DoItCharacter and: [ source atEnd ]) ifTrue: [ + ^self offEnd: 'Unmatched comment quote' ]. + buffer nextPut: self step ]. + self step. + (currentComment ifNil: [ + currentComment := OrderedCollection new ]) + add: buffer contents. + self scanToken! - | aStream stopChar | - stopChar := 30 asCharacter. - aStream := WriteStream on: (String new: 200). - self step. - [hereChar == $"] - whileFalse: - [(hereChar == stopChar and: [source atEnd]) - ifTrue: [^self offEnd: 'Unmatched comment quote']. - aStream nextPut: self step.]. - self step. - currentComment == nil - ifTrue: [currentComment := OrderedCollection with: aStream - contents] - ifFalse: [currentComment add: aStream contents]. - self scanToken! Item was changed: ----- Method: Scanner>>xSingleQuote (in category 'multi-character scans') ----- xSingleQuote "String." self step. buffer reset. [hereChar == $' and: [aheadChar == $' ifTrue: [self step. false] ifFalse: [true]]] whileFalse: [buffer nextPut: self step. + (hereChar == DoItCharacter and: [source atEnd]) - (hereChar asciiValue = 30 and: [source atEnd]) ifTrue: [^self offEnd: 'Unmatched string quote']]. self step. token := buffer contents. tokenType := #string!
1
0
0
0
The Trunk: Compiler-ul.203.mcz
by commits@source.squeak.org
02 Apr '11
02 Apr '11
Levente Uzonyi uploaded a new version of Compiler to project The Trunk:
http://source.squeak.org/trunk/Compiler-ul.203.mcz
==================== Summary ==================== Name: Compiler-ul.203 Author: ul Time: 2 April 2011, 12:50:39.599 am UUID: 9817f32d-943f-4745-81b8-0ecaeec259d4 Ancestors: Compiler-ul.202 Scanner changes: - introduced #character type for characters instead of #number (part 2) - updated comment - added a new class variable DoItCharacter. All checks for doIt characters (30) should use this. (part 1) - moved TypeTable initialization to a separate method - instance creation uses the common #initialize method instead of #initScanner - removed #initScanner and the class side #new - use #== instead of #= for symbol and character comparison - use #repeat instead of true and #whileTrue for loops - use #and: instead of #& =============== Diff against Compiler-ul.202 =============== Item was changed: Object subclass: #Scanner instanceVariableNames: 'source mark hereChar aheadChar token tokenType currentComment buffer typeTable' + classVariableNames: 'AllowUnderscoreAssignments AllowUnderscoreSelectors DoItCharacter TypeTable' - classVariableNames: 'AllowUnderscoreAssignments AllowUnderscoreSelectors TypeTable' poolDictionaries: '' category: 'Compiler-Kernel'! + !Scanner commentStamp: 'ul 3/27/2011 23:24' prior: 0! + I scan a string or text, picking out Smalltalk syntactic tokens. I look one character ahead. I put each token found into the instance variable, token, and its type (a Symbol) into the variable, tokenType. At the end of the input stream, I pretend to see an endless sequence of special characters called doIts. + + Instance Variables + aheadChar: <Character> + buffer: <WriteStream> + currentComment: <OrderedCollection> + hereChar: <Character> + mark: <Integer> + source: <ReadStream> + token: <Symbol|String|NumberCharacter|Boolean|nil> + tokenType: <Symbol> + typeTable: <Array> + + aheadChar + - the next character in the input stream + + buffer + - a reusable WriteStream on a String which is used for building strings. Shouldn't be used from multiple methods without resetting. + + currentComment + - an OrderedCollection of strings which contain all comments between the current token and the previous token or the beginning of the source. + + hereChar + - the current character + + mark + - the position of the current token in the source stream + + source + - xxxxx + + token + - the current token + + tokenType + - the type of the current token. The possible token types are: #binary, #character, #colon, #doIt, #keyword, #leftArrow, #leftBrace, #leftBracket, #leftParenthesis, #literal, #period, #rightBrace, #rightBracket, #rightParenthesis, #semicolon, #string, #upArrow, #verticalBar, #word, #xBinary, #xColon, #xDelimiter, #xDigit, #xDollar, #xDoubleQuote, #xLetter, #xLitQuote, #xSingleQuote, #xUnderscore + + typeTable + - an array that maps each an evaluable tokenType to each character with asciiValue between 0 and 255! - !Scanner commentStamp: '<historical>' prior: 0! - I scan a string or text, picking out Smalltalk syntactic tokens. I look one character ahead. I put each token found into the instance variable, token, and its type (a Symbol) into the variable, tokenType. At the end of the input stream, I pretend to see an endless sequence of special characters called doits.! Item was changed: ----- Method: Scanner class>>initialize (in category 'initialization') ----- initialize + + self initializeTypeTable. + DoItCharacter := Character value: 30! - | newTable | - newTable := Array new: 256 withAll: #xBinary. "default" - newTable atAll: #(9 10 12 13 32 ) put: #xDelimiter. "tab lf ff cr space" - newTable atAll: ($0 asciiValue to: $9 asciiValue) put: #xDigit. - - 1 to: 255 - do: [:index | - (Character value: index) isLetter - ifTrue: [newTable at: index put: #xLetter]]. - - newTable at: 30 put: #doIt. - newTable at: $" asciiValue put: #xDoubleQuote. - newTable at: $# asciiValue put: #xLitQuote. - newTable at: $$ asciiValue put: #xDollar. - newTable at: $' asciiValue put: #xSingleQuote. - newTable at: $: asciiValue put: #xColon. - newTable at: $( asciiValue put: #leftParenthesis. - newTable at: $) asciiValue put: #rightParenthesis. - newTable at: $. asciiValue put: #period. - newTable at: $; asciiValue put: #semicolon. - newTable at: $[ asciiValue put: #leftBracket. - newTable at: $] asciiValue put: #rightBracket. - newTable at: ${ asciiValue put: #leftBrace. - newTable at: $} asciiValue put: #rightBrace. - newTable at: $^ asciiValue put: #upArrow. - newTable at: $_ asciiValue put: #xUnderscore. - newTable at: $| asciiValue put: #verticalBar. - TypeTable := newTable "bon voyage!!" - - "Scanner initialize"! Item was added: + ----- Method: Scanner class>>initializeTypeTable (in category 'initialization') ----- + initializeTypeTable + "self initializeTypeTable" + + | newTable | + newTable := Array new: 256 withAll: #xBinary. "default" + newTable atAll: #(9 10 12 13 32 ) put: #xDelimiter. "tab lf ff cr space" + newTable atAll: ($0 asciiValue to: $9 asciiValue) put: #xDigit. + + 1 to: 255 + do: [:index | + (Character value: index) isLetter + ifTrue: [newTable at: index put: #xLetter]]. + + newTable at: 30 put: #doIt. + newTable at: $" asciiValue put: #xDoubleQuote. + newTable at: $# asciiValue put: #xLitQuote. + newTable at: $$ asciiValue put: #xDollar. + newTable at: $' asciiValue put: #xSingleQuote. + newTable at: $: asciiValue put: #xColon. + newTable at: $( asciiValue put: #leftParenthesis. + newTable at: $) asciiValue put: #rightParenthesis. + newTable at: $. asciiValue put: #period. + newTable at: $; asciiValue put: #semicolon. + newTable at: $[ asciiValue put: #leftBracket. + newTable at: $] asciiValue put: #rightBracket. + newTable at: ${ asciiValue put: #leftBrace. + newTable at: $} asciiValue put: #rightBrace. + newTable at: $^ asciiValue put: #upArrow. + newTable at: $_ asciiValue put: #xUnderscore. + newTable at: $| asciiValue put: #verticalBar. + TypeTable := newTable "bon voyage!!"! Item was removed: - ----- Method: Scanner class>>new (in category 'instance creation') ----- - new - - ^self basicNew initScanner! Item was removed: - ----- Method: Scanner>>initScanner (in category 'initialize-release') ----- - initScanner - - buffer := WriteStream on: (String new: 40). - typeTable := TypeTable! Item was added: + ----- Method: Scanner>>initialize (in category 'initialize-release') ----- + initialize + + super initialize. + buffer := WriteStream on: (String new: 40). + typeTable := TypeTable! Item was changed: ----- Method: Scanner>>scanAllTokenPositionsInto: (in category 'expression types') ----- scanAllTokenPositionsInto: aBlock "Evaluate aBlock with the start and end positions of all separate non-white-space tokens, including comments." | lastMark | lastMark := 1. [currentComment ifNotNil: [currentComment do: [:cmnt| | idx | idx := source originalContents indexOfSubCollection: cmnt startingAt: lastMark. (idx > 0 and: [idx < mark]) ifTrue: [aBlock value: idx - 1 value: (lastMark := idx + cmnt size)]]. currentComment := nil]. mark ifNotNil: [(token == #- and: [(self typeTableAt: hereChar) == #xDigit]) ifTrue: [| savedMark | savedMark := mark. self scanToken. token := token negated. mark := savedMark]. "Compensate for the fact that the parser uses two character lookahead. Normally we must remove the extra two characters. But this mustn't happen for the last token at the end of stream." aBlock value: mark value: (source atEnd ifTrue: [tokenType := #doIt. "to cause an immediate ^self" source position] ifFalse: [source position - 2])]. (tokenType == #rightParenthesis or: [tokenType == #doIt]) ifTrue: [^self]. tokenType == #leftParenthesis ifTrue: [self scanToken; scanAllTokenPositionsInto: aBlock] ifFalse: + [(tokenType == #word or: [tokenType == #keyword or: [tokenType == #colon]]) - [(tokenType == #word or: [tokenType == #keyword or: [tokenType = #colon]]) ifTrue: [self scanLitWord. token == #true ifTrue: [token := true]. token == #false ifTrue: [token := false]. token == #nil ifTrue: [token := nil]] ifFalse: [(token == #- and: [(self typeTableAt: hereChar) == #xDigit]) ifTrue: [self scanToken. token := token negated]]]. + self scanToken ] repeat! - self scanToken. - true] whileTrue! Item was changed: ----- Method: Scanner>>scanFieldNames: (in category 'public access') ----- scanFieldNames: stringOrArray "Answer an Array of Strings that are the identifiers in the input string, stringOrArray. If passed an Array, just answer with that Array, i.e., assume it has already been scanned." | strm | (stringOrArray isMemberOf: Array) ifTrue: [^stringOrArray]. self scan: (ReadStream on: stringOrArray asString). strm := WriteStream on: (Array new: 10). + [tokenType == #doIt] - [tokenType = #doIt] whileFalse: + [tokenType == #word ifTrue: [strm nextPut: token]. - [tokenType = #word ifTrue: [strm nextPut: token]. self scanToken]. ^strm contents "Scanner new scanFieldNames: 'abc def ghi' ('abc' 'def' 'ghi' )"! Item was changed: ----- Method: Scanner>>scanMessageParts: (in category 'public access') ----- scanMessageParts: sourceString "Return an array of the form (comment keyword comment arg comment keyword comment arg comment) for the message pattern of this method. Courtesy of Ted Kaehler, June 1999" | coll nonKeywords | coll := OrderedCollection new. self scan: (ReadStream on: sourceString asString). nonKeywords := 0. + [tokenType == #doIt] whileFalse: - [tokenType = #doIt] whileFalse: [(currentComment == nil or: [currentComment isEmpty]) ifTrue: [coll addLast: nil] ifFalse: [coll addLast: currentComment removeFirst. [currentComment isEmpty] whileFalse: [coll at: coll size put: (coll last, ' ', currentComment removeFirst)]]. + (token numArgs < 1 or: [token = #| and: [ coll size > 1 ] ]) - (token numArgs < 1 or: [(token = #|) & (coll size > 1)]) ifTrue: [(nonKeywords := nonKeywords + 1) > 1 ifTrue: [^ coll]] "done with header" ifFalse: [nonKeywords := 0]. coll addLast: token. self scanToken]. (currentComment == nil or: [currentComment isEmpty]) ifTrue: [coll addLast: nil] ifFalse: [coll addLast: currentComment removeFirst. [currentComment isEmpty] whileFalse: [ coll at: coll size put: (coll last, ' ', currentComment removeFirst)]]. ^ coll! Item was changed: ----- Method: Scanner>>scanStringStruct (in category 'expression types') ----- scanStringStruct | s | s := WriteStream on: (Array new: 16). + [tokenType == #rightParenthesis or: [tokenType == #doIt]] - [tokenType = #rightParenthesis or: [tokenType = #doIt]] whileFalse: + [tokenType == #leftParenthesis - [tokenType = #leftParenthesis ifTrue: [self scanToken; scanStringStruct] ifFalse: + [tokenType == #word ifFalse: - [tokenType = #word ifFalse: [^self error: 'only words and parens allowed']]. s nextPut: token. self scanToken]. token := s contents! Item was changed: ----- Method: Scanner>>typedScanTokens: (in category 'public access') ----- typedScanTokens: textOrString "Answer an Array that has been tokenized with literals mapped to literals, special characters mapped to symbols and variable names and keywords to strings. This methiod accepts _ (underscore) as an assignment token irrespective of whether the system prefers := as the assignment token." | s | self initScannerForTokenization. self scan: (ReadStream on: textOrString asString). s := WriteStream on: (Array new: 16). + [tokenType == #doIt] whileFalse: - [tokenType = #doIt] whileFalse: [(token == #- + and: [(self typeTableAt: hereChar) == #xDigit]) ifTrue: - and: [(self typeTableAt: hereChar) = #xDigit]) ifTrue: [self scanToken. token := token negated]. s nextPut: token. self scanToken]. ^s contents "Scanner new typedScanTokens: (Scanner sourceCodeAt: #typedScanTokens:)"! Item was changed: ----- Method: Scanner>>xColon (in category 'multi-character scans') ----- xColon "Allow := for assignment" + aheadChar == $= ifTrue: - aheadChar = $= ifTrue: [self step. tokenType := #leftArrow. self step. ^ token := #':=']. "Otherwise, just do what normal scan of colon would do" tokenType := #colon. ^ token := self step asSymbol! Item was changed: ----- Method: Scanner>>xDollar (in category 'multi-character scans') ----- xDollar "Form a Character literal." self step. "pass over $" token := self step. + tokenType := #character! - tokenType := #number "really should be Char, but rest of compiler doesn't know"!
1
0
0
0
The Trunk: Compiler-ul.203.mcz
by commits@source.squeak.org
02 Apr '11
02 Apr '11
Levente Uzonyi uploaded a new version of Compiler to project The Trunk:
http://source.squeak.org/trunk/Compiler-ul.203.mcz
==================== Summary ==================== Name: Compiler-ul.203 Author: ul Time: 2 April 2011, 12:50:39.599 am UUID: 9817f32d-943f-4745-81b8-0ecaeec259d4 Ancestors: Compiler-ul.202 Scanner changes: - introduced #character type for characters instead of #number (part 2) - updated comment - added a new class variable DoItCharacter. All checks for doIt characters (30) should use this. (part 1) - moved TypeTable initialization to a separate method - instance creation uses the common #initialize method instead of #initScanner - removed #initScanner and the class side #new - use #== instead of #= for symbol and character comparison - use #repeat instead of true and #whileTrue for loops - use #and: instead of #& =============== Diff against Compiler-ul.202 =============== Item was changed: Object subclass: #Scanner instanceVariableNames: 'source mark hereChar aheadChar token tokenType currentComment buffer typeTable' + classVariableNames: 'AllowUnderscoreAssignments AllowUnderscoreSelectors DoItCharacter TypeTable' - classVariableNames: 'AllowUnderscoreAssignments AllowUnderscoreSelectors TypeTable' poolDictionaries: '' category: 'Compiler-Kernel'! + !Scanner commentStamp: 'ul 3/27/2011 23:24' prior: 0! + I scan a string or text, picking out Smalltalk syntactic tokens. I look one character ahead. I put each token found into the instance variable, token, and its type (a Symbol) into the variable, tokenType. At the end of the input stream, I pretend to see an endless sequence of special characters called doIts. + + Instance Variables + aheadChar: <Character> + buffer: <WriteStream> + currentComment: <OrderedCollection> + hereChar: <Character> + mark: <Integer> + source: <ReadStream> + token: <Symbol|String|NumberCharacter|Boolean|nil> + tokenType: <Symbol> + typeTable: <Array> + + aheadChar + - the next character in the input stream + + buffer + - a reusable WriteStream on a String which is used for building strings. Shouldn't be used from multiple methods without resetting. + + currentComment + - an OrderedCollection of strings which contain all comments between the current token and the previous token or the beginning of the source. + + hereChar + - the current character + + mark + - the position of the current token in the source stream + + source + - xxxxx + + token + - the current token + + tokenType + - the type of the current token. The possible token types are: #binary, #character, #colon, #doIt, #keyword, #leftArrow, #leftBrace, #leftBracket, #leftParenthesis, #literal, #period, #rightBrace, #rightBracket, #rightParenthesis, #semicolon, #string, #upArrow, #verticalBar, #word, #xBinary, #xColon, #xDelimiter, #xDigit, #xDollar, #xDoubleQuote, #xLetter, #xLitQuote, #xSingleQuote, #xUnderscore + + typeTable + - an array that maps each an evaluable tokenType to each character with asciiValue between 0 and 255! - !Scanner commentStamp: '<historical>' prior: 0! - I scan a string or text, picking out Smalltalk syntactic tokens. I look one character ahead. I put each token found into the instance variable, token, and its type (a Symbol) into the variable, tokenType. At the end of the input stream, I pretend to see an endless sequence of special characters called doits.! Item was changed: ----- Method: Scanner class>>initialize (in category 'initialization') ----- initialize + + self initializeTypeTable. + DoItCharacter := Character value: 30! - | newTable | - newTable := Array new: 256 withAll: #xBinary. "default" - newTable atAll: #(9 10 12 13 32 ) put: #xDelimiter. "tab lf ff cr space" - newTable atAll: ($0 asciiValue to: $9 asciiValue) put: #xDigit. - - 1 to: 255 - do: [:index | - (Character value: index) isLetter - ifTrue: [newTable at: index put: #xLetter]]. - - newTable at: 30 put: #doIt. - newTable at: $" asciiValue put: #xDoubleQuote. - newTable at: $# asciiValue put: #xLitQuote. - newTable at: $$ asciiValue put: #xDollar. - newTable at: $' asciiValue put: #xSingleQuote. - newTable at: $: asciiValue put: #xColon. - newTable at: $( asciiValue put: #leftParenthesis. - newTable at: $) asciiValue put: #rightParenthesis. - newTable at: $. asciiValue put: #period. - newTable at: $; asciiValue put: #semicolon. - newTable at: $[ asciiValue put: #leftBracket. - newTable at: $] asciiValue put: #rightBracket. - newTable at: ${ asciiValue put: #leftBrace. - newTable at: $} asciiValue put: #rightBrace. - newTable at: $^ asciiValue put: #upArrow. - newTable at: $_ asciiValue put: #xUnderscore. - newTable at: $| asciiValue put: #verticalBar. - TypeTable := newTable "bon voyage!!" - - "Scanner initialize"! Item was added: + ----- Method: Scanner class>>initializeTypeTable (in category 'initialization') ----- + initializeTypeTable + "self initializeTypeTable" + + | newTable | + newTable := Array new: 256 withAll: #xBinary. "default" + newTable atAll: #(9 10 12 13 32 ) put: #xDelimiter. "tab lf ff cr space" + newTable atAll: ($0 asciiValue to: $9 asciiValue) put: #xDigit. + + 1 to: 255 + do: [:index | + (Character value: index) isLetter + ifTrue: [newTable at: index put: #xLetter]]. + + newTable at: 30 put: #doIt. + newTable at: $" asciiValue put: #xDoubleQuote. + newTable at: $# asciiValue put: #xLitQuote. + newTable at: $$ asciiValue put: #xDollar. + newTable at: $' asciiValue put: #xSingleQuote. + newTable at: $: asciiValue put: #xColon. + newTable at: $( asciiValue put: #leftParenthesis. + newTable at: $) asciiValue put: #rightParenthesis. + newTable at: $. asciiValue put: #period. + newTable at: $; asciiValue put: #semicolon. + newTable at: $[ asciiValue put: #leftBracket. + newTable at: $] asciiValue put: #rightBracket. + newTable at: ${ asciiValue put: #leftBrace. + newTable at: $} asciiValue put: #rightBrace. + newTable at: $^ asciiValue put: #upArrow. + newTable at: $_ asciiValue put: #xUnderscore. + newTable at: $| asciiValue put: #verticalBar. + TypeTable := newTable "bon voyage!!"! Item was removed: - ----- Method: Scanner class>>new (in category 'instance creation') ----- - new - - ^self basicNew initScanner! Item was removed: - ----- Method: Scanner>>initScanner (in category 'initialize-release') ----- - initScanner - - buffer := WriteStream on: (String new: 40). - typeTable := TypeTable! Item was added: + ----- Method: Scanner>>initialize (in category 'initialize-release') ----- + initialize + + super initialize. + buffer := WriteStream on: (String new: 40). + typeTable := TypeTable! Item was changed: ----- Method: Scanner>>scanAllTokenPositionsInto: (in category 'expression types') ----- scanAllTokenPositionsInto: aBlock "Evaluate aBlock with the start and end positions of all separate non-white-space tokens, including comments." | lastMark | lastMark := 1. [currentComment ifNotNil: [currentComment do: [:cmnt| | idx | idx := source originalContents indexOfSubCollection: cmnt startingAt: lastMark. (idx > 0 and: [idx < mark]) ifTrue: [aBlock value: idx - 1 value: (lastMark := idx + cmnt size)]]. currentComment := nil]. mark ifNotNil: [(token == #- and: [(self typeTableAt: hereChar) == #xDigit]) ifTrue: [| savedMark | savedMark := mark. self scanToken. token := token negated. mark := savedMark]. "Compensate for the fact that the parser uses two character lookahead. Normally we must remove the extra two characters. But this mustn't happen for the last token at the end of stream." aBlock value: mark value: (source atEnd ifTrue: [tokenType := #doIt. "to cause an immediate ^self" source position] ifFalse: [source position - 2])]. (tokenType == #rightParenthesis or: [tokenType == #doIt]) ifTrue: [^self]. tokenType == #leftParenthesis ifTrue: [self scanToken; scanAllTokenPositionsInto: aBlock] ifFalse: + [(tokenType == #word or: [tokenType == #keyword or: [tokenType == #colon]]) - [(tokenType == #word or: [tokenType == #keyword or: [tokenType = #colon]]) ifTrue: [self scanLitWord. token == #true ifTrue: [token := true]. token == #false ifTrue: [token := false]. token == #nil ifTrue: [token := nil]] ifFalse: [(token == #- and: [(self typeTableAt: hereChar) == #xDigit]) ifTrue: [self scanToken. token := token negated]]]. + self scanToken ] repeat! - self scanToken. - true] whileTrue! Item was changed: ----- Method: Scanner>>scanFieldNames: (in category 'public access') ----- scanFieldNames: stringOrArray "Answer an Array of Strings that are the identifiers in the input string, stringOrArray. If passed an Array, just answer with that Array, i.e., assume it has already been scanned." | strm | (stringOrArray isMemberOf: Array) ifTrue: [^stringOrArray]. self scan: (ReadStream on: stringOrArray asString). strm := WriteStream on: (Array new: 10). + [tokenType == #doIt] - [tokenType = #doIt] whileFalse: + [tokenType == #word ifTrue: [strm nextPut: token]. - [tokenType = #word ifTrue: [strm nextPut: token]. self scanToken]. ^strm contents "Scanner new scanFieldNames: 'abc def ghi' ('abc' 'def' 'ghi' )"! Item was changed: ----- Method: Scanner>>scanMessageParts: (in category 'public access') ----- scanMessageParts: sourceString "Return an array of the form (comment keyword comment arg comment keyword comment arg comment) for the message pattern of this method. Courtesy of Ted Kaehler, June 1999" | coll nonKeywords | coll := OrderedCollection new. self scan: (ReadStream on: sourceString asString). nonKeywords := 0. + [tokenType == #doIt] whileFalse: - [tokenType = #doIt] whileFalse: [(currentComment == nil or: [currentComment isEmpty]) ifTrue: [coll addLast: nil] ifFalse: [coll addLast: currentComment removeFirst. [currentComment isEmpty] whileFalse: [coll at: coll size put: (coll last, ' ', currentComment removeFirst)]]. + (token numArgs < 1 or: [token = #| and: [ coll size > 1 ] ]) - (token numArgs < 1 or: [(token = #|) & (coll size > 1)]) ifTrue: [(nonKeywords := nonKeywords + 1) > 1 ifTrue: [^ coll]] "done with header" ifFalse: [nonKeywords := 0]. coll addLast: token. self scanToken]. (currentComment == nil or: [currentComment isEmpty]) ifTrue: [coll addLast: nil] ifFalse: [coll addLast: currentComment removeFirst. [currentComment isEmpty] whileFalse: [ coll at: coll size put: (coll last, ' ', currentComment removeFirst)]]. ^ coll! Item was changed: ----- Method: Scanner>>scanStringStruct (in category 'expression types') ----- scanStringStruct | s | s := WriteStream on: (Array new: 16). + [tokenType == #rightParenthesis or: [tokenType == #doIt]] - [tokenType = #rightParenthesis or: [tokenType = #doIt]] whileFalse: + [tokenType == #leftParenthesis - [tokenType = #leftParenthesis ifTrue: [self scanToken; scanStringStruct] ifFalse: + [tokenType == #word ifFalse: - [tokenType = #word ifFalse: [^self error: 'only words and parens allowed']]. s nextPut: token. self scanToken]. token := s contents! Item was changed: ----- Method: Scanner>>typedScanTokens: (in category 'public access') ----- typedScanTokens: textOrString "Answer an Array that has been tokenized with literals mapped to literals, special characters mapped to symbols and variable names and keywords to strings. This methiod accepts _ (underscore) as an assignment token irrespective of whether the system prefers := as the assignment token." | s | self initScannerForTokenization. self scan: (ReadStream on: textOrString asString). s := WriteStream on: (Array new: 16). + [tokenType == #doIt] whileFalse: - [tokenType = #doIt] whileFalse: [(token == #- + and: [(self typeTableAt: hereChar) == #xDigit]) ifTrue: - and: [(self typeTableAt: hereChar) = #xDigit]) ifTrue: [self scanToken. token := token negated]. s nextPut: token. self scanToken]. ^s contents "Scanner new typedScanTokens: (Scanner sourceCodeAt: #typedScanTokens:)"! Item was changed: ----- Method: Scanner>>xColon (in category 'multi-character scans') ----- xColon "Allow := for assignment" + aheadChar == $= ifTrue: - aheadChar = $= ifTrue: [self step. tokenType := #leftArrow. self step. ^ token := #':=']. "Otherwise, just do what normal scan of colon would do" tokenType := #colon. ^ token := self step asSymbol! Item was changed: ----- Method: Scanner>>xDollar (in category 'multi-character scans') ----- xDollar "Form a Character literal." self step. "pass over $" token := self step. + tokenType := #character! - tokenType := #number "really should be Char, but rest of compiler doesn't know"!
1
0
0
0
The Trunk: Compiler-ul.203.mcz
by commits@source.squeak.org
02 Apr '11
02 Apr '11
Levente Uzonyi uploaded a new version of Compiler to project The Trunk:
http://source.squeak.org/trunk/Compiler-ul.203.mcz
==================== Summary ==================== Name: Compiler-ul.203 Author: ul Time: 2 April 2011, 12:50:39.599 am UUID: 9817f32d-943f-4745-81b8-0ecaeec259d4 Ancestors: Compiler-ul.202 Scanner changes: - introduced #character type for characters instead of #number (part 2) - updated comment - added a new class variable DoItCharacter. All checks for doIt characters (30) should use this. (part 1) - moved TypeTable initialization to a separate method - instance creation uses the common #initialize method instead of #initScanner - removed #initScanner and the class side #new - use #== instead of #= for symbol and character comparison - use #repeat instead of true and #whileTrue for loops - use #and: instead of #& =============== Diff against Compiler-ul.202 =============== Item was changed: Object subclass: #Scanner instanceVariableNames: 'source mark hereChar aheadChar token tokenType currentComment buffer typeTable' + classVariableNames: 'AllowUnderscoreAssignments AllowUnderscoreSelectors DoItCharacter TypeTable' - classVariableNames: 'AllowUnderscoreAssignments AllowUnderscoreSelectors TypeTable' poolDictionaries: '' category: 'Compiler-Kernel'! + !Scanner commentStamp: 'ul 3/27/2011 23:24' prior: 0! + I scan a string or text, picking out Smalltalk syntactic tokens. I look one character ahead. I put each token found into the instance variable, token, and its type (a Symbol) into the variable, tokenType. At the end of the input stream, I pretend to see an endless sequence of special characters called doIts. + + Instance Variables + aheadChar: <Character> + buffer: <WriteStream> + currentComment: <OrderedCollection> + hereChar: <Character> + mark: <Integer> + source: <ReadStream> + token: <Symbol|String|NumberCharacter|Boolean|nil> + tokenType: <Symbol> + typeTable: <Array> + + aheadChar + - the next character in the input stream + + buffer + - a reusable WriteStream on a String which is used for building strings. Shouldn't be used from multiple methods without resetting. + + currentComment + - an OrderedCollection of strings which contain all comments between the current token and the previous token or the beginning of the source. + + hereChar + - the current character + + mark + - the position of the current token in the source stream + + source + - xxxxx + + token + - the current token + + tokenType + - the type of the current token. The possible token types are: #binary, #character, #colon, #doIt, #keyword, #leftArrow, #leftBrace, #leftBracket, #leftParenthesis, #literal, #period, #rightBrace, #rightBracket, #rightParenthesis, #semicolon, #string, #upArrow, #verticalBar, #word, #xBinary, #xColon, #xDelimiter, #xDigit, #xDollar, #xDoubleQuote, #xLetter, #xLitQuote, #xSingleQuote, #xUnderscore + + typeTable + - an array that maps each an evaluable tokenType to each character with asciiValue between 0 and 255! - !Scanner commentStamp: '<historical>' prior: 0! - I scan a string or text, picking out Smalltalk syntactic tokens. I look one character ahead. I put each token found into the instance variable, token, and its type (a Symbol) into the variable, tokenType. At the end of the input stream, I pretend to see an endless sequence of special characters called doits.! Item was changed: ----- Method: Scanner class>>initialize (in category 'initialization') ----- initialize + + self initializeTypeTable. + DoItCharacter := Character value: 30! - | newTable | - newTable := Array new: 256 withAll: #xBinary. "default" - newTable atAll: #(9 10 12 13 32 ) put: #xDelimiter. "tab lf ff cr space" - newTable atAll: ($0 asciiValue to: $9 asciiValue) put: #xDigit. - - 1 to: 255 - do: [:index | - (Character value: index) isLetter - ifTrue: [newTable at: index put: #xLetter]]. - - newTable at: 30 put: #doIt. - newTable at: $" asciiValue put: #xDoubleQuote. - newTable at: $# asciiValue put: #xLitQuote. - newTable at: $$ asciiValue put: #xDollar. - newTable at: $' asciiValue put: #xSingleQuote. - newTable at: $: asciiValue put: #xColon. - newTable at: $( asciiValue put: #leftParenthesis. - newTable at: $) asciiValue put: #rightParenthesis. - newTable at: $. asciiValue put: #period. - newTable at: $; asciiValue put: #semicolon. - newTable at: $[ asciiValue put: #leftBracket. - newTable at: $] asciiValue put: #rightBracket. - newTable at: ${ asciiValue put: #leftBrace. - newTable at: $} asciiValue put: #rightBrace. - newTable at: $^ asciiValue put: #upArrow. - newTable at: $_ asciiValue put: #xUnderscore. - newTable at: $| asciiValue put: #verticalBar. - TypeTable := newTable "bon voyage!!" - - "Scanner initialize"! Item was added: + ----- Method: Scanner class>>initializeTypeTable (in category 'initialization') ----- + initializeTypeTable + "self initializeTypeTable" + + | newTable | + newTable := Array new: 256 withAll: #xBinary. "default" + newTable atAll: #(9 10 12 13 32 ) put: #xDelimiter. "tab lf ff cr space" + newTable atAll: ($0 asciiValue to: $9 asciiValue) put: #xDigit. + + 1 to: 255 + do: [:index | + (Character value: index) isLetter + ifTrue: [newTable at: index put: #xLetter]]. + + newTable at: 30 put: #doIt. + newTable at: $" asciiValue put: #xDoubleQuote. + newTable at: $# asciiValue put: #xLitQuote. + newTable at: $$ asciiValue put: #xDollar. + newTable at: $' asciiValue put: #xSingleQuote. + newTable at: $: asciiValue put: #xColon. + newTable at: $( asciiValue put: #leftParenthesis. + newTable at: $) asciiValue put: #rightParenthesis. + newTable at: $. asciiValue put: #period. + newTable at: $; asciiValue put: #semicolon. + newTable at: $[ asciiValue put: #leftBracket. + newTable at: $] asciiValue put: #rightBracket. + newTable at: ${ asciiValue put: #leftBrace. + newTable at: $} asciiValue put: #rightBrace. + newTable at: $^ asciiValue put: #upArrow. + newTable at: $_ asciiValue put: #xUnderscore. + newTable at: $| asciiValue put: #verticalBar. + TypeTable := newTable "bon voyage!!"! Item was removed: - ----- Method: Scanner class>>new (in category 'instance creation') ----- - new - - ^self basicNew initScanner! Item was removed: - ----- Method: Scanner>>initScanner (in category 'initialize-release') ----- - initScanner - - buffer := WriteStream on: (String new: 40). - typeTable := TypeTable! Item was added: + ----- Method: Scanner>>initialize (in category 'initialize-release') ----- + initialize + + super initialize. + buffer := WriteStream on: (String new: 40). + typeTable := TypeTable! Item was changed: ----- Method: Scanner>>scanAllTokenPositionsInto: (in category 'expression types') ----- scanAllTokenPositionsInto: aBlock "Evaluate aBlock with the start and end positions of all separate non-white-space tokens, including comments." | lastMark | lastMark := 1. [currentComment ifNotNil: [currentComment do: [:cmnt| | idx | idx := source originalContents indexOfSubCollection: cmnt startingAt: lastMark. (idx > 0 and: [idx < mark]) ifTrue: [aBlock value: idx - 1 value: (lastMark := idx + cmnt size)]]. currentComment := nil]. mark ifNotNil: [(token == #- and: [(self typeTableAt: hereChar) == #xDigit]) ifTrue: [| savedMark | savedMark := mark. self scanToken. token := token negated. mark := savedMark]. "Compensate for the fact that the parser uses two character lookahead. Normally we must remove the extra two characters. But this mustn't happen for the last token at the end of stream." aBlock value: mark value: (source atEnd ifTrue: [tokenType := #doIt. "to cause an immediate ^self" source position] ifFalse: [source position - 2])]. (tokenType == #rightParenthesis or: [tokenType == #doIt]) ifTrue: [^self]. tokenType == #leftParenthesis ifTrue: [self scanToken; scanAllTokenPositionsInto: aBlock] ifFalse: + [(tokenType == #word or: [tokenType == #keyword or: [tokenType == #colon]]) - [(tokenType == #word or: [tokenType == #keyword or: [tokenType = #colon]]) ifTrue: [self scanLitWord. token == #true ifTrue: [token := true]. token == #false ifTrue: [token := false]. token == #nil ifTrue: [token := nil]] ifFalse: [(token == #- and: [(self typeTableAt: hereChar) == #xDigit]) ifTrue: [self scanToken. token := token negated]]]. + self scanToken ] repeat! - self scanToken. - true] whileTrue! Item was changed: ----- Method: Scanner>>scanFieldNames: (in category 'public access') ----- scanFieldNames: stringOrArray "Answer an Array of Strings that are the identifiers in the input string, stringOrArray. If passed an Array, just answer with that Array, i.e., assume it has already been scanned." | strm | (stringOrArray isMemberOf: Array) ifTrue: [^stringOrArray]. self scan: (ReadStream on: stringOrArray asString). strm := WriteStream on: (Array new: 10). + [tokenType == #doIt] - [tokenType = #doIt] whileFalse: + [tokenType == #word ifTrue: [strm nextPut: token]. - [tokenType = #word ifTrue: [strm nextPut: token]. self scanToken]. ^strm contents "Scanner new scanFieldNames: 'abc def ghi' ('abc' 'def' 'ghi' )"! Item was changed: ----- Method: Scanner>>scanMessageParts: (in category 'public access') ----- scanMessageParts: sourceString "Return an array of the form (comment keyword comment arg comment keyword comment arg comment) for the message pattern of this method. Courtesy of Ted Kaehler, June 1999" | coll nonKeywords | coll := OrderedCollection new. self scan: (ReadStream on: sourceString asString). nonKeywords := 0. + [tokenType == #doIt] whileFalse: - [tokenType = #doIt] whileFalse: [(currentComment == nil or: [currentComment isEmpty]) ifTrue: [coll addLast: nil] ifFalse: [coll addLast: currentComment removeFirst. [currentComment isEmpty] whileFalse: [coll at: coll size put: (coll last, ' ', currentComment removeFirst)]]. + (token numArgs < 1 or: [token = #| and: [ coll size > 1 ] ]) - (token numArgs < 1 or: [(token = #|) & (coll size > 1)]) ifTrue: [(nonKeywords := nonKeywords + 1) > 1 ifTrue: [^ coll]] "done with header" ifFalse: [nonKeywords := 0]. coll addLast: token. self scanToken]. (currentComment == nil or: [currentComment isEmpty]) ifTrue: [coll addLast: nil] ifFalse: [coll addLast: currentComment removeFirst. [currentComment isEmpty] whileFalse: [ coll at: coll size put: (coll last, ' ', currentComment removeFirst)]]. ^ coll! Item was changed: ----- Method: Scanner>>scanStringStruct (in category 'expression types') ----- scanStringStruct | s | s := WriteStream on: (Array new: 16). + [tokenType == #rightParenthesis or: [tokenType == #doIt]] - [tokenType = #rightParenthesis or: [tokenType = #doIt]] whileFalse: + [tokenType == #leftParenthesis - [tokenType = #leftParenthesis ifTrue: [self scanToken; scanStringStruct] ifFalse: + [tokenType == #word ifFalse: - [tokenType = #word ifFalse: [^self error: 'only words and parens allowed']]. s nextPut: token. self scanToken]. token := s contents! Item was changed: ----- Method: Scanner>>typedScanTokens: (in category 'public access') ----- typedScanTokens: textOrString "Answer an Array that has been tokenized with literals mapped to literals, special characters mapped to symbols and variable names and keywords to strings. This methiod accepts _ (underscore) as an assignment token irrespective of whether the system prefers := as the assignment token." | s | self initScannerForTokenization. self scan: (ReadStream on: textOrString asString). s := WriteStream on: (Array new: 16). + [tokenType == #doIt] whileFalse: - [tokenType = #doIt] whileFalse: [(token == #- + and: [(self typeTableAt: hereChar) == #xDigit]) ifTrue: - and: [(self typeTableAt: hereChar) = #xDigit]) ifTrue: [self scanToken. token := token negated]. s nextPut: token. self scanToken]. ^s contents "Scanner new typedScanTokens: (Scanner sourceCodeAt: #typedScanTokens:)"! Item was changed: ----- Method: Scanner>>xColon (in category 'multi-character scans') ----- xColon "Allow := for assignment" + aheadChar == $= ifTrue: - aheadChar = $= ifTrue: [self step. tokenType := #leftArrow. self step. ^ token := #':=']. "Otherwise, just do what normal scan of colon would do" tokenType := #colon. ^ token := self step asSymbol! Item was changed: ----- Method: Scanner>>xDollar (in category 'multi-character scans') ----- xDollar "Form a Character literal." self step. "pass over $" token := self step. + tokenType := #character! - tokenType := #number "really should be Char, but rest of compiler doesn't know"!
1
0
0
0
The Trunk: Compiler-ul.202.mcz
by commits@source.squeak.org
02 Apr '11
02 Apr '11
Levente Uzonyi uploaded a new version of Compiler to project The Trunk:
http://source.squeak.org/trunk/Compiler-ul.202.mcz
==================== Summary ==================== Name: Compiler-ul.202 Author: ul Time: 2 April 2011, 12:44:12.796 am UUID: e7e9151c-c2e9-b14a-bb57-68e6ea8daea5 Ancestors: Compiler-nice.201 - reject literals with superfluous # at the beginning, like #$a #123 ##foo ##(1 2 3), etc - introduced #character type for characters instead of #number (part 1) =============== Diff against Compiler-nice.201 =============== Item was changed: ----- Method: Parser>>pragmaLiteral: (in category 'pragmas') ----- pragmaLiteral: selectorSoFar "Read a pragma literal. As a nicety we allow a variable name (rather than a literal string) as the second argument to primitive:error:" + (hereType == #string or: [ hereType == #literal or: [ hereType == #number or: [ hereType == #character ] ] ]) - (hereType == #string or: [ hereType == #literal or: [ hereType == #number ] ]) ifTrue: [ ^ self advance ]. (here == $# and: [ tokenType == #word ]) ifTrue: [ ^ self advance ]. (here == #- and: [ tokenType == #number ]) ifTrue: [ ^ (self advance; advance) negated ]. (here = 'true' or: [ here = 'false' or: [ here = 'nil' ] ]) ifTrue: [ ^ Compiler evaluate: self advance ]. "This nicety allows one to supply a primitive error temp as a variable name, rather than a string." ((selectorSoFar beginsWith: 'primitive:') and: [(selectorSoFar endsWith: 'error:') and: [hereType == #word]]) ifTrue: [^self advance]. ^self expected: 'Literal constant'! Item was changed: ----- Method: Parser>>primaryExpression (in category 'expression types') ----- primaryExpression hereType == #word ifTrue: [parseNode := self variable. (parseNode isUndefTemp and: [self interactive]) ifTrue: [self queryUndefined]. parseNode nowHasRef. ^ true]. hereType == #leftBracket ifTrue: [self advance. self blockExpression. ^true]. hereType == #leftBrace ifTrue: [self braceExpression. ^true]. hereType == #leftParenthesis ifTrue: [self advance. self expression ifFalse: [^self expected: 'expression']. (self match: #rightParenthesis) ifFalse: [^self expected: 'right parenthesis']. ^true]. + (hereType == #string or: [hereType == #number or: [hereType == #literal or: [hereType == #character]]]) - (hereType == #string or: [hereType == #number or: [hereType == #literal]]) ifTrue: [parseNode := encoder encodeLiteral: self advance. ^true]. (here == #- and: [tokenType == #number and: [1 + hereEnd = mark]]) ifTrue: [self advance. parseNode := encoder encodeLiteral: self advance negated. ^true]. ^false! Item was changed: ----- Method: Scanner>>xLitQuote (in category 'multi-character scans') ----- xLitQuote "Symbols and vectors: #(1 (4 5) 2 3) #ifTrue:ifFalse: #'abc'." | start | start := mark. self step. "litQuote" self scanToken. tokenType == #leftParenthesis ifTrue: [self scanToken; scanLitVec. mark := start + 1. tokenType == #doIt ifTrue: [self offEnd: 'Unmatched parenthesis']] ifFalse: [tokenType == #leftBracket ifTrue: [self scanToken; scanLitByteVec. mark := start + 1. tokenType == #doIt ifTrue: [self offEnd: 'Unmatched bracket']] ifFalse: [(tokenType == #word or: [tokenType == #keyword or: [tokenType == #colon]]) ifTrue: [self scanLitWord] + ifFalse: [tokenType == #string + ifTrue: [token := token asSymbol] + ifFalse: [(tokenType == #binary or: [ tokenType == #verticalBar ]) + ifFalse: [self notify: 'Invalid literal character' at: start + 1]]]]]. - ifFalse: ["tokenType == #literal - ifTrue: [token isSymbol - ifTrue: [""##word"" - token := token - ""May want to move toward ANSI - here ""]] - ifFalse: ["tokenType == #string - ifTrue: [token := token asSymbol]"]"]]]. mark := start. tokenType := #literal "#(Pen) #Pen #'Pen' ##Pen ###Pen "!
1
0
0
0
The Trunk: Compiler-ul.202.mcz
by commits@source.squeak.org
02 Apr '11
02 Apr '11
Levente Uzonyi uploaded a new version of Compiler to project The Trunk:
http://source.squeak.org/trunk/Compiler-ul.202.mcz
==================== Summary ==================== Name: Compiler-ul.202 Author: ul Time: 2 April 2011, 12:44:12.796 am UUID: e7e9151c-c2e9-b14a-bb57-68e6ea8daea5 Ancestors: Compiler-nice.201 - reject literals with superfluous # at the beginning, like #$a #123 ##foo ##(1 2 3), etc - introduced #character type for characters instead of #number (part 1) =============== Diff against Compiler-nice.201 =============== Item was changed: ----- Method: Parser>>pragmaLiteral: (in category 'pragmas') ----- pragmaLiteral: selectorSoFar "Read a pragma literal. As a nicety we allow a variable name (rather than a literal string) as the second argument to primitive:error:" + (hereType == #string or: [ hereType == #literal or: [ hereType == #number or: [ hereType == #character ] ] ]) - (hereType == #string or: [ hereType == #literal or: [ hereType == #number ] ]) ifTrue: [ ^ self advance ]. (here == $# and: [ tokenType == #word ]) ifTrue: [ ^ self advance ]. (here == #- and: [ tokenType == #number ]) ifTrue: [ ^ (self advance; advance) negated ]. (here = 'true' or: [ here = 'false' or: [ here = 'nil' ] ]) ifTrue: [ ^ Compiler evaluate: self advance ]. "This nicety allows one to supply a primitive error temp as a variable name, rather than a string." ((selectorSoFar beginsWith: 'primitive:') and: [(selectorSoFar endsWith: 'error:') and: [hereType == #word]]) ifTrue: [^self advance]. ^self expected: 'Literal constant'! Item was changed: ----- Method: Parser>>primaryExpression (in category 'expression types') ----- primaryExpression hereType == #word ifTrue: [parseNode := self variable. (parseNode isUndefTemp and: [self interactive]) ifTrue: [self queryUndefined]. parseNode nowHasRef. ^ true]. hereType == #leftBracket ifTrue: [self advance. self blockExpression. ^true]. hereType == #leftBrace ifTrue: [self braceExpression. ^true]. hereType == #leftParenthesis ifTrue: [self advance. self expression ifFalse: [^self expected: 'expression']. (self match: #rightParenthesis) ifFalse: [^self expected: 'right parenthesis']. ^true]. + (hereType == #string or: [hereType == #number or: [hereType == #literal or: [hereType == #character]]]) - (hereType == #string or: [hereType == #number or: [hereType == #literal]]) ifTrue: [parseNode := encoder encodeLiteral: self advance. ^true]. (here == #- and: [tokenType == #number and: [1 + hereEnd = mark]]) ifTrue: [self advance. parseNode := encoder encodeLiteral: self advance negated. ^true]. ^false! Item was changed: ----- Method: Scanner>>xLitQuote (in category 'multi-character scans') ----- xLitQuote "Symbols and vectors: #(1 (4 5) 2 3) #ifTrue:ifFalse: #'abc'." | start | start := mark. self step. "litQuote" self scanToken. tokenType == #leftParenthesis ifTrue: [self scanToken; scanLitVec. mark := start + 1. tokenType == #doIt ifTrue: [self offEnd: 'Unmatched parenthesis']] ifFalse: [tokenType == #leftBracket ifTrue: [self scanToken; scanLitByteVec. mark := start + 1. tokenType == #doIt ifTrue: [self offEnd: 'Unmatched bracket']] ifFalse: [(tokenType == #word or: [tokenType == #keyword or: [tokenType == #colon]]) ifTrue: [self scanLitWord] + ifFalse: [tokenType == #string + ifTrue: [token := token asSymbol] + ifFalse: [(tokenType == #binary or: [ tokenType == #verticalBar ]) + ifFalse: [self notify: 'Invalid literal character' at: start + 1]]]]]. - ifFalse: ["tokenType == #literal - ifTrue: [token isSymbol - ifTrue: [""##word"" - token := token - ""May want to move toward ANSI - here ""]] - ifFalse: ["tokenType == #string - ifTrue: [token := token asSymbol]"]"]]]. mark := start. tokenType := #literal "#(Pen) #Pen #'Pen' ##Pen ###Pen "!
1
0
0
0
← Newer
1
...
47
48
49
50
51
Older →
Jump to page:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
Results per page:
10
25
50
100
200