Hi All,
If I attempt to execute the following:
| rules |
rules := RBParseTreeRewriter new.
rules replace: 'state at: `@index put: `@expr' with: 'state at: `@index put: `@expr asC_unsigned_int'.
(rules executeTree: (MD5Plugin parseTreeFor: #md5ProcessBuffer:withState:)) ifTrue: [rules tree newSource inspect]. self error: 'parse tree rewrite failed'
I get "Error: End of statement list encountered" (stack trace below). The same works and produces the expected output in Pharo 7.
It looks like the Refactoring Browser has diverged quite a bit between Squeak and Pharo.
Who looks after RB in Squeak, and is there a plan to keep in sync with Pharo, or the other way around, or should this be fixed independently?
Thanks, Alistair
4 November 2018 8:01:29.599623 pm
VM: unix - Smalltalk Image: Squeak5.3alpha [latest update: #18261]
SecurityManager state: Restricted: false FileAccess: true SocketAccess: true Working Dir /home/alistair/vmmaker/opensmalltalk-vm/sqvmm.03 Trusted Dir /home/alistair/vmmaker/opensmalltalk-vm/sqvmm.03/secure Untrusted Dir /home/alistair/vmmaker/opensmalltalk-vm/sqvmm.03/My Squeak
RBPatternParser(Object)>>error: Receiver: a RBPatternParser Arguments and temporary variables: aString: 'End of statement list encounted' Receiver's instance variables: scanner: a RBPatternScanner currentToken: RBShortAssignmentToken nextToken: nil errorBlock: nil source: 'state at: `@index put: `@expr asC_unsigned_int' comments: an OrderedCollection() pragmas: nil
RBPatternParser(RBParser)>>parserError: Receiver: a RBPatternParser Arguments and temporary variables: aString: 'End of statement list encounted' Receiver's instance variables: scanner: a RBPatternScanner currentToken: RBShortAssignmentToken nextToken: nil errorBlock: nil source: 'state at: `@index put: `@expr asC_unsigned_int' comments: an OrderedCollection() pragmas: nil
RBPatternParser(RBParser)>>parseStatementList:into: Receiver: a RBPatternParser Arguments and temporary variables: pragmaBoolean: false sequenceNode: RBSequenceNode() statements: an OrderedCollection(RBMessageNode(state at: `@index put: `@expr as...etc... return: true periods: an OrderedCollection() returnPosition: nil node: RBMessageNode(state at: `@index put: `@expr asC) Receiver's instance variables: scanner: a RBPatternScanner currentToken: RBShortAssignmentToken nextToken: nil errorBlock: nil source: 'state at: `@index put: `@expr asC_unsigned_int' comments: an OrderedCollection() pragmas: nil
RBPatternParser(RBParser)>>parseStatements: Receiver: a RBPatternParser Arguments and temporary variables: pragmaBoolean: false args: #() leftBar: nil rightBar: nil Receiver's instance variables: scanner: a RBPatternScanner currentToken: RBShortAssignmentToken nextToken: nil errorBlock: nil source: 'state at: `@index put: `@expr asC_unsigned_int' comments: an OrderedCollection() pragmas: nil
RBPatternParser(RBParser)>>parseExpression: Receiver: a RBPatternParser Arguments and temporary variables: aString: 'state at: `@index put: `@expr asC_unsigned_int' node: nil Receiver's instance variables: scanner: a RBPatternScanner currentToken: RBShortAssignmentToken nextToken: nil errorBlock: nil source: 'state at: `@index put: `@expr asC_unsigned_int' comments: an OrderedCollection() pragmas: nil
RBPatternParser class(RBParser class)>>parseExpression:onError: Receiver: RBPatternParser Arguments and temporary variables: aString: 'state at: `@index put: `@expr asC_unsigned_int' aBlock: nil node: nil parser: a RBPatternParser Receiver's instance variables: superclass: RBParser methodDict: a MethodDictionary(#messageNodeClass->(RBPatternParser>>#messageNod...etc... format: 65543 instanceVariables: nil organization: ('private-classes' messageNodeClass methodNodeClass pragmaNodeClass...etc... subclasses: nil name: #RBPatternParser classPool: nil sharedPools: nil environment: Smalltalk category: #'AST-Core-Parser'
RBParser class>>parseRewriteExpression:onError: Receiver: RBParser Arguments and temporary variables: aString: 'state at: `@index put: `@expr asC_unsigned_int' aBlock: nil Receiver's instance variables: superclass: Object methodDict: a MethodDictionary(#addCommentsTo:->(RBParser>>#addCommentsTo: "a CompiledMethod...etc... format: 65543 instanceVariables: #('scanner' 'currentToken' 'nextToken' 'errorBlock' 'source'...etc... organization: ('private' addCommentsTo: nextToken patchLiteralArrayToken patchLiteralMessage...etc... subclasses: {RBPatternParser} name: #RBParser classPool: nil sharedPools: nil environment: Smalltalk category: #'AST-Core-Parser'
RBParser class>>parseRewriteExpression: Receiver: RBParser Arguments and temporary variables: aString: 'state at: `@index put: `@expr asC_unsigned_int' Receiver's instance variables: superclass: Object methodDict: a MethodDictionary(#addCommentsTo:->(RBParser>>#addCommentsTo: "a CompiledMethod...etc... format: 65543 instanceVariables: #('scanner' 'currentToken' 'nextToken' 'errorBlock' 'source'...etc... organization: ('private' addCommentsTo: nextToken patchLiteralArrayToken patchLiteralMessage...etc... subclasses: {RBPatternParser} name: #RBParser classPool: nil sharedPools: nil environment: Smalltalk category: #'AST-Core-Parser'
RBStringReplaceRule>>replaceString: Receiver: a RBStringReplaceRule Arguments and temporary variables: replaceString: 'state at: `@index put: `@expr asC_unsigned_int' Receiver's instance variables: searchTree: RBMessageNode(state at: `@index put: `@expr) owner: nil verificationBlock: [closure] in RBStringReplaceRule(RBReplaceRule)>>initialize replaceTree: nil
RBStringReplaceRule>>searchFor:replaceWith: Receiver: a RBStringReplaceRule Arguments and temporary variables: searchString: 'state at: `@index put: `@expr' replaceString: 'state at: `@index put: `@expr asC_unsigned_int' Receiver's instance variables: searchTree: RBMessageNode(state at: `@index put: `@expr) owner: nil verificationBlock: [closure] in RBStringReplaceRule(RBReplaceRule)>>initialize replaceTree: nil
RBStringReplaceRule class>>searchFor:replaceWith: Receiver: RBStringReplaceRule Arguments and temporary variables: searchString: 'state at: `@index put: `@expr' replaceString: 'state at: `@index put: `@expr asC_unsigned_int' Receiver's instance variables: superclass: RBReplaceRule methodDict: a MethodDictionary(#foundMatchFor:->(RBStringReplaceRule>>#foundMat...etc... format: 65540 instanceVariables: #('replaceTree') organization: ('matching' foundMatchFor:) ('initialize-release' methodReplaceString:...etc... subclasses: nil name: #RBStringReplaceRule classPool: nil sharedPools: nil environment: Smalltalk category: #'AST-Core-Matching'
RBParseTreeRewriter>>replace:with: Receiver: a RBParseTreeRewriter Arguments and temporary variables: searchString: 'state at: `@index put: `@expr' replaceString: 'state at: `@index put: `@expr asC_unsigned_int' Receiver's instance variables: searches: an OrderedCollection() answer: nil argumentSearches: an OrderedCollection() context: a RBSmallDictionary messages: nil tree: nil
UndefinedObject>>DoIt Receiver: nil Arguments and temporary variables: rules: a RBParseTreeRewriter Receiver's instance variables: nil
--- The full stack --- RBPatternParser(Object)>>error: RBPatternParser(RBParser)>>parserError: RBPatternParser(RBParser)>>parseStatementList:into: RBPatternParser(RBParser)>>parseStatements: RBPatternParser(RBParser)>>parseExpression: RBPatternParser class(RBParser class)>>parseExpression:onError: RBParser class>>parseRewriteExpression:onError: RBParser class>>parseRewriteExpression: RBStringReplaceRule>>replaceString: RBStringReplaceRule>>searchFor:replaceWith: RBStringReplaceRule class>>searchFor:replaceWith: RBParseTreeRewriter>>replace:with: UndefinedObject>>DoIt
On Sun, 4 Nov 2018, Alistair Grant wrote:
Hi All,
If I attempt to execute the following:
| rules |
rules := RBParseTreeRewriter new.
rules replace: 'state at: `@index put: `@expr' with: 'state at: `@index put: `@expr asC_unsigned_int'.
I presume the version you use doesn't support underscores in selectors. Does it work if you use a selector with no underscores in it? How did you load RB? Did you use the Metacello configuration? (it may not be up-to-date)
Levente
(rules executeTree: (MD5Plugin parseTreeFor: #md5ProcessBuffer:withState:)) ifTrue: [rules tree newSource inspect]. self error: 'parse tree rewrite failed'
I get "Error: End of statement list encountered" (stack trace below). The same works and produces the expected output in Pharo 7.
It looks like the Refactoring Browser has diverged quite a bit between Squeak and Pharo.
Who looks after RB in Squeak, and is there a plan to keep in sync with Pharo, or the other way around, or should this be fixed independently?
Thanks, Alistair
4 November 2018 8:01:29.599623 pm
VM: unix - Smalltalk Image: Squeak5.3alpha [latest update: #18261]
SecurityManager state: Restricted: false FileAccess: true SocketAccess: true Working Dir /home/alistair/vmmaker/opensmalltalk-vm/sqvmm.03 Trusted Dir /home/alistair/vmmaker/opensmalltalk-vm/sqvmm.03/secure Untrusted Dir /home/alistair/vmmaker/opensmalltalk-vm/sqvmm.03/My Squeak
RBPatternParser(Object)>>error: Receiver: a RBPatternParser Arguments and temporary variables: aString: 'End of statement list encounted' Receiver's instance variables: scanner: a RBPatternScanner currentToken: RBShortAssignmentToken nextToken: nil errorBlock: nil source: 'state at: `@index put: `@expr asC_unsigned_int' comments: an OrderedCollection() pragmas: nil
RBPatternParser(RBParser)>>parserError: Receiver: a RBPatternParser Arguments and temporary variables: aString: 'End of statement list encounted' Receiver's instance variables: scanner: a RBPatternScanner currentToken: RBShortAssignmentToken nextToken: nil errorBlock: nil source: 'state at: `@index put: `@expr asC_unsigned_int' comments: an OrderedCollection() pragmas: nil
RBPatternParser(RBParser)>>parseStatementList:into: Receiver: a RBPatternParser Arguments and temporary variables: pragmaBoolean: false sequenceNode: RBSequenceNode() statements: an OrderedCollection(RBMessageNode(state at: `@index put: `@expr as...etc... return: true periods: an OrderedCollection() returnPosition: nil node: RBMessageNode(state at: `@index put: `@expr asC) Receiver's instance variables: scanner: a RBPatternScanner currentToken: RBShortAssignmentToken nextToken: nil errorBlock: nil source: 'state at: `@index put: `@expr asC_unsigned_int' comments: an OrderedCollection() pragmas: nil
RBPatternParser(RBParser)>>parseStatements: Receiver: a RBPatternParser Arguments and temporary variables: pragmaBoolean: false args: #() leftBar: nil rightBar: nil Receiver's instance variables: scanner: a RBPatternScanner currentToken: RBShortAssignmentToken nextToken: nil errorBlock: nil source: 'state at: `@index put: `@expr asC_unsigned_int' comments: an OrderedCollection() pragmas: nil
RBPatternParser(RBParser)>>parseExpression: Receiver: a RBPatternParser Arguments and temporary variables: aString: 'state at: `@index put: `@expr asC_unsigned_int' node: nil Receiver's instance variables: scanner: a RBPatternScanner currentToken: RBShortAssignmentToken nextToken: nil errorBlock: nil source: 'state at: `@index put: `@expr asC_unsigned_int' comments: an OrderedCollection() pragmas: nil
RBPatternParser class(RBParser class)>>parseExpression:onError: Receiver: RBPatternParser Arguments and temporary variables: aString: 'state at: `@index put: `@expr asC_unsigned_int' aBlock: nil node: nil parser: a RBPatternParser Receiver's instance variables: superclass: RBParser methodDict: a MethodDictionary(#messageNodeClass->(RBPatternParser>>#messageNod...etc... format: 65543 instanceVariables: nil organization: ('private-classes' messageNodeClass methodNodeClass pragmaNodeClass...etc... subclasses: nil name: #RBPatternParser classPool: nil sharedPools: nil environment: Smalltalk category: #'AST-Core-Parser'
RBParser class>>parseRewriteExpression:onError: Receiver: RBParser Arguments and temporary variables: aString: 'state at: `@index put: `@expr asC_unsigned_int' aBlock: nil Receiver's instance variables: superclass: Object methodDict: a MethodDictionary(#addCommentsTo:->(RBParser>>#addCommentsTo: "a CompiledMethod...etc... format: 65543 instanceVariables: #('scanner' 'currentToken' 'nextToken' 'errorBlock' 'source'...etc... organization: ('private' addCommentsTo: nextToken patchLiteralArrayToken patchLiteralMessage...etc... subclasses: {RBPatternParser} name: #RBParser classPool: nil sharedPools: nil environment: Smalltalk category: #'AST-Core-Parser'
RBParser class>>parseRewriteExpression: Receiver: RBParser Arguments and temporary variables: aString: 'state at: `@index put: `@expr asC_unsigned_int' Receiver's instance variables: superclass: Object methodDict: a MethodDictionary(#addCommentsTo:->(RBParser>>#addCommentsTo: "a CompiledMethod...etc... format: 65543 instanceVariables: #('scanner' 'currentToken' 'nextToken' 'errorBlock' 'source'...etc... organization: ('private' addCommentsTo: nextToken patchLiteralArrayToken patchLiteralMessage...etc... subclasses: {RBPatternParser} name: #RBParser classPool: nil sharedPools: nil environment: Smalltalk category: #'AST-Core-Parser'
RBStringReplaceRule>>replaceString: Receiver: a RBStringReplaceRule Arguments and temporary variables: replaceString: 'state at: `@index put: `@expr asC_unsigned_int' Receiver's instance variables: searchTree: RBMessageNode(state at: `@index put: `@expr) owner: nil verificationBlock: [closure] in RBStringReplaceRule(RBReplaceRule)>>initialize replaceTree: nil
RBStringReplaceRule>>searchFor:replaceWith: Receiver: a RBStringReplaceRule Arguments and temporary variables: searchString: 'state at: `@index put: `@expr' replaceString: 'state at: `@index put: `@expr asC_unsigned_int' Receiver's instance variables: searchTree: RBMessageNode(state at: `@index put: `@expr) owner: nil verificationBlock: [closure] in RBStringReplaceRule(RBReplaceRule)>>initialize replaceTree: nil
RBStringReplaceRule class>>searchFor:replaceWith: Receiver: RBStringReplaceRule Arguments and temporary variables: searchString: 'state at: `@index put: `@expr' replaceString: 'state at: `@index put: `@expr asC_unsigned_int' Receiver's instance variables: superclass: RBReplaceRule methodDict: a MethodDictionary(#foundMatchFor:->(RBStringReplaceRule>>#foundMat...etc... format: 65540 instanceVariables: #('replaceTree') organization: ('matching' foundMatchFor:) ('initialize-release' methodReplaceString:...etc... subclasses: nil name: #RBStringReplaceRule classPool: nil sharedPools: nil environment: Smalltalk category: #'AST-Core-Matching'
RBParseTreeRewriter>>replace:with: Receiver: a RBParseTreeRewriter Arguments and temporary variables: searchString: 'state at: `@index put: `@expr' replaceString: 'state at: `@index put: `@expr asC_unsigned_int' Receiver's instance variables: searches: an OrderedCollection() answer: nil argumentSearches: an OrderedCollection() context: a RBSmallDictionary messages: nil tree: nil
UndefinedObject>>DoIt Receiver: nil Arguments and temporary variables: rules: a RBParseTreeRewriter Receiver's instance variables: nil
--- The full stack --- RBPatternParser(Object)>>error: RBPatternParser(RBParser)>>parserError: RBPatternParser(RBParser)>>parseStatementList:into: RBPatternParser(RBParser)>>parseStatements: RBPatternParser(RBParser)>>parseExpression: RBPatternParser class(RBParser class)>>parseExpression:onError: RBParser class>>parseRewriteExpression:onError: RBParser class>>parseRewriteExpression: RBStringReplaceRule>>replaceString: RBStringReplaceRule>>searchFor:replaceWith: RBStringReplaceRule class>>searchFor:replaceWith: RBParseTreeRewriter>>replace:with: UndefinedObject>>DoIt
Hi Levente,
On Mon, 5 Nov 2018 at 00:35, Levente Uzonyi leves@caesar.elte.hu wrote:
On Sun, 4 Nov 2018, Alistair Grant wrote:
Hi All,
If I attempt to execute the following:
| rules |
rules := RBParseTreeRewriter new.
rules replace: 'state at: `@index put: `@expr' with: 'state at: `@index put: `@expr asC_unsigned_int'.
I presume the version you use doesn't support underscores in selectors. Does it work if you use a selector with no underscores in it?
That's it, thanks! (it works without the underscores)
I hate it when I forget to include version information, sorry:
Squeak5.3alpha latest update: #18261 Image format 68021 (64 bit)
AST-Core-eem.97 Refactoring-Changes-eem.22 Refactoring-Core-ul.162 Refactoring-Environment-eem.11 Refactoring-Squeak-Platform-cwp.2 Refactoring-Tests-Changes-lr.16 Refactoring-Tests-Core-lr.56 Refactoring-Tests-Environment-eem.5
How did you load RB? Did you use the Metacello configuration? (it may not be up-to-date)
All the above appear to be the latest versions on http://www.squeaksource.com/rb.
This is a vmmaker image, they were all loaded through:
https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/image/BuildSqueak...
It sounds like there may be an updated version of RB that already addresses this?
Thanks again, Alistair
Levente
(rules executeTree: (MD5Plugin parseTreeFor: #md5ProcessBuffer:withState:)) ifTrue: [rules tree newSource inspect]. self error: 'parse tree rewrite failed'
I get "Error: End of statement list encountered" (stack trace below). The same works and produces the expected output in Pharo 7.
It looks like the Refactoring Browser has diverged quite a bit between Squeak and Pharo.
Who looks after RB in Squeak, and is there a plan to keep in sync with Pharo, or the other way around, or should this be fixed independently?
Thanks, Alistair
Hi Alistair,
I have found the cause of the problem, and it's easy to work the issue around. Just evaluate the following in a workspace:
Scanner allowUnderscoreAsAssignment: false. RBScanner initializeClassificationTable
The cause of the error is that RBScanner considers _ to be #special instead of #alphabetic when Scanner class >> #allowUnderscoreAsAssignment returns true. In Pharo this is not an issue, because underscore assignments have been abandoned and underscores in selectors are allowed permanently. But in Squeak everything can happen (so underscore can either be #special or #alphabetic depending on its context) and RBScanner would need some rewrite to support those cases.
I suggest the line
Scanner allowUnderscoreAsAssignment: false
be added to BuildSqueakSpurTrunkVMMakerImage.st before RB is loaded to avoid this issue in future images (because VMMaker code shouldn't rely on underscore assignments being supported).
Levente
On Mon, 5 Nov 2018, Alistair Grant wrote:
Hi Levente,
On Mon, 5 Nov 2018 at 00:35, Levente Uzonyi leves@caesar.elte.hu wrote:
On Sun, 4 Nov 2018, Alistair Grant wrote:
Hi All,
If I attempt to execute the following:
| rules |
rules := RBParseTreeRewriter new.
rules replace: 'state at: `@index put: `@expr' with: 'state at: `@index put: `@expr asC_unsigned_int'.
I presume the version you use doesn't support underscores in selectors. Does it work if you use a selector with no underscores in it?
That's it, thanks! (it works without the underscores)
I hate it when I forget to include version information, sorry:
Squeak5.3alpha latest update: #18261 Image format 68021 (64 bit)
AST-Core-eem.97 Refactoring-Changes-eem.22 Refactoring-Core-ul.162 Refactoring-Environment-eem.11 Refactoring-Squeak-Platform-cwp.2 Refactoring-Tests-Changes-lr.16 Refactoring-Tests-Core-lr.56 Refactoring-Tests-Environment-eem.5
How did you load RB? Did you use the Metacello configuration? (it may not be up-to-date)
All the above appear to be the latest versions on http://www.squeaksource.com/rb.
This is a vmmaker image, they were all loaded through:
https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/image/BuildSqueak...
It sounds like there may be an updated version of RB that already addresses this?
Thanks again, Alistair
Levente
(rules executeTree: (MD5Plugin parseTreeFor: #md5ProcessBuffer:withState:)) ifTrue: [rules tree newSource inspect]. self error: 'parse tree rewrite failed'
I get "Error: End of statement list encountered" (stack trace below). The same works and produces the expected output in Pharo 7.
It looks like the Refactoring Browser has diverged quite a bit between Squeak and Pharo.
Who looks after RB in Squeak, and is there a plan to keep in sync with Pharo, or the other way around, or should this be fixed independently?
Thanks, Alistair
Hi Levente,
On Mon, 5 Nov 2018 at 13:33, Levente Uzonyi leves@caesar.elte.hu wrote:
Hi Alistair,
I have found the cause of the problem, and it's easy to work the issue around. Just evaluate the following in a workspace:
Scanner allowUnderscoreAsAssignment: false. RBScanner initializeClassificationTable
Yep, that fixes it. Thanks very much for tracking this down.
The cause of the error is that RBScanner considers _ to be #special instead of #alphabetic when Scanner class >> #allowUnderscoreAsAssignment returns true. In Pharo this is not an issue, because underscore assignments have been abandoned and underscores in selectors are allowed permanently. But in Squeak everything can happen (so underscore can either be #special or #alphabetic depending on its context) and RBScanner would need some rewrite to support those cases.
I suggest the line
Scanner allowUnderscoreAsAssignment: false
be added to BuildSqueakSpurTrunkVMMakerImage.st before RB is loaded to avoid this issue in future images (because VMMaker code shouldn't rely on underscore assignments being supported).
Shall do.
Thanks again, Alistair
Levente
On Mon, 5 Nov 2018, Alistair Grant wrote:
Hi Levente,
On Mon, 5 Nov 2018 at 00:35, Levente Uzonyi leves@caesar.elte.hu wrote:
On Sun, 4 Nov 2018, Alistair Grant wrote:
Hi All,
If I attempt to execute the following:
| rules |
rules := RBParseTreeRewriter new.
rules replace: 'state at: `@index put: `@expr' with: 'state at: `@index put: `@expr asC_unsigned_int'.
I presume the version you use doesn't support underscores in selectors. Does it work if you use a selector with no underscores in it?
That's it, thanks! (it works without the underscores)
I hate it when I forget to include version information, sorry:
Squeak5.3alpha latest update: #18261 Image format 68021 (64 bit)
AST-Core-eem.97 Refactoring-Changes-eem.22 Refactoring-Core-ul.162 Refactoring-Environment-eem.11 Refactoring-Squeak-Platform-cwp.2 Refactoring-Tests-Changes-lr.16 Refactoring-Tests-Core-lr.56 Refactoring-Tests-Environment-eem.5
How did you load RB? Did you use the Metacello configuration? (it may not be up-to-date)
All the above appear to be the latest versions on http://www.squeaksource.com/rb.
This is a vmmaker image, they were all loaded through:
https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/image/BuildSqueak...
It sounds like there may be an updated version of RB that already addresses this?
Thanks again, Alistair
Levente
(rules executeTree: (MD5Plugin parseTreeFor: #md5ProcessBuffer:withState:)) ifTrue: [rules tree newSource inspect]. self error: 'parse tree rewrite failed'
I get "Error: End of statement list encountered" (stack trace below). The same works and produces the expected output in Pharo 7.
It looks like the Refactoring Browser has diverged quite a bit between Squeak and Pharo.
Who looks after RB in Squeak, and is there a plan to keep in sync with Pharo, or the other way around, or should this be fixed independently?
Thanks, Alistair
squeak-dev@lists.squeakfoundation.org