Hi Christoph --

+1

If we do not treat an NBSP as a delimiter, it could sneak into symbols without knowing or seeing it.

Best,
Marcel
Am 01.04.2022 18:41:18 schrieb commits at source.squeak.org <commits at source.squeak.org>:
A new version of Compiler was added to project The Inbox:
http://source.squeak.org/inbox/Compiler-ct.473.mcz

==================== Summary ====================

Name: Compiler-ct.473
Author: ct
Time: 1 April 2022, 6:41:06.522422 pm
UUID: fa612011-d500-8946-a772-b460d0743c67
Ancestors: Compiler-eem.472

Proposal: Treat non-breaking space (16ra0) as delimiter in parser/scanner.

Compiler evaluate: '2 +?3'.

=============== Diff against Compiler-eem.472 ===============

Item was changed:
----- Method: Scanner class>>initializeTypeTable (in category 'initialization') -----
initializeTypeTable
"self initializeTypeTable"

| newTable |
newTable := Array new: 256 withAll: #xBinary. "default"
+ newTable atAll: #(9 10 12 13 32 16ra0) put: #xDelimiter. "tab lf ff cr space nbsp"
- newTable atAll: #(9 10 12 13 32 ) put: #xDelimiter. "tab lf ff cr space"
newTable atAll: ($0 asciiValue to: $9 asciiValue) put: #xDigit.

1 to: 255
do: [:index |
(Character value: index) isLetter
ifTrue: [newTable at: index put: #xLetter]].

newTable at: $" asciiValue put: #xDoubleQuote.
newTable at: $# asciiValue put: #xLitQuote.
newTable at: $$ asciiValue put: #xDollar.
newTable at: $' asciiValue put: #xSingleQuote.
newTable at: $: asciiValue put: #xColon.
newTable at: $( asciiValue put: #leftParenthesis.
newTable at: $) asciiValue put: #rightParenthesis.
newTable at: $. asciiValue put: #period.
newTable at: $; asciiValue put: #semicolon.
newTable at: $[ asciiValue put: #leftBracket.
newTable at: $] asciiValue put: #rightBracket.
newTable at: ${ asciiValue put: #leftBrace.
newTable at: $} asciiValue put: #rightBrace.
newTable at: $^ asciiValue put: #upArrow.
newTable at: $_ asciiValue put: #xUnderscore.
newTable at: $| asciiValue put: #verticalBar.
TypeTable := newTable "bon voyage!!"!


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20220408/8ae1b162/attachment.html>