[squeak-dev] The Trunk: Multilingual-ul.115.mcz
commits at source.squeak.org
commits at source.squeak.org
Sat Mar 27 20:36:20 UTC 2010
Levente Uzonyi uploaded a new version of Multilingual to project The Trunk:
http://source.squeak.org/trunk/Multilingual-ul.115.mcz
==================== Summary ====================
Name: Multilingual-ul.115
Author: ul
Time: 27 March 2010, 9:30:44.52 pm
UUID: efcad9da-606e-ea4b-9cb8-a19d6e18a164
Ancestors: Multilingual-ul.114
- move separator skip methods to the TextConverters
=============== Diff against Multilingual-ul.114 ===============
Item was added:
+ ----- Method: TextConverter>>skipSeparatorsAndPeekNextFrom: (in category 'fileIn/Out') -----
+ skipSeparatorsAndPeekNextFrom: aStream
+ "Same as #skipSeparators, but returns the next character after the separators if such exists."
+
+ | state character |
+ [
+ state := self saveStateOf: aStream.
+ (character := aStream next)
+ ifNil: [ false ]
+ ifNotNil: [ character isSeparator ] ] whileTrue.
+ character ifNotNil: [
+ self restoreStateOf: aStream with: state.
+ ^character ].
+ ^aStream
+ !
Item was changed:
----- Method: UTF8TextConverter>>nextChunkFromStream: (in category 'fileIn/Out') -----
nextChunkFromStream: input
"Answer the contents of input, up to the next terminator character. Doubled terminators indicate an embedded terminator character."
+ self skipSeparatorsFrom: input.
- input skipSeparators.
^self
parseLangTagFor: (
+ self class decodeByteString: (
+ String new: 1000 streamContents: [ :stream |
+ [
+ stream nextPutAll: (input basicUpTo: $!!).
+ input basicNext == $!! ]
+ whileTrue: [
+ stream nextPut: $!! ].
+ input atEnd ifFalse: [ input skip: -1 ] ]))
- String new: 1000 streamContents: [ :stream |
- [
- stream nextPutAll: (input basicUpTo: $!!).
- input basicNext == $!! ]
- whileTrue: [
- stream nextPut: $!! ].
- input atEnd ifFalse: [ input skip: -1 ] ]) utf8ToSqueak
fromStream: input!
Item was added:
+ ----- Method: UTF8TextConverter>>skipSeparatorsAndPeekNextFrom: (in category 'fileIn/Out') -----
+ skipSeparatorsAndPeekNextFrom: aStream
+ "Same as #skipSeparators, but returns the next character after the separators if such exists."
+
+ | character |
+ [
+ ((character := aStream basicNext)
+ ifNil: [ ^aStream "backwards compatibility, should be nil" ])
+ isSeparator ] whileTrue.
+ aStream skip: -1.
+ ^character
+ !
Item was changed:
----- Method: MultiByteFileStream>>skipSeparatorsAndPeekNext (in category 'public') -----
skipSeparatorsAndPeekNext
+ "A special function to make nextChunk fast. Same as #skipSeparators, but returns the next character after the separators if such exists."
+
+ ^converter skipSeparatorsAndPeekNextFrom: self!
- "Same as #skipSeparators, but returns the next character after the separators if such exists."
-
- | state character |
- [
- state := converter saveStateOf: self.
- (character := self next)
- ifNil: [ false ]
- ifNotNil: [ character isSeparator ] ] whileTrue.
- character ifNotNil: [
- converter restoreStateOf: self with: state.
- ^character ].
- !
Item was added:
+ ----- Method: TextConverter>>skipSeparatorsFrom: (in category 'fileIn/Out') -----
+ skipSeparatorsFrom: aStream
+
+ | state character |
+ [
+ state := self saveStateOf: aStream.
+ (character := aStream next)
+ ifNil: [ false ]
+ ifNotNil: [ character isSeparator ] ] whileTrue.
+ character ifNotNil: [
+ self restoreStateOf: aStream with: state ]
+ !
Item was added:
+ ----- Method: UTF8TextConverter>>skipSeparatorsFrom: (in category 'fileIn/Out') -----
+ skipSeparatorsFrom: aStream
+
+ [ (aStream basicNext ifNil: [ ^self ]) isSeparator ] whileTrue.
+ aStream skip: -1!
Item was changed:
----- Method: MultiByteFileStream>>skipSeparators (in category 'public') -----
skipSeparators
+ converter skipSeparatorsFrom: self!
- | state character |
- [
- state := converter saveStateOf: self.
- (character := self next)
- ifNil: [ false ]
- ifNotNil: [ character isSeparator ] ] whileTrue.
- character ifNotNil: [
- converter restoreStateOf: self with: state ]!
Item was changed:
----- Method: UTF8TextConverter>>parseLangTagFor:fromStream: (in category 'fileIn/Out') -----
parseLangTagFor: aString fromStream: stream
+ | position |
+ position := stream position.
- | state |
- state := self saveStateOf: stream.
"Test for ]lang[ tag"
+ ((self skipSeparatorsAndPeekNextFrom: stream) == $] and: [
- (stream skipSeparatorsAndPeekNext == $] and: [
(stream basicNext: 6) = ']lang[' ]) ifTrue: [
^stream
decodeString: aString
andRuns: (self nextChunkFromStream: stream) ].
"no tag"
+ stream position: position.
- self restoreStateOf: stream with: state.
^aString!
Item was changed:
----- Method: UTF8TextConverter>>nextChunkTextFromStream: (in category 'fileIn/Out') -----
nextChunkTextFromStream: input
"Deliver the next chunk as a Text. Decode the following ]style[ chunk if present. Position at start of next real chunk."
+ | chunk position runs |
- | chunk state runs |
chunk := self nextChunkFromStream: input.
+ position := input position.
+ ((self skipSeparatorsAndPeekNextFrom: input) == $] and: [
- state := self saveStateOf: input.
- (input skipSeparatorsAndPeekNext == $] and: [
(input basicNext: 7) = ']style[' ])
ifTrue: [
runs := RunArray scanFrom: (self nextChunkFromStream: input) readStream ]
ifFalse: [
+ input position: position.
- self restoreStateOf: input with: state.
runs := RunArray new: chunk size withAll: #() ].
^Text string: chunk runs: runs!
More information about the Squeak-dev
mailing list
|