On Wed, 23 Jul 2008 18:42:15 +0200, Klaus D. Witzel wrote:

> On Wed, 23 Jul 2008 17:46:58 +0200, Yoshiki Ohshima wrote:
>
>> At Sat, 19 Jul 2008 14:55:45 +0200,
>> Klaus D. Witzel wrote:
>>>
>>> Hi,
>>>
>>> I'm trying to successfully fileOut and fileIn class and method comments
>>> with characters codePoint <= 255 (the "european" range of byte size
>>> characters).
>>>
>>> Fileout works but the file looks to have prefix from #writeBOMOn: of
>>> UTF8TextConverter and file suffix is ".st" or ".cs". On fileIn this  
>>> fails
>>> with syntax error :( and Google search did not help. Has anyone  
>>> experience
>>> with source code fileOut/fileIn for the "european" character range?
>>>
>>> Did a quick shot on FileStream class>>#writeSourceCodeFrom:*, with
>>> #isAsciiString replaced by #isWideString (followed by ifFalse:ifTrue:),
>>> filed out again and then fileIn was happy again.
>>>
>>> I understand this is perhaps not really sort of bug, but want a bit  
>>> more
>>> than just the ascii characters. So what do you Squeakers recommend, how
>>> should that be done in a Squeak platform compatible way?
>>
>>   What version is it?
>
> Found the method in 3.10 and checked it against 3.9 (same method), but  
> in 3.8 the method source code is garbled (decompiler shows it's the same  
> method).
>
> Meanwhile I did:
>
> filedIn unpatched 3.9 from fileOut of my patched 3.10 (has no BOM):  
> worked
> filedOut again from 3.9: (it wrote BOM as expected)
> filedIn 3.9 the fileOut from 3.9 (it has BOM): it worked
> filedIn 3.10 the fileOut from 3.9 (it has BOM): it worked
>
> Now I'm stuck, since the problem was not reproduced ? (did a lot of  
> changes meanwhile to the class comment in question). Will have to go  
> through the .changes and try to find the problematic chunk. FWIW the net  
> effect was the compiler raised a SyntaxError complaining about the BOM  
> characters.
>
> I'll let you know what I find.

Found that my .changes file also has some UTF8TextConverter problem (see  
log below) which lets me not find the class comment change :(

Will have to start with a fresh stock .image and, if something happens  
again, try to report it reproducable.

/Klaus

------------------
Error: Invalid utf8 input detected
25 July 2008 12:05:23 am

VM: Win32 - a SmalltalkImage
Image: Squeak3.10.2 [latest update: #7179]

SecurityManager state:
Restricted: false
FileAccess: true
SocketAccess: true
Working Dir ...\SqueakDev
Trusted Dir ...\SqueakDev\kWitzel
Untrusted Dir ...\kWitzel

UTF8TextConverter(Object)>>error:
	Receiver: an UTF8TextConverter
	Arguments and temporary variables:
		aString: 	'Invalid utf8 input detected'
	Receiver's instance variables:
		acceptingEncodings: 	nil
		currentCharSize: 	1
		forceToEncodingTag: 	nil

UTF8TextConverter>>errorMalformedInput
	Receiver: an UTF8TextConverter
	Arguments and temporary variables:

	Receiver's instance variables:
		acceptingEncodings: 	nil
		currentCharSize: 	1
		forceToEncodingTag: 	nil

UTF8TextConverter>>nextFromStream:
	Receiver: an UTF8TextConverter
	Arguments and temporary variables:
		aStream: 	MultiByteFileStream: '...\Squeak3.10.2...etc...
		character1: 	$‰
		value1: 	137
		character2: 	Character tab
		value2: 	9
		unicode: 	nil
		character3: 	Character tab
		value3: 	9
		character4: 	nil
		value4: 	nil
	Receiver's instance variables:
		acceptingEncodings: 	nil
		currentCharSize: 	1
		forceToEncodingTag: 	nil

MultiByteFileStream>>next
	Receiver: MultiByteFileStream:  
'...\Squeak3.10.2-7179-basic.changes'...etc...
	Arguments and temporary variables:
		char: 	nil
		secondChar: 	nil
		state: 	nil
	Receiver's instance variables:



--- The full stack ---
UTF8TextConverter(Object)>>error:
UTF8TextConverter>>errorMalformedInput
UTF8TextConverter>>nextFromStream:
MultiByteFileStream>>next
  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
MultiByteFileStream>>skipSeparators
MultiByteFileStream(PositionableStream)>>nextChunk
ChangeList class>>browseRecentLogOn:
ChangeList class>>browseRecentLogOnPath:
SimpleServiceEntry>>performServiceFor:
PluggableButtonMorph>>performAction
[] in PluggableButtonMorph>>mouseUp: {[:m | (m containsPoint: evt  
cursorPoint)   ifTrue: [m performAction]]}
Array(SequenceableCollection)>>do:
PluggableButtonMorph>>mouseUp:
PluggableButtonMorph(Morph)>>handleMouseUp:
MouseButtonEvent>>sentTo:
PluggableButtonMorph(Morph)>>handleEvent:
PluggableButtonMorph(Morph)>>handleFocusEvent:
[] in HandMorph>>sendFocusEvent:to:clear: {[ActiveHand := self.   
ActiveEvent := anEvent.  result := focusHolder     han...]}
[] in PasteUpMorph>>becomeActiveDuring: {[aBlock value]}
BlockContext>>on:do:
PasteUpMorph>>becomeActiveDuring:
HandMorph>>sendFocusEvent:to:clear:
HandMorph>>sendEvent:focus:clear:
HandMorph>>sendMouseEvent:
HandMorph>>handleEvent:
HandMorph>>processEvents
[] in WorldState>>doOneCycleNowFor: {[:h |  ActiveHand := h.  h  
processEvents.  capturingGesture := capturingGest...]}
Array(SequenceableCollection)>>do:
WorldState>>handsDo:
WorldState>>doOneCycleNowFor:
WorldState>>doOneCycleFor:
PasteUpMorph>>doOneCycle
[] in Project class>>spawnNewProcess {[[World doOneCycle.  Processor  
yield.  false] whileFalse.  nil]}
[] in BlockContext>>newProcess {[self value.  Processor terminateActive]}