[Seaside-dev] Encoding issues? Entering 'euro sign' into a text input breaks render

Philippe Marschall philippe.marschall at gmail.com
Thu Oct 18 18:35:25 UTC 2007 

2007/10/17, John Thornborrow <john at pinesoft.co.uk>:
> Right.. I've found the issue and resolved it.
>
> The BMP Table on WAHtmlEncoder does not contain an element at value
> 8364, or 8365 (Euro's value is 8364) but it is less than the size of the
> table (65536) so it sends nil to RWBinaryOrTextStream>>nextPut:; which has:
>
> RWBinaryOrTextStream>>nextPut: charOrByte
>
>         super nextPut: charOrByte asCharacter
>
> This is where it fails - nil does not understand asCharacter.
>
> I have inspected the table, and noted that it was a majority of nil's -
> so I inspected it, and ran: "self at: 8365" but did not receive a
> notification..
>
> So then I ran WAHtmlEncoder class>>initializeBMP and checked the table
> again..
>
> Now when I run the tests - it works. I run my example, it works. I run
> my clients application, it works :)
>
> I'm not entirely sure what cause the table to have so many nil values,
> but it has now been repopulated correctly.. so the moral of the story:
> next time I'll blame monticello (and/or SqueakMap/Package Universe) for
> not calling class #initliaze when installing :)
>
> Sorry for wasting your time.

Np. Gee, I was already beginning to see ghosts.

Cheers
Philippe

> Regards,
> John.
>
>
> John Thornborrow wrote:
> > P.S. those are the characters being received by WAKom>>handleRequest: in
> > the post fields.
> >
> > John Thornborrow wrote:
> >> No, I meant #processHttpRequest: on HttpService class. Though I did also
> >> walkthrough WAKom handleRequest:
> >>
> >> Regards,
> >> John.
> >>
> >> Philippe Marschall wrote:
> >>> 2007/10/17, John Thornborrow <john at pinesoft.co.uk>:
> >>>> I've been walking-through the kom request process after halting in
> >>>> HttpService>>#handleRequest: and can see that the field finalises as
> >>>> unencoded string of three characters (which cannot be copy/pasted due to
> >>>> obvious encoding issues)
> >>> Do you mean WAKom>>#handleRequest:? If then what you observe is the
> >>> correct behaviour. The normal WAKom should represent the euro sign as
> >>> a ByteString with the three Characters (actually only bytes) 16rE2
> >>> 16r82 16rAC. Only the WAKomEncoded should represent it as a WideString
> >>> with one Character with value 16r20AC.
> >>>
> >>> Cheers
> >>> Philippe
> >>>
> >>>> However, it is just odd that I can challenge this with what I mentioned
> >>>> before and it will convert easily. I shall discuss with my colleague
> >>>> when he returns from sick leave, as this is edging me on to believe it
> >>>> is a squeak encoding issue - and he has a far superiour understanding of
> >>>> squeak's encoding than I. I will let you know the outcome; but in the
> >>>> meantime I shall continue to look/walk through Kom.
> >>>>
> >>>> Regards,
> >>>> John.
> >>>>
> >>>> Philippe Marschall wrote:
> >>>>> 2007/10/17, John Thornborrow <john at pinesoft.co.uk>:
> >>>>>> I have 3.10.2 vm, Windows XP (32Bit)
> >>>>>>
> >>>>>> Seaside 2.8beta from SqueakMap, with "Yes" to all options (Kom,
> >>>>>> Scriptaculous, RSS) Just reinstalled to confirm.
> >>>>>>
> >>>>>> KomHttpServer-pmm.2 - yes.
> >>>>>> Seaside2.8a1-pmm.476 - yes
> >>>>>>
> >>>>>> /seaside/tests/alltests (encoding) multipart fails for Euro symbol, and
> >>>>>> other unicode chars. (prime, left side quote, etc) Same error(s) as before.
> >>>>>>
> >>>>>> The sample code I provided in my first mail also breaks with the same
> >>>>>> result.
> >>>>> Wow, this is getting really strange, I just did the same and still can
> >>>>> not reproduce:
> >>>>> http://img502.imageshack.us/img502/3742/shotih6.png
> >>>>>
> >>>>> You did not change something in the session or browser properties? Is
> >>>>> there some way how you could build a clean image that exposes the bug
> >>>>> and give it to me?
> >>>>>
> >>>>> Cheers
> >>>>> Philippe
> >>>>>
> >>>>>> Thanks and regards,
> >>>>>> John.
> >>>>>>
> >>>>>> Philippe Marschall wrote:
> >>>>>>> 2007/10/16, John Thornborrow <john at pinesoft.co.uk>:
> >>>>>>>> So what am I to use instead? I've not seen anything about an alternative.
> >>>>>>>>
> >>>>>>>> I used SeasideInstaller - I always did, yet it left me with what I have
> >>>>>>>> been having trouble with today - the encoding test failing, and Seaside
> >>>>>>>> rendering breaking on any "special" character. I only installed other
> >>>>>>>> versions of KomHttpServer in a bid to try and fix this problem.
> >>>>>>> Ok, this is different then, you have my full attention. So lets recap:
> >>>>>>> - you use Squeak 3.9, correct?
> >>>>>>> - you use SeasideInstaller from SqueakMap, which version, 2.8beta?
> >>>>>>> - you use WAKom
> >>>>>>> - /seaside/tests/alltests the "Encoding" tests fails?
> >>>>>>> Anything else? I just did all this for Windows XP and a 3.10.6 VM and
> >>>>>>> the tests work. Can you confirm that you have KomHttpSever-pmm.2 and
> >>>>>>> Seaside2.8a1-pmm.476? This is configuration that is really supposed to
> >>>>>>> work. If this really doesn't work for you then I need your image
> >>>>>>> because otherwise I'm not able to reproduce this bug.
> >>>>>>>
> >>>>>>> Cheers
> >>>>>>> Philippe
> >>>>>>>
> >>>>>>>> Regards,
> >>>>>>>> John
> >>>>>>>>
> >>>>>>>> Philippe Marschall wrote:
> >>>>>>>>> 2007/10/16, John Thornborrow <john at pinesoft.co.uk>:
> >>>>>>>>>> I orignally had the Package Universe version of KomHttpServer (6.2) and
> >>>>>>>>>> received the same results - the test does not pass.
> >>>>>>>>> Then you did use WAKom in a Squeak 3.9 image. As noted several times
> >>>>>>>>> this is not supported. This is a bug in KomHttpServer. We contacted
> >>>>>>>>> the maintainer and sent him patches (2 methods). He's too busy.
> >>>>>>>>>
> >>>>>>>>>> The same is also true for many other characters, unicode 2000+ also
> >>>>>>>>>> cause the test and Seaside to fail rendering.
> >>>>>>>>>>
> >>>>>>>>>> This is proving to be havoc for my clients who are copying from Word
> >>>>>>>>>> documents made by publishers who make use of left/right-side quotes,
> >>>>>>>>>> primes etc. in their documents and wish to publish them on the web in
> >>>>>>>>>> HTML form. I agree, this is not your problem, it is mine, but surely you
> >>>>>>>>>> must be able to see that it is something to consider, rather than just
> >>>>>>>>>> sticking to the still-not-yet-fully-supported-by-browsers (X)HTML standards?
> >>>>>>>>> XHTML has _nothing_ at all to do with it. Just that the specified and
> >>>>>>>>> actual encoding don't match. You want options? I'll give you options.
> >>>>>>>>> They all work with all the languages you get on
> >>>>>>>>> http://www.columbia.edu/kermit/utf8.html, not only the euro symbol.
> >>>>>>>>>
> >>>>>>>>> - use the attached KomHttpServer version. This is the one from the
> >>>>>>>>> SeasideInstaller. This makes WAKom and WAKomEncoded work on Squeak 3.9
> >>>>>>>>> - use http://squeak.krampe.se/KomHttpServer-gk.14.mcz this makes
> >>>>>>>>> WAKomEncoded39 work on Squeak 3.9
> >>>>>>>>>
> >>>>>>>>> Anything beyond that I don't care.
> >>>>>>>>>
> >>>>>>>>> Cheers
> >>>>>>>>> Philippe
> >>>>>>>>>
> >>>>>>>>>> Regards,
> >>>>>>>>>> John
> >>>>>>>>>>
> >>>>>>>>>> Philippe Marschall wrote:
> >>>>>>>>>>> 2007/10/16, John Thornborrow <john at pinesoft.co.uk>:
> >>>>>>>>>>>> The /seaside/tests/alltests for encoding doesn't pass - it fails with:
> >>>>>>>>>>>>
> >>>>>>>>>>>> "Internal Error
> >>>>>>>>>>>>
> >>>>>>>>>>>> MessageNotUnderstood: UndefinedObject>>asCharacter
> >>>>>>>>>>>>
> >>>>>>>>>>>> RWBinaryOrTextStream>>nextPut:
> >>>>>>>>>>>> WAHtmlEncoder(WAEncoder)>>nextPutAll:
> >>>>>>>>>>>> ByteString(String)>>encodeOn:
> >>>>>>>>>>>> WAHtmlStreamDocument>>print:
> >>>>>>>>>>>> WARenderCanvas(WAHtmlCanvas)>>text:
> >>>>>>>>>>>> ByteString(String)>>renderOn:
> >>>>>>>>>>>> [] in WAGenericTag(WATagBrush)>>with: {[self before. anObject renderOn:
> >>>>>>>>>>>> canvas. self after]}
> >>>>>>>>>>>>
> >>>>>>>>>>>> etc.."
> >>>>>>>>>>>>
> >>>>>>>>>>>> I've got the lastest KomHttpServer from squeaksource.com and I'm not
> >>>>>>>>>>>> sure what version WAKom* is - I thought this was part of the seaside
> >>>>>>>>>>>> package?
> >>>>>>>>>>> No, we only support the official one on SqueakMap and the one from
> >>>>>>>>>>> netstyle.ch which comes with the SeasideInstaller.
> >>>>>>>>>>>
> >>>>>>>>>>>> I've made a work around, and that is to include:
> >>>>>>>>>>> Use one of the supported KomHttpServers. They should pass the
> >>>>>>>>>>> /seaside/tests/alltests at least in ecoding mode.
> >>>>>>>>>>>
> >>>>>>>>>>> If you use an other version of Kom we don't care. We're sorry but
> >>>>>>>>>>> already supporting these is enough of a mess.
> >>>>>>>>>>>
> >>>>>>>>>>> Cheers
> >>>>>>>>>>> Philippe
> >>>>>>>>>>>
> >>>>>>>>>>>> aCharacter = (Character value: 1069555884) ifTrue: [^ aStream
> >>>>>>>>>>>> nextPutAll: '&euro;' ] .
> >>>>>>>>>>>>
> >>>>>>>>>>>> to the list of characters in WAHtmlEncoder class>>encode:on: but given
> >>>>>>>>>>>> the ourcome of the last time I made changes to this method, it is not
> >>>>>>>>>>>> the preferred/accepted solution.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Regards,
> >>>>>>>>>>>> John
> >>>>>>>>>>>>
> >>>>>>>>>>>> Philippe Marschall wrote:
> >>>>>>>>>>>>> 2007/10/16, John Thornborrow <john at pinesoft.co.uk>:
> >>>>>>>>>>>>>> Hello all,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I've discussed this briefly before on the list, following my
> >>>>>>>>>>>>>> contribution regarding encoding more characters than what the (X)HTML
> >>>>>>>>>>>>>> spec defines - but did not have an example at the time.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> However, I have just stumbled upon a significant problem which I wanted
> >>>>>>>>>>>>>> to raise as an issue to the dev-list; that is that Seasides rendering
> >>>>>>>>>>>>>> does not like the Euro-symbol.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> The simplest example:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> renderContentOn: html
> >>>>>>>>>>>>>>   html form with: [
> >>>>>>>>>>>>>>     html textInput callback: [ :x | text := x ].
> >>>>>>>>>>>>>>     html paragraph with: text.
> >>>>>>>>>>>>>>     html submitButton
> >>>>>>>>>>>>>>   ]
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On the form, enter a Euro symbol (ALT+0128 on Windows) in the text box
> >>>>>>>>>>>>>> and submit.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> After submitting, it is very obvious that there is a problem (the screen
> >>>>>>>>>>>>>> will be filled with characters that cannot be rendered.)
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Just incase it is something I am doing wrong, I have attached a file-out
> >>>>>>>>>>>>>> of the example component. If anyone could confirm this is a problem/bug
> >>>>>>>>>>>>>> then I shall submit a bug-report, providing it is not already reported :).
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> However, I've also noticed that the WAEncoder>>initializeBMP does not
> >>>>>>>>>>>>>> reach the range necessary to encode the euro (Character value: 1069555884)
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> P.S. Tested with the latest version available in Monticello as of today,
> >>>>>>>>>>>>>> and this is on Squeak 32bit 3.10 vm on Windows platform.
> >>>>>>>>>>>>> What combination of WAKom* and KomHttpServer do you run?
> >>>>>>>>>>>>> WAEncodingTest under /seaside/test/alltests should test this exact
> >>>>>>>>>>>>> scenario and run.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Cheers
> >>>>>>>>>>>>> Philippe
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Regards,
> >>>>>>>>>>>>>> John
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Pinesoft Computers are registered in England, Registered number: 2914825. Registered office: 266-268 High Street, Waltham Cross, Herts, EN8 7EA
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> This message has been scanned for viruses by BlackSpider MailControl - www.blackspider.com
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> 'From Squeak3.9 of 7 November 2006 [latest update: #7067] on 16 October 2007 at 11:50:34 am'!
> >>>>>>>>>>>>>> WAComponent subclass: #BrokenEuro
> >>>>>>>>>>>>>>         instanceVariableNames: 'text'
> >>>>>>>>>>>>>>         classVariableNames: ''
> >>>>>>>>>>>>>>         poolDictionaries: ''
> >>>>>>>>>>>>>>         category: 'MyStuff'!
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> !BrokenEuro methodsFor: 'rendering' stamp: 'jmt 10/16/2007 10:58'!
> >>>>>>>>>>>>>> renderContentOn: html
> >>>>>>>>>>>>>>         html form with: [
> >>>>>>>>>>>>>>                 html textInput callback: [ :x | text := x ].
> >>>>>>>>>>>>>>                 html paragraph with: text.
> >>>>>>>>>>>>>>                 html submitButton
> >>>>>>>>>>>>>>         ]! !
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> "-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- "!
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> BrokenEuro class
> >>>>>>>>>>>>>>         instanceVariableNames: ''!
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> !BrokenEuro class methodsFor: 'as yet unclassified' stamp: 'jmt 10/16/2007 11:49'!
> >>>>>>>>>>>>>> canBeRoot
> >>>>>>>>>>>>>>         ^true! !
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> _______________________________________________
> >>>>>>>>>>>>>> seaside-dev mailing list
> >>>>>>>>>>>>>> seaside-dev at lists.squeakfoundation.org
> >>>>>>>>>>>>>> http://lists.squeakfoundation.org/mailman/listinfo/seaside-dev
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> _______________________________________________
> >>>>>>>>>>>>> seaside-dev mailing list
> >>>>>>>>>>>>> seaside-dev at lists.squeakfoundation.org
> >>>>>>>>>>>>> http://lists.squeakfoundation.org/mailman/listinfo/seaside-dev
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>> _______________________________________________
> >>>>>>>>>>>> seaside-dev mailing list
> >>>>>>>>>>>> seaside-dev at lists.squeakfoundation.org
> >>>>>>>>>>>> http://lists.squeakfoundation.org/mailman/listinfo/seaside-dev
> >>>>>>>>>>>>
> >>>>>>>>>>> _______________________________________________
> >>>>>>>>>>> seaside-dev mailing list
> >>>>>>>>>>> seaside-dev at lists.squeakfoundation.org
> >>>>>>>>>>> http://lists.squeakfoundation.org/mailman/listinfo/seaside-dev
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>> _______________________________________________
> >>>>>>>>>> seaside-dev mailing list
> >>>>>>>>>> seaside-dev at lists.squeakfoundation.org
> >>>>>>>>>> http://lists.squeakfoundation.org/mailman/listinfo/seaside-dev
> >>>>>>>>>>
> >>>>>>>>> ------------------------------------------------------------------------
> >>>>>>>>>
> >>>>>>>>> _______________________________________________
> >>>>>>>>> seaside-dev mailing list
> >>>>>>>>> seaside-dev at lists.squeakfoundation.org
> >>>>>>>>> http://lists.squeakfoundation.org/mailman/listinfo/seaside-dev
> >>>>>>>> _______________________________________________
> >>>>>>>> seaside-dev mailing list
> >>>>>>>> seaside-dev at lists.squeakfoundation.org
> >>>>>>>> http://lists.squeakfoundation.org/mailman/listinfo/seaside-dev
> >>>>>>>>
> >>>>>>> _______________________________________________
> >>>>>>> seaside-dev mailing list
> >>>>>>> seaside-dev at lists.squeakfoundation.org
> >>>>>>> http://lists.squeakfoundation.org/mailman/listinfo/seaside-dev
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>> _______________________________________________
> >>>>>> seaside-dev mailing list
> >>>>>> seaside-dev at lists.squeakfoundation.org
> >>>>>> http://lists.squeakfoundation.org/mailman/listinfo/seaside-dev
> >>>>>>
> >>>>> _______________________________________________
> >>>>> seaside-dev mailing list
> >>>>> seaside-dev at lists.squeakfoundation.org
> >>>>> http://lists.squeakfoundation.org/mailman/listinfo/seaside-dev
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>> _______________________________________________
> >>>> seaside-dev mailing list
> >>>> seaside-dev at lists.squeakfoundation.org
> >>>> http://lists.squeakfoundation.org/mailman/listinfo/seaside-dev
> >>>>
> >>> _______________________________________________
> >>> seaside-dev mailing list
> >>> seaside-dev at lists.squeakfoundation.org
> >>> http://lists.squeakfoundation.org/mailman/listinfo/seaside-dev
> >>>
> >>>
> >>>
> >>>
> >> _______________________________________________
> >> seaside-dev mailing list
> >> seaside-dev at lists.squeakfoundation.org
> >> http://lists.squeakfoundation.org/mailman/listinfo/seaside-dev
> >>
> >>
> >>
> >>
> > _______________________________________________
> > seaside-dev mailing list
> > seaside-dev at lists.squeakfoundation.org
> > http://lists.squeakfoundation.org/mailman/listinfo/seaside-dev
> >
> >
> >
> >
> _______________________________________________
> seaside-dev mailing list
> seaside-dev at lists.squeakfoundation.org
> http://lists.squeakfoundation.org/mailman/listinfo/seaside-dev
>

More information about the seaside-dev mailing list