[Seaside-dev] A little bit more on encodings

[Julian Fitzell]

On Mon, Jun 15, 2009 at 1:04 PM, Michael
Lucas-Smith<mlucas-smith at cincom.com> wrote:
> Hi All,
>
> I'm digging in to why WAEncodingFunctionalTest was failing for me and
> discovered that the WABufferedResponse is not set up the way I would expect.
>
> It has a header that says the content-type's charset is utf8, however the
> actual contentsStream is a WANullCodecStream wrapped around a plain
> WriteStream containing a collection of a TwoByteString of characters.
>
> Basically... no encoding has gone on at all - so naturally when I try to
> send the response back to the browser, it's not bytes and it's not utf8, so
> i get gibberish.
>
> Where was the content type of the response even supposed to have been setup?
> What's puzzling is that the codec of a WAServerAdaptor always seems to be a
> WANullCodec, which flows on down in to the rest of the response handling
> mechanisms and never appears to change?
>
> I must be missing something....

Well, you need to configure the codec you want the server adaptor to
use, for a start. WANullCodec is fundamentally pretty broken as
Philippe will tell you at any opportunity. It's also the behaviour
that older versions (pre-2.6 or something?) of Seaside used and means
a lot of people have these probably-UTF-8-encoded strings in their
image. It's also (pragmatically speaking as Lukas will tell you) you
the codec that a lot of people still want to use rather than mucking
around with conversions all over the place.

WANullCodec simply assumes that you know what you are putting in the
Response and that you know what you are getting in the Request. You
need to set the header appropriately. Since most people have been
using this behaviour for ages and most browsers send utf-8 data, most
people have utf-8 encoded bytes in their image as strings and unless
they want to recode all of that data, they need to continue sending
those strings out to the browser without any encoding happening. They
just send the right bytes, set the header the right way, and it works.

Is it pretty? Absolutely not. Legacy data is a female dog...

(this incidentally is why I would really like to get a better codec
implementation fully sorted before releasing this version - I'd like
to have a base in place so that we can really start pushing people
towards proper encoding support in the release after)

Julian