[Seaside] Zinc-Seaside: Problem with POSTing UTF-8 JSON
Tomas Kukol
tomas.kukol at gmail.com
Wed Jun 26 11:20:40 UTC 2013
Hi Sven,
you are right. Seaside needs it as a string. But I've found another problem
with output (maybe because Seaside uses string???) which leads to error
'Improper store into indexable object'.
I have JSON on output. It is again string and there is this problem. In
ZnZincServerAdaptor>>responseFrom: we are asking for contents.
ZnZincServerAdaptor>>responseFrom: aRequestContext
| partialHeaders cookies fullHeaders seasideResponse contents entity
contentType |
seasideResponse := aRequestContext response.
partialHeaders := seasideResponse headers.
cookies := seasideResponse cookies.
fullHeaders := ZnHeaders defaultResponseHeaders.
partialHeaders keysAndValuesDo: [ :key :value |
fullHeaders at: key put: value ].
cookies do: [ :each |
fullHeaders at: 'Set-Cookie' add: each oldNetscapeString.
fullHeaders at: 'Set-Cookie2' add: each rfc2965String ].
contentType := seasideResponse contentType greaseString.
contents := seasideResponse contents. "<------------------------ HERE - we
are asking for contents"
entity := (ZnEntity bytes: contents) contentType: contentType; yourself.
^ ZnResponse new
statusLine: (ZnStatusLine code: seasideResponse status);
headers: fullHeaders;
entity: entity;
yourself
It calls:
WABufferedResponse>>contents
^ contentsStream contents "<------------------------ HERE - normally,
asking underlying stream for contents"
An the error is coming here:
RWBinaryOrTextStream>>contents
"Answer with a copy of my collection from 1 to readLimit."
| newArray |
isBinary ifFalse: [^ super contents]. "String"
readLimit := readLimit max: position.
newArray := ByteArray new: readLimit. "<------------------------ HERE - we
are creating array of bytes (SmallInteger 0 - 255)"
^ newArray replaceFrom: 1
to: readLimit
with: collection "<------------------------ HERE - but we are having
WideString in collection - characters"
startingAt: 1.
Now we are calling primitive 105:
ByteArray>>replaceFrom: start to: stop with: replacement startingAt:
repStart
"Primitive. This destructively replaces elements from start to stop in the
receiver starting at index, repStart, in the collection, replacement.
Answer the receiver. Range checks are performed in the primitive only.
Optional. See Object documentation whatIsAPrimitive."
<primitive: 105>
super replaceFrom: start to: stop with: replacement startingAt: repStart
"<------------------------ HERE - it fails and continues here"
It calls loop in SequenceableCollection:
ByteArray(SequenceableCollection)>>replaceFrom: start to: stop with:
replacement startingAt: repStart
"This destructively replaces elements from start to stop in the receiver
starting at index, repStart, in the sequenceable collection,
replacementCollection. Answer the receiver. No range checks are
performed."
| index repOff |
repOff := repStart - start.
index := start - 1.
[(index := index + 1) <= stop]
whileTrue: [self at: index put: (replacement at: repOff + index)]
"<------------------------ HERE - here we are putting character into
ByteArray"
Stops here at Object at primitive 61:
ByteArray(Object)>>at: index put: value
"Primitive. Assumes receiver is indexable. Store the argument value in
the indexable element of the receiver indicated by index. Fail if the
index is not an Integer or is out of bounds. Or fail if the value is not of
the right type for this kind of collection. Answer the value that was
stored. Essential. See Object documentation whatIsAPrimitive."
<primitive: 61>
index isInteger ifTrue:
[self class isVariable
ifTrue: [(index >= 1 and: [index <= self size])
ifTrue: [self errorImproperStore] "<------------------------ HERE - ends
here"
ifFalse: [self errorSubscriptBounds: index]]
ifFalse: [self errorNotIndexable]].
index isNumber
ifTrue: [^self at: index asInteger put: value]
ifFalse: [self errorNonIntegerIndex]
I've seen this problem mostly with Seaside-REST, but also in some other
posts from Seaside.
Any idea how to fix it?
Regards,
Tomas
On Wed, Jun 26, 2013 at 12:17 AM, Sven Van Caekenberghe <sven at stfx.eu>wrote:
> Hi Tomas,
>
> Thanks for the report: this is an important area indeed.
>
> However, the comment says it: this is intentional, and as far as I know,
> it is correct. You see, Zinc will correctly decode/encode any HTTP payload,
> using the supplied mime-type and/or charsets. Seaside is written such that
> it does not want this: it insists on doing this on its own. That is why
> ZnZincServerAdaptor is _not_ using the normal request reading code of Zn,
> but uses a special option to read everything binary. The stupid thing is
> that even though Seaside needs bytes, it wants them as a String. That is
> the reason for the otherwise brain dead #asString (and the implicit copy is
> inefficient as well).
>
> Of course, you will see your special characters there if you do UTF8
> decoding, but that is because you already know what is inside.
>
> What normally happens, is that later on in the processing, Seaside will
> access the WARequest payload using proper decoding, using its own framework
> (much like what Zn would do). AFAIK this whole process works. You can
> actually test this using some of the functional tests.
>
> I am not sure that Seaside-REST is doing the right thing (there were some
> issues with SmalltalkHub as well), but I would think so.
>
> Are you sure you have set the correct encoding on the adaptor ?
>
> Are you sure you are posting as application/json;charset=utf-8 and if you
> do not set the charset, are you sure utf-8 is the default ?
>
> Are you sure your REST handler and/or JSON parser does the right thing ?
>
> It is too late right now, but if we want to get further with this, I will
> need a failing unit test - if these exist in Seaside-REST, but I would
> assume so. I have no experience running Seaside-REST, I am using Zinc-REST
> myself, but I would like to learn.
>
> Regards,
>
> Sven
>
> On 25 Jun 2013, at 23:24, Tomas Kukol <tomas.kukol at gmail.com> wrote:
>
> > Hi Sven.
> >
> > I've had a problem when POSTing non-ascii UTF-8 characters in JSON to
> Seaside REST service. I've located the problem in the method
> ZnZincServerAdaptor>>requestBodyFor: where the body of ZnRequest is
> translated to body of WARequest. I use Pharo 1.4 with Seaside 3.0.8 and
> Zinc-Seaside-SvenVanCaekenberghe.40.
> >
> > When the POSTed JSON contains non-ascii UTF-8 characters (Czech
> characters), they are corrupted. The problem is on the "MARKED" line, where
> the array of bytes changed to string by asString.
> >
> > "Problematic" code:
> >
> > ZnZincServerAdaptor>>requestBodyFor: aZincRequest
> > ^ (aZincRequest method ~= #TRACE
> > and: [ aZincRequest hasEntity
> > and: [ aZincRequest entity isEmpty not
> > and: [ (aZincRequest entity contentType
> matches: ZnMimeType applicationFormUrlEncoded) not
> > and: [ (aZincRequest entity
> contentType matches: ZnMimeType multiPartFormData) not ] ] ] ])
> > ifTrue: [
> > "Seaside wants to do its own text
> conversions"
> > aZincRequest entity bytes asString
> "MARKED" ]
> > ifFalse: [
> > String new ]
> >
> > I did a quick correction, which is not nice, but works for me:
> >
> > ZnZincServerAdaptor>>requestBodyFor: aZincRequest
> > ^ (aZincRequest method ~= #TRACE
> > and: [ aZincRequest hasEntity
> > and: [ aZincRequest entity isEmpty not
> > and: [ (aZincRequest entity contentType
> matches: ZnMimeType applicationFormUrlEncoded) not
> > and: [ (aZincRequest entity
> contentType matches: ZnMimeType multiPartFormData) not ] ] ] ])
> > ifTrue: [
> > "Seaside wants to do its own text
> conversions"
> > ZnUTF8Encoder new decodeBytes:
> aZincRequest entity bytes "CORRECTED" ]
> > ifFalse: [
> > String new ]
> >
> > My correction tries to decode byte array with ZnUTF8Encoder and the
> result is OK.
> >
> > Maybe I would recommend to use GRPharoUtf8Codec (although I like
> ZnUTF8Encoder more) or even better self codec (self = ZnZincServerAdaptor)
> to try to decode the bytes.
> >
> > Regards,
> > Tomas Kukol
> > _______________________________________________
> > seaside mailing list
> > seaside at lists.squeakfoundation.org
> > http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>
>
>
> --
> Sven Van Caekenberghe
> http://stfx.eu
> Smalltalk is the Red Pill
>
> _______________________________________________
> seaside mailing list
> seaside at lists.squeakfoundation.org
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/seaside/attachments/20130626/c9f73957/attachment-0001.htm
More information about the seaside
mailing list