<div dir="ltr">Hi Sven,<div><br></div><div style>you are right. Seaside needs it as a string. But I've found another problem with output (maybe because Seaside uses string???) which leads to error 'Improper store into indexable object'.</div>
<div style><br></div><div style>I have JSON on output. It is again string and there is this problem. In ZnZincServerAdaptor>>responseFrom: we are asking for contents.</div><div style><br></div><div style><div><font face="courier new, monospace">ZnZincServerAdaptor>>responseFrom: aRequestContext</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>| partialHeaders cookies fullHeaders seasideResponse contents entity contentType |</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>seasideResponse := aRequestContext response.</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>partialHeaders := seasideResponse headers.</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>cookies := seasideResponse cookies.</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>fullHeaders := ZnHeaders defaultResponseHeaders.</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>partialHeaders keysAndValuesDo: [ :key :value |</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>fullHeaders at: key put: value ].</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>cookies do: [ :each |</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>fullHeaders at: 'Set-Cookie' add: each oldNetscapeString.</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>fullHeaders at: 'Set-Cookie2' add: each rfc2965String ].</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>contentType := seasideResponse contentType greaseString.</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>contents := seasideResponse contents. "<------------------------ HERE - we are asking for contents"</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>entity := (ZnEntity bytes: contents) contentType: contentType; yourself.</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>^ ZnResponse new</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>statusLine: (ZnStatusLine code: seasideResponse status);</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>headers: fullHeaders;</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>entity: entity;</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>yourself</font></div>
<div><br></div><div style>It calls:</div><div style><br></div><div style><div><font face="courier new, monospace">WABufferedResponse>>contents</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>^ contentsStream contents "<------------------------ HERE - normally, asking underlying stream for contents"</font></div>
<div><br></div><div style>An the error is coming here:</div><div style><br></div><div style><div><font face="courier new, monospace">RWBinaryOrTextStream>>contents</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>"Answer with a copy of my collection from 1 to readLimit."</font></div>
<div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>| newArray |</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>isBinary ifFalse: [^ super contents].<span class="" style="white-space:pre"> </span>"String"</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>readLimit := readLimit max: position.</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>newArray := ByteArray new: readLimit. "<------------------------ HERE - we are creating array of bytes (SmallInteger 0 - 255)"</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>^ newArray replaceFrom: 1</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>to: readLimit</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>with: collection "<------------------------ HERE - but we are having WideString in collection - characters"</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>startingAt: 1.</font></div><div><br></div><div style>Now we are calling primitive 105:</div><div style><br></div><div style><div><font face="courier new, monospace">ByteArray>>replaceFrom: start to: stop with: replacement startingAt: repStart </font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>"Primitive. This destructively replaces elements from start to stop in the receiver starting at index, repStart, in the collection, replacement. Answer the receiver. Range checks are performed in the primitive only. Optional. See Object documentation whatIsAPrimitive."</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span><primitive: 105></font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>super replaceFrom: start to: stop with: replacement startingAt: repStart "<------------------------ HERE - it fails and continues here"</font></div>
<div><br></div><div style>It calls loop in SequenceableCollection:</div><div style><br></div><div style><div><font face="courier new, monospace">ByteArray(SequenceableCollection)>>replaceFrom: start to: stop with: replacement startingAt: repStart </font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>"This destructively replaces elements from start to stop in the receiver </font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>starting at index, repStart, in the sequenceable collection, </font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>replacementCollection. Answer the receiver. No range checks are </font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>performed."</font></div>
<div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>| index repOff |</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>repOff := repStart - start.</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>index := start - 1.</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>[(index := index + 1) <= stop]</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>whileTrue: [self at: index put: (replacement at: repOff + index)] "<------------------------ HERE - here we are putting character into ByteArray"</font></div>
<div><br></div><div style>Stops here at Object at primitive 61:</div><div style><br></div><div style><div><font face="courier new, monospace">ByteArray(Object)>>at: index put: value </font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>"Primitive. Assumes receiver is indexable. Store the argument value in </font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>the indexable element of the receiver indicated by index. Fail if the </font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>index is not an Integer or is out of bounds. Or fail if the value is not of </font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>the right type for this kind of collection. Answer the value that was </font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>stored. Essential. See Object documentation whatIsAPrimitive."</font></div>
<div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span><primitive: 61></font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>index isInteger ifTrue:</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>[self class isVariable</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>ifTrue: [(index >= 1 and: [index <= self size])</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>ifTrue: [self errorImproperStore] "<------------------------ HERE - ends here"</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>ifFalse: [self errorSubscriptBounds: index]]</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>ifFalse: [self errorNotIndexable]].</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>index isNumber</font></div>
<div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>ifTrue: [^self at: index asInteger put: value]</font></div><div><font face="courier new, monospace"><span class="" style="white-space:pre"> </span>ifFalse: [self errorNonIntegerIndex]</font></div>
<div><font face="courier new, monospace"><br></font></div><div style><font face="arial, helvetica, sans-serif">I've seen this problem mostly with Seaside-REST, but also in some other posts from Seaside.</font></div><div style>
<font face="arial, helvetica, sans-serif"><br></font></div><div style><font face="arial, helvetica, sans-serif">Any idea how to fix it?</font></div><div style><font face="arial, helvetica, sans-serif"><br></font></div><div style>
<font face="arial, helvetica, sans-serif">Regards,</font></div><div style><font face="arial, helvetica, sans-serif">Tomas</font></div></div></div></div></div></div></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">
On Wed, Jun 26, 2013 at 12:17 AM, Sven Van Caekenberghe <span dir="ltr"><<a href="mailto:sven@stfx.eu" target="_blank">sven@stfx.eu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi Tomas,<br>
<br>
Thanks for the report: this is an important area indeed.<br>
<br>
However, the comment says it: this is intentional, and as far as I know, it is correct. You see, Zinc will correctly decode/encode any HTTP payload, using the supplied mime-type and/or charsets. Seaside is written such that it does not want this: it insists on doing this on its own. That is why ZnZincServerAdaptor is _not_ using the normal request reading code of Zn, but uses a special option to read everything binary. The stupid thing is that even though Seaside needs bytes, it wants them as a String. That is the reason for the otherwise brain dead #asString (and the implicit copy is inefficient as well).<br>
<br>
Of course, you will see your special characters there if you do UTF8 decoding, but that is because you already know what is inside.<br>
<br>
What normally happens, is that later on in the processing, Seaside will access the WARequest payload using proper decoding, using its own framework (much like what Zn would do). AFAIK this whole process works. You can actually test this using some of the functional tests.<br>
<br>
I am not sure that Seaside-REST is doing the right thing (there were some issues with SmalltalkHub as well), but I would think so.<br>
<br>
Are you sure you have set the correct encoding on the adaptor ?<br>
<br>
Are you sure you are posting as application/json;charset=utf-8 and if you do not set the charset, are you sure utf-8 is the default ?<br>
<br>
Are you sure your REST handler and/or JSON parser does the right thing ?<br>
<br>
It is too late right now, but if we want to get further with this, I will need a failing unit test - if these exist in Seaside-REST, but I would assume so. I have no experience running Seaside-REST, I am using Zinc-REST myself, but I would like to learn.<br>
<br>
Regards,<br>
<br>
Sven<br>
<div><div class="h5"><br>
On 25 Jun 2013, at 23:24, Tomas Kukol <<a href="mailto:tomas.kukol@gmail.com">tomas.kukol@gmail.com</a>> wrote:<br>
<br>
> Hi Sven.<br>
><br>
> I've had a problem when POSTing non-ascii UTF-8 characters in JSON to Seaside REST service. I've located the problem in the method ZnZincServerAdaptor>>requestBodyFor: where the body of ZnRequest is translated to body of WARequest. I use Pharo 1.4 with Seaside 3.0.8 and Zinc-Seaside-SvenVanCaekenberghe.40.<br>
><br>
> When the POSTed JSON contains non-ascii UTF-8 characters (Czech characters), they are corrupted. The problem is on the "MARKED" line, where the array of bytes changed to string by asString.<br>
><br>
> "Problematic" code:<br>
><br>
> ZnZincServerAdaptor>>requestBodyFor: aZincRequest<br>
> ^ (aZincRequest method ~= #TRACE<br>
> and: [ aZincRequest hasEntity<br>
> and: [ aZincRequest entity isEmpty not<br>
> and: [ (aZincRequest entity contentType matches: ZnMimeType applicationFormUrlEncoded) not<br>
> and: [ (aZincRequest entity contentType matches: ZnMimeType multiPartFormData) not ] ] ] ])<br>
> ifTrue: [<br>
> "Seaside wants to do its own text conversions"<br>
> aZincRequest entity bytes asString "MARKED" ]<br>
> ifFalse: [<br>
> String new ]<br>
><br>
> I did a quick correction, which is not nice, but works for me:<br>
><br>
> ZnZincServerAdaptor>>requestBodyFor: aZincRequest<br>
> ^ (aZincRequest method ~= #TRACE<br>
> and: [ aZincRequest hasEntity<br>
> and: [ aZincRequest entity isEmpty not<br>
> and: [ (aZincRequest entity contentType matches: ZnMimeType applicationFormUrlEncoded) not<br>
> and: [ (aZincRequest entity contentType matches: ZnMimeType multiPartFormData) not ] ] ] ])<br>
> ifTrue: [<br>
> "Seaside wants to do its own text conversions"<br>
> ZnUTF8Encoder new decodeBytes: aZincRequest entity bytes "CORRECTED" ]<br>
> ifFalse: [<br>
> String new ]<br>
><br>
> My correction tries to decode byte array with ZnUTF8Encoder and the result is OK.<br>
><br>
> Maybe I would recommend to use GRPharoUtf8Codec (although I like ZnUTF8Encoder more) or even better self codec (self = ZnZincServerAdaptor) to try to decode the bytes.<br>
><br>
> Regards,<br>
> Tomas Kukol<br>
</div></div>> _______________________________________________<br>
> seaside mailing list<br>
> <a href="mailto:seaside@lists.squeakfoundation.org">seaside@lists.squeakfoundation.org</a><br>
> <a href="http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside" target="_blank">http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside</a><br>
<br>
<br>
<br>
--<br>
Sven Van Caekenberghe<br>
<a href="http://stfx.eu" target="_blank">http://stfx.eu</a><br>
Smalltalk is the Red Pill<br>
<br>
_______________________________________________<br>
seaside mailing list<br>
<a href="mailto:seaside@lists.squeakfoundation.org">seaside@lists.squeakfoundation.org</a><br>
<a href="http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside" target="_blank">http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside</a><br>
</blockquote></div><br></div>