[Seaside] random maddening bug - "corrupted" file upload contents

Cyrille Delaunay cy.delaunay at gmail.com
Wed Jan 10 09:42:43 UTC 2018


Hello,

Has anyone been able to reproduce my little scenario ?
As a reminder, suspicion is that something "wrong" has been introduced with
Pharo6
The scenario to reproduce:

"Download Pharo 6.1 on my Mac (Sierra 10.12.6): https://pharo.org/
web/download

Then, iterate over this process till spotting the issue:

=> start the pharo image
=> execute this piece of code in a playground

ZnServer startDefaultOn: 1701.
ZnServer default maximumEntitySize: 80* 1024 * 1024.
'/Users/cdelaunay/myzip.zip' asFileReference writeStreamDo: [ :out |
out binary; nextPutAll: #[80 75 3 4 10 0 0 0 0 0 125 83 67 73 0 0 0 0 0 0].
18202065 timesRepeat: [ out nextPut: 0 ]
].

=> Open a web browser page on: http://localhost:1701/form-test-3
=> Upload the file zip file previously generated ('myzip.zip').
=> If the web page displays: "contents=000000000a00..." (or anything
unexpected), THIS IS THE ISSUE !
=> If the web page displays: "contents=504b03040a00..", the upload worked
fine. You can close the image (without saving)
Mettre à jour <http://37.139.2.203/redmine/issues/906/edit> Saisir temps
<http://37.139.2.203/redmine/issues/906/time_entries/new> Surveiller
<http://37.139.2.203/redmine/watchers/watch?object_id=906&object_type=issue>
 Copier <http://37.139.2.203/redmine/projects/general/issues/906/copy>
Supprimer <http://37.139.2.203/redmine/issues/906>


Usually, after starting 3 / 4 times pharo image, I am getting the problem."


2018-01-05 10:23 GMT+01:00 Cyrille Delaunay <cy.delaunay at gmail.com>:

> I made some experiments which might help to restrict the scope:
>
> I applied my little "manual" scenario with a Pharo 5 image (
> https://files.pharo.org/get-files/50/)
> After 7 tries, the issue never show up.
>
> With a Pharo 6.1 image, the issue shows up after maximum 3 / 4 tries.
>
> My feeling: something changed between these 2 versions.
> This idea would tend to be confirmed by the period we started to
> experiment the issue: We recently migrated some production images from
> Pharo 4 to Pharo 6.
> Is it Zinc or some Pharo inside bricks used by Zinc ? I could not say.
>
>
>
>
>
>
> 2018-01-04 16:48 GMT+01:00 Cyrille Delaunay <cy.delaunay at gmail.com>:
>
>> I tried several times and the test I wrote always passed.
>>
>> All I have been able to do, is to experiment some failures by playing
>> with the client timeout.
>> My test initially contains the following statement: *client timeout:
>> 120000.*
>> If I comment this statement, sometimes the server received a request with
>> a nil entity.
>> The test then get an unexpected response.
>> But I am not sure it an unexpected behavior, as there is a lot of data
>> being sent.
>>
>>
>>
>>
>>
>> 2018-01-02 20:28 GMT+01:00 Cyrille Delaunay <cy.delaunay at gmail.com>:
>>
>>>
>>>
>>> 2018-01-02 17:37 GMT+01:00 Sven Van Caekenberghe <sven at stfx.eu>:
>>>
>>>>
>>>>
>>>> > On 2 Jan 2018, at 12:53, Cyrille Delaunay <cy.delaunay at gmail.com>
>>>> wrote:
>>>> >
>>>> > Hi Sven,
>>>> >
>>>> > I am back on this issue :)
>>>> >
>>>> > I have kind of a "clean" scenario that could help to reproduce.
>>>> > It is still random, but at one point, it shows up for me.
>>>> >
>>>> > Here is how I am doing:
>>>> > Download Pharo 6.1 on my Mac (Sierra 10.12.6):
>>>> https://pharo.org/web/download
>>>> > Then, iterate over this process till spotting the issue:
>>>> >
>>>> > => start the pharo image
>>>> > => execute this piece of code in a playground
>>>> >
>>>> > ZnServer startDefaultOn: 1701.
>>>> > ZnServer default maximumEntitySize: 80* 1024 * 1024.
>>>> > '/Users/cdelaunay/myzip.zip' asFileReference writeStreamDo: [ :out |
>>>> > out binary; nextPutAll: #[80 75 3 4 10 0 0 0 0 0 125 83 67 73 0 0 0 0
>>>> 0 0].
>>>> > 18202065 timesRepeat: [ out nextPut: 0 ]
>>>> > ].
>>>> >
>>>> > => Open a web browser page on: http://localhost:1701/form-test-3
>>>> > => Upload the file zip file previously generated ('myzip.zip').
>>>> > => If the web page displays: "contents=000000000a00..." (or anything
>>>> unexpected), THIS IS THE ISSUE !
>>>> > => If the web page displays: "contents=504b03040a00..", the upload
>>>> worked fine. You can close the image (without saving)
>>>> >
>>>> > Mettre à jour Saisir temps Surveiller Copier Supprimer
>>>> >
>>>> >
>>>> > Usually, after starting 3 / 4 times pharo image, I am getting the
>>>> problem.
>>>> > I have been able to reproduce with chrome and firefox.
>>>> >
>>>> >
>>>> > Seing that I was able to reproduce within a fresh pharo image, I
>>>> tried to automate this scenario.
>>>> > I wrote the following test:
>>>> >
>>>> >
>>>> > ZnServerTests >>testFormTest4
>>>> >       | inputs client part fileReference fileContents entity |
>>>> >       fileReference := 'test-zip.zip' asFileReference.
>>>> >       fileReference
>>>> >               writeStreamDo: [ :out |
>>>> >                       out
>>>> >                               binary;
>>>> >                               nextPutAll: #[80 75 3 4 10 0 0 0 0 0
>>>> 125 83 67 73 0 0 0 0 0 0].
>>>> >                       18202065 timesRepeat: [ out nextPut: 0 ] ].
>>>> >       fileContents := fileReference binaryReadStreamDo: [ :in | in
>>>> upToEnd ].
>>>> >       15
>>>> >               timesRepeat: [ self
>>>> >                               withServerDo: [ :server |
>>>> >                                       server maximumEntitySize: 80 *
>>>> 1024 * 1024.
>>>> >                                       server debugMode: true.
>>>> >                                       (client := ZnClient new)
>>>> >                                               url: server localUrl;
>>>> >                                               addPathSegment:
>>>> 'form-test-3'.
>>>> >                                       client timeout: 120000.
>>>> >                                       entity := (ZnByteArrayEntity
>>>> type: (ZnMimeType main: 'application' sub: 'zip') length: fileContents
>>>> size) bytes: fileContents.
>>>> >                                       part := ZnMimePart fieldName:
>>>> 'file' fileName: fileReference basename entity: entity.
>>>> >                                       client
>>>> >                                               resetEntity;
>>>> >                                               addPart: part;
>>>> >                                               post.
>>>> >                                       self assert: client isSuccess.
>>>> >                                       self assert: (client contents
>>>> includesSubstring: '504b03040a') ] ]
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > In order to make this test run correctly I had to modify this method.
>>>> > Without this change, I always got an empty html page as response
>>>> >
>>>> >
>>>> >
>>>> > ZnDefaultServerDelegate >> formTest3: request
>>>> >       | contents filename contentType page |
>>>> >       contents := filename := contentType := ''.
>>>> >       (request hasEntity and: [ request contentType matches:
>>>> ZnMimeType multiPartFormData ])
>>>> >               ifTrue: [
>>>> >                       (request entity partNamed: #file ifNone: [ nil
>>>> ])
>>>> >                               ifNotNil: [ :part |
>>>> >                                       filename := part fileName.
>>>> >                                       contents := part contents.
>>>> >                                       contentType := part
>>>> contentType.
>>>> >                                       contentType isBinary ifTrue: [
>>>> contents := contents hex ] ] ].
>>>> >       page := ZnHtmlOutputStream streamContents: [ :html |
>>>> >               html page: 'Form Test 3' do: [
>>>> >                       html
>>>> >                               tag: #form
>>>> >                               attributes: #(action 'form-test-3'
>>>> 'accept-charset' 'utf-8'
>>>> >
>>>>  enctype 'multipart/form-data' method POST)
>>>> >                               do: [
>>>> >                                       html
>>>> >                                               str: 'File'; space;
>>>> >                                               tag: #input attributes:
>>>> #(type file name file); space;
>>>> >                                               tag: #input attributes:
>>>> #(type submit) ];
>>>> >                               tag: #p do: [ html str: 'filename = ';
>>>> str: filename ];
>>>> >                               tag: #p do: [ html str: 'content-type =
>>>> '; str: contentType asString ];
>>>> >                               tag: #p do: [ html str: 'contents = ';
>>>> str: (contents copyFrom: 1 to: 20) asString ] ] ].
>>>> >       ^ ZnResponse ok: (ZnEntity html: page)
>>>> >
>>>> >
>>>> >
>>>> > Unfortunately, the tests always passed till now !
>>>>
>>>> So you have to upload 80Mb ?
>>>>
>>>
>>> Sometimes yes.
>>> But here the file I build and use to reproduce is only 18 mb.
>>> (the same size as the original file highlighting the issue on my
>>> machine).
>>>
>>>
>>>> And if you script it in Pharo and repeat it 15 times it keeps on
>>>> working ?
>>>> Really ?
>>>>
>>>
>>> Yes indeed.
>>> But again, it is completely random.
>>> I could not ensure that the problem would never show up with the script.
>>> I will try to make more tries tomorrow.
>>>
>>>
>>>>
>>>> It seems that it only fails when a manually operated browser is
>>>> involved.
>>>>
>>>
>>> Yes, exactly
>>>
>>>
>>>>
>>>> Have you tried doing the upload with curl ?
>>>>
>>>
>>> No.
>>> How would I do that ?
>>>
>>>
>>>>
>>>> > I will try to go on refining the context parameters that could make
>>>> the issue show up or not
>>>> >
>>>> >
>>>> >
>>>> > 2017-12-22 18:05 GMT+01:00 Sven Van Caekenberghe <sven at stfx.eu>:
>>>> > Well, I can't repeat it, for now.
>>>> >
>>>> > Try to make a minimal self-contained example that fails (reliably),
>>>> like how I did it, you can even try to do the upload in Pharo too (see
>>>> #testFormTest3).
>>>> >
>>>> > > On 22 Dec 2017, at 18:00, Cyril Ferlicot <cyril.ferlicot at gmail.com>
>>>> wrote:
>>>> > >
>>>> > >
>>>> > > On ven. 22 déc. 2017 at 17:50, Sven Van Caekenberghe <sven at stfx.eu>
>>>> wrote:
>>>> > > Cyrille,
>>>> > >
>>>> > > I wrote a file like this:
>>>> > >
>>>> > > FileLocator temp / 'myzip.zip' writeStreamDo: [ :out |
>>>> > >   out binary; nextPutAll: #[80 75 3 4 10 0 0 0 0 0 125 83 67 73 0 0
>>>> 0 0 0 0] ].
>>>> > >
>>>> > > Which I can read as follows:
>>>> > >
>>>> > > FileLocator temp / 'myzip.zip' binaryReadStreamDo: [ :in | in
>>>> upToEnd ].
>>>> > >
>>>> > >   => #[80 75 3 4 10 0 0 0 0 0 125 83 67 73 0 0 0 0 0 0]
>>>> > >
>>>> > > If I upload this file to http://localhost:1701/form-test-3 this
>>>> works perfectly (macOS 10.13.2 Pharo #60402, Safari. Firefox & Chrome),
>>>> first time and second time.
>>>> > >
>>>> > >
>>>> > >
>>>> > > Maybe you should try on a co-workers machine just to make sure it
>>>> is not a local problem on your machine ?
>>>> > >
>>>> > > Hi Sven,
>>>> > >
>>>> > > It happen on Cyrille's machine, my machine, Yann's machine and our
>>>> boss's machine.
>>>> > >
>>>> > > We first saw it on Windows and never on mac/Linux. But now we see
>>>> it also on mac.
>>>> > >
>>>> > > I did not see it yet on Linux but we use it less for now.
>>>> > >
>>>> > > Also, when it happen, it tend to happen a lot afterward.
>>>> > >
>>>> > > Also, by random it means that we can have 1/2 week without problem,
>>>> then 4 failure in 2h.
>>>> > >
>>>> > >
>>>> > >
>>>> > > Sven
>>>> > >
>>>> > > --
>>>> > > Cyril Ferlicot
>>>> > > https://ferlicot.fr
>>>> > >
>>>> > > http://www.synectique.eu
>>>> > > 2 rue Jacques Prévert 01,
>>>> > > 59650 Villeneuve d'ascq France
>>>> > > <Screen Shot 2017-12-22 at 17.48.31.png>_________________
>>>> ______________________________
>>>> > > seaside mailing list
>>>> > > seaside at lists.squeakfoundation.org
>>>> > > http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>>>> >
>>>> > _______________________________________________
>>>> > seaside mailing list
>>>> > seaside at lists.squeakfoundation.org
>>>> > http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > Cyrille Delaunay
>>>> > _______________________________________________
>>>> > seaside mailing list
>>>> > seaside at lists.squeakfoundation.org
>>>> > http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>>>>
>>>> _______________________________________________
>>>> seaside mailing list
>>>> seaside at lists.squeakfoundation.org
>>>> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>>>>
>>>
>>>
>>>
>>> --
>>> Cyrille Delaunay
>>>
>>
>>
>>
>> --
>> Cyrille Delaunay
>>
>
>
>
> --
> Cyrille Delaunay
>



-- 
Cyrille Delaunay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/seaside/attachments/20180110/f207ab74/attachment-0001.html>


More information about the seaside mailing list