[squeak-dev] Re: [Vm-dev] Bug in writing compressed stream when saving an mcz (was: New Cog VMs available...)

Nicolas Cellier nicolas.cellier.aka.nice at gmail.com
Thu Jul 17 22:24:29 UTC 2014


2014-07-13 16:22 GMT+02:00 David T. Lewis <lewis at mail.msen.com>:

> On Sun, Jul 13, 2014 at 09:55:41AM -0400, David T. Lewis wrote:
> > On Sun, Jul 06, 2014 at 12:47:25PM -0400, David T. Lewis wrote:
> > > On Sun, Jul 06, 2014 at 12:23:11PM -0400, David T. Lewis wrote:
> > > > On Sun, Jul 06, 2014 at 10:34:15AM +0200, Nicolas Cellier wrote:
> > > > > 2014-07-04 0:47 GMT+02:00 Nicolas Cellier <
> > > > > nicolas.cellier.aka.nice at gmail.com>:
> > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > 2014-07-04 0:09 GMT+02:00 Eliot Miranda <eliot.miranda at gmail.com
> >:
> > > > > >
> > > > > >
> > > > > >>
> > > > > >>
> > > > > >> On Thu, Jul 3, 2014 at 12:33 PM, Nicolas Cellier <
> > > > > >> nicolas.cellier.aka.nice at gmail.com> wrote:
> > > > > >>
> > > > > >>> The bug is repeatable, i simply have to execute this snippet
> with my
> > > > > >>> test file:
> > > > > >>>
> > > > > >>> (FileStream fileNamed: 'snapshot.bin') binary
> contentsOfEntireFile
> > > > > >>> asString zipped.
> > > > > >>>
> > > > > >>> The file is too big for an attachment here - 7.3 Mbytes - or
> 1.7 Mbytes
> > > > > >>> gzipped by external tool.
> > > > > >>> If someone can suggest an upload site, or want it by mail,
> just ask.
> > > > > >>>
> > > > > >>
> > > > > >> You're welcome to put it on ftp.mirandanamda.org, cogftpuser,
> pw cogging
> > > > > >> with 0's & 1's.
> > > > > >>
> > > > > >>
> > > > > > done, pollution of your site engaged...
> > > > > > Thanks Eliot!
> > > > > >
> > > > > >
> > > > > >>
> > > > > I inquired a bit more about this bug.
> > > > > An important clue is that it does not happen in Pharo3.0!
> > > > >
> > > > > But Pharo3.0 did not fundamentally change compression
> > > > > (except some FileSystem related changes, separation of CRC stuff
> in another
> > > > > package, some other cosmetic changes like replacing some self
> assert: ...
> > > > > by [...] assert and a potential bug in
> > > > > DeflateStream>>nextPutAll:startingAt: introduced by CamilleTeruel).
> > > > > Squeak behavior is presumably not due to a Compression change.
> > > > >
> > > > > Neither is it a VM problem, the bugs still shows up when running
> the Squeak
> > > > > image with Pharo VM.
> > > > >
> > > >
> > > > Yes, it is definite a problem in the Squeak trunk image.
> > > >
> > > >
> > > > > So the difference lies somewhere else: in our beloved Stream.
> > > > > I remembered a recent change of Eliot related to handling of
> Stream created
> > > > > with on:from:to: (Collections-eem.567).
> > > > > Reverting those changes just make the snippet pass!
> > > > >
> > > > > Ah ah! At the time, i found the change of Eliot quite minimal and
> nice.
> > > > > But I wish I had time to analyze this small innocent change deeper.
> > > > > Indeed, I remembered I previously broke my teeth on this one 2
> years or so
> > > > > ago, and preferred to adopt another strategy: avoid using
> on:from:to: and
> > > > > ReadWriteStream as much as possible.
> > > > > Why? because when analyzing the usage of inst.var. in Stream
> hierarchy, it
> > > > > gave me headaches, some subclass are just ignoring superclass
> variables, or
> > > > > worse are bending the semantics of superclass variables.
> > > > > I came to the conclusion that I could not decently master a change
> of
> > > > > on:from:to:
> > > >
> > > > I can not confirm this. I have an image in which I can reliably
> reproduce the
> > > > failure in writing the MCZ to a file. I tried reverting the methods
> that were
> > > > introduced in Collections-eem.567 and I am still getting the same
> failure.
> > > >
> > > > Dave
> > >
> > > Oops, I spoke too soon. Indeed the problem appears as of
> Collections-eem.567,
> > > and does not appear in Collections-nice.566.
> > >
> > > So I am confirming Nicolas' observations ... now to find the bug :)
> > >
> > > Dave
> > >
> >
> > This seems to be a rather tricky bug, and I don't think any of us knows
> the
> > cause right now. In case anyone else wants to have a look at it, here is
> a
> > recipe for reproducing the bug in a clean 4.5 image:
> >
> > - Start with a clean Squeak 4.5 image in and empty working directory.
> >       ftp://ftp.squeak.org/4.5/Squeak4.5-13680.zip
> >
> > - Open the image, do not update it.
> >
> > - Open an MC browser and add the http://source.squeak.org/trunk
> repository for the Help-Squeak-Project package and Collections package.
> >
> > - Load Help-Squeak-Project-kfr.18 from trunk.
> >
> > - Load Collections-eem.567 from trunk.
> >
> > - Open a browser on class SqueakToolsDebuggerHelp, and delete the class
> side method #showDebuggerMenuForm.
> >
> > - Remove the http://source.squeak.org/trunk repository form the
> Help-Squeak-Project package in the MC browser.
> >
> > - Highlight your package-cache repository and try to save
> Help-Squeak-project to the package-cache repository.
> >
> > - Enter author initials 'dtl'.
> >
> > - For the package comment, enter 'Remove unreferenced method'.
> >
> > - Package save will fail part way through saving the MCZ.
> >
> > - To reproduce the failure in this image, do the following in a
> workspace:
> >
> >       mcv := MCVersion allInstances detect: [:e | e name = 'a
> MCVersion(Help-Squeak-Project-dtl.19)'].
> >       [f := FileStream fileNamed: 'junk.mcz'.
> >       mcv fileOutOn: f]
> >               ensure: [f close].
> >
>
> In addition to the above recipe, here are some other observations:
>
> - Reverting from Collections-eem.567 to Collections-nice.566 makes the
>   problem go away, but ONLY if the DeflatePlugin is active.
>
> - The failure appears while executing the fallback code for
> #primitiveDeflateBlock
>   in ZipWriteStream>>deflateBlock:chainLength:goodMatch:
>
> - If this primitive is deactivate by commenting out the
>   <primitive: 'primitiveDeflateBlock' module: 'ZipPlugin'> in
>   ZipWriteStream>>deflateBlock:chainLength:goodMatch: then the problem will
>   occur regardless of which version of Collections is loaded.
>
> - When Collections-eem.567 is loaded, the primitive will fail on the 22nd
>   call to #deflateBlock:chainLength:goodMatch: and a debugger appears
> during
>   execution of the fallback code.
>
> - When Collections-nice.566 is loaded, the primitive does not fail on the
> 22nd
>   call to #deflateBlock:chainLength:goodMatch:, the fallback code is never
> run,
>   and no error appears.
>
> - When Collections-nice.566 is loaded and the primitive is disabled to
> force
>   use of the fallback code, the error appears on on the 22nd call to
>   #deflateBlock:chainLength:goodMatch: in exactlly the same place as when
>   Collections-eem.567 is loaded.
>
> - If I inspect the ZipWriteStream at the failure point (when the debugger
> pops
>   up after the problem), I see no obvious difference in the state of the
> stream
>   between the Collections-eem.567 case versus the Collections-nice.566 with
>   primitive disabled case.
>
> Dave
>
>
> Very good mining work!
When I hear  Stream, I think fluid...
But these snags make it so viscous, it's like our Stream is going to freeze
soon.
Or is it more than frozen? The vein you're after sounds as hard as diamond,
we gonna need a sharp peak pickaxe.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20140718/39534544/attachment-0001.htm


More information about the Squeak-dev mailing list