[Vm-dev] Reproducible crashing the VM with a zipped fuel serialization

Mariano Martinez Peck marianopeck at gmail.com
Sun Jan 14 20:10:03 UTC 2018


Hi Max,

Great debugging capabilities! Since you are already in deep waters....could
you check if it's the same problem I reported in the email "Problem with
ZipArchive (probably bug due to large file)"?

Thanks a lot in advance,



On Sun, Jan 14, 2018 at 4:52 PM, Max Leske <maxleske at gmail.com> wrote:

>
> So, I seem to have identified the problem. According to the RFC on Deflate
> (https://tools.ietf.org/html/rfc1951#page-5) the maximum number of
> non-compressible blocks is limited to (2^16) - 1:
>
> "A compressed data set consists of a series of blocks, corresponding
>    to successive blocks of input data.  The block sizes are arbitrary,
>    except that non-compressible blocks are limited to 65,535 bytes."
>
> Stephan's case exceeds this limit, which leads to an error when validating
> the match in DeflateStream>>validateMatchAt:from:to:. I assume that the
> compiler optimization removes the trap for the validation failure in this
> case (note that I simply used the Smalltalk implementation to trace the
> problem by replacing the primitive send with a super send).
>
> With Fuel loaded (Pharo 7, 64-bit) the following produces a reproducible
> validation error, which is misleading, the problem is actually that the
> number of blocks was exceeded (I think that's because of the 16 bit bit
> mask that is being used in the implementation):
>
> (FLGZipStrategy newWithTarget: FLByteArrayStreamStrategy new)
> writeStreamDo: [ :stream |
> FLSerializer newDefault
> serialize: ((1 to: (2 raisedTo: 16) - 82) collect: [ :i | 1 ]) asByteArray
> on: stream ]
>
> I'm not sure what should be done in this case but it appears to me that
> all remaining data could simply be appended to the last block, at least in
> the implementation I found in GCC (https://gcc.gnu.org/ml/gcc/
> 2009-09/txt00001.txt):
>
> /* Stored blocks are limited to 0xffff bytes, pending_buf is limited
>
>      * to pending_buf_size, and each stored block has a 5 byte header:
>
>      */
>
>     ulg max_block_size = 0xffff;
>
>     ulg max_start;
>
>
>
>     if (max_block_size > s->pending_buf_size - 5) {
>
>         max_block_size = s->pending_buf_size - 5;
>
>     }
>
>
> Where should we track this issue? Can someone take care to open an issue
> or should I do it?
>
> In any case, I will close the Fuel issue, as this is clearly problem of
> the Deflate / Zip implementation.
>
>
>
> Cheers,
> Max
>
>
>
> On 13 January 2018 at 18:31:09, Max Leske (maxleske at gmail.com) wrote:
>
> Hi Stephan,
>
> I've opened an issue here: https://github.com/theseion/Fuel/issues/229.
> I'll try to reproduce and see what could be the problem.
>
> Cheers,
> Max
>
> On 13 January 2018 at 17:13:23, vm-dev-request at lists.squeakfoundation.org
> (vm-dev-request at lists.squeakfoundation.org) wrote:
>
> On 21-09-17 14:51 <http://airmail.calendar/2017-09-21%2014:51:00%20CEST>,
> stephan wrote:
> > Still reproducible with current versions of Pharo7 and vm
> I tried again after Pharo build #420
>
> The Pull Request #654 was integrated: "Updated Fuel to 2.2.0 with the 64
> bit adaptations"
>
> Ubuntu 16.04LTS
>
> vm: opensmalltalk / vm / cog / 201801121936 <//201801121936> /
> cog_linux64x64_pharo.cog.spur_201801121936
>
> image: Pharo-7.0.0-alpha.build.425.sha.eb0a6fb.arch.64bit.image
>
>
>


-- 
Mariano
http://marianopeck.wordpress.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20180114/474644de/attachment-0001.html>


More information about the Vm-dev mailing list