[Vm-dev] Reproducible crashing the VM with a zipped fuel serialization

Max Leske maxleske at gmail.com
Sun Jan 14 19:52:44 UTC 2018


So, I seem to have identified the problem. According to the RFC on Deflate (
https://tools.ietf.org/html/rfc1951#page-5) the maximum number of
non-compressible blocks is limited to (2^16) - 1:

"A compressed data set consists of a series of blocks, corresponding
   to successive blocks of input data.  The block sizes are arbitrary,
   except that non-compressible blocks are limited to 65,535 bytes."

Stephan's case exceeds this limit, which leads to an error when validating
the match in DeflateStream>>validateMatchAt:from:to:. I assume that the
compiler optimization removes the trap for the validation failure in this
case (note that I simply used the Smalltalk implementation to trace the
problem by replacing the primitive send with a super send).

With Fuel loaded (Pharo 7, 64-bit) the following produces a reproducible
validation error, which is misleading, the problem is actually that the
number of blocks was exceeded (I think that's because of the 16 bit bit
mask that is being used in the implementation):

(FLGZipStrategy newWithTarget: FLByteArrayStreamStrategy new)
writeStreamDo: [ :stream |
FLSerializer newDefault
serialize: ((1 to: (2 raisedTo: 16) - 82) collect: [ :i | 1 ]) asByteArray
on: stream ]

I'm not sure what should be done in this case but it appears to me that all
remaining data could simply be appended to the last block, at least in the
implementation I found in GCC (
https://gcc.gnu.org/ml/gcc/2009-09/txt00001.txt):

/* Stored blocks are limited to 0xffff bytes, pending_buf is limited

     * to pending_buf_size, and each stored block has a 5 byte header:

     */

    ulg max_block_size = 0xffff;

    ulg max_start;



    if (max_block_size > s->pending_buf_size - 5) {

        max_block_size = s->pending_buf_size - 5;

    }


Where should we track this issue? Can someone take care to open an issue or
should I do it?

In any case, I will close the Fuel issue, as this is clearly problem of the
Deflate / Zip implementation.



Cheers,
Max



On 13 January 2018 at 18:31:09, Max Leske (maxleske at gmail.com) wrote:

Hi Stephan,

I've opened an issue here: https://github.com/theseion/Fuel/issues/229.
I'll try to reproduce and see what could be the problem.

Cheers,
Max

On 13 January 2018 at 17:13:23, vm-dev-request at lists.squeakfoundation.org (
vm-dev-request at lists.squeakfoundation.org) wrote:

On 21-09-17 14:51 <http://airmail.calendar/2017-09-21%2014:51:00%20CEST>,
stephan wrote:
> Still reproducible with current versions of Pharo7 and vm
I tried again after Pharo build #420

The Pull Request #654 was integrated: "Updated Fuel to 2.2.0 with the 64
bit adaptations"

Ubuntu 16.04LTS

vm: opensmalltalk / vm / cog / 201801121936 <//201801121936> /
cog_linux64x64_pharo.cog.spur_201801121936

image: Pharo-7.0.0-alpha.build.425.sha.eb0a6fb.arch.64bit.image
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20180114/acb01fa0/attachment.html>


More information about the Vm-dev mailing list