<html><head><style>body{font-family:Helvetica,Arial;font-size:13px}</style></head><body style="word-wrap:break-word;line-break:after-white-space"><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">So, I seem to have identified the problem. According to the RFC on Deflate (<a href="https://tools.ietf.org/html/rfc1951#page-5">https://tools.ietf.org/html/rfc1951#page-5</a>) the maximum number of non-compressible blocks is limited to (2^16) - 1:</div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto"><br></div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">"A compressed data set consists of a series of blocks, corresponding</div><div id="bloop_customfont" style="margin:0px"> to successive blocks of input data. The block sizes are arbitrary,</div><div id="bloop_customfont" style="margin:0px"> except that non-compressible blocks are limited to 65,535 bytes."</div> <div><br></div>Stephan's case exceeds this limit, which leads to an error when validating the match in DeflateStream>>validateMatchAt:from:to:. I assume that the compiler optimization removes the trap for the validation failure in this case (note that I simply used the Smalltalk implementation to trace the problem by replacing the primitive send with a super send).<div><br></div><div>With Fuel loaded (Pharo 7, 64-bit) the following produces a reproducible validation error, which is misleading, the problem is actually that the number of blocks was exceeded (I think that's because of the 16 bit bit mask that is being used in the implementation):</div><div><br></div><div><div>(FLGZipStrategy newWithTarget: FLByteArrayStreamStrategy new) writeStreamDo: [ :stream |</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>FLSerializer newDefault</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>serialize: ((1 to: (2 raisedTo: 16) - 82) collect: [ :i | 1 ]) asByteArray</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>on: stream ]</div><div><br></div><div>I'm not sure what should be done in this case but it appears to me that all remaining data could simply be appended to the last block, at least in the implementation I found in GCC (<a href="https://gcc.gnu.org/ml/gcc/2009-09/txt00001.txt">https://gcc.gnu.org/ml/gcc/2009-09/txt00001.txt</a>):</div><div><br></div><div><div>/* Stored blocks are limited to 0xffff bytes, pending_buf is limited</div><div><br></div><div> * to pending_buf_size, and each stored block has a 5 byte header:</div><div><br></div><div> */</div><div><br></div><div> ulg max_block_size = 0xffff;</div><div><br></div><div> ulg max_start;</div><div><br></div><div><br></div><div><br></div><div> if (max_block_size > s->pending_buf_size - 5) {</div><div><br></div><div> max_block_size = s->pending_buf_size - 5;</div><div><br></div><div> }</div></div><div><br></div><div><br></div><div>Where should we track this issue? Can someone take care to open an issue or should I do it?</div><div><br></div><div>In any case, I will close the Fuel issue, as this is clearly problem of the Deflate / Zip implementation.</div><div><br></div><div><br></div><div><br></div><div>Cheers,</div><div>Max</div><div><br></div> <div id="bloop_sign_1515957338926422016" class="bloop_sign"> <br></div> <br><p class="airmail_on">On 13 January 2018 at 18:31:09, Max Leske (<a href="mailto:maxleske@gmail.com">maxleske@gmail.com</a>) wrote:</p> <blockquote type="cite" class="clean_bq"><span><div style="word-wrap:break-word;line-break:after-white-space"><div></div><div>
<title></title>
<div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">
Hi Stephan,</div>
<div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">
<br></div>
<div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">
I've opened an issue here: <a href="https://github.com/theseion/Fuel/issues/229">https://github.com/theseion/Fuel/issues/229</a>.</div>
<div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">
I'll try to reproduce and see what could be the problem.</div>
<div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">
<br></div>
<div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">
Cheers,</div>
<div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">
Max</div>
<div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">
<br></div>
<div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">
On 13 January 2018 at 17:13:23,
<a href="mailto:vm-dev-request@lists.squeakfoundation.org">vm-dev-request@lists.squeakfoundation.org</a> (<a href="mailto:vm-dev-request@lists.squeakfoundation.org">vm-dev-request@lists.squeakfoundation.org</a>)
wrote:</div>
<blockquote type="cite" class="clean_bq">
<div><span><a href="http://airmail.calendar/2017-09-21%2014:51:00%20CEST" style="font-family:helvetica;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
On 21-09-17 14:51</a><span style="color:rgb(0,0,0);font-family:helvetica;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;display:inline!important;float:none">,
stephan wrote:</span><br style="color:rgb(0,0,0);font-family:helvetica;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
<span style="color:rgb(0,0,0);font-family:helvetica;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;display:inline!important;float:none">
> Still reproducible with current versions of Pharo7 and
vm</span><br style="color:rgb(0,0,0);font-family:helvetica;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
<span style="color:rgb(0,0,0);font-family:helvetica;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;display:inline!important;float:none">
I tried again after Pharo build #420</span><br style="color:rgb(0,0,0);font-family:helvetica;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
<br style="color:rgb(0,0,0);font-family:helvetica;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
<span style="color:rgb(0,0,0);font-family:inherit!important;font-size:inherit!important;font-style:inherit!important;font-variant-caps:inherit!important;font-weight:inherit!important;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;display:inline!important;margin:0px!important;padding:0px!important;opacity:1!important;float:inherit!important;font-stretch:inherit!important;line-height:inherit!important">
The Pull Request #654 was integrated: "Updated<span class="Apple-converted-space"> </span><span style="background-image:-webkit-gradient(linear,0% 0%,0% 100%,from(rgba(244,234,38,0.701961)),to(rgba(237,206,0,0.701961)))!important;display:inline!important;margin:0px!important;padding:0px!important;opacity:1!important;float:inherit!important;font-family:inherit!important;font-size:inherit!important;font-style:inherit!important;font-variant-caps:inherit!important;font-weight:inherit!important;font-stretch:inherit!important;line-height:inherit!important;background-position:initial initial!important;background-repeat:initial initial!important">Fuel</span><span class="Apple-converted-space"> </span>to
2.2.0 with the 64<span class="Apple-converted-space"> </span></span><br style="color:rgb(0,0,0);font-family:helvetica;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
<span style="color:rgb(0,0,0);font-family:helvetica;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;display:inline!important;float:none">
bit adaptations"</span><br style="color:rgb(0,0,0);font-family:helvetica;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
<br style="color:rgb(0,0,0);font-family:helvetica;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
<span style="color:rgb(0,0,0);font-family:helvetica;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;display:inline!important;float:none">
Ubuntu 16.04LTS</span><br style="color:rgb(0,0,0);font-family:helvetica;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
<br style="color:rgb(0,0,0);font-family:helvetica;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
<span style="color:rgb(0,0,0);font-family:helvetica;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;display:inline!important;float:none">
vm: opensmalltalk / vm / cog /<span class="Apple-converted-space"> </span></span><a href="tel://201801121936" style="font-family:helvetica;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">201801121936</a><span style="color:rgb(0,0,0);font-family:helvetica;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;display:inline!important;float:none"><span class="Apple-converted-space"> </span>/<span class="Apple-converted-space"> </span></span><br style="color:rgb(0,0,0);font-family:helvetica;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
<span style="color:rgb(0,0,0);font-family:helvetica;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;display:inline!important;float:none">
cog_linux64x64_pharo.cog.spur_201801121936</span><br style="color:rgb(0,0,0);font-family:helvetica;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
<br style="color:rgb(0,0,0);font-family:helvetica;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
<span style="color:rgb(0,0,0);font-family:helvetica;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;display:inline!important;float:none">
image:
Pharo-7.0.0-alpha.build.425.sha.eb0a6fb.arch.64bit.image</span></span></div>
</blockquote>
</div></div></span></blockquote></div></body></html>