On Sat, Dec 10, 2011 at 2:16 AM, David T. Lewis lewis@mail.msen.com wrote:

Hi Mariano,

This looks like a nasty bug in the plugin, but I cannot reproduce it. I have Fuel loaded in my Squeak image running on Linux, and I ran FLMethodContextSerializationTest>>testMethodContextThisContext without a crash. I tried it on both Cog and a standard VM, and did not get a crash in either case. The test fails (probably for some Squeak/Pharo issue) but it gets through the serialize/materialize part of the test without any problem.

Am I perhaps running a different version of Fuel that does not exercise the ZipPlugin in the same way?

Yes, exactly. Sorry for not saying that before. If you want to reproduce it, you must also install the package FuelCompression and then change FLSerializationTest >> #setUp to send the message #useGzipInMemoryStream rather than #useStandardFileStream. Now if you run #testMethodContextThisContext it would do it using a GZip stream.

Anyway, I have also sent an image to Eliot since he asked it.

Here it is the image: https://gforge.inria.fr/frs/download.php/29895/Pharo-CrashGzipThisContext.im... I found a couple of interesting things:

1) it only fails the test for thisContext (testMethodContextThisContext) , since the rest are ok. 2) I tried with StackVM as well as previous CogVMs and it also crash. However, there is a VM which it doesn't crash: Squeak5.8b4

With the attached image, you have opened the test to run (testMethodContextThisContext). In my case I can reproduce it always.

Let me know if you could reproduce it.

@John: indeed, it is very much likely there is a problem with the size, because it only crashes with #testMethodContextThisContext (and few others) where the graph we serialize is really big (in this case because we serialize thisContext which means we seralize the whole stack of senders and senders). And for the rest of the tests (which are smaller) it usually does not crash. I will try to let you know the exact size.

Thanks in advance,

Dave

On Fri, Dec 09, 2011 at 07:31:56PM +0100, Mariano Martinez Peck wrote:

...

So...after debugging the VM, it looks it is crashing in DeflatePlugin >> encodeMatch: length distance: dist

encodeMatch: length distance: dist "Encode the given match of length length starting at dist bytes

ahead"

...

| literal distance |
<inline: true>
zipLiterals at: zipLiteralCount put: length - DeflateMinMatch.
zipDistances at: zipLiteralCount put: dist.
literal := (zipMatchLengthCodes at: length - DeflateMinMatch).
zipLiteralFreq at: literal put: (zipLiteralFreq at: literal) + 1.
dist < 257
    ifTrue:[distance := zipDistanceCodes at: dist - 1]
    ifFalse:[distance := zipDistanceCodes at: 256 + (dist - 1

bitShift:

...

-7)]. zipDistanceFreq at: distance put: (zipDistanceFreq at: distance) + 1. zipLiteralCount := zipLiteralCount + 1. zipMatchCount := zipMatchCount + 1. ^zipLiteralCount = zipLiteralSize "We *must* flush" or:[(zipLiteralCount bitAnd: 16rFFF) = 0 "Only check every N

kbytes"

...

        and:[self shouldFlush]]

When it does: zipDistanceFreq at: distance put: (zipDistanceFreq at: distance) + 1. it gives a EXC_BAD_ACCESS. distance value is 1444965993

The sender is #deflateBlock: lastIndex chainLength: chainLength

goodMatch:

...

goodMatch and sender #primitiveDeflateBlock

If I do a call printAllStacks(), I get this part:

(gdb) call printAllStacks() Process 0x1f70c924 priority 40 0xbff65968 M GZipWriteStream(DeflateStream)>deflateBlock 555574892: a(n) GZipWriteStream 0xbff6598c M GZipWriteStream(DeflateStream)>nextPutAll: 555574892: a(n) GZipWriteStream 0xbff659a8 M GZipWriteStream(DeflateStream)>next:putAll:startingAt: 555574892: a(n) GZipWriteStream 0xbff659cc M FLBufferedWriteStream>flushBuffer 556072120: a(n) FLBufferedWriteStream 0xbff659e4 M FLBufferedWriteStream>nextBytesPutAll: 556072120: a(n) FLBufferedWriteStream 0xbff65a04 M FLBufferedWriteStream>nextStringPut: 556072120: a(n) FLBufferedWriteStream 0xbff65a24 M [] in FLVariablesMapping>serializeMappingOn: 557179132: a(n) FLVariablesMapping 0xbff65a44 M OrderedCollection>do: 557179148: a(n) OrderedCollection 0xbff65a64 M FLVariablesMapping>serializeMappingOn: 557179132: a(n) FLVariablesMapping 0xbff65a80 M FLFixedObjectCluster(FLPointerObjectCluster)>serializeInstancesStepWith: 556081188: a(n) FLFixedObjectCluster 0xbff65a9c M FLSerialization>clusterInstancesStepOf: 556076884: a(n) FLSerialization 0xbff65ab8 M [] in FLSerialization>instancesStep 556076884: a(n) FLSerialization

And from the gdb I get:

#0 0x000b3721 in deflateBlockchainLengthgoodMatch at ZipPlugin.c:303 #1 0x000b4a62 in primitiveDeflateBlock at ZipPlugin.c:719 #2 0x1f400858 in ?? #3 0x00067820 in threadSchedulingLoop at gcc3x-cointerpmt.c:45853 #4 0x0003d79b in initialEnterSmalltalkExecutive at gcc3x-cointerpmt.c:18372 #5 0x0003e45f in initStackPagesAndInterpret at

gcc3x-cointerpmt.c:18839

...

#6 0x00022ae8 in interpret at gcc3x-cointerpmt.c:2076 #7 0x0006df20 in -[sqSqueakMainApplication runSqueak] at sqSqueakMainApplication.m:174 #8 0x92ee186c in __NSFirePerformWithOrder #9 0x967fadd2 in __CFRunLoopDoObservers #10 0x967b6ced in __CFRunLoopRun #11 0x967b63c4 in CFRunLoopRunSpecific #12 0x967b61f1 in CFRunLoopRunInMode #13 0x98262e04 in RunCurrentEventLoopInMode #14 0x98262af5 in ReceiveNextEventCommon #15 0x98262a3e in BlockUntilNextEventMatchingListInMode #16 0x93159595 in _DPSNextEvent #17 0x93158dd6 in -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] #18 0x9311b1f3 in -[NSApplication run] #19 0x93113289 in NSApplicationMain #20 0x0006bec9 in main at main.m:52

Any pointer is appreaciated.

THanks

On Fri, Dec 2, 2011 at 11:57 PM, Diogenes Moreira < diogenes.moreira@gmail.com> wrote:

...

en que SO estas corriendo..primitiva ?? por otro lado que tenes en el PharoDebug.Log

http://about.me/diogenes.moreira

2011/12/2 Mariano Martinez Peck marianopeck@gmail.com:

...

Hi guys. We are doing some experiments with Fuel and GZip streams...

Now

...

...

I

...

have a beatiful test like this:

testMethodContextThisContext

| materializedContext context |
context := thisContext copy.
materializedContext := self resultOfSerializeAndMaterialize:

context.

...

...

...

and it crashes like crazy. From the VM crash stack, I can see:

Smalltalk stack dump: 0xbff66ba4 M GZipWriteStream(DeflateStream)>deflateBlock 447551436:

a(n)

...

...

...

GZipWriteStream 0xbff66bc8 M GZipWriteStream(DeflateStream)>nextPutAll: 447551436:

a(n)

...

...

...

GZipWriteStream 0xbff66be4 M GZipWriteStream(DeflateStream)>next:putAll:startingAt: 447551436: a(n) GZipWriteStream 0xbff66c08 M FLBufferedWriteStream>flushBuffer 448048664: a(n) FLBufferedWriteStream 0xbff66c20 M FLBufferedWriteStream>nextBytesPutAll: 448048664: a(n) FLBufferedWriteStream 0xbff66c40 M FLBufferedWriteStream>nextStringPut: 448048664: a(n) FLBufferedWriteStream 0xbff66c5c M FLByteStringCluster>serialize:on: 448054196: a(n) FLByteStringCluster 0xbff66c7c M

FLByteStringCluster(FLPrimitiveCluster)>serializeInstance:with:

...

448054196: a(n) FLByteStringCluster 0xbff66ca0 M [] in FLByteStringCluster(FLIteratingCluster)>serializeInstances:

448054196:

...

...

a(n)

...

FLByteStringCluster 0xbff66cc8 M IdentitySet(Set)>do: 451711680: a(n) IdentitySet 0xbff66ce4 M

FLByteStringCluster(FLIteratingCluster)>serializeInstances:

...

...

...

448054196: a(n) FLByteStringCluster 0xbff66d00 M FLByteStringCluster(FLIteratingCluster)>serializeInstancesStepWith: 448054196: a(n) FLByteStringCluster 0xbff66d1c M FLSerialization>clusterInstancesStepOf: 448053428: a(n) FLSerialization 0xbff66d38 M [] in FLSerialization>instancesStep 448053428: a(n) FLSerialization 0xbff66d58 M OrderedCollection>do: 449989736: a(n) OrderedCollection 0xbff66d74 M FLSerialization>instancesStep 448053428: a(n)

FLSerialization

...

0xbff66d8c M FLSerialization>run 448053428: a(n) FLSerialization 0xbff66db0 I [] in FLSerializer>defaultSerialization 447550464: a(n) FLSerializer 0xbff66dd8 M [] in FLSerializer>serialize:on: 447550464: a(n)

FLSerializer

...

0xbff66df8 M BlockClosure>ensure: 448052884: a(n) BlockClosure 0xbff66e18 M FLSerializer>serialize:on: 447550464: a(n) FLSerializer 0xbff66e3c M [] in FLMethodContextSerializationTest(FLSerializationTest)>serialize:

447235380:

...

a(n) FLMethodContextSerializationTest 0xbff66e60 M [] in GZipWriteStream class(PositionableStream

class)>on:do:

...

...

...

419845144: a(n) GZipWriteStream class 0xbff66e80 M BlockClosure>ensure: 448048616: a(n) BlockClosure 0xbff66ea0 M GZipWriteStream class(PositionableStream class)>on:do: 419845144: a(n) GZipWriteStream class 0xbff66ec4 M [] in FLGZipStrategy>writeStreamDo: 447550896: a(n) FLGZipStrategy 0xbff66ee0 M FLByteArrayStreamStrategy>writeStreamDo: 447550884: a(n) FLByteArrayStreamStrategy 0xbff66efc M FLGZipStrategy>writeStreamDo: 447550896: a(n)

FLGZipStrategy

...

...

...

0xbff66f18 M FLMethodContextSerializationTest(FLSerializationTest)>serialize:

447235380:

...

a(n) FLMethodContextSerializationTest 0xbff66f34 M

FLMethodContextSerializationTest(FLSerializationTest)>resultOfSerializeAndMaterialize:

...

...

...

447235380: a(n) FLMethodContextSerializationTest 0xbff66f58 M

FLMethodContextSerializationTest>testMethodContextThisContext

...

447235380: a(n) FLMethodContextSerializationTest

any idea of where I should start to look at?

Thanks

-- Mariano http://marianopeck.wordpress.com

-- Mariano http://marianopeck.wordpress.com

On Sat, Dec 10, 2011 at 5:34 PM, David T. Lewis lewis@mail.msen.com wrote:

On Sat, Dec 10, 2011 at 03:58:29PM +0100, Mariano Martinez Peck wrote:

...

On Sat, Dec 10, 2011 at 2:16 AM, David T. Lewis lewis@mail.msen.com

wrote:

...

...

This looks like a nasty bug in the plugin, but I cannot reproduce it. I have Fuel loaded in my Squeak image running on Linux, and I ran FLMethodContextSerializationTest>>testMethodContextThisContext without a crash. I tried it on both Cog and a standard VM, and did not get a crash in either case. The test fails (probably for some Squeak/Pharo issue) but it gets through the serialize/materialize part of the test without any problem.

Am I perhaps running a different version of Fuel that does not exercise the ZipPlugin in the same way?

Yes, exactly. Sorry for not saying that before. If you want to reproduce it, you must also install the package FuelCompression and then change FLSerializationTest >> #setUp to send the message #useGzipInMemoryStream rather than #useStandardFileStream. Now if you run #testMethodContextThisContext it would do it using a GZip stream.

I loaded this in my Squeak image on Linux and can now confirm the crash for both Cog and standard interpreter VM.

Thanks Dave for the confirmation. We haven't use GZip before so I hope we are not doing something wrong ourselfs.

On Sat, Dec 10, 2011 at 05:38:24PM +0100, Mariano Martinez Peck wrote:

On Sat, Dec 10, 2011 at 5:34 PM, David T. Lewis lewis@mail.msen.com wrote:

...

On Sat, Dec 10, 2011 at 03:58:29PM +0100, Mariano Martinez Peck wrote:

...

On Sat, Dec 10, 2011 at 2:16 AM, David T. Lewis lewis@mail.msen.com

wrote:

...

...

This looks like a nasty bug in the plugin, but I cannot reproduce it. I have Fuel loaded in my Squeak image running on Linux, and I ran FLMethodContextSerializationTest>>testMethodContextThisContext without a crash. I tried it on both Cog and a standard VM, and did not get a crash in either case. The test fails (probably for some Squeak/Pharo issue) but it gets through the serialize/materialize part of the test without any problem.

Am I perhaps running a different version of Fuel that does not exercise the ZipPlugin in the same way?

Yes, exactly. Sorry for not saying that before. If you want to reproduce it, you must also install the package FuelCompression and then change FLSerializationTest >> #setUp to send the message #useGzipInMemoryStream rather than #useStandardFileStream. Now if you run #testMethodContextThisContext it would do it using a GZip stream.

I loaded this in my Squeak image on Linux and can now confirm the crash for both Cog and standard interpreter VM.

Thanks Dave for the confirmation. We haven't use GZip before so I hope we are not doing something wrong ourselfs.

This is starting to look like an interesting bug. There certainly is some kind of problem in the serialization phase of the Fuel test, but of course it is hard to debug that when the VM is crashing.

Good news: I figured out how to make it stop crashing, which should make it possible for you to go back to debugging your Fuel test.

Bad news: The way to make the VM stop crashing is to compile it with optimization turned off. That means that perhaps the plugin itself is not directly at fault, but maybe it is something induced by gcc optimization. Or possibly the reverse, it may be a bug in the plugin that is exposed by the optimizer.

I found this by accident because I built a VM with debugging on and optimization off so that I would be able to catch the crash in a debugger. Wonder of wonders, it stopped crashing. Specifically, it worked with 'CFLAGS=-g' and it crashes with 'CFLAGS=-g -O1' or with 'CFLAGS=-g -O2'.

The issue is specific to the plugin itself. If I compile the VM with optimization on, and the plugin with optimization off, then there is no crash.

I am using: Linux version 2.6.34.7-0.7-desktop (geeko@buildhost) (gcc version 4.5.0 20100604 [gcc-4_5-branch revision 160292] (SUSE Linux) ) #1 SMP PREEMPT 2010-12-13 11:13:53 +0100

This is with a standard interpreter VM compiled with gcc 4.5.0. I did not try reproducing this with Cog, but since the previously reported symptoms were basically the same for both Cog and the interpreter VM, I expect that turning off optimization for the ZipPlugin will give the same result for Cog.

Dave

On Sun, Dec 11, 2011 at 02:10:25PM +0100, Mariano Martinez Peck wrote:

On Sun, Dec 11, 2011 at 6:46 AM, David T. Lewis lewis@mail.msen.com wrote:

...

This is starting to look like an interesting bug. There certainly is some kind of problem in the serialization phase of the Fuel test, but of course it is hard to debug that when the VM is crashing.

Yes! What should happen with that test is that it should fail because it doesn't find an instVar called 'testSelector'. If that happens, then it is perfect ;) this is because of something we have recently changed and we need to fix it.

For what it's worth, I when I run the test with a non-optimized plugin (so it does not crash), I am able to get to a debugger that suggests some problem related to array bounds (see attached). No surprise there, but it suggests that the test is probably feeding some invalid parameters to a primitive that crashes it in one case but not the other. This is just speculation of course, but perhaps this is the reason that no one has encountered the bug up to this time.

...

Good news: I figured out how to make it stop crashing, which should make it possible for you to go back to debugging your Fuel test.

Bad news: The way to make the VM stop crashing is to compile it with optimization turned off. That means that perhaps the plugin itself is not directly at fault, but maybe it is something induced by gcc optimization. Or possibly the reverse, it may be a bug in the plugin that is exposed by the optimizer.

I found this by accident because I built a VM with debugging on and optimization off so that I would be able to catch the crash in a debugger. Wonder of wonders, it stopped crashing. Specifically, it worked with 'CFLAGS=-g' and it crashes with 'CFLAGS=-g -O1' or with 'CFLAGS=-g -O2'.

The issue is specific to the plugin itself. If I compile the VM with optimization on, and the plugin with optimization off, then there is no crash.

Thanks Dave for digging in the issue. I tried to do the same as you did with the CMakeVMMaker but I it cashed anyway, so I think I am doing something wrong here. Igor, could you take a look? I have implemented:

configureDeflatePlugin: maker

maker addDefinitions: '-g -O0'.

I was using Ian's CMake build (the standard interpreter VM one for unix), and I hacked around it by editing this file: ZipPlugin/CMakeFiles/ZipPlugin.dir/flags.make

You can probably do something similar with Igor's CMake build as a workaround for your testing. Just ignore the "do not edit this file" warning ;)

Dave