I can now reprocuce corrupt sessions. The trick is to debug DFSession>>numberOfMethodCalls: 1. start new session 2. debug "DFSession uniqueInstance printString" 3. step into #numberOfMehthodCalls 4. proceed 5. stop the session 5. right click on the new session in the session browser and click "Export session" 6. boom!
The reason is of course, that now the compiled method of #numberOfMethodCalls is referenced by the session and will be serialized by Fuel and when Fuel gets to the corrupt BlockClosure, serialization fails.
So the real question still is: how did that BlockClosure obtain a wrong start PC? What I find particularly interesting, is that even changing the method does not fix the BlockClosure. I added the line "Transcript show: 'foo'." before the block (so the block should get a new start PC). Then running the procedure detailed above still produced an open debugger. To illustrate that the block did indeed not get fixed I ran "BlockClosure allInstances select: [ :e | e startpc = 59 ]", with 59 being the new startPC ("55 <8F 01 00 03> closureNumCopied: 0 numArgs: 1 bytes 59 to 61"). The result does not include the block.
Any ideas?
Max
On 24.05.2013, at 08:10, roberto.minelli@usi.ch wrote:
Hi,
Thanks all of you for interest ;)
On May 24, 2013, at 7:15 AM, Max Leske maxleske@gmail.com wrote:
I played around with reverting the method and serializing but that doesn't change anything (the existing BlockClosure is obviously not modified). Then I thought that maybe the method was changed during a running DFSession but still couldn't reproduce.
Me neither!!
@Roberto Can you tell us anything about what you did befor / during / after running your session?
Nope. Unfortunately I've a couple of "broken sessions" I cannot serialize because of that bug.. But all of them refers sooner or later to the DFSession>>#numberOfMethodCalls method. Dunno why.
@Roberto Is it possible to reproduce the stack with a new session? Or did this only happen once?
I have no clue how to reproduce, but it did not happen only once. As I said, there are 3/4 sessions I couldn't export due to this SubscriptOutOfBounds.
Cheers, Roby
On 24.05.2013, at 06:44, Max Leske maxleske@gmail.com wrote:
On 23.05.2013, at 22:58, Camille Teruel camille.teruel@gmail.com wrote:
Hi,
This is indeed strange, the BlockClosure has an incorrect startpc (31 instead of 43). You can fix it with: theBlock instVar: 2 put: 43. The wrong startpc corresponds to the beginning of the block in the previous version of the method (DFSession>>#numberOfMethodCalls), but I don't know why.
Would it help to recompile the method?
I don't know fuel enough but could it be because the method of the block changed between serialization and materialization?
No, there was never a materialization of that session.
Camille
On 23 mai 2013, at 21:02, Max Leske wrote:
Hi Marcus
Roberto sent this mail to the Fuel-dev list. When we looked at the problem we noticed that serialization fails because of a corrupt BlockClosure. Since that's not really our territorry Mariano suggested to forward this to you.
The image with the corrupt BlockClosure is available here: https://dl.dropboxusercontent.com/u/6281855/DFlow-SubscriptOutOfBounds.zip. To see the stack, right click on the only entry in the right window and click "export session". You'll find the corrupt BlockClosure at BlockClosure>>fuelAccept:
@Roberto Is it possible to reproduce the stack with a new session? Or did this only happen once?
Cheers, Max
Begin forwarded message:
From: Mariano Martinez Peck marianopeck@gmail.com Subject: Re: [Pharo-fuel] [Fuel] SubscriptOutOfBounds: 27 Date: 23. Mai 2013 14:23:45 MESZ To: The Fuel Project pharo-fuel@lists.gforge.inria.fr Reply-To: The Fuel Project pharo-fuel@lists.gforge.inria.fr
Indeed, it would be nice if you can known which CompiledMethod and blockclosure are having the problem. Not only the source code but the real bytecodes. Also, can you isolate and just try to serialize them alone and reproduce the problem? In other words, a reproducible test case? :) Thanks!
On Thu, May 23, 2013 at 8:41 AM, Max Leske maxleske@gmail.com wrote: What's the compiled method? Is it a special one? Which selector in which class?
Max
On 23.05.2013, at 12:45, "roberto.minelli@usi.ch" roberto.minelli@usi.ch wrote:
> Hi, > > While using Fuel in a 2.0 image to serialize some objects (including CompiledMethods) I'm getting this error: SubscriptOutOfBounds: 27. > > I spent hours in the debugger before writing this email, but I'm not able to figure out why this is happening. I attach the full stack of the error (STACK#1). > > I understood that it has something to do with a particular CompiledMethod in my image and/or a BlockClosure which for some reason cannot be printed (i.e., the printString returns <error in printString: evaluate "self printString" to debug>) and when I try to printString to debug I got the STACK#2 which in turn does not bring me to any possible solution. > > Hope some of you will have an intuition on that or at least tell me where to look or how to script a possible workaround. > > Thanks in advance, > Roby > > ############################################################################################ > ######################################### STACK#1 ########################################### > ########################################################################################### > CompiledMethod(Object)>>errorSubscriptBounds: > CompiledMethod(Object)>>at: > InstructionStream>>interpretNextInstructionFor: > CompiledMethod>>abstractBytecodeMessageAt: in Block: [(InstructionStream new method: self pc: pc)... > BlockClosure>>on:do: > CompiledMethod>>abstractBytecodeMessageAt: > BlockClosure>>blockCreationBytecodeMessage > BlockClosure>>endPC > BlockClosure>>abstractBytecodeMessagesDo: > BlockClosure>>isClean > BlockClosure>>shouldBeSubstitutedByCleanCopy > BlockClosure>>fuelAccept: > FLFullGeneralMapper(FLLightGeneralMapper)>>mapAndTrace: > FLFullGlobalMapper>>mapAndTrace: in Block: [(anObject class == CompiledMethod... > FLLargeIdentityDictionary>>at:ifAbsent: > FLFullGlobalMapper>>mapAndTrace: > FLPluggableSubstitutionMapper>>mapAndTrace: > FLPluggableSubstitutionMapper>>mapAndTrace: > FLAnalysis>>mapAndTrace: > FLAnalysis>>run > FLAnalyzer>>setDefaultAnalysis in Block: [:anObject | (FLAnalysis... > FLAnalyzer>>analysisFor: > FLSerialization>>analysisStep > FLSerialization>>run > DFSerializer(FLSerializer)>>setDefaultSerialization in Block: [:anObject :anEncoder | (FLSerialization... > DFSerializer(FLSerializer)>>serialize:on: in Block: [:anEncoder | ... > FLEncoder class>>on:globalEnvironment:do: in Block: [aBlock value: anEncoder] > BlockClosure>>ensure: > FLEncoder class>>on:globalEnvironment:do: > DFSerializer(FLSerializer)>>serialize:on: > > > ############################################################################################ > ######################################### STACK#2 ########################################### > ############################################################################################ > Decompiler(Object)>>error: > Decompiler>>decompileBlock: > BlockClosure>>decompile > BlockClosure>>printOn: > BlockClosure(Object)>>printStringLimitedTo: in Block: [:s | self printOn: s] > String class(SequenceableCollection class)>>streamContents:limitedTo: > BlockClosure(Object)>>printStringLimitedTo: > BlockClosure(Object)>>printString > BlockClosure>>DoIt > Compiler>>evaluate:in:to:notifying:ifFail:logged: > SmalltalkEditor>>evaluateSelectionAndDo: in Block: [rcvr class evaluatorClass new... > BlockClosure>>on:do: > SmalltalkEditor>>evaluateSelectionAndDo: > SmalltalkEditor>>evaluateSelection > PluggableTextMorph>>doIt in Block: [textMorph editor evaluateSelection] > PluggableTextMorph>>handleEdit: in Block: [result := editBlock value] > TextMorphForEditView(TextMorph)>>handleEdit: > PluggableTextMorph>>handleEdit: > PluggableTextMorph>>doIt > SmalltalkEditor class>>buildSmalltalkEditorKeymappingsOn: in Block: [:morph | morph doIt] > BlockClosure>>cull: > BlockClosure>>cull:cull: > BlockClosure>>cull:cull:cull: > KMCategoryTarget>>completeMatch:buffer: > KMKeymap>>notifyCompleteMatchTo:buffer: in Block: [:l | l completeMatch: self buffer: aBuffer] > Array(SequenceableCollection)>>do: > KMKeymap>>notifyCompleteMatchTo:buffer: > KMKeymap>>onMatchWith:notify:andDo: > KMCategory>>onMatchWith:notify:andDo: in Block: [:entry | entry... > Set>>do: > > > > _______________________________________________ > Pharo-fuel mailing list > Pharo-fuel@lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-fuel
Pharo-fuel mailing list Pharo-fuel@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-fuel
-- Mariano http://marianopeck.wordpress.com _______________________________________________ Pharo-fuel mailing list Pharo-fuel@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-fuel
Pharo-fuel mailing list Pharo-fuel@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-fuel
Pharo-fuel mailing list Pharo-fuel@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-fuel
Bump.
Doesn't anybody feel that corrupt blocks are a problem? IMHO that's something that should *never* happen.
Max
On 25.05.2013, at 17:56, Max Leske maxleske@gmail.com wrote:
I can now reprocuce corrupt sessions. The trick is to debug DFSession>>numberOfMethodCalls:
- start new session
- debug "DFSession uniqueInstance printString"
- step into #numberOfMehthodCalls
- proceed
- stop the session
- right click on the new session in the session browser and click "Export session"
- boom!
The reason is of course, that now the compiled method of #numberOfMethodCalls is referenced by the session and will be serialized by Fuel and when Fuel gets to the corrupt BlockClosure, serialization fails.
So the real question still is: how did that BlockClosure obtain a wrong start PC? What I find particularly interesting, is that even changing the method does not fix the BlockClosure. I added the line "Transcript show: 'foo'." before the block (so the block should get a new start PC). Then running the procedure detailed above still produced an open debugger. To illustrate that the block did indeed not get fixed I ran "BlockClosure allInstances select: [ :e | e startpc = 59 ]", with 59 being the new startPC ("55 <8F 01 00 03> closureNumCopied: 0 numArgs: 1 bytes 59 to 61"). The result does not include the block.
Any ideas?
Max
On 24.05.2013, at 08:10, roberto.minelli@usi.ch wrote:
Hi,
Thanks all of you for interest ;)
On May 24, 2013, at 7:15 AM, Max Leske maxleske@gmail.com wrote:
I played around with reverting the method and serializing but that doesn't change anything (the existing BlockClosure is obviously not modified). Then I thought that maybe the method was changed during a running DFSession but still couldn't reproduce.
Me neither!!
@Roberto Can you tell us anything about what you did befor / during / after running your session?
Nope. Unfortunately I've a couple of "broken sessions" I cannot serialize because of that bug.. But all of them refers sooner or later to the DFSession>>#numberOfMethodCalls method. Dunno why.
@Roberto Is it possible to reproduce the stack with a new session? Or did this only happen once?
I have no clue how to reproduce, but it did not happen only once. As I said, there are 3/4 sessions I couldn't export due to this SubscriptOutOfBounds.
Cheers, Roby
On 24.05.2013, at 06:44, Max Leske maxleske@gmail.com wrote:
On 23.05.2013, at 22:58, Camille Teruel camille.teruel@gmail.com wrote:
Hi,
This is indeed strange, the BlockClosure has an incorrect startpc (31 instead of 43). You can fix it with: theBlock instVar: 2 put: 43. The wrong startpc corresponds to the beginning of the block in the previous version of the method (DFSession>>#numberOfMethodCalls), but I don't know why.
Would it help to recompile the method?
I don't know fuel enough but could it be because the method of the block changed between serialization and materialization?
No, there was never a materialization of that session.
Camille
On 23 mai 2013, at 21:02, Max Leske wrote:
Hi Marcus
Roberto sent this mail to the Fuel-dev list. When we looked at the problem we noticed that serialization fails because of a corrupt BlockClosure. Since that's not really our territorry Mariano suggested to forward this to you.
The image with the corrupt BlockClosure is available here: https://dl.dropboxusercontent.com/u/6281855/DFlow-SubscriptOutOfBounds.zip. To see the stack, right click on the only entry in the right window and click "export session". You'll find the corrupt BlockClosure at BlockClosure>>fuelAccept:
@Roberto Is it possible to reproduce the stack with a new session? Or did this only happen once?
Cheers, Max
Begin forwarded message:
> From: Mariano Martinez Peck marianopeck@gmail.com > Subject: Re: [Pharo-fuel] [Fuel] SubscriptOutOfBounds: 27 > Date: 23. Mai 2013 14:23:45 MESZ > To: The Fuel Project pharo-fuel@lists.gforge.inria.fr > Reply-To: The Fuel Project pharo-fuel@lists.gforge.inria.fr > > Indeed, it would be nice if you can known which CompiledMethod and blockclosure are having the problem. Not only the source code but the real bytecodes. > Also, can you isolate and just try to serialize them alone and reproduce the problem? In other words, a reproducible test case? :) > Thanks! > > > On Thu, May 23, 2013 at 8:41 AM, Max Leske maxleske@gmail.com wrote: > What's the compiled method? Is it a special one? Which selector in which class? > > Max > > On 23.05.2013, at 12:45, "roberto.minelli@usi.ch" roberto.minelli@usi.ch wrote: > >> Hi, >> >> While using Fuel in a 2.0 image to serialize some objects (including CompiledMethods) I'm getting this error: SubscriptOutOfBounds: 27. >> >> I spent hours in the debugger before writing this email, but I'm not able to figure out why this is happening. I attach the full stack of the error (STACK#1). >> >> I understood that it has something to do with a particular CompiledMethod in my image and/or a BlockClosure which for some reason cannot be printed (i.e., the printString returns <error in printString: evaluate "self printString" to debug>) and when I try to printString to debug I got the STACK#2 which in turn does not bring me to any possible solution. >> >> Hope some of you will have an intuition on that or at least tell me where to look or how to script a possible workaround. >> >> Thanks in advance, >> Roby >> >> ############################################################################################ >> ######################################### STACK#1 ########################################### >> ########################################################################################### >> CompiledMethod(Object)>>errorSubscriptBounds: >> CompiledMethod(Object)>>at: >> InstructionStream>>interpretNextInstructionFor: >> CompiledMethod>>abstractBytecodeMessageAt: in Block: [(InstructionStream new method: self pc: pc)... >> BlockClosure>>on:do: >> CompiledMethod>>abstractBytecodeMessageAt: >> BlockClosure>>blockCreationBytecodeMessage >> BlockClosure>>endPC >> BlockClosure>>abstractBytecodeMessagesDo: >> BlockClosure>>isClean >> BlockClosure>>shouldBeSubstitutedByCleanCopy >> BlockClosure>>fuelAccept: >> FLFullGeneralMapper(FLLightGeneralMapper)>>mapAndTrace: >> FLFullGlobalMapper>>mapAndTrace: in Block: [(anObject class == CompiledMethod... >> FLLargeIdentityDictionary>>at:ifAbsent: >> FLFullGlobalMapper>>mapAndTrace: >> FLPluggableSubstitutionMapper>>mapAndTrace: >> FLPluggableSubstitutionMapper>>mapAndTrace: >> FLAnalysis>>mapAndTrace: >> FLAnalysis>>run >> FLAnalyzer>>setDefaultAnalysis in Block: [:anObject | (FLAnalysis... >> FLAnalyzer>>analysisFor: >> FLSerialization>>analysisStep >> FLSerialization>>run >> DFSerializer(FLSerializer)>>setDefaultSerialization in Block: [:anObject :anEncoder | (FLSerialization... >> DFSerializer(FLSerializer)>>serialize:on: in Block: [:anEncoder | ... >> FLEncoder class>>on:globalEnvironment:do: in Block: [aBlock value: anEncoder] >> BlockClosure>>ensure: >> FLEncoder class>>on:globalEnvironment:do: >> DFSerializer(FLSerializer)>>serialize:on: >> >> >> ############################################################################################ >> ######################################### STACK#2 ########################################### >> ############################################################################################ >> Decompiler(Object)>>error: >> Decompiler>>decompileBlock: >> BlockClosure>>decompile >> BlockClosure>>printOn: >> BlockClosure(Object)>>printStringLimitedTo: in Block: [:s | self printOn: s] >> String class(SequenceableCollection class)>>streamContents:limitedTo: >> BlockClosure(Object)>>printStringLimitedTo: >> BlockClosure(Object)>>printString >> BlockClosure>>DoIt >> Compiler>>evaluate:in:to:notifying:ifFail:logged: >> SmalltalkEditor>>evaluateSelectionAndDo: in Block: [rcvr class evaluatorClass new... >> BlockClosure>>on:do: >> SmalltalkEditor>>evaluateSelectionAndDo: >> SmalltalkEditor>>evaluateSelection >> PluggableTextMorph>>doIt in Block: [textMorph editor evaluateSelection] >> PluggableTextMorph>>handleEdit: in Block: [result := editBlock value] >> TextMorphForEditView(TextMorph)>>handleEdit: >> PluggableTextMorph>>handleEdit: >> PluggableTextMorph>>doIt >> SmalltalkEditor class>>buildSmalltalkEditorKeymappingsOn: in Block: [:morph | morph doIt] >> BlockClosure>>cull: >> BlockClosure>>cull:cull: >> BlockClosure>>cull:cull:cull: >> KMCategoryTarget>>completeMatch:buffer: >> KMKeymap>>notifyCompleteMatchTo:buffer: in Block: [:l | l completeMatch: self buffer: aBuffer] >> Array(SequenceableCollection)>>do: >> KMKeymap>>notifyCompleteMatchTo:buffer: >> KMKeymap>>onMatchWith:notify:andDo: >> KMCategory>>onMatchWith:notify:andDo: in Block: [:entry | entry... >> Set>>do: >> >> >> >> _______________________________________________ >> Pharo-fuel mailing list >> Pharo-fuel@lists.gforge.inria.fr >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-fuel > > _______________________________________________ > Pharo-fuel mailing list > Pharo-fuel@lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-fuel > > > > -- > Mariano > http://marianopeck.wordpress.com > _______________________________________________ > Pharo-fuel mailing list > Pharo-fuel@lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-fuel
Pharo-fuel mailing list Pharo-fuel@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-fuel
Pharo-fuel mailing list Pharo-fuel@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-fuel
On Fri, May 31, 2013 at 12:54 AM, Max Leske maxleske@gmail.com wrote:
Bump.
Doesn't anybody feel that corrupt blocks are a problem? IMHO that's something that should *never* happen.
But what's happening here is that the Fuel serializer is allowing the serialization of blocks without serializing their outer context and method, and later the reconstruction of blocks against the *wrong* version of a method. This is a Fuel bug. IMO the right solution is for Fuel to serialize the method of the block, and on reconstruction replace the reconstructed method with the method in the system iff the reconstructed method is the same as that in the system, but substitute the recinstructed method if it differs from the system's version (or of course if the system;s version is missing). this mimics the system's behaviour with unbound methods, where e.g. a block holds onto a method and the code of a class is recompiled, resulting iun that block holding onto an unbound method.
Max
On 25.05.2013, at 17:56, Max Leske maxleske@gmail.com wrote:
I can now reprocuce corrupt sessions. The trick is to debug
DFSession>>numberOfMethodCalls:
- start new session
- debug "DFSession uniqueInstance printString"
- step into #numberOfMehthodCalls
- proceed
- stop the session
- right click on the new session in the session browser and click
"Export session"
- boom!
The reason is of course, that now the compiled method of
#numberOfMethodCalls is referenced by the session and will be serialized by Fuel and when Fuel gets to the corrupt BlockClosure, serialization fails.
So the real question still is: how did that BlockClosure obtain a wrong
start PC? What I find particularly interesting, is that even changing the method does not fix the BlockClosure. I added the line "Transcript show: 'foo'." before the block (so the block should get a new start PC). Then running the procedure detailed above still produced an open debugger.
To illustrate that the block did indeed not get fixed I ran
"BlockClosure allInstances select: [ :e | e startpc = 59 ]", with 59 being the new startPC ("55 <8F 01 00 03> closureNumCopied: 0 numArgs: 1 bytes 59 to 61"). The result does not include the block.
Any ideas?
Max
On 24.05.2013, at 08:10, roberto.minelli@usi.ch wrote:
Hi,
Thanks all of you for interest ;)
On May 24, 2013, at 7:15 AM, Max Leske maxleske@gmail.com wrote:
I played around with reverting the method and serializing but that
doesn't change anything (the existing BlockClosure is obviously not modified).
Then I thought that maybe the method was changed during a running
DFSession but still couldn't reproduce.
Me neither!!
@Roberto Can you tell us anything about what you did befor / during / after
running your session?
Nope. Unfortunately I've a couple of "broken sessions" I cannot
serialize because of that bug.. But all of them refers sooner or later to the DFSession>>#numberOfMethodCalls method. Dunno why.
@Roberto Is it possible to reproduce the stack with a new session? Or did this
only happen once?
I have no clue how to reproduce, but it did not happen only once. As I
said, there are 3/4 sessions I couldn't export due to this SubscriptOutOfBounds.
Cheers, Roby
On 24.05.2013, at 06:44, Max Leske maxleske@gmail.com wrote:
On 23.05.2013, at 22:58, Camille Teruel camille.teruel@gmail.com
wrote:
Hi,
This is indeed strange, the BlockClosure has an incorrect startpc
(31 instead of 43). You can fix it with:
theBlock instVar: 2 put: 43. The wrong startpc corresponds to the beginning of the block in the
previous version of the method (DFSession>>#numberOfMethodCalls), but I don't know why.
Would it help to recompile the method?
I don't know fuel enough but could it be because the method of the
block changed between serialization and materialization?
No, there was never a materialization of that session.
Camille
On 23 mai 2013, at 21:02, Max Leske wrote:
> Hi Marcus > > Roberto sent this mail to the Fuel-dev list. When we looked at the
problem we noticed that serialization fails because of a corrupt BlockClosure. Since that's not really our territorry Mariano suggested to forward this to you.
> > The image with the corrupt BlockClosure is available here:
https://dl.dropboxusercontent.com/u/6281855/DFlow-SubscriptOutOfBounds.zip .
> To see the stack, right click on the only entry in the right window
and click "export session". You'll find the corrupt BlockClosure at BlockClosure>>fuelAccept:
> > @Roberto > Is it possible to reproduce the stack with a new session? Or did
this only happen once?
> > > Cheers, > Max > > > Begin forwarded message: > >> From: Mariano Martinez Peck marianopeck@gmail.com >> Subject: Re: [Pharo-fuel] [Fuel] SubscriptOutOfBounds: 27 >> Date: 23. Mai 2013 14:23:45 MESZ >> To: The Fuel Project pharo-fuel@lists.gforge.inria.fr >> Reply-To: The Fuel Project pharo-fuel@lists.gforge.inria.fr >> >> Indeed, it would be nice if you can known which CompiledMethod and
blockclosure are having the problem. Not only the source code but the real bytecodes.
>> Also, can you isolate and just try to serialize them alone and
reproduce the problem? In other words, a reproducible test case? :)
>> Thanks! >> >> >> On Thu, May 23, 2013 at 8:41 AM, Max Leske maxleske@gmail.com
wrote:
>> What's the compiled method? Is it a special one? Which selector in
which class?
>> >> Max >> >> On 23.05.2013, at 12:45, "roberto.minelli@usi.ch" <
roberto.minelli@usi.ch> wrote:
>> >>> Hi, >>> >>> While using Fuel in a 2.0 image to serialize some objects
(including CompiledMethods) I'm getting this error: SubscriptOutOfBounds: 27.
>>> >>> I spent hours in the debugger before writing this email, but I'm
not able to figure out why this is happening. I attach the full stack of the error (STACK#1).
>>> >>> I understood that it has something to do with a particular
CompiledMethod in my image and/or a BlockClosure which for some reason cannot be printed (i.e., the printString returns <error in printString: evaluate "self printString" to debug>) and when I try to printString to debug I got the STACK#2 which in turn does not bring me to any possible solution.
>>> >>> Hope some of you will have an intuition on that or at least tell
me where to look or how to script a possible workaround.
>>> >>> Thanks in advance, >>> Roby >>> >>>
############################################################################################
>>> ######################################### STACK#1
###########################################
>>>
###########################################################################################
>>> CompiledMethod(Object)>>errorSubscriptBounds: >>> CompiledMethod(Object)>>at: >>> InstructionStream>>interpretNextInstructionFor: >>> CompiledMethod>>abstractBytecodeMessageAt: in Block:
[(InstructionStream new method: self pc: pc)...
>>> BlockClosure>>on:do: >>> CompiledMethod>>abstractBytecodeMessageAt: >>> BlockClosure>>blockCreationBytecodeMessage >>> BlockClosure>>endPC >>> BlockClosure>>abstractBytecodeMessagesDo: >>> BlockClosure>>isClean >>> BlockClosure>>shouldBeSubstitutedByCleanCopy >>> BlockClosure>>fuelAccept: >>> FLFullGeneralMapper(FLLightGeneralMapper)>>mapAndTrace: >>> FLFullGlobalMapper>>mapAndTrace: in Block: [(anObject class ==
CompiledMethod...
>>> FLLargeIdentityDictionary>>at:ifAbsent: >>> FLFullGlobalMapper>>mapAndTrace: >>> FLPluggableSubstitutionMapper>>mapAndTrace: >>> FLPluggableSubstitutionMapper>>mapAndTrace: >>> FLAnalysis>>mapAndTrace: >>> FLAnalysis>>run >>> FLAnalyzer>>setDefaultAnalysis in Block: [:anObject |
(FLAnalysis...
>>> FLAnalyzer>>analysisFor: >>> FLSerialization>>analysisStep >>> FLSerialization>>run >>> DFSerializer(FLSerializer)>>setDefaultSerialization in Block:
[:anObject :anEncoder | (FLSerialization...
>>> DFSerializer(FLSerializer)>>serialize:on: in Block: [:anEncoder |
...
>>> FLEncoder class>>on:globalEnvironment:do: in Block: [aBlock
value: anEncoder]
>>> BlockClosure>>ensure: >>> FLEncoder class>>on:globalEnvironment:do: >>> DFSerializer(FLSerializer)>>serialize:on: >>> >>> >>>
############################################################################################
>>> ######################################### STACK#2
###########################################
>>>
############################################################################################
>>> Decompiler(Object)>>error: >>> Decompiler>>decompileBlock: >>> BlockClosure>>decompile >>> BlockClosure>>printOn: >>> BlockClosure(Object)>>printStringLimitedTo: in Block: [:s | self
printOn: s]
>>> String class(SequenceableCollection
class)>>streamContents:limitedTo:
>>> BlockClosure(Object)>>printStringLimitedTo: >>> BlockClosure(Object)>>printString >>> BlockClosure>>DoIt >>> Compiler>>evaluate:in:to:notifying:ifFail:logged: >>> SmalltalkEditor>>evaluateSelectionAndDo: in Block: [rcvr class
evaluatorClass new...
>>> BlockClosure>>on:do: >>> SmalltalkEditor>>evaluateSelectionAndDo: >>> SmalltalkEditor>>evaluateSelection >>> PluggableTextMorph>>doIt in Block: [textMorph editor
evaluateSelection]
>>> PluggableTextMorph>>handleEdit: in Block: [result := editBlock
value]
>>> TextMorphForEditView(TextMorph)>>handleEdit: >>> PluggableTextMorph>>handleEdit: >>> PluggableTextMorph>>doIt >>> SmalltalkEditor class>>buildSmalltalkEditorKeymappingsOn: in
Block: [:morph | morph doIt]
>>> BlockClosure>>cull: >>> BlockClosure>>cull:cull: >>> BlockClosure>>cull:cull:cull: >>> KMCategoryTarget>>completeMatch:buffer: >>> KMKeymap>>notifyCompleteMatchTo:buffer: in Block: [:l | l
completeMatch: self buffer: aBuffer]
>>> Array(SequenceableCollection)>>do: >>> KMKeymap>>notifyCompleteMatchTo:buffer: >>> KMKeymap>>onMatchWith:notify:andDo: >>> KMCategory>>onMatchWith:notify:andDo: in Block: [:entry | entry... >>> Set>>do: >>> >>> >>> >>> _______________________________________________ >>> Pharo-fuel mailing list >>> Pharo-fuel@lists.gforge.inria.fr >>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-fuel >> >> _______________________________________________ >> Pharo-fuel mailing list >> Pharo-fuel@lists.gforge.inria.fr >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-fuel >> >> >> >> -- >> Mariano >> http://marianopeck.wordpress.com >> _______________________________________________ >> Pharo-fuel mailing list >> Pharo-fuel@lists.gforge.inria.fr >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-fuel >
Pharo-fuel mailing list Pharo-fuel@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-fuel
Pharo-fuel mailing list Pharo-fuel@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-fuel
On Fri, May 31, 2013 at 1:11 PM, Eliot Miranda eliot.miranda@gmail.comwrote:
On Fri, May 31, 2013 at 12:54 AM, Max Leske maxleske@gmail.com wrote:
Bump.
Doesn't anybody feel that corrupt blocks are a problem? IMHO that's something that should *never* happen.
But what's happening here is that the Fuel serializer is allowing the serialization of blocks without serializing their outer context and method, and later the reconstruction of blocks against the *wrong* version of a method. This is a Fuel bug. IMO the right solution is for Fuel to serialize the method of the block, and on reconstruction replace the reconstructed method with the method in the system iff the reconstructed method is the same as that in the system, but substitute the recinstructed method if it differs from the system's version (or of course if the system;s version is missing). this mimics the system's behaviour with unbound methods, where e.g. a block holds onto a method and the code of a class is recompiled, resulting iun that block holding onto an unbound method.
Sorry, "without serializing their outer context and method" is wrong; "without serializing their method"
Max
On 25.05.2013, at 17:56, Max Leske maxleske@gmail.com wrote:
I can now reprocuce corrupt sessions. The trick is to debug
DFSession>>numberOfMethodCalls:
- start new session
- debug "DFSession uniqueInstance printString"
- step into #numberOfMehthodCalls
- proceed
- stop the session
- right click on the new session in the session browser and click
"Export session"
- boom!
The reason is of course, that now the compiled method of
#numberOfMethodCalls is referenced by the session and will be serialized by Fuel and when Fuel gets to the corrupt BlockClosure, serialization fails.
So the real question still is: how did that BlockClosure obtain a wrong
start PC? What I find particularly interesting, is that even changing the method does not fix the BlockClosure. I added the line "Transcript show: 'foo'." before the block (so the block should get a new start PC). Then running the procedure detailed above still produced an open debugger.
To illustrate that the block did indeed not get fixed I ran
"BlockClosure allInstances select: [ :e | e startpc = 59 ]", with 59 being the new startPC ("55 <8F 01 00 03> closureNumCopied: 0 numArgs: 1 bytes 59 to 61"). The result does not include the block.
Any ideas?
Max
On 24.05.2013, at 08:10, roberto.minelli@usi.ch wrote:
Hi,
Thanks all of you for interest ;)
On May 24, 2013, at 7:15 AM, Max Leske maxleske@gmail.com wrote:
I played around with reverting the method and serializing but that
doesn't change anything (the existing BlockClosure is obviously not modified).
Then I thought that maybe the method was changed during a running
DFSession but still couldn't reproduce.
Me neither!!
@Roberto Can you tell us anything about what you did befor / during / after
running your session?
Nope. Unfortunately I've a couple of "broken sessions" I cannot
serialize because of that bug.. But all of them refers sooner or later to the DFSession>>#numberOfMethodCalls method. Dunno why.
@Roberto Is it possible to reproduce the stack with a new session? Or did this
only happen once?
I have no clue how to reproduce, but it did not happen only once. As I
said, there are 3/4 sessions I couldn't export due to this SubscriptOutOfBounds.
Cheers, Roby
On 24.05.2013, at 06:44, Max Leske maxleske@gmail.com wrote:
On 23.05.2013, at 22:58, Camille Teruel camille.teruel@gmail.com
wrote:
> Hi, > > This is indeed strange, the BlockClosure has an incorrect startpc
(31 instead of 43). You can fix it with:
> theBlock instVar: 2 put: 43. > The wrong startpc corresponds to the beginning of the block in the
previous version of the method (DFSession>>#numberOfMethodCalls), but I don't know why.
Would it help to recompile the method?
> I don't know fuel enough but could it be because the method of the
block changed between serialization and materialization?
No, there was never a materialization of that session.
> > Camille > > On 23 mai 2013, at 21:02, Max Leske wrote: > >> Hi Marcus >> >> Roberto sent this mail to the Fuel-dev list. When we looked at the
problem we noticed that serialization fails because of a corrupt BlockClosure. Since that's not really our territorry Mariano suggested to forward this to you.
>> >> The image with the corrupt BlockClosure is available here:
https://dl.dropboxusercontent.com/u/6281855/DFlow-SubscriptOutOfBounds.zip .
>> To see the stack, right click on the only entry in the right
window and click "export session". You'll find the corrupt BlockClosure at BlockClosure>>fuelAccept:
>> >> @Roberto >> Is it possible to reproduce the stack with a new session? Or did
this only happen once?
>> >> >> Cheers, >> Max >> >> >> Begin forwarded message: >> >>> From: Mariano Martinez Peck marianopeck@gmail.com >>> Subject: Re: [Pharo-fuel] [Fuel] SubscriptOutOfBounds: 27 >>> Date: 23. Mai 2013 14:23:45 MESZ >>> To: The Fuel Project pharo-fuel@lists.gforge.inria.fr >>> Reply-To: The Fuel Project pharo-fuel@lists.gforge.inria.fr >>> >>> Indeed, it would be nice if you can known which CompiledMethod
and blockclosure are having the problem. Not only the source code but the real bytecodes.
>>> Also, can you isolate and just try to serialize them alone and
reproduce the problem? In other words, a reproducible test case? :)
>>> Thanks! >>> >>> >>> On Thu, May 23, 2013 at 8:41 AM, Max Leske maxleske@gmail.com
wrote:
>>> What's the compiled method? Is it a special one? Which selector
in which class?
>>> >>> Max >>> >>> On 23.05.2013, at 12:45, "roberto.minelli@usi.ch" <
roberto.minelli@usi.ch> wrote:
>>> >>>> Hi, >>>> >>>> While using Fuel in a 2.0 image to serialize some objects
(including CompiledMethods) I'm getting this error: SubscriptOutOfBounds: 27.
>>>> >>>> I spent hours in the debugger before writing this email, but I'm
not able to figure out why this is happening. I attach the full stack of the error (STACK#1).
>>>> >>>> I understood that it has something to do with a particular
CompiledMethod in my image and/or a BlockClosure which for some reason cannot be printed (i.e., the printString returns <error in printString: evaluate "self printString" to debug>) and when I try to printString to debug I got the STACK#2 which in turn does not bring me to any possible solution.
>>>> >>>> Hope some of you will have an intuition on that or at least tell
me where to look or how to script a possible workaround.
>>>> >>>> Thanks in advance, >>>> Roby >>>> >>>>
############################################################################################
>>>> ######################################### STACK#1
###########################################
>>>>
###########################################################################################
>>>> CompiledMethod(Object)>>errorSubscriptBounds: >>>> CompiledMethod(Object)>>at: >>>> InstructionStream>>interpretNextInstructionFor: >>>> CompiledMethod>>abstractBytecodeMessageAt: in Block:
[(InstructionStream new method: self pc: pc)...
>>>> BlockClosure>>on:do: >>>> CompiledMethod>>abstractBytecodeMessageAt: >>>> BlockClosure>>blockCreationBytecodeMessage >>>> BlockClosure>>endPC >>>> BlockClosure>>abstractBytecodeMessagesDo: >>>> BlockClosure>>isClean >>>> BlockClosure>>shouldBeSubstitutedByCleanCopy >>>> BlockClosure>>fuelAccept: >>>> FLFullGeneralMapper(FLLightGeneralMapper)>>mapAndTrace: >>>> FLFullGlobalMapper>>mapAndTrace: in Block: [(anObject class ==
CompiledMethod...
>>>> FLLargeIdentityDictionary>>at:ifAbsent: >>>> FLFullGlobalMapper>>mapAndTrace: >>>> FLPluggableSubstitutionMapper>>mapAndTrace: >>>> FLPluggableSubstitutionMapper>>mapAndTrace: >>>> FLAnalysis>>mapAndTrace: >>>> FLAnalysis>>run >>>> FLAnalyzer>>setDefaultAnalysis in Block: [:anObject |
(FLAnalysis...
>>>> FLAnalyzer>>analysisFor: >>>> FLSerialization>>analysisStep >>>> FLSerialization>>run >>>> DFSerializer(FLSerializer)>>setDefaultSerialization in Block:
[:anObject :anEncoder | (FLSerialization...
>>>> DFSerializer(FLSerializer)>>serialize:on: in Block: [:anEncoder
| ...
>>>> FLEncoder class>>on:globalEnvironment:do: in Block: [aBlock
value: anEncoder]
>>>> BlockClosure>>ensure: >>>> FLEncoder class>>on:globalEnvironment:do: >>>> DFSerializer(FLSerializer)>>serialize:on: >>>> >>>> >>>>
############################################################################################
>>>> ######################################### STACK#2
###########################################
>>>>
############################################################################################
>>>> Decompiler(Object)>>error: >>>> Decompiler>>decompileBlock: >>>> BlockClosure>>decompile >>>> BlockClosure>>printOn: >>>> BlockClosure(Object)>>printStringLimitedTo: in Block: [:s | self
printOn: s]
>>>> String class(SequenceableCollection
class)>>streamContents:limitedTo:
>>>> BlockClosure(Object)>>printStringLimitedTo: >>>> BlockClosure(Object)>>printString >>>> BlockClosure>>DoIt >>>> Compiler>>evaluate:in:to:notifying:ifFail:logged: >>>> SmalltalkEditor>>evaluateSelectionAndDo: in Block: [rcvr class
evaluatorClass new...
>>>> BlockClosure>>on:do: >>>> SmalltalkEditor>>evaluateSelectionAndDo: >>>> SmalltalkEditor>>evaluateSelection >>>> PluggableTextMorph>>doIt in Block: [textMorph editor
evaluateSelection]
>>>> PluggableTextMorph>>handleEdit: in Block: [result := editBlock
value]
>>>> TextMorphForEditView(TextMorph)>>handleEdit: >>>> PluggableTextMorph>>handleEdit: >>>> PluggableTextMorph>>doIt >>>> SmalltalkEditor class>>buildSmalltalkEditorKeymappingsOn: in
Block: [:morph | morph doIt]
>>>> BlockClosure>>cull: >>>> BlockClosure>>cull:cull: >>>> BlockClosure>>cull:cull:cull: >>>> KMCategoryTarget>>completeMatch:buffer: >>>> KMKeymap>>notifyCompleteMatchTo:buffer: in Block: [:l | l
completeMatch: self buffer: aBuffer]
>>>> Array(SequenceableCollection)>>do: >>>> KMKeymap>>notifyCompleteMatchTo:buffer: >>>> KMKeymap>>onMatchWith:notify:andDo: >>>> KMCategory>>onMatchWith:notify:andDo: in Block: [:entry |
entry...
>>>> Set>>do: >>>> >>>> >>>> >>>> _______________________________________________ >>>> Pharo-fuel mailing list >>>> Pharo-fuel@lists.gforge.inria.fr >>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-fuel >>> >>> _______________________________________________ >>> Pharo-fuel mailing list >>> Pharo-fuel@lists.gforge.inria.fr >>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-fuel >>> >>> >>> >>> -- >>> Mariano >>> http://marianopeck.wordpress.com >>> _______________________________________________ >>> Pharo-fuel mailing list >>> Pharo-fuel@lists.gforge.inria.fr >>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-fuel >> >
Pharo-fuel mailing list Pharo-fuel@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-fuel
Pharo-fuel mailing list Pharo-fuel@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-fuel
-- best, Eliot
Thanks for your reply Eliot.
Maybe you're right and serializing blocks in the way we do is not the best. In my opinion however, that's not the issue here. I don't have any deep knowledge about bytecodes and closures but as I described, the block is corrupt without *any* interaction from Fuel (at least as far as I can tell from debugging the issue). Fuel can not finish the serialization, hence there's no materialization (or reconstruction as you call it) later either. The block is simply corrupt and even changing the method so that the startpc of the block changes, does not fix the block (the value of the startpc doesn't change and is still that of the first version of the method).
So what I'm worried about has nothing to do with Fuel but with the fact that it's possible to (somehow) create corrupt closures.
Bump.
Doesn't anybody feel that corrupt blocks are a problem? IMHO that's something that should *never* happen.
But what's happening here is that the Fuel serializer is allowing the serialization of blocks without serializing their outer context and method, and later the reconstruction of blocks against the *wrong* version of a method. This is a Fuel bug. IMO the right solution is for Fuel to serialize the method of the block, and on reconstruction replace the reconstructed method with the method in the system iff the reconstructed method is the same as that in the system, but substitute the recinstructed method if it differs from the system's version (or of course if the system;s version is missing). this mimics the system's behaviour with unbound methods, where e.g. a block holds onto a method and the code of a class is recompiled, resulting iun that block holding onto an unbound method.
Sorry, "without serializing their outer context and method" is wrong; "without serializing their method"
On Tue, Jun 4, 2013 at 11:08 AM, Max Leske maxleske@gmail.com wrote:
Thanks for your reply Eliot.
Maybe you're right and serializing blocks in the way we do is not the best. In my opinion however, that's not the issue here. I don't have any deep knowledge about bytecodes and closures but as I described, the block is corrupt without *any* interaction from Fuel (at least as far as I can tell from debugging the issue). Fuel can not finish the serialization, hence there's no materialization (or reconstruction as you call it) later either. The block is simply corrupt and even changing the method so that the startpc of the block changes, does not fix the block (the value of the startpc doesn't change and is still that of the first version of the method).
So what I'm worried about has nothing to do with Fuel but with the fact that it's possible to (somehow) create corrupt closures.
How does that happen? It doesn't happen in normal use. I can see how Fuel does it. I can't see how non-Fuel use would do it (other than deliberate construction). A closure is created by evaluating the push-closure bytecode in a specific method, and when this closure is created, it refers, through its outer context to the method object containing the evaluated push-closure bytecode. Since the closure's startpc is derived from the pc of the push-closure bytecode, there is always a match.
However, when Fuel serializes (IIRC) it serializes a *reference* to a method in the form of a class-name,selector pair, and this approach can obviously yield an invalid method.
therefore I think it much more likely that what you;re seeing is a result of a bug in Fuel than some systemic problem with closures.
Bump.
Doesn't anybody feel that corrupt blocks are a problem? IMHO that's something that should *never* happen.
But what's happening here is that the Fuel serializer is allowing the serialization of blocks without serializing their outer context and
method,
and later the reconstruction of blocks against the *wrong* version of a method. This is a Fuel bug. IMO the right solution is for Fuel to serialize the method of the block, and on reconstruction replace the reconstructed method with the method in the system iff the reconstructed method is the same as that in the system, but substitute the
recinstructed
method if it differs from the system's version (or of course if the system;s version is missing). this mimics the system's behaviour with unbound methods, where e.g. a block holds onto a method and the code of
a
class is recompiled, resulting iun that block holding onto an unbound method.
Sorry, "without serializing their outer context and method" is wrong; "without serializing their method"
I still can't tell you how the closure become corrupted but I now know how to get rid of it. I'm also pretty sure that closures work fine (as you say) and that I probably mistook the same corrupt instance for different instances.
Cheers, Max
How does that happen? It doesn't happen in normal use. I can see how Fuel does it. I can't see how non-Fuel use would do it (other than deliberate construction). A closure is created by evaluating the push-closure bytecode in a specific method, and when this closure is created, it refers, through its outer context to the method object containing the evaluated push-closure bytecode. Since the closure's startpc is derived from the pc of the push-closure bytecode, there is always a match.
However, when Fuel serializes (IIRC) it serializes a *reference* to a method in the form of a class-name,selector pair, and this approach can obviously yield an invalid method.
therefore I think it much more likely that what you;re seeing is a result of a bug in Fuel than some systemic problem with closures.
On Sun, Jun 9, 2013 at 12:38 PM, Max Leske maxleske@gmail.com wrote:
I still can't tell you how the closure become corrupted but I now know how to get rid of it. I'm also pretty sure that closures work fine (as you say) and that I probably mistook the same corrupt instance for different instances.
Yes, but can you see how Fuel can construct bad instances? Is so, you must see that Fuel needs fixing.
Cheers, Max
How does that happen? It doesn't happen in normal use. I can see how
Fuel
does it. I can't see how non-Fuel use would do it (other than deliberate construction). A closure is created by evaluating the push-closure bytecode in a specific method, and when this closure is created, it
refers,
through its outer context to the method object containing the evaluated push-closure bytecode. Since the closure's startpc is derived from the
pc
of the push-closure bytecode, there is always a match.
However, when Fuel serializes (IIRC) it serializes a *reference* to a method in the form of a class-name,selector pair, and this approach can obviously yield an invalid method.
therefore I think it much more likely that what you;re seeing is a result of a bug in Fuel than some systemic problem with closures.
I still can't tell you how the closure become corrupted but I now know how to get rid of it. I'm also pretty sure that closures work fine (as you say) and that I probably mistook the same corrupt instance for different instances.
Yes, but can you see how Fuel can construct bad instances? Is so, you must see that Fuel needs fixing.
I'll have to look into that a bit more. From what I know now, Fuel signals an exception when it detects a method with changed bytecodes (when installing compiled methods). So it seems unlikely to me that Fuel should have created the corrupt closure.
Max
vm-dev@lists.squeakfoundation.org
-
Eliot Miranda -
Max Leske