<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
</head>
<body>
Hi Eliot,<br>
<br>
- Regarding yesterday's email: yesterday I took the time with Pablo
to read and understand the entire scavenging and I understood that
my first email was nonsense :).<br>
<br>
* We were thinking on enhancing the comment in
#scavengeUnfiredEphemeronsInRememberedSet to explain how the
remembered set is managed. It was a bit unclear at first for us that
the ephemerons to fire were being swapped to occupy the first
places, which was important afterwards for firing them.<br>
<br>
* Also, there is something that we think is a bug: in the whole
ephemeron processing during scavenging, we check always if the key
has survived the scavenge. However, we never check if the key is
old. If the key is old it looks like the algorithm is treating the
ephemerons as ephemerons to fire, while it should not, isn't it?<br>
<br>
<br>
- Regarding the images to test, I'm actually using a plain Pharo
image with your changes (+ modifications in the class builder to
create Ephemeric classes). The snippets that I share crash those
images.<br>
<br>
I'll try to push these changes into Pharo to make it even easier to
reproduce.<br>
<br>
- Then, about the finalizer is just a facility as you say. The
important thing is that with those snippets I can reproduce the
crashes with almost 100% probability and no manual intervention :).<br>
<br>
Guille<br>
<br>
<div class="moz-cite-prefix">-------- Original Message --------<br>
</div>
<blockquote
cite="mid:CAC20JE3bZHpxok0G2_=q+apf1HcZW86OQC5p=q5-uo3w2P222g@mail.gmail.com"
type="cite">
<pre wrap=""> </pre>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<div dir="ltr"><br>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Wed, May 25, 2016 at 10:46 AM,
Eliot Miranda <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:eliot.miranda@gmail.com" target="_blank">eliot.miranda@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">Hi Guille,
<div class="gmail_extra"><br>
<div class="gmail_quote">
<div>
<div class="h5">On Wed, May 25, 2016 at 10:43 AM,
Eliot Miranda <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:eliot.miranda@gmail.com"
target="_blank"><a class="moz-txt-link-abbreviated" href="mailto:eliot.miranda@gmail.com">eliot.miranda@gmail.com</a></a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0
0 0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div dir="ltr">Hi Guille,
<div><br>
</div>
<div> first, v important. can you put the
image(s) you're working with somewhere I
can download, and send me a URL? I can be
more helpful if I can run the tests too.</div>
<div><br>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Wed, May 25,
2016 at 6:24 AM, Guille Polito <span
dir="ltr"><<a
moz-do-not-send="true"
href="mailto:guillermopolito@gmail.com"
target="_blank"><a class="moz-txt-link-abbreviated" href="mailto:guillermopolito@gmail.com">guillermopolito@gmail.com</a></a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"> <br>
<div text="#000000"
bgcolor="#FFFFFF"> Hi again!<br>
<br>
I snip most of the old thread to
keep the relevant.<br>
<br>
<div>-------- Original Message
--------<br>
</div>
<blockquote type="cite">
<div dir="ltr"><span>1 - The
mourn table is being
collected because it is not
marked upon creation<br>
</span>
<div class="gmail_extra">
<div class="gmail_quote">
<blockquote
class="gmail_quote"
style="margin:0px 0px
0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div text="#000000"
bgcolor="#FFFFFF">
===============<br>
<br>
</div>
</blockquote>
<span>
<div>I decided to move
the assignment of
marking out into
markObjects:</div>
</span></div>
</div>
</div>
</blockquote>
<br>
Ok, it looks good for me, I did
not know what would be a good
place to put them without breaking
something :)<br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote"><span>
<div><br>
</div>
<div><br>
</div>
<blockquote
class="gmail_quote"
style="margin:0px 0px
0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div text="#000000"
bgcolor="#FFFFFF">2
- Accesses to
unscannedEphemerons
are unaligned<br>
===============<br>
<br>
</div>
</blockquote>
</span><span>
<div>Looks good. I've
changed all the
ephemeron methods to
use bytesPerOop, cuz
the ephemeron queue
contains oops, but
your fix is just as
good.</div>
</span></div>
</div>
</div>
</blockquote>
<br>
cool! I thought so that would be
the best.<br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div> </div>
<blockquote
class="gmail_quote"
style="margin:0px 0px
0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div text="#000000"
bgcolor="#FFFFFF"><span>
<br>
<br>
3 - While debugging
I also had a DNU
#intAt: in
#primitiveIntegerAt<br>
==========================<br>
<br>
</span><span>
Question: besides
reviewing, how
should I commit
these fixes? one
issue per commit?
all at once?</span></div>
</blockquote>
<span>
<div><br>
</div>
<div>Typically all at
once. What I try and
do is commit a large
set of related changes
in a single commit, so
I wouldn't mix the
fixes to ephemerons
with, say, fixes to
time. But equally I'd
commit any changes I
made to help debugging
in the same commit.
And all rules are made
to be broken ;-).</div>
</span></div>
</div>
</div>
</blockquote>
ok!<span><br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div><br>
</div>
<div>Hey, _thank you_!
This is really great
and ephemerons won't
be usable without
testing like this.
I'm in your debt.</div>
</div>
</div>
</div>
</blockquote>
<br>
</span> Ha, there is no debt (or
at least it would be in the
opposite direction).<br>
<br>
Now I'm working on another crash
that I can easily reproduce using
this script:<br>
<br>
Smalltalk
supportsQueueingFinalization:
true.<br>
e := (1 to: 200000) collect: [ :i
| Ephemeron key: (ObjectFinalizer
receiver: 'test', 'asd' selector:
#logCr) value: Object new ].<br>
Smalltalk garbageCollect.<br>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
<div><br>
</div>
</div>
</div>
<div>Why the extra level of indirection? The
Ephemeron /is/ the object finaliser. It can send
finalise to its key. Why bother with the extra
level of wrapper? It's wasteful; especially if
we're attaching lots of ephemerons to things cuz
we want to finalise something that has lots of
instances.</div>
</div>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>Ignore this stupid question. I see it's an old
facility. And it is a helpful example :-). Sorry for the
noise!</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div class="h5">
<div> </div>
<blockquote class="gmail_quote" style="margin:0
0 0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div dir="ltr">
<div>
<div class="gmail_extra">
<div class="gmail_quote">
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div text="#000000"
bgcolor="#FFFFFF">While debugging
that in the simulator, we saw with
clement that during
#fireEphemeronsInRememberedSet,
#fireEphemeronsOnEphemeronList we
are always firing, tenuring and
scavenging ephemerons regardless
they are marked or not.<br>
<br>
fireEphemeronsInRememberedSet<br>
[SNIP]<br>
coInterpreter
fireEphemeron: ephemeron.<br>
manager<br>
storePointerUnchecked:
0<br>
ofObject: ephemeron<br>
withValue: (self
copyAndForward: (manager
keyOfEphemeron: ephemeron)).<br>
(self scavengeReferentsOf:
ephemeron)<br>
[SNIP]<br>
<br>
So I put an assertion to check
that the key of the ephemeron is
marked when scavenging and it is
not always the case...<br>
self assert: (manager
isMarked: (manager keyOfEphemeron:
ephemeron)).<br>
<br>
Is the code wrong? or I'm missing
something?<br>
</div>
</blockquote>
<div><br>
</div>
<div>The implementation of ephemerons
in the scavenger and the
implementation in mark-sweep are
quite different.</div>
<div><br>
</div>
<div>Remember that what we're trying
to do is find out if an object is
only referenced from the transitive
closure of ephemerons or not. And
the way that we do that in both
cases is avoid processing ephemerons
whose keys are not yet reachable
from the roots until all objects
reachable from the roots have been
reached. We do this by putting
"unscanned" ephemerons in a queue,
saving them until later.</div>
<div><br>
</div>
<div>In the scavenger the roots are
the objects in the remembered set,
the interpreter state (newMethod
etc) and the stack zone, and GC is
performed by copying all objects
reachable from these roots in past
and new spaces into future space,
possibly tenuring overflowed objects
into old space. In the mark-sweep
the roots are the
specialObjectsArray, the interpreter
state (newMethod etc) and the stack
zone.</div>
<div><br>
</div>
<div>In the scavenger (a copying
collector) this means that when we
process the unscanned ephemerons
their keys will either have been
copied into future space or tenured
to old space, in which case they
were reachable from the roots, or
they will not have been copied yet,
in which case they are reachable
only from ephemerons. So in the
scavenger marking is irrelevant; in
fact /no/ objects should be marked
when scavenging. The important
thing is whether a key is in past
and new spaces or is in future and
old spaces.</div>
<div><br>
</div>
<div>In the mark-sweep this means that
when we process the unscanned
ephemerons their keys will either be
marked, in which case they were
reachable from the roots, or
unmarked, in which case they are
reachable only from ephemerons. So
the assert for marked-ness only
makes sense in ephemeron processing
in the mark-sweep collector.</div>
<div><br>
</div>
<div>HTH</div>
<div><br>
</div>
<div> </div>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div text="#000000"
bgcolor="#FFFFFF">Guille
<div>
<div><br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div> </div>
<blockquote
class="gmail_quote"
style="margin:0px
0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div text="#000000"
bgcolor="#FFFFFF">
<div>
<div><br>
<br>
Guille<br>
<br>
<div>--------
Original
Message
--------<br>
</div>
<blockquote
type="cite"> I
see, I
probably I
need to leave
the simulator
run for a
longer time...
Because
running the
image using
the assertion
VM took some
time to crash,
it may be the
same case.<br>
<br>
<div>--------
Original
Message
--------<br>
</div>
<blockquote
type="cite">
Some advance
on my side,<br>
<br>
- Yes,
apparently the
image was
already
broken. I
started a new
one and I can
open it with
the simulator.
It seems that
as soon as I
activate
Ephemerons and
I have alive
ephemerons the
heap corrupts
(and since
saving the
image fires a
gc...).<br>
<br>
- Then, to
catch the
moment where
the image
breaks, I
prepared my
image for
debug with the
following
script:<br>
<br>
Smalltalk
snapshotPrimitive.<br>
f :=
ObjectFinalizer
receiver:
'Hello'
selector:
#logCr.<br>
d :=
EphemeronDictionary
new.<br>
d at: f put:
f.<br>
f := nil.<br>
Smalltalk
supportsQueueingFinalization:
true.<br>
1 to: 2 do: [
:i | Smalltalk
garbageCollect].<br>
<br>
Then I opened
it with the
simulator
(latest
version:
VMMaker.oscog-tfel.1864)
and I had
several
assertion
failures in<br>
<br>
keyOfEphemeron:
objOop<br>
"Answer
the object the
ephemeron
guards. This
is its first
element."<br>
self
assert: ((self
isNonImmediate:
objOop) and:
[self
isEphemeron:
objOop]).<br>
^self
fetchPointer:
0 ofObject:
objOop<br>
<br>
followedKeyOfEphemeron:
objOop<br>
"Answer
the object the
ephemeron
guards. This
is its first
element."<br>
self
assert: ((self
isNonImmediate:
objOop) and:
[self
isEphemeron:
objOop]).<br>
^self
followOopField:
0 ofObject:
objOop<br>
<br>
So, I noticed
the following:<br>
<br>
- Ephemerons
in the
unscanned list
are marked,
fired, and the
format of
ephemerons is
changed to a
non-ephemeron
format<br>
- But they are
not removed
from the
unscanned list<br>
- then the
mark
#markAllUnscannedEphemerons
fails tons of
assertions
because it
expects
ephemeron
objects and
the
(not-anymore-)ephemeron
is still there
in the
unscanned list<br>
<br>
self
markInactiveEphemerons
ifFalse:<br>
[self
fireAllUnscannedEphemerons].<br>
self
markAllUnscannedEphemerons<br>
<br>
The strange
thing is that
if I ignore
the assertions
by proceeding,
the thing
continues
normally and
it the image
does not
break...<br>
<br>
So I then
compiled the
VM, with the
latest
version, and
in trace mode
to compare. I
attach the log
of the stdout.
But here there
is a résumé<br>
<br>
* First I see
the same
assertions I
saw in the
simulator<br>
<br>
(isNonImmediate(objOop))
&&
(isEphemeron(objOop))
41155<br>
<br>
(isNonImmediate(objOop))
&&
(isEphemeron(objOop))
55477<br>
<br>
* Then there
are tons of
repetitions of
the same
assertion:<br>
<br>
!(((GIV(mournQueue)
!=
GIV(nilObj))
&&
(isonObjStack(anEphemeronOrWeakArray,
GIV(mournQueue)))))
60030<br>
<br>
Which I
understand it
means that we
are adding to
the mourn
queue all the
time the same
object. Which
is strange
because it is
a WeakArray
with a single
slot which
turns out to
be nil (at
least at the
moment I
inspected it).<br>
<br>
* Then
something
breaks (!!)
Looks like we
are trying to
mark a
forwarder. And
then I have a
follow up of
assertions
which are
probably
caused by the
same thing.<br>
<br>
!(isForwarded(field))
49489<br>
<br>
(classIndexOf(objToScan))
>
(isForwardedObjectClassIndexPun())
49537<br>
<br>
(classIndex
<=
(tagMask()))
|| (classIndex
>=
(arrayClassIndexPun()))
49403<br>
<br>
objCouldBeClassObj(classObj)
49415<br>
<br>
addressCouldBeOop(objOop)
59997<br>
<br>
addressCouldBeObj(topOfObjStack((0
==
(fetchPointerofObject(ObjStackTopx,
objStack)) ?
fetchPointerofObject(ObjStackNextx,
objStack) :
objStack)))
60002<br>
<br>
addressCouldBeOop(objOop)
59997<br>
<br>
* Finally
there are some
more <br>
<br>
!(isForwarded(field))
49489<br>
<br>
until a
segmentation
fault arrives
while
executing a
fullGC<br>
<br>
../results/pharo[0x80ceaa4]<br>
../results/pharo[0x80cfde4]<br>
../results/pharo(fullGC+0x9b)[0x80c95ea]<br>
../results/pharo[0x80e2d49]<br>
../results/pharo[0x80e5f4a]<br>
../results/pharo[0x80ebca3]<br>
../results/pharo(ceStackOverflow+0xd7)[0x809356a]<br>
[0x91002c6]<br>
../results/pharo(interpret+0x128)[0x8082b38]<br>
../results/pharo(main+0x23a)[0x810a7db]<br>
/lib/i386-linux-gnu/i686/cmov/libc.so.6(__libc_start_main+0xf3)[0xf73b5a63]<br>
[0x1000]<br>
<br>
I'll continue
digging a bit
more,<br>
Guille<br>
<br>
<div>--------
Original
Message
--------<br>
</div>
<blockquote
type="cite">
<div dir="ltr">Hi
Guille,
<div
class="gmail_extra"><br>
<div
class="gmail_quote">On
Thu, May 19,
2016 at 12:53
AM, Guille
Polito <span
dir="ltr"><<a
moz-do-not-send="true" href="mailto:guillermopolito@gmail.com"
target="_blank"><a class="moz-txt-link-abbreviated" href="mailto:guillermopolito@gmail.com">guillermopolito@gmail.com</a></a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0px
0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div><span> <br>
<br>
<div>--------
Original
Message
--------<br>
</div>
<blockquote
type="cite">
<div dir="ltr">Hi
Guille,
<div
class="gmail_extra"><br>
<div
class="gmail_quote">On
Wed, May 18,
2016 at 12:46
AM, Guille
Polito <span
dir="ltr"><<a
moz-do-not-send="true" href="mailto:guillermopolito@gmail.com"
target="_blank"><a class="moz-txt-link-abbreviated" href="mailto:guillermopolito@gmail.com">guillermopolito@gmail.com</a></a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0px
0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div> Hello,<span><br>
<br>
<div>--------
Original
Message
--------<br>
</div>
<blockquote
type="cite">
<pre> </pre>
<br>
<fieldset></fieldset>
<br>
<div dir="ltr">Hi
Guille, Hi
Pablo (and
welcome),<br>
<div
class="gmail_extra"><br>
<div
class="gmail_quote">On
Tue, May 17,
2016 at 8:37
AM, Guille
Polito <span
dir="ltr"><<a
moz-do-not-send="true" href="mailto:guillermopolito@gmail.com"
target="_blank"><a class="moz-txt-link-abbreviated" href="mailto:guillermopolito@gmail.com">guillermopolito@gmail.com</a></a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0px
0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><br>
Hi Eliot, list<br>
<br>
I'm working
here with
Pablo (Tesone)
on moving
forward the
Ephemeron
implementation.</blockquote>
</div>
</div>
</div>
</blockquote>
</span></div>
</blockquote>
<div><br>
</div>
<div><br>
</div>
<div>Where's
"here"? Are
you in Lille?</div>
</div>
</div>
</div>
</blockquote>
</span> Yup.
Pablo is a new
Phd student
here :)<span><br>
<blockquote
type="cite">
<div dir="ltr">
<div
class="gmail_extra">
<div
class="gmail_quote">
<div><br>
</div>
<div><br>
</div>
<blockquote
class="gmail_quote"
style="margin:0px
0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div><span>
<blockquote
type="cite">
<div dir="ltr">
<div
class="gmail_extra">
<div
class="gmail_quote">
<blockquote
class="gmail_quote"
style="margin:0px
0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
We first
installed
Eliot's
changeset,
added a #mourn
method and an
EphemeronDictionary
collection,
and then
started
testing
something like
this:<br>
<br>
f :=
ObjectFinalizer
receiver:
'Hello'
selector:
#logCr.<br>
d :=
EphemeronDictionary
new.<br>
<br>
d at: f put:
f.<br>
<br>
f := nil.<br>
Smalltalk
garbageCollect.<br>
</blockquote>
<div><br>
</div>
<div>So this
looks like
something
simulate
able. Are you
able to use
the
simulator? If
not, why not?
<br>
</div>
</div>
</div>
</div>
</blockquote>
<br>
</span> For
some reason I
have that
bytesToShift
when opening
the image is
negative.<br>
</div>
</blockquote>
<div><br>
</div>
<div>That is
to be
expected. In
the real VM
the heap is
located
somewhere well
above the
bottom of the
address space,
typically
above the
program code.
In the
simulator the
heap is
located either
at 0 (an
interpreter or
stack VM) or
immediately
above the code
zone (in a
Cogit VM). So
when an image
that has been
saved on the
real VM is
loaded into
the simulator
all oops have
to be adjusted
down and hence
bytesToShift
is negative.</div>
<div><br>
</div>
<blockquote
class="gmail_quote"
style="margin:0px
0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div> <br>
bytesToShift
:=
objectMemory
memoryBaseForImageRead
-
oldBaseAddr.
"adjust
pointers for
zero base
address"<br>
<br>
So I cannot
continue
loading
because
addresses
become
negative and I
have "Improper
Store into
indexable
object kind of
errors".</div>
</blockquote>
<div><br>
</div>
<div>Can you
post a back
trace? Where
is this
happening? </div>
</div>
</div>
</div>
</blockquote>
</span> I'm
working in
latest pharo
(5.0), using
the same
source code
that is used
in the Pharo
CI jobs to
build the VM
(pharo
branded). I
would expect
this branch to
be stable,
maybe it could
be a bit
behind your
branch of
development,
but I think
Esteban merges
regularly.<br>
</div>
</blockquote>
<div><br>
</div>
<div>Esteban
is very busy
trying to get
UFFI working,
and having
just got Pharo
5 released and
working on
Pharo 6 to be
released. I'd
prefer it if
you used the
tip of
VMMaker.oscog.
Further,
that's the
package that
Clément is
using and he's
your expert
source of
local help.</div>
<div><br>
</div>
<div> </div>
<blockquote
class="gmail_quote"
style="margin:0px
0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div> <br>
I'm executing
this,<br>
<br>
vm :=
StackInterpreterSimulator
newWithOptions:
#(#ObjectMemory
#Spur32BitMemoryManager
).<br>
vm
desiredNumStackPages:
8. "Makes
simulation
faster by
creating fewer
stack pages."<br>
(vm openOn:
'/home/guille/Pharo/temp/ephemerons/broken.image')<br>
openAsMorph;
run.<br>
<br>
And here is
the trace. The
error happens
while
swizzling.<br>
<br>
LittleEndianBitmap(Object)>>error:<br>
LittleEndianBitmap(Object)>>errorImproperStore<br>
LittleEndianBitmap(Object)>>at:put:<br>
Spur32BitMMLESimulator>>longAt:put:<br>
Spur32BitMMLESimulator(SpurMemoryManager)>>swizzleFieldsOfObject:<br>
Spur32BitMMLESimulator(SpurMemoryManager)>>adjustAllOopsBy:<br>
Spur32BitMMLESimulator(SpurMemoryManager)>>initializeObjectMemory:<br>
StackInterpreterSimulatorLSB(StackInterpreter)>>initializeInterpreter:<br>
[ self
initializeInterpreter:
bytesToShift ]
in
StackInterpreterSimulatorLSB(StackInterpreterSimulator)>>openOn:extraMemory:
in Block: [
self
initializeInterpreter:
bytesToShift ]<span><br>
</span></div>
</blockquote>
<div><br>
</div>
<div> So the
image is
presumably
already
broken? Looks
to me like it
was corrupted
when saved.
Can you
reproduce the
problem that
led to this
image being
broken from a
working image?
</div>
<blockquote
class="gmail_quote"
style="margin:0px
0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div><span>
<blockquote
type="cite">
<div dir="ltr">
<div
class="gmail_extra">
<div
class="gmail_quote">
<div>What
version of
VMMaker.oscog
are you using?</div>
</div>
</div>
</div>
</blockquote>
</span>
Monticello
reports
VMMaker.oscog-eem.1855.<br>
However, I'm
not sure I
have the right
number,
because source
code is now in
git. <br>
</div>
</blockquote>
<div><br>
</div>
<div>It would
be so great if
you could use
Monticello,
not git, and
hence be able
to contribute
back.</div>
<div> </div>
<blockquote
class="gmail_quote"
style="margin:0px
0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div><span>
<blockquote
type="cite">
<div dir="ltr">
<div
class="gmail_extra">
<div
class="gmail_quote">
<div> Are you
running in
Pharo or
Squeak? If
you're in
Lille you
could perhaps
visit
Clément's
office and get
him to take a
look.
Clément, would
that be ok?</div>
</div>
</div>
</div>
</blockquote>
</span> I'll
maybe see it
with clement,
but today I
have to work
on some other
project...<span><br>
<blockquote
type="cite">
<div dir="ltr">
<div
class="gmail_extra">
<div
class="gmail_quote">
<blockquote
class="gmail_quote"
style="margin:0px
0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div><span>
<blockquote
type="cite">
<div dir="ltr">
<div
class="gmail_extra">
<div
class="gmail_quote">
<div>When
debugging the
VM there are
two main
levels of
support, one
is the
simulator
where there is
maximum
support for
debugging:</div>
<div>- asserts
on all the
time</div>
<div>-
arbitrary
breakpoints</div>
<div>-
attempting
every GC in a
copy of the
heap before
doing the real
GC so that
bugs in the GC
can be
investigated
without
needing to
construct a
reproducible
case after a
crash</div>
<div>- the
Smalltalk
environment to
inspect and
browse</div>
<div><br>
</div>
<div>The next
level is the
assert and
debug VMs. If
you look in
the build
directories on
the Cog svn
branch you'll
see that all
of them build
three VMs, a
production VM
with maximum
optimisation
and asserts
excluded, an
assert VM with
-O1 and
asserts
enabled, and a
debug VM with
-O0 and
asserts
enabled. So
if you either
don't see the
bug in the
simulator, or
the simulator
is too slow
for the case
being
examined, or
if the bug
doesn't show
up in the
simulator (the
worst of all
worlds), build
both assert
and debug VMs
and run with
the assert VM
first.</div>
<div><br>
</div>
</div>
</div>
</div>
</blockquote>
</span> Well
so far we were
using a VM
compiled for
debug with a
graphical C
debugger. It
was not so
bad. However,
I cannot say
I'm missing a
better
debugger.</div>
</blockquote>
<div><br>
</div>
<div>"Compiled
for debug" is
vague. Do you
mean it is
compiled with
-g -O0, or in
addition is
compiled with
-g -O0
-DDEBUGVM=0
-DNDEBUG=1?</div>
<div> </div>
</div>
</div>
</div>
</blockquote>
</span> That
would exactly
be:<br>
<br>
{ '-g3'. <br>
'-O0'.
<br>
'-msse2'. <br>
'-D_GNU_SOURCE'.
<br>
'-DITIMER_HEARTBEAT=1'.
<br>
'-DNO_VM_PROFILE=1'.
<br>
'-DDEBUGVM=1'.<br>
'-DNDEBUG' }<br>
<br>
And Esteban
told me to
compile it
with
'-UNDEBUG' to
get more
information,
but I did not
try this yet.<br>
</div>
</blockquote>
<div><br>
</div>
<div>Yes.
NDEBUG is what
turns off
asserts. You
don't have a
proper assert
or debug VM
unless you do
not use
-NDEBUG.</div>
<div><br>
</div>
<blockquote
class="gmail_quote"
style="margin:0px
0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div>Thanks!
<div>
<div><br>
<blockquote
type="cite">
<div dir="ltr">
<div
class="gmail_extra">
<div
class="gmail_quote">
<blockquote
class="gmail_quote"
style="margin:0px
0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div><span>
<blockquote
type="cite">
<div dir="ltr">
<div
class="gmail_extra">
<div
class="gmail_quote">
<div>Note that
there is a
heap leak
checker which
can be enabled
both in the
simulator and
the assert and
debug VMs.
See the
checkForLeaks
method and the
-leakcheck
argument.</div>
</div>
</div>
</div>
</blockquote>
<br>
</span> ok!<span><br>
<blockquote
type="cite">
<div dir="ltr">
<div
class="gmail_extra">
<div
class="gmail_quote">
<div><br>
</div>
<div>Without
the simulator
or the assert
and debug VMs
you are flying
blind. It is
/really/
productive to
use the
simulator for
debugging,
provided the
bug is
reproducible
within a short
amount of
time, as for
example your
case is above.</div>
</div>
</div>
</div>
</blockquote>
<br>
</span> Ok,
gotcha! By
this afternoon
I'll have some
news probably.<br>
<br>
Thanks a lot!
<div>
<div><br>
<blockquote
type="cite">
<div dir="ltr">
<div
class="gmail_extra">
<div
class="gmail_quote">
<div><br>
</div>
<div><br>
</div>
<blockquote
class="gmail_quote"
style="margin:0px
0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
<br>
However, as
soon as we
garbage
collect twice,
we have a VM
crash. We
started
debugging the
VM to see if
we could have
some more
clues.<br>
<br>
The first
thing we
noticed is
that the first
time the GC
runs, the
mournQueue is
nil. This is
of course
expected
because the
new
finalization
mechanism was
not active and
then there was
no need to
create the
mournQueue. We
saw that the
mournQueue is
actually
created in a
lazy fashion
when putting
queuing a
mourned object
(I refer
myself to
#queueMourner:
and
#ensureRoomOnObjStackAt:).
So the second
time the GC
passes, the
mournQueue is
there. So far
ok, but still
crashing.<br>
<br>
The crash
happens in the
call to<br>
<br>
markAndTraceObjStackandContents(GIV(mournQueue),
1);<br>
<br>
after the<br>
<br>
if
(!markAndTraceContents)
{<br>
return;<br>
}<br>
<br>
But when
understanding
why, it starts
being less
clear to us
:). We used
the
printObjStack()
function and
we saw that:<br>
<br>
call
printObjStack(markStack)<br>
call
printObjStack(weaklingStack)<br>
<br>
and we saw in
the console
some output
that makes
sense.
However,
printing the
mournQueue in
the same
manner
produces some
strange output<br>
<br>
call
printObjStack(mournQueue)<br>
<br>
head
0xb06e980 cx
18 (18) fmt 10
(10) sz 4092
(4092) myx:
4098 (4098)
unmkd<br>
topx: 14
next:
0x0 free:
0x0<br>
<br>
We noticed
that free and
next are 0x0
while the
others are
not...<br>
<br>
Finally we saw
there is
isValidObjStack(),
that gave us
the following
results:<br>
<br>
call
isValidObjStack(markStack)
=> 1<br>
<br>
call
isValidObjStack(weaklingStack)
=> 0<br>
p
objStackInvalidBecause
= "marking but
page is
unmarked"<br>
<br>
call
isValidObjStack(mournQueue)
=> 0<br>
p
objStackInvalidBecause
= "marking but
page is
unmarked"<br>
<br>
<br>
So we assume
that the stack
creation is
wrong? We are
a bit lost in
here.<br>
<br>
Guille and
Pablo<br>
</blockquote>
</div>
<br>
<br
clear="all">
<div><br>
</div>
-- <br>
<div>
<div dir="ltr">
<div><span
style="font-size:small;border-collapse:separate">
<div>_,,,^..^,,,_<br>
</div>
<div>best, Eliot</div>
</span></div>
</div>
</div>
</div>
</div>
</blockquote>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
<br
clear="all">
<div><br>
</div>
-- <br>
<div>
<div dir="ltr">
<div><span
style="font-size:small;border-collapse:separate">
<div>_,,,^..^,,,_<br>
</div>
<div>best, Eliot</div>
</span></div>
</div>
</div>
</div>
</div>
</blockquote>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
<br
clear="all">
<div><br>
</div>
-- <br>
<div>
<div dir="ltr">
<div><span
style="font-size:small;border-collapse:separate">
<div>_,,,^..^,,,_<br>
</div>
<div>best, Eliot</div>
</span></div>
</div>
</div>
</div>
</div>
</blockquote>
<br>
</blockquote>
<br>
</blockquote>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<div><br>
</div>
-- <br>
<div>
<div dir="ltr">
<div><span
style="font-size:small;border-collapse:separate">
<div>_,,,^..^,,,_<br>
</div>
<div>best, Eliot</div>
</span></div>
</div>
</div>
</div>
</div>
</blockquote>
<br>
</div>
</div>
</div>
<br>
</blockquote>
</div>
<span><font color="#888888"><br>
<br clear="all">
<div><br>
</div>
-- <br>
<div>
<div dir="ltr">
<div><span
style="font-size:small;border-collapse:separate">
<div>_,,,^..^,,,_<br>
</div>
<div>best, Eliot</div>
</span></div>
</div>
</div>
</font></span></div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
<span class="HOEnZb"><font color="#888888"><br>
<br clear="all">
<div><br>
</div>
-- <br>
<div>
<div dir="ltr">
<div><span
style="font-size:small;border-collapse:separate">
<div>_,,,^..^,,,_<br>
</div>
<div>best, Eliot</div>
</span></div>
</div>
</div>
</font></span></div>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<div><br>
</div>
-- <br>
<div class="gmail_signature">
<div dir="ltr">
<div><span
style="font-size:small;border-collapse:separate">
<div>_,,,^..^,,,_<br>
</div>
<div>best, Eliot</div>
</span></div>
</div>
</div>
</div>
</div>
</blockquote>
<br>
</body>
</html>