[Vm-dev] Re: [Pharo-project] [Ann] Ephemerons for Cog

[Igor Stasenko]

On 24 May 2011 19:31, Eliot Miranda <eliot.miranda at gmail.com> wrote:
>
>
>
> On Tue, May 24, 2011 at 9:56 AM, Igor Stasenko <siguctua at gmail.com> wrote:
>>
>> On 24 May 2011 18:38, Eliot Miranda <eliot.miranda at gmail.com> wrote:
>> >
>> >
>> >
>> > On Tue, May 24, 2011 at 9:09 AM, Igor Stasenko <siguctua at gmail.com> wrote:
>> >>
>> >> On 24 May 2011 16:58, Javier Burroni <javier.burroni at gmail.com> wrote:
>> >> >
>> >> > Hi Igor,
>> >> > It's great you have implemented Ephemerons. I'll take a look to the code asap.
>> >> >
>> >> > I've sent the following mail to the VSWE-L list like a month ago. It
>> >> > may (or may not) be useful for you
>> >> > saludos
>> >> > jb
>> >> >
>> >> The mail you posted is hard to follow for me, because
>> >> i don't really understand what 'rescued ephemeron' means.
>> >> Is it the one who initially seen without its key being marked, but
>> >> then after full graph traversal
>> >> has it key marked?
>> >>
>> >> The ephemerons extension to GC are quite simple (it explains why it
>> >> took me 1 day to get initial implementation working).
>> >>
>> >> The only implication of ephemerons,
>> >> that you need to fulfill is that you mark all reachable objects before
>> >> analyzing ephemerons queue.
>> >> And during analysis you have to make sure that again, you are marking
>> >> all reachable objects
>> >> discovered through ephemerons, whose keys are marked.
>> >> If you obey the above rules, then there's not much to fear about. It
>> >> will just work :)
>> >
>> > I think there are two implications, and the second one is non-obvious because the original implementor of ephemerons in VW, who was a GC expert, got this wrong.  The first implication is indeed that you mark all reachable objects before analysing ephemerons with unmarked/unreachable keys (ephemerons with marked/reachable keys can and should be processed just like ordinary objects).  The second implication is that the tracing of ephemerons with unreachable keys continues until a fixed-point is reached where no new ephemerons have been added to the ephemeron queue.  i.e. when tracing ephemerons with unreachable keys one may reach new ephemerons which themselves may have either marked/reachable or unmarked/unreachable keys.
>>
>> I got it from another perspective: we are keep analyzing queue, as
>> long as there are some marking activity happen during previous
>> analysis.
>>
>> This is what this loop for:
>>
>>        "Process ephemerons"
>>        [ self processEphemeronsQueue ] whileTrue: [
>>                "some stack pages may be reachable only through ephemerons"
>>                fullGCFlag
>>                        ifTrue: [ self markAndTraceRestStackPages ].
>>        ].
>>
>> #processEphemeronsQueue aswers true if during processing the queue,
>> some ephemerons are marked their values (because it is found that
>> their keys are marked).
>>
>> > This is really important and a tad tricky.
>>
>> And it is there. Don't worry i got it right.  :)
>>
>>
>> My only concern now is about #markAndTraceOrFreeMachineCode:
>> i found that it sends #markAndTrace: , and so, potentially it could
>> mark some previously unmarked ephemeron's keys. Then we are in
>> trouble,
>> since after that we should again mark values for those ephemerons.
>> So, i need your comment about that.
>
> Tracing object references in machine code methods is conceptually the same as finding them in objects.  A machine code method contains references to other objects and so these must be traced and the references updated when the referents move.  This happens via markAndTrace: just as it does for references from stack pages or ordinary objects.  There's no magic here.  There is an optimization, avoiding the tracing for incremental GCs unless the method contains references to young objects, since tracing machine code is much more expensive than tracing objects.  And of course the machine code method isn't a real object, but a proxy for its compiled method, so it gets discarded when its compiled method is collected.  But apart from the cost, the young optimization, and the being a proxy it is really the same process as tracing ordinary objects.

So, from your explanation my first conclusion is that existing
implementation is wrong. (not saying about ephemerons).
Since before tracing machine code, you tracing stack pages and freeing them.
Then what happens if by occasion, a machine code (indirectly) will
point to page which already freed?

Also, i am a bit wonder, what other objects a machine code could
reference to, in addition to what it's compiled method (from which it
was generated)?
Isn't it would be simpler to treat machine code to be 'alive' as long
as its corresponding compiled method are alive as well?
And objects , reachable directly from compiled method are the same as
reachable from machine code. No?

Ah, ok.. i think i know the answer: inline caches could reference
other methods and so on..

So, then i need to deal with that..
Would it be ok to do it like that:


	fullGCFlag
		ifTrue: [ self markAndTraceRestStackPages ].
        self markAndTraceOrFreeMachineCode: fullGCFlag.

	"Process ephemerons"
	[ self processEphemeronsQueue ] whileTrue: [
		"some stack pages may be reachable only through ephemerons"
		fullGCFlag
			ifTrue: [ self markAndTraceRestStackPages ].
	        self markAndTraceOrFreeMachineCode: fullGCFlag.
	].

	self unlinkEphemeronsStillInQueue.
	
	"Free unused stack pages. Only safe to free stack pages after all
have been traced. "

	fullGCFlag ifTrue: [ self freeUnusedStackPages  ].

or it will require some more serious hacking?	

-- 
Best regards,
Igor Stasenko AKA sig.