[Vm-dev] Re: [Pharo-dev] [ANN] Ephemeron Support is Ready

Wed Jun 8 07:36:27 UTC 2016

Hi Eliot,

Pharo does AFAIK the same with the source files when you're navigating 
source code.

Now, I remember that while adapting the finalization scheme that you 
sent me to Pharo (because there are indeed subtle differences), I 
noticed that there was a missing loop.

[looking for the code... found!]

finalizationProcess
     "The finalization process arranges to send mourn to each element of 
the VM's finalization queue,
      which is accessed via primitiveFetchMourner.  The VM signals 
FinalizationSemaphore whenever
      the queue is non-empty.  This process loops, waiting on the 
semaphore, fetches the first element
      of the queue and then spawns a process at a higher priority to 
acually send the mourn messages.
      If an error occurs in the higher priority mourn loop process then 
this process will simply spawn
      another process, hence ensuring that errors in finalization 
methods don't break finalization.

      In addition this process also runs the old finalization scheme, 
supporting clients of the older,
      WeakRegistry based scheme.  Hopefully this will go away when all 
cleints have moved over."
     | throttle firstMourner |
     throttle := Semaphore new.
     [FinalizationSemaphore wait; initSignals.
      "Support the old scheme until things have changed over..."
      self doOldFinalization.
      [firstMourner := self primitiveFetchMourner.
       firstMourner notNil] whileTrue:
         [[throttle signal.
           self mournLoopWith: firstMourner] forkAt: Processor 
activePriority + 1.
          throttle wait]]

At first I was using that code that you sent me and I noticed that the 
finalization process in there is a loop that is never evaluated! So I 
updated it to the following using a [true] whileTrue:

finalizationProcess
     "The finalization process arranges to send mourn to each element of 
the VM's finalization queue,
      which is accessed via primitiveFetchMourner.  The VM signals 
FinalizationSemaphore whenever
      the queue is non-empty.  This process loops, waiting on the 
semaphore, fetches the first element
      of the queue and then spawns a process at a higher priority to 
acually send the mourn messages.
      If an error occurs in the higher priority mourn loop process then 
this process will simply spawn
      another process, hence ensuring that errors in finalization 
methods don't break finalization.

      In addition this process also runs the old finalization scheme, 
supporting clients of the older,
      WeakRegistry based scheme.  Hopefully this will go away when all 
cleints have moved over."
     | throttle firstMourner |
     throttle := Semaphore new.
     [true] whileTrue: [FinalizationSemaphore wait; initSignals.
      "Support the old scheme until things have changed over..."
      self doOldFinalization.
      [firstMourner := self primitiveFetchMourner.
       firstMourner notNil] whileTrue:
         [[throttle signal.
           self mournLoopWith: firstMourner] forkAt: Processor 
activePriority + 1.
          throttle wait]]

Maybe that's the reason of weak arrays not being finalized in squeak?

Guille

-------- Original Message --------
>   
>
>
> Hi Guille,
>
>    good news!  But I'm seeing something wrong with finalisation of 
> weak arrays using the new scheme.  In Squeak method source access is 
> done by default by opening a new read-only file for each method's 
> source read (crazy, but that's not the issue).  So by default 
> something that accesses lots of source ends up running out of file 
> descriptors, which causes primOpen:writable: to fail.  The surrounding 
> code then uses retryWithGC:until:forFileNamed: to do a GC to try and 
> reclaim non-longer referenced files, close file descriptors and continue:
>
>         StandardFileStream retryWithGC:[self primOpen: f writable: 
> writeMode]
> until:[:id| id notNil]
> forFileNamed: fileName.
>
> But in my tests I'm not seeing any files reclaimed.  I wonder whether 
> the new finalisation code is failing to finalise weak arrays 
> properly.  I wonder whether your weak tests work properly with the new 
> scheme or not.
>
> Anyway, I think I can reproduce the pathology in the simulator if I 
> modify it to implement a small limit on the number of file 
> descriptors.  I'm going to try that to shed light on the problem.  I 
> can't easily debug in a running image because...I run out of file 
> descriptors ;-)
>
> On Tue, Jun 7, 2016 at 2:32 AM, Guille Polito 
> <guillermopolito at gmail.com <mailto:guillermopolito at gmail.com>> wrote:
>
>
>
>
>
>
>
>     Hi All,
>
>
>
>     Since this morning, in Pharo #60065, Ephemeron support is in the
>     image. Most of the changes are infrastructural, so far transparent
>     for the users. It is important to notice that even while the support
>     is there, it is not enabled by default. Also, this required changes
>     in the virtual machine that are not yet distributed everywhere. For
>     the ones that would like more detail, I invite you to read the
>     following :)
>
>
>
>     * On the infrastructure side
>
>       - There is support to create Ephemeric classes and load them from
>     monticello
>
>       - There is a new finalization mechanism (by default disabled) that
>     will process Ephemerons using a finalization queue. This will avoid
>     scanning collections in look for weak objects to finalize ,as it
>     happens now with the WeakDependent mechanism in WeakArray.
>
>       - System-Finalization features two new classes *Ephemeron* and
>     *EphemeronRegistry*. For the ones that want more details on
>     Ephemerons, you can read the associated paper [1], or the class
>     comment of Ephemeron:
>
>
>
>         I represent ephemeric key-value objects. Ephemerons are
>         key-value objects (subclasses of Association) with special
>         semantics during garbage collection.  My special behavior can
>         resumed as follows:
>
>
>
>         - The garbage collection will iterate my instances only if the key
>         is not referenced strongly by another object.
>
>         - Then, if no strong references to the key are found, then the
>         values of this ephemeron are hold weakly.
>
>         - Otherwise, the values are hold strongly.
>
>
>
>         In this implementation, an Ephemeron can hold more than one value,
>         which are all treated in the same manner. This ephemeron instance
>         knows its container, which allows the ephemeron to remove itself
>         from a container (such as a Dictionary) upon finalization.
>
>
>
>         !! Example usages
>
>
>
>         In general terms, do not use myself directly. Use instead an
>         Ephemeric container like EphemeronRegistry. An Ephemeron registry
>         will guarantee the collection of keys and values of the object
>         inside the Ephemeron.
>
>
>
>         Otherwise, if you want to use it, you can create an Ephemeron as
>         any association:
>
>
>
>         ephemeron := Ephemeron key: aKey value: aValue.
>
>         ephemeron container: aContainer.
>
>
>
>         !! Ephemeron Finalization
>
>
>
>         When an ephemeron's key is hold strongly just by the ephemeron
>         itself, the Ephemeron will be mourned (finalized). That means that
>         the VM will:
>
>         - put the Ephemeron in the mourning queue waiting for the image to
>         take care of mourning
>
>         - make the Ephemeron non ephemeric. That is, the ephemeron
>         instance cannot be reused.
>
>
>
>         On the image side, the finalization process will send the message
>         #mourn to an Ephemeron.  #mourn will #finalize the Ephemeron's
>         key, and remove the Ephemeron from it's container to allow its
>         collection during a subsequent garbage collection.
>
>
>
>         !! More Documentation
>
>
>
>         You can read the associated paper to understand better the
>         semantics of ephemerons:
>
>
>
>         [1]Ephemerons: A New Finalization Mechanism. Barry Hayes. OOPSLA
>         '97
>
>
>
>
>
>
>       - WARNING: to be able to use ephemerons, you need to use the
>     *latestVm* that has several fixes for making ephemerons work, and
>     you need to enable ephemerons on the image side by evaluating:
>
>
>
>                 Smalltalk supportsQueueingFinalization: true.
>
>
>
>       - With latest vm and ephemerons enabled, tests should be green,
>     otherwise they are skipped
>
>
>
>
>
>
>
>     * From the user point of view:
>
>
>
>       - The Weak registries were not yet migrated to the new
>     finalization mechanism.
>
>       - We expect nothing will change from the user point of view. Just
>     less memory leaks.
>
>
>
>     * Next steps (in order)
>
>       1) Bless the latest vm as stable
>
>       2) Enable queueing finalization by default
>
>       3) Replace Weak Registry by Ephemeron Registry.
>
>
>
>
>
>     Informed by: Guille
>
>
>
>
>
>
> -- 
> _,,,^..^,,,_
> best, Eliot
>
>
>
>
>

-------------- next part --------------
Skipped content of type multipart/related