[Vm-dev] Re: [Pharo-dev] threading in Pharo

Eliot Miranda eliot.miranda at gmail.com
Wed Mar 26 17:46:40 UTC 2014


Hi Ben,


On Wed, Mar 26, 2014 at 7:04 AM, Ben Coman <btc at openinworld.com> wrote:

>  Eliot Miranda wrote:
>
>
>
>
> On Tue, Mar 25, 2014 at 10:21 AM, Eliot Miranda <eliot.miranda at gmail.com>wrote:
>
>>  Hi Igor,
>>
>>
>>      you have a point but I disagree.  The scheduler is defined in the
>> implementation section of the blue book.  It could be more explicit, but
>> the blue book scheduler is a known and simple system.  In my threading work
>> I've made sure to preserve its semantics (cooperative within priorities,
>> preemptive across priorities, thread switch at activation of non-primitive
>> sends and backward branches).  The only serious bug I know of (that
>> preempting sends a process to the back of its run-queue) was addressed in
>> Cog and the VW VM.  We can and should write up the semantics, but they're
>> not undefined, they're simple and they do work.
>>
>
> Oops.  I lie.  There is a serious bug with Semaphore>>critical: and the
> like.  There's a suspension point after signal and before block evaluation
> that can result in deadlock if a higher priority process signals the
> semaphore.  Since the higher priority process is running the lower priority
> process never makes progress to run the ensure block or cvlear the flag or
> whatever it needs to do, and hence the system deadlocks.  This requires
> thought ;-)  So I do know of one outstanding bug.
>
> My grasp of concurrency controls hasn't been tested in 20 years, so
> naively I would say: If a lower priority process "L" holds** a semaphore
> when the higher priority process "H" signals the semaphore, temporarily
> raise the priority of the "L", or temporarily lower the priority of "H".
> Is there something simple that I missing?
>

There are a couple of obvious solutions.  One is to avoid the suspension
point somehow, another is adding priority inversion. Adding priority
inversion to the scheduler is a significant change which would increase
complexity.  You yourself have glossed over detail by saying "temporarily".
 I've glossed over a lot with "somehow".  How would these "temporarily"s
and "somehow"s be implemented?  Your design sketches are appreciated.

cc'ing vm-dev and squeak dev lists.  this is an important discussion.


> cheers -ben
>
>
>
>
>> Eliot (phone)
>>
>> On Mar 25, 2014, at 10:11 AM, Igor Stasenko <siguctua at gmail.com> wrote:
>>
>>
>>
>>
>> On 25 March 2014 17:31, Eliot Miranda <eliot.miranda at gmail.com> wrote:
>>
>>> Hi Igor,
>>>
>>>
>>>  On Tue, Mar 25, 2014 at 5:05 AM, Igor Stasenko <siguctua at gmail.com>wrote:
>>>
>>>>
>>>>
>>>>
>>>> On 24 March 2014 22:54, phil at highoctane.be <phil at highoctane.be> wrote:
>>>>
>>>>>   On Mon, Mar 24, 2014 at 8:23 PM, Alexandre Bergel <
>>>>> alexandre.bergel at me.com> wrote:
>>>>>
>>>>>> >> I am working on a memory model for expandable collection in Pharo.
>>>>>> Currently, OrderedCollection, Dictionary and other expandable collections
>>>>>> use a internal array to store their data. My new collection library recycle
>>>>>> these array instead of letting the garbage collector dispose them. I simply
>>>>>> insert the arrays in an ordered collection when an array is not necessary
>>>>>> anymore. And I remove one when I need one.
>>>>>> >
>>>>>> > Hm, is that really going to be worth the trouble?
>>>>>>
>>>>>> This technique reduces the consumption of about 15% of memory.
>>>>>>
>>>>>> >> At the end, #add:  and #remove: are performed on these polls of
>>>>>> arrays. I haven't been able to spot any problem regarding concurrency and I
>>>>>> made no effort in preventing them. I have a simple global collection and
>>>>>> each call site of "OrderedCollection new" can pick an element of my global
>>>>>> collection.
>>>>>> >>
>>>>>> >> I have the impression that I simply need to guard the access to
>>>>>> the global poll, which is basically guarding #add:  #remove: and #includes:
>>>>>> >
>>>>>> > One of the AtomicCollections might be the right things for you?
>>>>>>
>>>>>> I will have a look at it.
>>>>>>
>>>>>> >> What is funny, is that I did not care at all about multi-threading
>>>>>> and concurrency, and I have not spotted any problem so far.
>>>>>> >
>>>>>> > There isn't any 'multi-threading' like in Java, you got a much more
>>>>>> control version: cooperative on the same priority, preemptive between
>>>>>> priorities.
>>>>>> > So, I am not surprised. And well, these operations are likely not
>>>>>> to be problematic when they are racy, except when the underling data
>>>>>> structure could get into an inconsistent state itself. The overall
>>>>>> operations (adding/removing/searing) are racy on the application level
>>>>>> anyway.
>>>>>> >
>>>>>> > However, much more interesting would be to know what kind of
>>>>>> benefit do you see for such reuse?
>>>>>> > And especially, with Spur around the corner, will it still pay off
>>>>>> then? Or is it an application-specific optimization?
>>>>>>
>>>>>> I am exploring a new design of the collection library of Pharo. Not
>>>>>> all the (academic) ideas will be worth porting into the mainstream of
>>>>>> Pharo. But some of them yes.
>>>>>>
>>>>>> Thanks for all your help guys! You're great!
>>>>>>
>>>>>> Cheers,
>>>>>> Alexandre
>>>>>>
>>>>>> --
>>>>>> _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:
>>>>>> Alexandre Bergel  http://www.bergel.eu
>>>>>> ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>   An interesting method I stumbled upon which may help in
>>>>> understanding how these things do work.
>>>>>
>>>>>  BlockClosure>>valueUnpreemptively
>>>>>  "Evaluate the receiver (block), without the possibility of
>>>>> preemption by higher priority processes. Use this facility VERY sparingly!"
>>>>>  "Think about using Block>>valueUninterruptably first, and think
>>>>> about using Semaphore>>critical: before that, and think about redesigning
>>>>> your application even before that!
>>>>>  After you've done all that thinking, go right ahead and use it..."
>>>>>  | activeProcess oldPriority result semaphore |
>>>>>  activeProcess := Processor activeProcess.
>>>>>  oldPriority := activeProcess priority.
>>>>>  activeProcess priority: Processor highestPriority.
>>>>>  result := self ensure: [activeProcess priority: oldPriority].
>>>>>
>>>>>
>>>>  I would not recommend you to use this method for anything.
>>>>  This method heavily relies on how process scheduler works, and in
>>>> case of any changes, it may break everything.
>>>>  For the sake of programming, one shall never assume there is a way to
>>>> "stop the world while i busy doing something".
>>>>
>>>
>>>   Really?  Surely any system as interactive as Smalltalk can benefit
>>> from a stop-the-rest-of-the-world scheduling facility, and surely packaging
>>> it as BlockClosure>>valueUnpreemptively would be a convenient way of doing
>>> so.  Surely the right attitude for an implementor of a threading system for
>>> Smalltalk would be "Sure, I can implement that, even in a truly concurrent,
>>> multi-processor environment".  It may take some doing but it's an important
>>> facility to have.  It shouldn't be abused, but when you need it, you need
>>> it.
>>>
>>>
>> There should be hard guarantees from VM to do it. Right now there's none.
>> That's my point.
>> Like special primitive(s) for disabling interrupts/scheduling and
>> enabling it back again.
>>  Let us be realistic: the above implementation is based on insider's
>> knowledge how scheduling works, lacking any notion of contract between VM
>> and image.
>>  Right now it just based on implementation detail rather than on
>> guaranteed and well defined semantic.
>>
>>  It is no doubt, sometimes you may need such hammer to stop the world.
>> And it is no doubt (to me) that one should avoid using it unless it is
>> impossible to do otherwise.
>>
>>
>>
>>>   --
>>> best,
>>> Eliot
>>>
>>
>>
>>
>> --
>> Best regards,
>> Igor Stasenko.
>>
>>
>
>
>  --
> best,
> Eliot
>
>
>


-- 
best,
Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20140326/ed8348db/attachment-0001.htm


More information about the Vm-dev mailing list