Hello.
I hear about new Monitor implementation based on new primitives. Where to get it?
Hello,
Eliot, please, you told me you had the code and Denis is interested.
It uses 3 primitives for performance.
2016-01-07 13:24 GMT+01:00 Denis Kudriashov dionisiydk@gmail.com:
Hello.
I hear about new Monitor implementation based on new primitives. Where to get it?
Hi Denis, Hi Clément, Hi Frank,
On Thu, Jan 7, 2016 at 5:34 AM, Clément Bera bera.clement@gmail.com wrote:
Hello,
Eliot, please, you told me you had the code and Denis is interested.
It uses 3 primitives for performance.
Forgive the delay. I thought it proper to ask permission since the code was written while I was at Qwaq. I'm attaching the code in a fairly raw state, see the attached. The code is MIT, but copyright 3DICC.
It is a plugin replacement for Squeak's Mutex, and with a little ingenuity could be a replacement for Squeak's Monitor. It is quicker because it uses three new primitives to manage entering a critical section and setting the owner, exiting the critical section and releasing the owner, and testing if a critical section, entering if the section is unowned. The use of the primitives means fewer block activations and ensure: blocks in entering and exiting the critical section, and that's the actual cause of the speed-up.
You can benchmark the code as is. Here are some results on 32-bit Spur, on my 2.2GHz Core i7
{Mutex new. Monitor new. CriticalSection new} collect: [:cs| | n | n := 0. [cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. n ] bench]
{Mutex new. Monitor new. CriticalSection new} collect: [:cs| | n | n := 0. cs class name, ' -> ', [cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. n ] bench]
#( 'Mutex -> 440,000 per second. 2.27 microseconds per run.' 'Monitor -> 688,000 per second. 1.45 microseconds per run.' 'CriticalSection -> 1,110,000 per second. 900 nanoseconds per run.')
Replacement is probably trivial; rename Mutex to OldMutex, rename CriticalSection to Mutex, recompile. But there are lots of mutexes in the system and these are potentially owned. Transforming unowned ones is trivial, but transforming owned ones is, I think, impossible. But at least in my system there are no owned mutexes or monitors.
Frank (or anyone else), would you be interested in creating a replacement for Squeak's Monitor based on CriticalSection?
Here are the two business methods: *CriticalSection methods for mutual exclusion* *critical:* aBlock "Evaluate aBlock protected by the receiver." ^self primitiveEnterCriticalSection ifTrue: [aBlock value] ifFalse: [aBlock ensure: [self primitiveExitCriticalSection]]
*critical:* aBlock *ifLocked:* lockedBlock "*Answer the evaluation of aBlock protected by the receiver. If it is already in a critical* * section on behalf of some other process answer the evaluation of lockedBlock.*" ^self primitiveTestAndSetOwnershipOfCriticalSection ifNil: [lockedBlock value] ifNotNil:[:alreadyOwner| alreadyOwner ifTrue: [aBlock value] ifFalse: [aBlock ensure: [self primitiveExitCriticalSection]]]
and the primitives: *CriticalSection methods for private-primitives* *primitiveEnterCriticalSection* "Primitive. The receiver must be unowned or owned by the current process to proceed. Answer if the process is owned by the current process." <primitive: 186> self primitiveFailed "In the spirit of the following" "[owner ifNil: [owner := Processor activeProcess. ^false]. owner = Processor activeProcess ifTrue: [^true]. self addLast: Processor activeProcess. Processor activeProcess suspend] valueUnpreemptively"
*primitiveExitCriticalSection* "Primitive. Set te receiver to unowned and if any processes are waiting on the receiver then proceed the first one, indicating that the receiver is unowned." <primitive: 185> self primitiveFailed "In the spirit of the following" "[owner := nil. self isEmpty ifFalse: [process := self removeFirst. process resume]] valueUnpreemptively"
*primitiveTestAndSetOwnershipOfCriticalSection* "Primitive. Attempt to set the ownership of the receiver. If the receiver is unowned set its owningProcess to the activeProcess and answer false. If the receiver is owned by the activeProcess answer true. If the receiver is owned by some other process answer nil." <primitive: 187> self primitiveFail "In the spirit of the following" "[owner ifNil: [owningProcess := Processor activeProcess. ^false]. owner = Processor activeProcess ifTrue: [^true]. ^nil] valueUnpreemptively"
2016-01-07 13:24 GMT+01:00 Denis Kudriashov dionisiydk@gmail.com:
Hello.
I hear about new Monitor implementation based on new primitives. Where to get it?
_,,,^..^,,,_ best, Eliot
and here's a version with a better class comment
On Thu, Jan 7, 2016 at 9:12 AM, Eliot Miranda eliot.miranda@gmail.com wrote:
Hi Denis, Hi Clément, Hi Frank,
On Thu, Jan 7, 2016 at 5:34 AM, Clément Bera bera.clement@gmail.com wrote:
Hello,
Eliot, please, you told me you had the code and Denis is interested.
It uses 3 primitives for performance.
Forgive the delay. I thought it proper to ask permission since the code was written while I was at Qwaq. I'm attaching the code in a fairly raw state, see the attached. The code is MIT, but copyright 3DICC.
It is a plugin replacement for Squeak's Mutex, and with a little ingenuity could be a replacement for Squeak's Monitor. It is quicker because it uses three new primitives to manage entering a critical section and setting the owner, exiting the critical section and releasing the owner, and testing if a critical section, entering if the section is unowned. The use of the primitives means fewer block activations and ensure: blocks in entering and exiting the critical section, and that's the actual cause of the speed-up.
You can benchmark the code as is. Here are some results on 32-bit Spur, on my 2.2GHz Core i7
{Mutex new. Monitor new. CriticalSection new} collect: [:cs| | n | n := 0. [cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n
- 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1].
n ] bench]
{Mutex new. Monitor new. CriticalSection new} collect: [:cs| | n | n := 0. cs class name, ' -> ', [cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n
- 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1].
n ] bench]
#( 'Mutex -> 440,000 per second. 2.27 microseconds per run.' 'Monitor -> 688,000 per second. 1.45 microseconds per run.' 'CriticalSection -> 1,110,000 per second. 900 nanoseconds per run.')
Replacement is probably trivial; rename Mutex to OldMutex, rename CriticalSection to Mutex, recompile. But there are lots of mutexes in the system and these are potentially owned. Transforming unowned ones is trivial, but transforming owned ones is, I think, impossible. But at least in my system there are no owned mutexes or monitors.
Frank (or anyone else), would you be interested in creating a replacement for Squeak's Monitor based on CriticalSection?
Here are the two business methods: *CriticalSection methods for mutual exclusion* *critical:* aBlock "Evaluate aBlock protected by the receiver." ^self primitiveEnterCriticalSection ifTrue: [aBlock value] ifFalse: [aBlock ensure: [self primitiveExitCriticalSection]]
*critical:* aBlock *ifLocked:* lockedBlock "*Answer the evaluation of aBlock protected by the receiver. If it is already in a critical*
- section on behalf of some other process answer the evaluation of
lockedBlock.*" ^self primitiveTestAndSetOwnershipOfCriticalSection ifNil: [lockedBlock value] ifNotNil:[:alreadyOwner| alreadyOwner ifTrue: [aBlock value] ifFalse: [aBlock ensure: [self primitiveExitCriticalSection]]]
and the primitives: *CriticalSection methods for private-primitives* *primitiveEnterCriticalSection* "Primitive. The receiver must be unowned or owned by the current process to proceed. Answer if the process is owned by the current process." <primitive: 186> self primitiveFailed "In the spirit of the following" "[owner ifNil: [owner := Processor activeProcess. ^false]. owner = Processor activeProcess ifTrue: [^true]. self addLast: Processor activeProcess. Processor activeProcess suspend] valueUnpreemptively"
*primitiveExitCriticalSection* "Primitive. Set te receiver to unowned and if any processes are waiting on the receiver then proceed the first one, indicating that the receiver is unowned." <primitive: 185> self primitiveFailed "In the spirit of the following" "[owner := nil. self isEmpty ifFalse: [process := self removeFirst. process resume]] valueUnpreemptively"
*primitiveTestAndSetOwnershipOfCriticalSection* "Primitive. Attempt to set the ownership of the receiver. If the receiver is unowned set its owningProcess to the activeProcess and answer false. If the receiver is owned by the activeProcess answer true. If the receiver is owned by some other process answer nil." <primitive: 187> self primitiveFail "In the spirit of the following" "[owner ifNil: [owningProcess := Processor activeProcess. ^false]. owner = Processor activeProcess ifTrue: [^true]. ^nil] valueUnpreemptively"
2016-01-07 13:24 GMT+01:00 Denis Kudriashov dionisiydk@gmail.com:
Hello.
I hear about new Monitor implementation based on new primitives. Where to get it?
_,,,^..^,,,_ best, Eliot
On Fri, Jan 8, 2016 at 1:20 AM, Eliot Miranda eliot.miranda@gmail.com wrote:
and here's a version with a better class comment
On Thu, Jan 7, 2016 at 9:12 AM, Eliot Miranda eliot.miranda@gmail.com wrote:
Hi Denis, Hi Clément, Hi Frank,
On Thu, Jan 7, 2016 at 5:34 AM, Clément Bera bera.clement@gmail.com wrote:
Hello,
Eliot, please, you told me you had the code and Denis is interested.
It uses 3 primitives for performance.
Forgive the delay. I thought it proper to ask permission since the code was written while I was at Qwaq. I'm attaching the code in a fairly raw state, see the attached. The code is MIT, but copyright 3DICC.
It is a plugin replacement for Squeak's Mutex, and with a little ingenuity could be a replacement for Squeak's Monitor. It is quicker because it uses three new primitives to manage entering a critical section and setting the owner, exiting the critical section and releasing the owner, and testing if a critical section, entering if the section is unowned. The use of the primitives means fewer block activations and ensure: blocks in entering and exiting the critical section, and that's the actual cause of the speed-up.
You can benchmark the code as is. Here are some results on 32-bit Spur, on my 2.2GHz Core i7
{Mutex new. Monitor new. CriticalSection new} collect: [:cs| | n | n := 0. [cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. n ] bench]
{Mutex new. Monitor new. CriticalSection new} collect: [:cs| | n | n := 0. cs class name, ' -> ', [cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. n ] bench]
#( 'Mutex -> 440,000 per second. 2.27 microseconds per run.' 'Monitor -> 688,000 per second. 1.45 microseconds per run.' 'CriticalSection -> 1,110,000 per second. 900 nanoseconds per run.')
This is great Eliot. Thank you and 3DICC. After loading the changeset into Pharo-50515 (32 bit Spur) I get the following results on my laptop i5-2520M @ 2.50GHz
#('Mutex -> 254,047 per second' 'Monitor -> 450,442 per second' 'CriticalSection -> 683,393 per second')
In a fresh Image "Mutex allInstances basicInspect" lists just two mutexes... 1. NetNameResolver-->ResolverMutex 2. ThreadSafeTranscript-->accessSemaphore
cheers -ben
Replacement is probably trivial; rename Mutex to OldMutex, rename CriticalSection to Mutex, recompile. But there are lots of mutexes in the system and these are potentially owned. Transforming unowned ones is trivial, but transforming owned ones is, I think, impossible. But at least in my system there are no owned mutexes or monitors.
Frank (or anyone else), would you be interested in creating a replacement for Squeak's Monitor based on CriticalSection?
Here are the two business methods: CriticalSection methods for mutual exclusion critical: aBlock "Evaluate aBlock protected by the receiver." ^self primitiveEnterCriticalSection ifTrue: [aBlock value] ifFalse: [aBlock ensure: [self primitiveExitCriticalSection]]
critical: aBlock ifLocked: lockedBlock "Answer the evaluation of aBlock protected by the receiver. If it is already in a critical section on behalf of some other process answer the evaluation of lockedBlock." ^self primitiveTestAndSetOwnershipOfCriticalSection ifNil: [lockedBlock value] ifNotNil:[:alreadyOwner| alreadyOwner ifTrue: [aBlock value] ifFalse: [aBlock ensure: [self primitiveExitCriticalSection]]]
and the primitives: CriticalSection methods for private-primitives primitiveEnterCriticalSection "Primitive. The receiver must be unowned or owned by the current process to proceed. Answer if the process is owned by the current process." <primitive: 186> self primitiveFailed "In the spirit of the following" "[owner ifNil: [owner := Processor activeProcess. ^false]. owner = Processor activeProcess ifTrue: [^true]. self addLast: Processor activeProcess. Processor activeProcess suspend] valueUnpreemptively"
primitiveExitCriticalSection "Primitive. Set te receiver to unowned and if any processes are waiting on the receiver then proceed the first one, indicating that the receiver is unowned." <primitive: 185> self primitiveFailed "In the spirit of the following" "[owner := nil. self isEmpty ifFalse: [process := self removeFirst. process resume]] valueUnpreemptively"
primitiveTestAndSetOwnershipOfCriticalSection "Primitive. Attempt to set the ownership of the receiver. If the receiver is unowned set its owningProcess to the activeProcess and answer false. If the receiver is owned by the activeProcess answer true. If the receiver is owned by some other process answer nil." <primitive: 187> self primitiveFail "In the spirit of the following" "[owner ifNil: [owningProcess := Processor activeProcess. ^false]. owner = Processor activeProcess ifTrue: [^true]. ^nil] valueUnpreemptively"
2016-01-07 13:24 GMT+01:00 Denis Kudriashov dionisiydk@gmail.com:
Hello.
I hear about new Monitor implementation based on new primitives. Where to get it?
_,,,^..^,,,_ best, Eliot
-- _,,,^..^,,,_ best, Eliot
Hi Ben,
On Thu, Jan 7, 2016 at 10:39 AM, Ben Coman btc@openinworld.com wrote:
On Fri, Jan 8, 2016 at 1:20 AM, Eliot Miranda eliot.miranda@gmail.com wrote:
and here's a version with a better class comment
On Thu, Jan 7, 2016 at 9:12 AM, Eliot Miranda eliot.miranda@gmail.com
wrote:
Hi Denis, Hi Clément, Hi Frank,
On Thu, Jan 7, 2016 at 5:34 AM, Clément Bera bera.clement@gmail.com
wrote:
Hello,
Eliot, please, you told me you had the code and Denis is interested.
It uses 3 primitives for performance.
Forgive the delay. I thought it proper to ask permission since the
code was written while I was at Qwaq. I'm attaching the code in a fairly raw state, see the attached. The code is MIT, but copyright 3DICC.
It is a plugin replacement for Squeak's Mutex, and with a little
ingenuity could be a replacement for Squeak's Monitor. It is quicker because it uses three new primitives to manage entering a critical section and setting the owner, exiting the critical section and releasing the owner, and testing if a critical section, entering if the section is unowned. The use of the primitives means fewer block activations and ensure: blocks in entering and exiting the critical section, and that's the actual cause of the speed-up.
You can benchmark the code as is. Here are some results on 32-bit
Spur, on my 2.2GHz Core i7
{Mutex new. Monitor new. CriticalSection new} collect: [:cs| | n | n := 0. [cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n
:= n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1].
cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n
:= n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1].
n ] bench]
{Mutex new. Monitor new. CriticalSection new} collect: [:cs| | n | n := 0. cs class name, ' -> ', [cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n
:= n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1].
cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n
:= n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1].
n ] bench]
#( 'Mutex -> 440,000 per second. 2.27 microseconds per run.' 'Monitor -> 688,000 per second. 1.45 microseconds per run.' 'CriticalSection -> 1,110,000 per second. 900 nanoseconds per run.')
This is great Eliot. Thank you and 3DICC. After loading the changeset into Pharo-50515 (32 bit Spur) I get the following results on my laptop i5-2520M @ 2.50GHz
#('Mutex -> 254,047 per second' 'Monitor -> 450,442 per second' 'CriticalSection -> 683,393 per second')
In a fresh Image "Mutex allInstances basicInspect" lists just two mutexes...
- NetNameResolver-->ResolverMutex
- ThreadSafeTranscript-->accessSemaphore
I hate myself for getting distracted but I'm finding this is un. One can migrate to the new representation using normal Monticello loads by
In the first version redefine Mutex and Monitor to subclass LinkedList and have their owner/ownerProcess inst var first (actually third after firstLink & lastLink), and add the primitives.
In the next version check that all Mutex and Monitor instanes are unowned and then redefine to discard excess inst vars
Let me test this before committing, and see that all tests are ok.
cheers -ben
Replacement is probably trivial; rename Mutex to OldMutex, rename
CriticalSection to Mutex, recompile. But there are lots of mutexes in the system and these are potentially owned. Transforming unowned ones is trivial, but transforming owned ones is, I think, impossible. But at least in my system there are no owned mutexes or monitors.
Frank (or anyone else), would you be interested in creating a
replacement for Squeak's Monitor based on CriticalSection?
Here are the two business methods: CriticalSection methods for mutual exclusion critical: aBlock "Evaluate aBlock protected by the receiver." ^self primitiveEnterCriticalSection ifTrue: [aBlock value] ifFalse: [aBlock ensure: [self primitiveExitCriticalSection]]
critical: aBlock ifLocked: lockedBlock "Answer the evaluation of aBlock protected by the receiver. If it is
already in a critical
section on behalf of some other process answer the evaluation of
lockedBlock."
^self primitiveTestAndSetOwnershipOfCriticalSection ifNil: [lockedBlock value] ifNotNil:[:alreadyOwner| alreadyOwner ifTrue: [aBlock value] ifFalse: [aBlock ensure: [self primitiveExitCriticalSection]]]
and the primitives: CriticalSection methods for private-primitives primitiveEnterCriticalSection "Primitive. The receiver must be unowned or owned by the current
process to proceed.
Answer if the process is owned by the current process." <primitive: 186> self primitiveFailed "In the spirit of the following" "[owner ifNil: [owner := Processor activeProcess. ^false]. owner = Processor activeProcess ifTrue: [^true]. self addLast: Processor activeProcess. Processor activeProcess suspend] valueUnpreemptively"
primitiveExitCriticalSection "Primitive. Set te receiver to unowned and if any processes are waiting
on
the receiver then proceed the first one, indicating that the receiver
is unowned."
<primitive: 185> self primitiveFailed "In the spirit of the following" "[owner := nil. self isEmpty ifFalse: [process := self removeFirst. process resume]] valueUnpreemptively"
primitiveTestAndSetOwnershipOfCriticalSection "Primitive. Attempt to set the ownership of the receiver. If the receiver is unowned set its owningProcess to the activeProcess and answer false. If the receiver is owned by the activeProcess answer true. If the receiver is owned by some other process answer nil." <primitive: 187> self primitiveFail "In the spirit of the following" "[owner ifNil: [owningProcess := Processor activeProcess. ^false]. owner = Processor activeProcess ifTrue: [^true]. ^nil] valueUnpreemptively"
2016-01-07 13:24 GMT+01:00 Denis Kudriashov dionisiydk@gmail.com:
Hello.
I hear about new Monitor implementation based on new primitives. Where to get it?
_,,,^..^,,,_ best, Eliot
-- _,,,^..^,,,_ best, Eliot
On Fri, Jan 8, 2016 at 2:51 AM, Eliot Miranda eliot.miranda@gmail.com wrote:
Hi Ben,
On Thu, Jan 7, 2016 at 10:39 AM, Ben Coman btc@openinworld.com wrote:
On Fri, Jan 8, 2016 at 1:20 AM, Eliot Miranda eliot.miranda@gmail.com wrote:
and here's a version with a better class comment
On Thu, Jan 7, 2016 at 9:12 AM, Eliot Miranda eliot.miranda@gmail.com wrote:
Hi Denis, Hi Clément, Hi Frank,
On Thu, Jan 7, 2016 at 5:34 AM, Clément Bera bera.clement@gmail.com wrote:
Hello,
Eliot, please, you told me you had the code and Denis is interested.
It uses 3 primitives for performance.
Forgive the delay. I thought it proper to ask permission since the code was written while I was at Qwaq. I'm attaching the code in a fairly raw state, see the attached. The code is MIT, but copyright 3DICC.
It is a plugin replacement for Squeak's Mutex, and with a little ingenuity could be a replacement for Squeak's Monitor. It is quicker because it uses three new primitives to manage entering a critical section and setting the owner, exiting the critical section and releasing the owner, and testing if a critical section, entering if the section is unowned. The use of the primitives means fewer block activations and ensure: blocks in entering and exiting the critical section, and that's the actual cause of the speed-up.
You can benchmark the code as is. Here are some results on 32-bit Spur, on my 2.2GHz Core i7
{Mutex new. Monitor new. CriticalSection new} collect: [:cs| | n | n := 0. [cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. n ] bench]
{Mutex new. Monitor new. CriticalSection new} collect: [:cs| | n | n := 0. cs class name, ' -> ', [cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. n ] bench]
#( 'Mutex -> 440,000 per second. 2.27 microseconds per run.' 'Monitor -> 688,000 per second. 1.45 microseconds per run.' 'CriticalSection -> 1,110,000 per second. 900 nanoseconds per run.')
This is great Eliot. Thank you and 3DICC. After loading the changeset into Pharo-50515 (32 bit Spur) I get the following results on my laptop i5-2520M @ 2.50GHz
#('Mutex -> 254,047 per second' 'Monitor -> 450,442 per second' 'CriticalSection -> 683,393 per second')
In a fresh Image "Mutex allInstances basicInspect" lists just two mutexes...
- NetNameResolver-->ResolverMutex
- ThreadSafeTranscript-->accessSemaphore
I hate myself for getting distracted but I'm finding this is un. One can migrate to the new representation using normal Monticello loads by
In the first version redefine Mutex and Monitor to subclass LinkedList and have their owner/ownerProcess inst var first (actually third after firstLink & lastLink), and add the primitives.
In the next version check that all Mutex and Monitor instanes are unowned and then redefine to discard excess inst vars
Let me test this before committing, and see that all tests are ok.
Should Mutex and Monitor both directly subclass LinkedList and duplicate the primitives in each?
Or should they both subclass CriticalSection which subclasses LinkedList so the primitives are only defined once?
What effect would using the primitives from the superclass have on performance? If any, I'd vote to optimise for duplication rather than "nice" design, but our comments should document this.
cheers -ben
Hi Ben,
On Thu, Jan 7, 2016 at 4:40 PM, Ben Coman btc@openinworld.com wrote:
On Fri, Jan 8, 2016 at 2:51 AM, Eliot Miranda eliot.miranda@gmail.com wrote:
Hi Ben,
On Thu, Jan 7, 2016 at 10:39 AM, Ben Coman btc@openinworld.com wrote:
On Fri, Jan 8, 2016 at 1:20 AM, Eliot Miranda eliot.miranda@gmail.com wrote:
and here's a version with a better class comment
On Thu, Jan 7, 2016 at 9:12 AM, Eliot Miranda <
eliot.miranda@gmail.com>
wrote:
Hi Denis, Hi Clément, Hi Frank,
On Thu, Jan 7, 2016 at 5:34 AM, Clément Bera <bera.clement@gmail.com
wrote:
Hello,
Eliot, please, you told me you had the code and Denis is interested.
It uses 3 primitives for performance.
Forgive the delay. I thought it proper to ask permission since the code was written while I was at Qwaq. I'm attaching the code in a
fairly raw
state, see the attached. The code is MIT, but copyright 3DICC.
It is a plugin replacement for Squeak's Mutex, and with a little ingenuity could be a replacement for Squeak's Monitor. It is quicker because it uses three new primitives to manage entering a critical
section
and setting the owner, exiting the critical section and releasing
the owner,
and testing if a critical section, entering if the section is
unowned. The
use of the primitives means fewer block activations and ensure:
blocks in
entering and exiting the critical section, and that's the actual
cause of
the speed-up.
You can benchmark the code as is. Here are some results on 32-bit Spur, on my 2.2GHz Core i7
{Mutex new. Monitor new. CriticalSection new} collect: [:cs| | n | n := 0. [cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical:
[n
:= n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. n ] bench]
{Mutex new. Monitor new. CriticalSection new} collect: [:cs| | n | n := 0. cs class name, ' -> ', [cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical:
[n
:= n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. n ] bench]
#( 'Mutex -> 440,000 per second. 2.27 microseconds per run.' 'Monitor -> 688,000 per second. 1.45 microseconds per run.' 'CriticalSection -> 1,110,000 per second. 900 nanoseconds per run.')
This is great Eliot. Thank you and 3DICC. After loading the changeset into Pharo-50515 (32 bit Spur) I get the following results on my laptop i5-2520M @ 2.50GHz
#('Mutex -> 254,047 per second' 'Monitor -> 450,442 per second' 'CriticalSection -> 683,393 per second')
In a fresh Image "Mutex allInstances basicInspect" lists just two mutexes...
- NetNameResolver-->ResolverMutex
- ThreadSafeTranscript-->accessSemaphore
I hate myself for getting distracted but I'm finding this is un. One can migrate to the new representation using normal Monticello loads by
In the first version redefine Mutex and Monitor to subclass LinkedList
and
have their owner/ownerProcess inst var first (actually third after
firstLink
& lastLink), and add the primitives.
In the next version check that all Mutex and Monitor instanes are unowned and then redefine to discard excess inst vars
Let me test this before committing, and see that all tests are ok.
Should Mutex and Monitor both directly subclass LinkedList and duplicate the primitives in each?
Or should they both subclass CriticalSection which subclasses LinkedList so the primitives are only defined once?
That's a good idea. Feel free to change the code, but test that the Monticello load handles this case properly first :-). Actually, given that the default state of all the Mutex and Monitor instances in the image is unowned (owner process is nil) then it'll just work anyway. If we do that, we must make sure to include the ICC copyright in CriticalSection's class comment, and can eliminate it from the primitives.
What effect would using the primitives from the superclass have on
performance? If any, I'd vote to optimise for duplication rather than "nice" design, but our comments should document this.
Likely in the noise. The inline cacheing machinery in the VM is far cheaper than the real overheads here which are in block creation, process switch, interpreter primitive invocation.
cheers -ben
Now I implement pragma approach to set up local variable during termination process. So I can write such methods:
critical: aBlock "Evaluate aBlock protected by the receiver." | lockAcquired | <lockAt: #lock trackStateAt: 1> lockAcquired := false. ^[ lockAcquired := true. lockAcquired := lock wait. aBlock value ] ensure: [lockAcquired ifTrue: [lock signal]].
And Process>>terminate detects waiting on such methods and push false to variable lockAcquired (which is 1 temp here). This approach allow me to use multiple "locks" (semaphores or whatever) in single method which I need for ReadWriteLock.
2016-01-08 18:31 GMT+01:00 Eliot Miranda eliot.miranda@gmail.com:
Hi Ben,
On Thu, Jan 7, 2016 at 4:40 PM, Ben Coman btc@openinworld.com wrote:
On Fri, Jan 8, 2016 at 2:51 AM, Eliot Miranda eliot.miranda@gmail.com wrote:
Hi Ben,
On Thu, Jan 7, 2016 at 10:39 AM, Ben Coman btc@openinworld.com wrote:
On Fri, Jan 8, 2016 at 1:20 AM, Eliot Miranda <eliot.miranda@gmail.com
wrote:
and here's a version with a better class comment
On Thu, Jan 7, 2016 at 9:12 AM, Eliot Miranda <
eliot.miranda@gmail.com>
wrote:
Hi Denis, Hi Clément, Hi Frank,
On Thu, Jan 7, 2016 at 5:34 AM, Clément Bera <
bera.clement@gmail.com>
wrote: > > Hello, > > Eliot, please, you told me you had the code and Denis is
interested.
> > It uses 3 primitives for performance.
Forgive the delay. I thought it proper to ask permission since the code was written while I was at Qwaq. I'm attaching the code in a
fairly raw
state, see the attached. The code is MIT, but copyright 3DICC.
It is a plugin replacement for Squeak's Mutex, and with a little ingenuity could be a replacement for Squeak's Monitor. It is
quicker
because it uses three new primitives to manage entering a critical
section
and setting the owner, exiting the critical section and releasing
the owner,
and testing if a critical section, entering if the section is
unowned. The
use of the primitives means fewer block activations and ensure:
blocks in
entering and exiting the critical section, and that's the actual
cause of
the speed-up.
You can benchmark the code as is. Here are some results on 32-bit Spur, on my 2.2GHz Core i7
{Mutex new. Monitor new. CriticalSection new} collect: [:cs| | n | n := 0. [cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical:
[n
:= n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical:
[n
:= n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. n ] bench]
{Mutex new. Monitor new. CriticalSection new} collect: [:cs| | n | n := 0. cs class name, ' -> ', [cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical:
[n
:= n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical:
[n
:= n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. n ] bench]
#( 'Mutex -> 440,000 per second. 2.27 microseconds per run.' 'Monitor -> 688,000 per second. 1.45 microseconds per run.' 'CriticalSection -> 1,110,000 per second. 900 nanoseconds per run.')
This is great Eliot. Thank you and 3DICC. After loading the changeset into Pharo-50515 (32 bit Spur) I get the following results on my laptop i5-2520M @ 2.50GHz
#('Mutex -> 254,047 per second' 'Monitor -> 450,442 per second' 'CriticalSection -> 683,393 per second')
In a fresh Image "Mutex allInstances basicInspect" lists just two mutexes...
- NetNameResolver-->ResolverMutex
- ThreadSafeTranscript-->accessSemaphore
I hate myself for getting distracted but I'm finding this is un. One
can
migrate to the new representation using normal Monticello loads by
In the first version redefine Mutex and Monitor to subclass LinkedList
and
have their owner/ownerProcess inst var first (actually third after
firstLink
& lastLink), and add the primitives.
In the next version check that all Mutex and Monitor instanes are
unowned
and then redefine to discard excess inst vars
Let me test this before committing, and see that all tests are ok.
Should Mutex and Monitor both directly subclass LinkedList and duplicate the primitives in each?
Or should they both subclass CriticalSection which subclasses LinkedList so the primitives are only defined once?
That's a good idea. Feel free to change the code, but test that the Monticello load handles this case properly first :-). Actually, given that the default state of all the Mutex and Monitor instances in the image is unowned (owner process is nil) then it'll just work anyway. If we do that, we must make sure to include the ICC copyright in CriticalSection's class comment, and can eliminate it from the primitives.
What effect would using the primitives from the superclass have on
performance? If any, I'd vote to optimise for duplication rather than "nice" design, but our comments should document this.
Likely in the noise. The inline cacheing machinery in the VM is far cheaper than the real overheads here which are in block creation, process switch, interpreter primitive invocation.
cheers -ben
-- _,,,^..^,,,_ best, Eliot
I have a (stupid) question. Is the code running without the primitives? Are the code below the primitives correct? I asked that because we can have 100 eyes and brains on the smalltalk level and far less on the VM primitive level.
Stef
Hi Ben,
On Thu, Jan 7, 2016 at 10:39 AM, Ben Coman btc@openinworld.comwrote:
On Fri, Jan 8, 2016 at 1:20 AM, Eliot Miranda eliot.miranda@gmail.com wrote:
and here's a version with a better class comment
On Thu, Jan 7, 2016 at 9:12 AM, Eliot Miranda eliot.miranda@gmail.com wrote:
Hi Denis, Hi Clément, Hi Frank,
On Thu, Jan 7, 2016 at 5:34 AM, Clément Bera bera.clement@gmail.com wrote:
Hello,
Eliot, please, you told me you had the code and Denis is interested.
It uses 3 primitives for performance.
Forgive the delay. I thought it proper to ask permission since the code was written while I was at Qwaq. I'm attaching the code in a fairly raw state, see the attached. The code is MIT, but copyright 3DICC.
It is a plugin replacement for Squeak's Mutex, and with a little ingenuity could be a replacement for Squeak's Monitor. It is quicker because it uses three new primitives to manage entering a critical section and setting the owner, exiting the critical section and releasing the owner, and testing if a critical section, entering if the section is unowned. The use of the primitives means fewer block activations and ensure: blocks in entering and exiting the critical section, and that's the actual cause of the speed-up.
You can benchmark the code as is. Here are some results on 32-bit Spur, on my 2.2GHz Core i7
{Mutex new. Monitor new. CriticalSection new} collect: [:cs| | n | n := 0. [cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. n ] bench]
{Mutex new. Monitor new. CriticalSection new} collect: [:cs| | n | n := 0. cs class name, ' -> ', [cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. n ] bench]
#( 'Mutex -> 440,000 per second. 2.27 microseconds per run.' 'Monitor -> 688,000 per second. 1.45 microseconds per run.' 'CriticalSection -> 1,110,000 per second. 900 nanoseconds per run.')
This is great Eliot. Thank you and 3DICC. After loading the changeset into Pharo-50515 (32 bit Spur) I get the following results on my laptop i5-2520M @ 2.50GHz
#('Mutex -> 254,047 per second' 'Monitor -> 450,442 per second' 'CriticalSection -> 683,393 per second')
In a fresh Image "Mutex allInstances basicInspect" lists just two mutexes...
- NetNameResolver-->ResolverMutex
- ThreadSafeTranscript-->accessSemaphore
I hate myself for getting distracted but I'm finding this is un. One can migrate to the new representation using normal Monticello loads by
In the first version redefine Mutex and Monitor to subclass LinkedList and have their owner/ownerProcess inst var first (actually third after firstLink & lastLink), and add the primitives.
In the next version check that all Mutex and Monitor instanes are unowned and then redefine to discard excess inst vars
Let me test this before committing, and see that all tests are ok.
cheers -ben
Replacement is probably trivial; rename Mutex to OldMutex, rename CriticalSection to Mutex, recompile. But there are lots of mutexes in the system and these are potentially owned. Transforming unowned ones is trivial, but transforming owned ones is, I think, impossible. But at least in my system there are no owned mutexes or monitors.
Frank (or anyone else), would you be interested in creating a replacement for Squeak's Monitor based on CriticalSection?
Here are the two business methods: CriticalSection methods for mutual exclusion critical: aBlock "Evaluate aBlock protected by the receiver." ^self primitiveEnterCriticalSection ifTrue: [aBlock value] ifFalse: [aBlock ensure: [self primitiveExitCriticalSection]]
critical: aBlock ifLocked: lockedBlock "Answer the evaluation of aBlock protected by the receiver. If it is already in a critical section on behalf of some other process answer the evaluation of lockedBlock." ^self primitiveTestAndSetOwnershipOfCriticalSection ifNil: [lockedBlock value] ifNotNil:[:alreadyOwner| alreadyOwner ifTrue: [aBlock value] ifFalse: [aBlock ensure: [self primitiveExitCriticalSection]]]
and the primitives: CriticalSection methods for private-primitives primitiveEnterCriticalSection "Primitive. The receiver must be unowned or owned by the current process to proceed. Answer if the process is owned by the current process." <primitive: 186> self primitiveFailed "In the spirit of the following" "[owner ifNil: [owner := Processor activeProcess. ^false]. owner = Processor activeProcess ifTrue: [^true]. self addLast: Processor activeProcess. Processor activeProcess suspend] valueUnpreemptively"
primitiveExitCriticalSection "Primitive. Set te receiver to unowned and if any processes are waiting on the receiver then proceed the first one, indicating that the receiver is unowned." <primitive: 185> self primitiveFailed "In the spirit of the following" "[owner := nil. self isEmpty ifFalse: [process := self removeFirst. process resume]] valueUnpreemptively"
primitiveTestAndSetOwnershipOfCriticalSection "Primitive. Attempt to set the ownership of the receiver. If the receiver is unowned set its owningProcess to the activeProcess and answer false. If the receiver is owned by the activeProcess answer true. If the receiver is owned by some other process answer nil." <primitive: 187> self primitiveFail "In the spirit of the following" "[owner ifNil: [owningProcess := Processor activeProcess. ^false]. owner = Processor activeProcess ifTrue: [^true]. ^nil] valueUnpreemptively"
2016-01-07 13:24 GMT+01:00 Denis Kudriashov dionisiydk@gmail.com:
Hello.
I hear about new Monitor implementation based on new primitives. Where to get it?
_,,,^..^,,,_ best, Eliot
-- _,,,^..^,,,_ best, Eliot
-- _,,,^..^,,,_ best, Eliot
On Fri, Jan 8, 2016 at 5:42 PM, stephane ducasse stephane.ducasse@gmail.com wrote:
I have a (stupid) question. Is the code running without the primitives? Are the code below the primitives correct? I asked that because we can have 100 eyes and brains on the smalltalk level and far less on the VM primitive level.
Because: 1. Concurrency bugs can be subtle and the *exact* conditions can be hard to reproduce for debugging. For example, the solution to a couple of problems with Delay [1] [2] were solved by moving away from Semaphore>>critical: to use signalling.
2. The in-image atomicity of determining whether a signal was actually consumed or not during process suspension/termination is awkward. Its seems hard to *really* know for sure it right (but I haven't looked in depth into Denis' latest proposals.)
3. The existing in-image implementation of Semaphore>>critical messes around in Process>>terminate in a *special* way that is not easy for those 100 eyes to understand. For example, I personally am not comfortable with understanding how the special Semaphore handling in Process>>terminate works, but I can easily follow how primitiveEnterCriticalSection just looking at the code [3].
4. Its faster (for me, the least argument given that.
btw, Looking at the following... CriticalSection>>critical: aBlock ^self primitiveEnterCriticalSection ifTrue: [aBlock value] ifFalse: [aBlock ensure: [self primitiveExitCriticalSection]]
without intimate VM knowledge but seeing Eliot recently say "invocations of methods with primitives [...] are not suspension points, unless their primitives fail" -- I'm paranoid that since #ensure: primitive 198 always fail, there might be some some small window for a race where a process might be terminated before #primitiveExitCriticalSection can be executed. I'll take a refutation of this to improve the method comment.
The following instils more confidence...
CriticalSection2>>critical: aBlock | reEntered | reEntered := false. [ reEntered := self primitiveEnterCriticalSection. aBlock value ] ensure: [ reEntered ifFalse: [ self primitiveExitCriticalSection] ]
but performance is reduced... "#('Mutex -> 282,511 per second' 'Monitor -> 479,217 per second' 'CriticalSection -> 729,049 per second' 'CriticalSection2 -> 571,631 per second')"
Then again, any termination from within a critical section is anyhow problematic since potentially data structures are left in a indeterminate state.
[1] http://forum.world.st/Super-fast-delay-td4787257.html [2] https://pharo.fogbugz.com/default.asp?13755 [3] https://git.io/vuDnA
cheers -ben
Stef
Hi Ben,
On Thu, Jan 7, 2016 at 10:39 AM, Ben Coman btc@openinworld.comwrote:
On Fri, Jan 8, 2016 at 1:20 AM, Eliot Miranda eliot.miranda@gmail.com wrote:
and here's a version with a better class comment
On Thu, Jan 7, 2016 at 9:12 AM, Eliot Miranda eliot.miranda@gmail.com wrote:
Hi Denis, Hi Clément, Hi Frank,
On Thu, Jan 7, 2016 at 5:34 AM, Clément Bera bera.clement@gmail.com wrote:
Hello,
Eliot, please, you told me you had the code and Denis is interested.
It uses 3 primitives for performance.
Forgive the delay. I thought it proper to ask permission since the code was written while I was at Qwaq. I'm attaching the code in a fairly raw state, see the attached. The code is MIT, but copyright 3DICC.
It is a plugin replacement for Squeak's Mutex, and with a little ingenuity could be a replacement for Squeak's Monitor. It is quicker because it uses three new primitives to manage entering a critical section and setting the owner, exiting the critical section and releasing the owner, and testing if a critical section, entering if the section is unowned. The use of the primitives means fewer block activations and ensure: blocks in entering and exiting the critical section, and that's the actual cause of the speed-up.
You can benchmark the code as is. Here are some results on 32-bit Spur, on my 2.2GHz Core i7
{Mutex new. Monitor new. CriticalSection new} collect: [:cs| | n | n := 0. [cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. n ] bench]
{Mutex new. Monitor new. CriticalSection new} collect: [:cs| | n | n := 0. cs class name, ' -> ', [cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. n ] bench]
#( 'Mutex -> 440,000 per second. 2.27 microseconds per run.' 'Monitor -> 688,000 per second. 1.45 microseconds per run.' 'CriticalSection -> 1,110,000 per second. 900 nanoseconds per run.')
This is great Eliot. Thank you and 3DICC. After loading the changeset into Pharo-50515 (32 bit Spur) I get the following results on my laptop i5-2520M @ 2.50GHz
#('Mutex -> 254,047 per second' 'Monitor -> 450,442 per second' 'CriticalSection -> 683,393 per second')
In a fresh Image "Mutex allInstances basicInspect" lists just two mutexes...
- NetNameResolver-->ResolverMutex
- ThreadSafeTranscript-->accessSemaphore
I hate myself for getting distracted but I'm finding this is un. One can migrate to the new representation using normal Monticello loads by
In the first version redefine Mutex and Monitor to subclass LinkedList and have their owner/ownerProcess inst var first (actually third after firstLink & lastLink), and add the primitives.
In the next version check that all Mutex and Monitor instanes are unowned and then redefine to discard excess inst vars
Let me test this before committing, and see that all tests are ok.
cheers -ben
Replacement is probably trivial; rename Mutex to OldMutex, rename CriticalSection to Mutex, recompile. But there are lots of mutexes in the system and these are potentially owned. Transforming unowned ones is trivial, but transforming owned ones is, I think, impossible. But at least in my system there are no owned mutexes or monitors.
Frank (or anyone else), would you be interested in creating a replacement for Squeak's Monitor based on CriticalSection?
Here are the two business methods: CriticalSection methods for mutual exclusion critical: aBlock "Evaluate aBlock protected by the receiver." ^self primitiveEnterCriticalSection ifTrue: [aBlock value] ifFalse: [aBlock ensure: [self primitiveExitCriticalSection]]
critical: aBlock ifLocked: lockedBlock "Answer the evaluation of aBlock protected by the receiver. If it is already in a critical section on behalf of some other process answer the evaluation of lockedBlock." ^self primitiveTestAndSetOwnershipOfCriticalSection ifNil: [lockedBlock value] ifNotNil:[:alreadyOwner| alreadyOwner ifTrue: [aBlock value] ifFalse: [aBlock ensure: [self primitiveExitCriticalSection]]]
and the primitives: CriticalSection methods for private-primitives primitiveEnterCriticalSection "Primitive. The receiver must be unowned or owned by the current process to proceed. Answer if the process is owned by the current process." <primitive: 186> self primitiveFailed "In the spirit of the following" "[owner ifNil: [owner := Processor activeProcess. ^false]. owner = Processor activeProcess ifTrue: [^true]. self addLast: Processor activeProcess. Processor activeProcess suspend] valueUnpreemptively"
primitiveExitCriticalSection "Primitive. Set te receiver to unowned and if any processes are waiting on the receiver then proceed the first one, indicating that the receiver is unowned." <primitive: 185> self primitiveFailed "In the spirit of the following" "[owner := nil. self isEmpty ifFalse: [process := self removeFirst. process resume]] valueUnpreemptively"
primitiveTestAndSetOwnershipOfCriticalSection "Primitive. Attempt to set the ownership of the receiver. If the receiver is unowned set its owningProcess to the activeProcess and answer false. If the receiver is owned by the activeProcess answer true. If the receiver is owned by some other process answer nil." <primitive: 187> self primitiveFail "In the spirit of the following" "[owner ifNil: [owningProcess := Processor activeProcess. ^false]. owner = Processor activeProcess ifTrue: [^true]. ^nil] valueUnpreemptively"
2016-01-07 13:24 GMT+01:00 Denis Kudriashov dionisiydk@gmail.com:
Hello.
I hear about new Monitor implementation based on new primitives. Where to get it?
2016-01-08 14:39 GMT+01:00 Ben Coman btc@openinworld.com:
The in-image atomicity of determining whether a signal was actually consumed or not during process suspension/termination is awkward. Its seems hard to *really* know for sure it right (but I haven't looked in depth into Denis' latest proposals.)
I realize that my proposals was wrong because it is based on extra message send which can be interrupted. So I think about different solution. Current solution which silently skip ensure block not suitable for case of ReadWriteLock where there are two semaphores inside same critical method
On Fri, Jan 8, 2016 at 9:39 PM, Ben Coman btc@openinworld.com wrote:
On Fri, Jan 8, 2016 at 5:42 PM, stephane ducasse stephane.ducasse@gmail.com wrote:
I have a (stupid) question. Is the code running without the primitives? Are the code below the primitives correct? I asked that because we can have 100 eyes and brains on the smalltalk level and far less on the VM primitive level.
Because:
- Concurrency bugs can be subtle and the *exact* conditions can be
hard to reproduce for debugging. For example, the solution to a couple of problems with Delay [1] [2] were solved by moving away from Semaphore>>critical: to use signalling.
- The in-image atomicity of determining whether a signal was actually
consumed or not during process suspension/termination is awkward. Its seems hard to *really* know for sure it right (but I haven't looked in depth into Denis' latest proposals.)
- The existing in-image implementation of Semaphore>>critical messes
around in Process>>terminate in a *special* way that is not easy for those 100 eyes to understand. For example, I personally am not comfortable with understanding how the special Semaphore handling in Process>>terminate works, but I can easily follow how primitiveEnterCriticalSection just looking at the code [3].
Points 2 & 3 might possibly be addressed by having new primitiveWaitReturned *always* return true, so if the process is terminated while waiting, the assignment to signalConsumed doesn't occur...
critical: mutuallyExcludedBlock signalConsumed := false. [ signalConsumed := self primitiveWaitReturned. blockValue := mutuallyExcludedBlock value ] ensure: [ signalConsumed ifTrue: [self signal] ]. ^blockValue
where primitiveWait (https://git.io/vuDjd) is copied and (just guessing) the marked line added...
primitiveWaitReturned | sema excessSignals activeProc inInterpreter | sema := self stackTop. "rcvr" "==>>" self pop: argumentCount + 1 thenPush: objectMemory trueObject. "<<==" excessSignals := self fetchInteger: ExcessSignalsIndex ofObject: sema. excessSignals > 0 ifTrue: [self storeInteger: ExcessSignalsIndex ofObject: sema withValue: excessSignals - 1] ifFalse: inInterpreter := instructionPointer >= objectMemory startOfMemory. activeProc := self activeProcess. self addLastLink: activeProc toList: sema. self transferTo: self wakeHighestPriority from: CSWait. self forProcessPrimitiveReturnToExecutivePostContextSwitch: inInterpreter]
which I guess could be added quickly if Esteban could compile the latest pharo-spur-vm ;)
cheers -ben
- Its faster
btw, Looking at the following... CriticalSection>>critical: aBlock ^self primitiveEnterCriticalSection ifTrue: [aBlock value] ifFalse: [aBlock ensure: [self primitiveExitCriticalSection]]
without intimate VM knowledge but seeing Eliot recently say "invocations of methods with primitives [...] are not suspension points, unless their primitives fail" -- I'm paranoid that since #ensure: primitive 198 always fail, there might be some some small window for a race where a process might be terminated before #primitiveExitCriticalSection can be executed. I'll take a refutation of this to improve the method comment.
The following instils more confidence...
CriticalSection2>>critical: aBlock | reEntered | reEntered := false. [ reEntered := self primitiveEnterCriticalSection. aBlock value ] ensure: [ reEntered ifFalse: [ self primitiveExitCriticalSection] ]
but performance is reduced... "#('Mutex -> 282,511 per second' 'Monitor -> 479,217 per second' 'CriticalSection -> 729,049 per second' 'CriticalSection2 -> 571,631 per second')"
Then again, any termination from within a critical section is anyhow problematic since potentially data structures are left in a indeterminate state.
[1] http://forum.world.st/Super-fast-delay-td4787257.html [2] https://pharo.fogbugz.com/default.asp?13755 [3] https://git.io/vuDnA
cheers -ben
Stef
Hi Ben,
On Thu, Jan 7, 2016 at 10:39 AM, Ben Coman btc@openinworld.comwrote:
On Fri, Jan 8, 2016 at 1:20 AM, Eliot Miranda eliot.miranda@gmail.com wrote:
and here's a version with a better class comment
On Thu, Jan 7, 2016 at 9:12 AM, Eliot Miranda eliot.miranda@gmail.com wrote:
Hi Denis, Hi Clément, Hi Frank,
On Thu, Jan 7, 2016 at 5:34 AM, Clément Bera bera.clement@gmail.com wrote:
Hello,
Eliot, please, you told me you had the code and Denis is interested.
It uses 3 primitives for performance.
Forgive the delay. I thought it proper to ask permission since the code was written while I was at Qwaq. I'm attaching the code in a fairly raw state, see the attached. The code is MIT, but copyright 3DICC.
It is a plugin replacement for Squeak's Mutex, and with a little ingenuity could be a replacement for Squeak's Monitor. It is quicker because it uses three new primitives to manage entering a critical section and setting the owner, exiting the critical section and releasing the owner, and testing if a critical section, entering if the section is unowned. The use of the primitives means fewer block activations and ensure: blocks in entering and exiting the critical section, and that's the actual cause of the speed-up.
You can benchmark the code as is. Here are some results on 32-bit Spur, on my 2.2GHz Core i7
{Mutex new. Monitor new. CriticalSection new} collect: [:cs| | n | n := 0. [cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. n ] bench]
{Mutex new. Monitor new. CriticalSection new} collect: [:cs| | n | n := 0. cs class name, ' -> ', [cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. n ] bench]
#( 'Mutex -> 440,000 per second. 2.27 microseconds per run.' 'Monitor -> 688,000 per second. 1.45 microseconds per run.' 'CriticalSection -> 1,110,000 per second. 900 nanoseconds per run.')
This is great Eliot. Thank you and 3DICC. After loading the changeset into Pharo-50515 (32 bit Spur) I get the following results on my laptop i5-2520M @ 2.50GHz
#('Mutex -> 254,047 per second' 'Monitor -> 450,442 per second' 'CriticalSection -> 683,393 per second')
In a fresh Image "Mutex allInstances basicInspect" lists just two mutexes...
- NetNameResolver-->ResolverMutex
- ThreadSafeTranscript-->accessSemaphore
I hate myself for getting distracted but I'm finding this is un. One can migrate to the new representation using normal Monticello loads by
In the first version redefine Mutex and Monitor to subclass LinkedList and have their owner/ownerProcess inst var first (actually third after firstLink & lastLink), and add the primitives.
In the next version check that all Mutex and Monitor instanes are unowned and then redefine to discard excess inst vars
Let me test this before committing, and see that all tests are ok.
cheers -ben
Replacement is probably trivial; rename Mutex to OldMutex, rename CriticalSection to Mutex, recompile. But there are lots of mutexes in the system and these are potentially owned. Transforming unowned ones is trivial, but transforming owned ones is, I think, impossible. But at least in my system there are no owned mutexes or monitors.
Frank (or anyone else), would you be interested in creating a replacement for Squeak's Monitor based on CriticalSection?
Here are the two business methods: CriticalSection methods for mutual exclusion critical: aBlock "Evaluate aBlock protected by the receiver." ^self primitiveEnterCriticalSection ifTrue: [aBlock value] ifFalse: [aBlock ensure: [self primitiveExitCriticalSection]]
critical: aBlock ifLocked: lockedBlock "Answer the evaluation of aBlock protected by the receiver. If it is already in a critical section on behalf of some other process answer the evaluation of lockedBlock." ^self primitiveTestAndSetOwnershipOfCriticalSection ifNil: [lockedBlock value] ifNotNil:[:alreadyOwner| alreadyOwner ifTrue: [aBlock value] ifFalse: [aBlock ensure: [self primitiveExitCriticalSection]]]
and the primitives: CriticalSection methods for private-primitives primitiveEnterCriticalSection "Primitive. The receiver must be unowned or owned by the current process to proceed. Answer if the process is owned by the current process." <primitive: 186> self primitiveFailed "In the spirit of the following" "[owner ifNil: [owner := Processor activeProcess. ^false]. owner = Processor activeProcess ifTrue: [^true]. self addLast: Processor activeProcess. Processor activeProcess suspend] valueUnpreemptively"
primitiveExitCriticalSection "Primitive. Set te receiver to unowned and if any processes are waiting on the receiver then proceed the first one, indicating that the receiver is unowned." <primitive: 185> self primitiveFailed "In the spirit of the following" "[owner := nil. self isEmpty ifFalse: [process := self removeFirst. process resume]] valueUnpreemptively"
primitiveTestAndSetOwnershipOfCriticalSection "Primitive. Attempt to set the ownership of the receiver. If the receiver is unowned set its owningProcess to the activeProcess and answer false. If the receiver is owned by the activeProcess answer true. If the receiver is owned by some other process answer nil." <primitive: 187> self primitiveFail "In the spirit of the following" "[owner ifNil: [owningProcess := Processor activeProcess. ^false]. owner = Processor activeProcess ifTrue: [^true]. ^nil] valueUnpreemptively"
2016-01-07 13:24 GMT+01:00 Denis Kudriashov dionisiydk@gmail.com: > > > Hello. > > I hear about new Monitor implementation based on new primitives. > Where to get it?
Hi Ben,
On Fri, Jan 8, 2016 at 7:25 AM, Ben Coman btc@openinworld.com wrote:
On Fri, Jan 8, 2016 at 9:39 PM, Ben Coman btc@openinworld.com wrote:
On Fri, Jan 8, 2016 at 5:42 PM, stephane ducasse stephane.ducasse@gmail.com wrote:
I have a (stupid) question. Is the code running without the primitives? Are the code below the primitives correct? I asked that because we can have 100 eyes and brains on the smalltalk
level and far less on the VM primitive level.
Because:
- Concurrency bugs can be subtle and the *exact* conditions can be
hard to reproduce for debugging. For example, the solution to a couple of problems with Delay [1] [2] were solved by moving away from Semaphore>>critical: to use signalling.
- The in-image atomicity of determining whether a signal was actually
consumed or not during process suspension/termination is awkward. Its seems hard to *really* know for sure it right (but I haven't looked in depth into Denis' latest proposals.)
- The existing in-image implementation of Semaphore>>critical messes
around in Process>>terminate in a *special* way that is not easy for those 100 eyes to understand. For example, I personally am not comfortable with understanding how the special Semaphore handling in Process>>terminate works, but I can easily follow how primitiveEnterCriticalSection just looking at the code [3].
Points 2 & 3 might possibly be addressed by having new primitiveWaitReturned *always* return true, so if the process is terminated while waiting, the assignment to signalConsumed doesn't occur...
critical: mutuallyExcludedBlock signalConsumed := false. [ signalConsumed := self primitiveWaitReturned. blockValue := mutuallyExcludedBlock value ] ensure: [ signalConsumed ifTrue: [self signal] ]. ^blockValue
where primitiveWait (https://git.io/vuDjd) is copied and (just guessing) the marked line added...
That looks like a good idea. I'll try and take a look early next week.
primitiveWaitReturned | sema excessSignals activeProc inInterpreter | sema := self stackTop. "rcvr" "==>>" self pop: argumentCount + 1 thenPush: objectMemory trueObject. "<<==" excessSignals := self fetchInteger: ExcessSignalsIndex ofObject: sema. excessSignals > 0 ifTrue: [self storeInteger: ExcessSignalsIndex ofObject: sema withValue: excessSignals - 1] ifFalse: inInterpreter := instructionPointer >= objectMemory startOfMemory. activeProc := self activeProcess. self addLastLink: activeProc toList: sema. self transferTo: self wakeHighestPriority from: CSWait. self forProcessPrimitiveReturnToExecutivePostContextSwitch: inInterpreter]
which I guess could be added quickly if Esteban could compile the latest pharo-spur-vm ;)
If you built a Squeak VMMaker image via http://www.squeakvm.org/svn/squeak/branches/Cog/image/BuildSqueakSpurTrunkVM... and built using the svn source tree you could do this yourself now.
cheers -ben
- Its faster
btw, Looking at the following... CriticalSection>>critical: aBlock ^self primitiveEnterCriticalSection ifTrue: [aBlock value] ifFalse: [aBlock ensure: [self primitiveExitCriticalSection]]
without intimate VM knowledge but seeing Eliot recently say "invocations of methods with primitives [...] are not suspension points, unless their primitives fail" -- I'm paranoid that since #ensure: primitive 198 always fail, there might be some some small window for a race where a process might be terminated before #primitiveExitCriticalSection can be executed. I'll take a refutation of this to improve the method comment.
The following instils more confidence...
CriticalSection2>>critical: aBlock | reEntered | reEntered := false. [ reEntered := self primitiveEnterCriticalSection. aBlock value ] ensure: [ reEntered ifFalse: [ self primitiveExitCriticalSection] ]
but performance is reduced... "#('Mutex -> 282,511 per second' 'Monitor -> 479,217 per second' 'CriticalSection -> 729,049 per second' 'CriticalSection2 -> 571,631 per second')"
Then again, any termination from within a critical section is anyhow problematic since potentially data structures are left in a indeterminate state.
[1] http://forum.world.st/Super-fast-delay-td4787257.html [2] https://pharo.fogbugz.com/default.asp?13755 [3] https://git.io/vuDnA
cheers -ben
Stef
Hi Ben,
On Thu, Jan 7, 2016 at 10:39 AM, Ben Coman btc@openinworld.comwrote:
On Fri, Jan 8, 2016 at 1:20 AM, Eliot Miranda eliot.miranda@gmail.com
wrote:
and here's a version with a better class comment
On Thu, Jan 7, 2016 at 9:12 AM, Eliot Miranda <
eliot.miranda@gmail.com> wrote:
Hi Denis, Hi Clément, Hi Frank,
On Thu, Jan 7, 2016 at 5:34 AM, Clément Bera <
bera.clement@gmail.com> wrote:
> > Hello, > > Eliot, please, you told me you had the code and Denis is
interested.
> > It uses 3 primitives for performance.
Forgive the delay. I thought it proper to ask permission since the
code was written while I was at Qwaq. I'm attaching the code in a fairly raw state, see the attached. The code is MIT, but copyright 3DICC.
It is a plugin replacement for Squeak's Mutex, and with a little
ingenuity could be a replacement for Squeak's Monitor. It is quicker because it uses three new primitives to manage entering a critical section and setting the owner, exiting the critical section and releasing the owner, and testing if a critical section, entering if the section is unowned. The use of the primitives means fewer block activations and ensure: blocks in entering and exiting the critical section, and that's the actual cause of the speed-up.
You can benchmark the code as is. Here are some results on 32-bit
Spur, on my 2.2GHz Core i7
{Mutex new. Monitor new. CriticalSection new} collect: [:cs| | n | n := 0. [cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical:
[n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1].
cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical:
[n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1].
n ] bench]
{Mutex new. Monitor new. CriticalSection new} collect: [:cs| | n | n := 0. cs class name, ' -> ', [cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical:
[n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1].
cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical:
[n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1].
n ] bench]
#( 'Mutex -> 440,000 per second. 2.27 microseconds per run.' 'Monitor -> 688,000 per second. 1.45 microseconds per run.' 'CriticalSection -> 1,110,000 per second. 900 nanoseconds per run.')
This is great Eliot. Thank you and 3DICC. After loading the changeset into Pharo-50515 (32 bit Spur) I get the following results on my laptop i5-2520M @ 2.50GHz
#('Mutex -> 254,047 per second' 'Monitor -> 450,442 per second' 'CriticalSection -> 683,393 per second')
In a fresh Image "Mutex allInstances basicInspect" lists just two
mutexes...
- NetNameResolver-->ResolverMutex
- ThreadSafeTranscript-->accessSemaphore
I hate myself for getting distracted but I'm finding this is un. One
can migrate to the new representation using normal Monticello loads by
In the first version redefine Mutex and Monitor to subclass LinkedList
and have their owner/ownerProcess inst var first (actually third after firstLink & lastLink), and add the primitives.
In the next version check that all Mutex and Monitor instanes are
unowned and then redefine to discard excess inst vars
Let me test this before committing, and see that all tests are ok.
cheers -ben
Replacement is probably trivial; rename Mutex to OldMutex, rename
CriticalSection to Mutex, recompile. But there are lots of mutexes in the system and these are potentially owned. Transforming unowned ones is trivial, but transforming owned ones is, I think, impossible. But at least in my system there are no owned mutexes or monitors.
Frank (or anyone else), would you be interested in creating a
replacement for Squeak's Monitor based on CriticalSection?
Here are the two business methods: CriticalSection methods for mutual exclusion critical: aBlock "Evaluate aBlock protected by the receiver." ^self primitiveEnterCriticalSection ifTrue: [aBlock value] ifFalse: [aBlock ensure: [self primitiveExitCriticalSection]]
critical: aBlock ifLocked: lockedBlock "Answer the evaluation of aBlock protected by the receiver. If it
is already in a critical
section on behalf of some other process answer the evaluation of
lockedBlock."
^self primitiveTestAndSetOwnershipOfCriticalSection ifNil: [lockedBlock value] ifNotNil:[:alreadyOwner| alreadyOwner ifTrue: [aBlock value] ifFalse: [aBlock ensure: [self primitiveExitCriticalSection]]]
and the primitives: CriticalSection methods for private-primitives primitiveEnterCriticalSection "Primitive. The receiver must be unowned or owned by the current
process to proceed.
Answer if the process is owned by the current process." <primitive: 186> self primitiveFailed "In the spirit of the following" "[owner ifNil: [owner := Processor activeProcess. ^false]. owner = Processor activeProcess ifTrue: [^true]. self addLast: Processor activeProcess. Processor activeProcess suspend] valueUnpreemptively"
primitiveExitCriticalSection "Primitive. Set te receiver to unowned and if any processes are
waiting on
the receiver then proceed the first one, indicating that the
receiver is unowned."
<primitive: 185> self primitiveFailed "In the spirit of the following" "[owner := nil. self isEmpty ifFalse: [process := self removeFirst. process resume]] valueUnpreemptively"
primitiveTestAndSetOwnershipOfCriticalSection "Primitive. Attempt to set the ownership of the receiver. If the receiver is unowned set its owningProcess to the activeProcess and answer false. If the receiver is owned by the activeProcess answer true. If the receiver is owned by some other process answer nil." <primitive: 187> self primitiveFail "In the spirit of the following" "[owner ifNil: [owningProcess := Processor activeProcess. ^false]. owner = Processor activeProcess ifTrue: [^true]. ^nil] valueUnpreemptively"
> 2016-01-07 13:24 GMT+01:00 Denis Kudriashov <dionisiydk@gmail.com
:
>> >> >> Hello. >> >> I hear about new Monitor implementation based on new primitives. >> Where to get it?
On 08 Jan 2016, at 4:25 , Ben Coman btc@openInWorld.com wrote:
On Fri, Jan 8, 2016 at 9:39 PM, Ben Coman <btc@openinworld.com mailto:btc@openinworld.com> wrote:
On Fri, Jan 8, 2016 at 5:42 PM, stephane ducasse stephane.ducasse@gmail.com wrote:
I have a (stupid) question. Is the code running without the primitives? Are the code below the primitives correct? I asked that because we can have 100 eyes and brains on the smalltalk level and far less on the VM primitive level.
Because:
- Concurrency bugs can be subtle and the *exact* conditions can be
hard to reproduce for debugging. For example, the solution to a couple of problems with Delay [1] [2] were solved by moving away from Semaphore>>critical: to use signalling.
- The in-image atomicity of determining whether a signal was actually
consumed or not during process suspension/termination is awkward. Its seems hard to *really* know for sure it right (but I haven't looked in depth into Denis' latest proposals.)
- The existing in-image implementation of Semaphore>>critical messes
around in Process>>terminate in a *special* way that is not easy for those 100 eyes to understand. For example, I personally am not comfortable with understanding how the special Semaphore handling in Process>>terminate works, but I can easily follow how primitiveEnterCriticalSection just looking at the code [3].
Points 2 & 3 might possibly be addressed by having new primitiveWaitReturned *always* return true, so if the process is terminated while waiting, the assignment to signalConsumed doesn't occur...
critical: mutuallyExcludedBlock signalConsumed := false. [ signalConsumed := self primitiveWaitReturned. blockValue := mutuallyExcludedBlock value ] ensure: [ signalConsumed ifTrue: [self signal] ]. ^blockValue
where primitiveWait (https://git.io/vuDjd https://git.io/vuDjd) is copied and (just guessing) the marked line added...
primitiveWaitReturned | sema excessSignals activeProc inInterpreter | sema := self stackTop. "rcvr" "==>>" self pop: argumentCount + 1 thenPush: objectMemory trueObject. "<<==" excessSignals := self fetchInteger: ExcessSignalsIndex ofObject: sema. excessSignals > 0 ifTrue: [self storeInteger: ExcessSignalsIndex ofObject: sema withValue: excessSignals - 1] ifFalse: inInterpreter := instructionPointer >= objectMemory startOfMemory. activeProc := self activeProcess. self addLastLink: activeProc toList: sema. self transferTo: self wakeHighestPriority from: CSWait. self forProcessPrimitiveReturnToExecutivePostContextSwitch: inInterpreter]
which I guess could be added quickly if Esteban could compile the latest pharo-spur-vm ;)
cheers -ben
Won't work, there's no guarantee thread has actually ran and signalConsumed been assigned the primitive result after Semaphore resumed the waiting thread, before a higher priority thread runs and terminates it. (which is exactly the case handled by special code in #terminate)
Cheers, Henry
Hello. I publish slice 17373 https://pharo.fogbugz.com/f/cases/17373/Mutex-should-be-based-on-VM-primitives-and-implement-critical-methods-from-Semaphore .
I introduce LockOwnership class which implements VM primitives as: - acquire - tryAcquire - release (It comment saves copyright from CriticalSection)
Mutext uses it as ownership instance variable to implement critical methods correctly: - critical: - critical:ifLocked: - critical:ifError: - critical:ifCurtailed:
For the integration process old Mutex instance variables are not removed. So loading this code should not broke current Mutex instances. But when we integrate it Mutex will have only variable ownership.
If you look critical implementation you will understand why I not like semantic of primitives. it should be inverted in future. #acquire should return true when ownership is acquired right now.
Process>>terminate now detects waiting on LockOwnership and ask it to handle wait termination.Then LockOwnership inject right value into lock state variable. Such variables should be marked with special pragma <lockAt: #ownershipVariableName tracksStateAt: 1> "index of local variable" Method can contain mulpible pragmas to reference all ownerships in method. ReadWriteLock for example needs this.
2016-01-11 13:30 GMT+01:00 Henrik Johansen henrik.s.johansen@veloxit.no:
On 08 Jan 2016, at 4:25 , Ben Coman <btc@openInWorld.com btc@openinworld.com> wrote:
On Fri, Jan 8, 2016 at 9:39 PM, Ben Coman btc@openinworld.com wrote:
On Fri, Jan 8, 2016 at 5:42 PM, stephane ducasse stephane.ducasse@gmail.com wrote:
I have a (stupid) question. Is the code running without the primitives? Are the code below the primitives correct? I asked that because we can have 100 eyes and brains on the smalltalk level and far less on the VM primitive level.
Because:
- Concurrency bugs can be subtle and the *exact* conditions can be
hard to reproduce for debugging. For example, the solution to a couple of problems with Delay [1] [2] were solved by moving away from Semaphore>>critical: to use signalling.
- The in-image atomicity of determining whether a signal was actually
consumed or not during process suspension/termination is awkward. Its seems hard to *really* know for sure it right (but I haven't looked in depth into Denis' latest proposals.)
- The existing in-image implementation of Semaphore>>critical messes
around in Process>>terminate in a *special* way that is not easy for those 100 eyes to understand. For example, I personally am not comfortable with understanding how the special Semaphore handling in Process>>terminate works, but I can easily follow how primitiveEnterCriticalSection just looking at the code [3].
Points 2 & 3 might possibly be addressed by having new primitiveWaitReturned *always* return true, so if the process is terminated while waiting, the assignment to signalConsumed doesn't occur...
critical: mutuallyExcludedBlock signalConsumed := false. [ signalConsumed := self primitiveWaitReturned. blockValue := mutuallyExcludedBlock value ] ensure: [ signalConsumed ifTrue: [self signal] ]. ^blockValue
where primitiveWait (https://git.io/vuDjd) is copied and (just guessing) the marked line added...
primitiveWaitReturned | sema excessSignals activeProc inInterpreter | sema := self stackTop. "rcvr" "==>>" self pop: argumentCount + 1 thenPush: objectMemory trueObject. "<<==" excessSignals := self fetchInteger: ExcessSignalsIndex ofObject: sema. excessSignals > 0 ifTrue: [self storeInteger: ExcessSignalsIndex ofObject: sema withValue: excessSignals - 1] ifFalse: inInterpreter := instructionPointer >= objectMemory startOfMemory. activeProc := self activeProcess. self addLastLink: activeProc toList: sema. self transferTo: self wakeHighestPriority from: CSWait. self forProcessPrimitiveReturnToExecutivePostContextSwitch: inInterpreter]
which I guess could be added quickly if Esteban could compile the latest pharo-spur-vm ;)
cheers -ben
Won't work, there's no guarantee thread has actually ran and signalConsumed been assigned the primitive result after Semaphore resumed the waiting thread, before a higher priority thread runs and terminates it. (which is exactly the case handled by special code in #terminate)
Cheers, Henry
Eliot I found that there are no methods for simulation code (if it right name for it). So stepping over new primitives failed (which is infinite recursion for Pharo case)
Context>>doPrimitive: primitiveIndex method: meth receiver: aReceiver args: arguments ... "Mutex>>primitiveEnterCriticalSection Mutex>>primitiveTestAndSetOwnershipOfCriticalSection" (primitiveIndex = 186 or: [primitiveIndex = 187]) ifTrue: [| active effective | active := Processor activeProcess. effective := active effectiveProcess. "active == effective" value := primitiveIndex = 186 ifTrue: [aReceiver *primitiveEnterCriticalSectionOnBehalfOf*: effective] ifFalse: [aReceiver *primitiveTestAndSetOwnershipOfCriticalSectionOnBehalfOf*: effective]. ^(self isPrimFailToken: value) ifTrue: [value] ifFalse: [self push: value]].
How this methods should be implemented?
2016-01-11 13:59 GMT+01:00 Denis Kudriashov dionisiydk@gmail.com:
Hello. I publish slice 17373 https://pharo.fogbugz.com/f/cases/17373/Mutex-should-be-based-on-VM-primitives-and-implement-critical-methods-from-Semaphore .
I introduce LockOwnership class which implements VM primitives as:
- acquire
- tryAcquire
- release
(It comment saves copyright from CriticalSection)
Mutext uses it as ownership instance variable to implement critical methods correctly:
- critical:
- critical:ifLocked:
- critical:ifError:
- critical:ifCurtailed:
For the integration process old Mutex instance variables are not removed. So loading this code should not broke current Mutex instances. But when we integrate it Mutex will have only variable ownership.
If you look critical implementation you will understand why I not like semantic of primitives. it should be inverted in future. #acquire should return true when ownership is acquired right now.
Process>>terminate now detects waiting on LockOwnership and ask it to handle wait termination.Then LockOwnership inject right value into lock state variable. Such variables should be marked with special pragma <lockAt: #ownershipVariableName tracksStateAt: 1> "index of local variable" Method can contain mulpible pragmas to reference all ownerships in method. ReadWriteLock for example needs this.
2016-01-11 13:30 GMT+01:00 Henrik Johansen henrik.s.johansen@veloxit.no:
On 08 Jan 2016, at 4:25 , Ben Coman <btc@openInWorld.com btc@openinworld.com> wrote:
On Fri, Jan 8, 2016 at 9:39 PM, Ben Coman btc@openinworld.com wrote:
On Fri, Jan 8, 2016 at 5:42 PM, stephane ducasse stephane.ducasse@gmail.com wrote:
I have a (stupid) question. Is the code running without the primitives? Are the code below the primitives correct? I asked that because we can have 100 eyes and brains on the smalltalk level and far less on the VM primitive level.
Because:
- Concurrency bugs can be subtle and the *exact* conditions can be
hard to reproduce for debugging. For example, the solution to a couple of problems with Delay [1] [2] were solved by moving away from Semaphore>>critical: to use signalling.
- The in-image atomicity of determining whether a signal was actually
consumed or not during process suspension/termination is awkward. Its seems hard to *really* know for sure it right (but I haven't looked in depth into Denis' latest proposals.)
- The existing in-image implementation of Semaphore>>critical messes
around in Process>>terminate in a *special* way that is not easy for those 100 eyes to understand. For example, I personally am not comfortable with understanding how the special Semaphore handling in Process>>terminate works, but I can easily follow how primitiveEnterCriticalSection just looking at the code [3].
Points 2 & 3 might possibly be addressed by having new primitiveWaitReturned *always* return true, so if the process is terminated while waiting, the assignment to signalConsumed doesn't occur...
critical: mutuallyExcludedBlock signalConsumed := false. [ signalConsumed := self primitiveWaitReturned. blockValue := mutuallyExcludedBlock value ] ensure: [ signalConsumed ifTrue: [self signal] ]. ^blockValue
where primitiveWait (https://git.io/vuDjd) is copied and (just guessing) the marked line added...
primitiveWaitReturned | sema excessSignals activeProc inInterpreter | sema := self stackTop. "rcvr" "==>>" self pop: argumentCount + 1 thenPush: objectMemory trueObject. "<<==" excessSignals := self fetchInteger: ExcessSignalsIndex ofObject: sema. excessSignals > 0 ifTrue: [self storeInteger: ExcessSignalsIndex ofObject: sema withValue: excessSignals - 1] ifFalse: inInterpreter := instructionPointer >= objectMemory startOfMemory. activeProc := self activeProcess. self addLastLink: activeProc toList: sema. self transferTo: self wakeHighestPriority from: CSWait. self forProcessPrimitiveReturnToExecutivePostContextSwitch: inInterpreter]
which I guess could be added quickly if Esteban could compile the latest pharo-spur-vm ;)
cheers -ben
Won't work, there's no guarantee thread has actually ran and signalConsumed been assigned the primitive result after Semaphore resumed the waiting thread, before a higher priority thread runs and terminates it. (which is exactly the case handled by special code in #terminate)
Cheers, Henry
I write it such way:
primitiveEnterCriticalSectionOnBehalfOf: activeProcess [owningProcess ifNil: [owningProcess := activeProcess. ^false]. owningProcess = activeProcess ifTrue: [^true]. self addLast: Processor activeProcess. activeProcess suspend] valueUnpreemptively
primitiveTestAndSetOwnershipOfCriticalSectionOnBehalfOf: activeProcess [owningProcess ifNil: [owningProcess := activeProcess. ^false]. owningProcess = activeProcess ifTrue: [^true]. ^nil] valueUnpreemptively
2016-01-11 14:58 GMT+01:00 Denis Kudriashov dionisiydk@gmail.com:
Eliot I found that there are no methods for simulation code (if it right name for it). So stepping over new primitives failed (which is infinite recursion for Pharo case)
Context>>doPrimitive: primitiveIndex method: meth receiver: aReceiver args: arguments ... "Mutex>>primitiveEnterCriticalSection Mutex>>primitiveTestAndSetOwnershipOfCriticalSection" (primitiveIndex = 186 or: [primitiveIndex = 187]) ifTrue: [| active effective | active := Processor activeProcess. effective := active effectiveProcess. "active == effective" value := primitiveIndex = 186 ifTrue: [aReceiver *primitiveEnterCriticalSectionOnBehalfOf*: effective] ifFalse: [aReceiver *primitiveTestAndSetOwnershipOfCriticalSectionOnBehalfOf*: effective]. ^(self isPrimFailToken: value) ifTrue: [value] ifFalse: [self push: value]].
How this methods should be implemented?
2016-01-11 13:59 GMT+01:00 Denis Kudriashov dionisiydk@gmail.com:
Hello. I publish slice 17373 https://pharo.fogbugz.com/f/cases/17373/Mutex-should-be-based-on-VM-primitives-and-implement-critical-methods-from-Semaphore .
I introduce LockOwnership class which implements VM primitives as:
- acquire
- tryAcquire
- release
(It comment saves copyright from CriticalSection)
Mutext uses it as ownership instance variable to implement critical methods correctly:
- critical:
- critical:ifLocked:
- critical:ifError:
- critical:ifCurtailed:
For the integration process old Mutex instance variables are not removed. So loading this code should not broke current Mutex instances. But when we integrate it Mutex will have only variable ownership.
If you look critical implementation you will understand why I not like semantic of primitives. it should be inverted in future. #acquire should return true when ownership is acquired right now.
Process>>terminate now detects waiting on LockOwnership and ask it to handle wait termination.Then LockOwnership inject right value into lock state variable. Such variables should be marked with special pragma <lockAt: #ownershipVariableName tracksStateAt: 1> "index of local variable" Method can contain mulpible pragmas to reference all ownerships in method. ReadWriteLock for example needs this.
2016-01-11 13:30 GMT+01:00 Henrik Johansen henrik.s.johansen@veloxit.no :
On 08 Jan 2016, at 4:25 , Ben Coman <btc@openInWorld.com btc@openinworld.com> wrote:
On Fri, Jan 8, 2016 at 9:39 PM, Ben Coman btc@openinworld.com wrote:
On Fri, Jan 8, 2016 at 5:42 PM, stephane ducasse stephane.ducasse@gmail.com wrote:
I have a (stupid) question. Is the code running without the primitives? Are the code below the primitives correct? I asked that because we can have 100 eyes and brains on the smalltalk level and far less on the VM primitive level.
Because:
- Concurrency bugs can be subtle and the *exact* conditions can be
hard to reproduce for debugging. For example, the solution to a couple of problems with Delay [1] [2] were solved by moving away from Semaphore>>critical: to use signalling.
- The in-image atomicity of determining whether a signal was actually
consumed or not during process suspension/termination is awkward. Its seems hard to *really* know for sure it right (but I haven't looked in depth into Denis' latest proposals.)
- The existing in-image implementation of Semaphore>>critical messes
around in Process>>terminate in a *special* way that is not easy for those 100 eyes to understand. For example, I personally am not comfortable with understanding how the special Semaphore handling in Process>>terminate works, but I can easily follow how primitiveEnterCriticalSection just looking at the code [3].
Points 2 & 3 might possibly be addressed by having new primitiveWaitReturned *always* return true, so if the process is terminated while waiting, the assignment to signalConsumed doesn't occur...
critical: mutuallyExcludedBlock signalConsumed := false. [ signalConsumed := self primitiveWaitReturned. blockValue := mutuallyExcludedBlock value ] ensure: [ signalConsumed ifTrue: [self signal] ]. ^blockValue
where primitiveWait (https://git.io/vuDjd) is copied and (just guessing) the marked line added...
primitiveWaitReturned | sema excessSignals activeProc inInterpreter | sema := self stackTop. "rcvr" "==>>" self pop: argumentCount + 1 thenPush: objectMemory trueObject. "<<==" excessSignals := self fetchInteger: ExcessSignalsIndex ofObject: sema. excessSignals > 0 ifTrue: [self storeInteger: ExcessSignalsIndex ofObject: sema withValue: excessSignals - 1] ifFalse: inInterpreter := instructionPointer >= objectMemory startOfMemory. activeProc := self activeProcess. self addLastLink: activeProc toList: sema. self transferTo: self wakeHighestPriority from: CSWait. self forProcessPrimitiveReturnToExecutivePostContextSwitch: inInterpreter]
which I guess could be added quickly if Esteban could compile the latest pharo-spur-vm ;)
cheers -ben
Won't work, there's no guarantee thread has actually ran and signalConsumed been assigned the primitive result after Semaphore resumed the waiting thread, before a higher priority thread runs and terminates it. (which is exactly the case handled by special code in #terminate)
Cheers, Henry
Thanks Denis,
On Mon, Jan 11, 2016 at 6:54 AM, Denis Kudriashov dionisiydk@gmail.com wrote:
I write it such way:
primitiveEnterCriticalSectionOnBehalfOf: activeProcess [owningProcess ifNil: [owningProcess := activeProcess. ^false]. owningProcess = activeProcess ifTrue: [^true]. self addLast: Processor activeProcess. activeProcess suspend] valueUnpreemptively
primitiveTestAndSetOwnershipOfCriticalSectionOnBehalfOf: activeProcess [owningProcess ifNil: [owningProcess := activeProcess. ^false]. owningProcess = activeProcess ifTrue: [^true]. ^nil] valueUnpreemptively
This looks good. Do you have methods you've written? I'd like to use yours as they have your time stamp.
2016-01-11 14:58 GMT+01:00 Denis Kudriashov dionisiydk@gmail.com:
Eliot I found that there are no methods for simulation code (if it right name for it). So stepping over new primitives failed (which is infinite recursion for Pharo case)
Context>>doPrimitive: primitiveIndex method: meth receiver: aReceiver args: arguments ... "Mutex>>primitiveEnterCriticalSection Mutex>>primitiveTestAndSetOwnershipOfCriticalSection" (primitiveIndex = 186 or: [primitiveIndex = 187]) ifTrue: [| active effective | active := Processor activeProcess. effective := active effectiveProcess. "active == effective" value := primitiveIndex = 186 ifTrue: [aReceiver *primitiveEnterCriticalSectionOnBehalfOf*: effective] ifFalse: [aReceiver *primitiveTestAndSetOwnershipOfCriticalSectionOnBehalfOf*: effective]. ^(self isPrimFailToken: value) ifTrue: [value] ifFalse: [self push: value]].
How this methods should be implemented?
2016-01-11 13:59 GMT+01:00 Denis Kudriashov dionisiydk@gmail.com:
Hello. I publish slice 17373 https://pharo.fogbugz.com/f/cases/17373/Mutex-should-be-based-on-VM-primitives-and-implement-critical-methods-from-Semaphore .
I introduce LockOwnership class which implements VM primitives as:
- acquire
- tryAcquire
- release
(It comment saves copyright from CriticalSection)
Mutext uses it as ownership instance variable to implement critical methods correctly:
- critical:
- critical:ifLocked:
- critical:ifError:
- critical:ifCurtailed:
For the integration process old Mutex instance variables are not removed. So loading this code should not broke current Mutex instances. But when we integrate it Mutex will have only variable ownership.
If you look critical implementation you will understand why I not like semantic of primitives. it should be inverted in future. #acquire should return true when ownership is acquired right now.
Process>>terminate now detects waiting on LockOwnership and ask it to handle wait termination.Then LockOwnership inject right value into lock state variable. Such variables should be marked with special pragma <lockAt: #ownershipVariableName tracksStateAt: 1> "index of local variable" Method can contain mulpible pragmas to reference all ownerships in method. ReadWriteLock for example needs this.
2016-01-11 13:30 GMT+01:00 Henrik Johansen <henrik.s.johansen@veloxit.no
:
On 08 Jan 2016, at 4:25 , Ben Coman <btc@openInWorld.com btc@openinworld.com> wrote:
On Fri, Jan 8, 2016 at 9:39 PM, Ben Coman btc@openinworld.com wrote:
On Fri, Jan 8, 2016 at 5:42 PM, stephane ducasse stephane.ducasse@gmail.com wrote:
I have a (stupid) question. Is the code running without the primitives? Are the code below the primitives correct? I asked that because we can have 100 eyes and brains on the smalltalk level and far less on the VM primitive level.
Because:
- Concurrency bugs can be subtle and the *exact* conditions can be
hard to reproduce for debugging. For example, the solution to a couple of problems with Delay [1] [2] were solved by moving away from Semaphore>>critical: to use signalling.
- The in-image atomicity of determining whether a signal was actually
consumed or not during process suspension/termination is awkward. Its seems hard to *really* know for sure it right (but I haven't looked in depth into Denis' latest proposals.)
- The existing in-image implementation of Semaphore>>critical messes
around in Process>>terminate in a *special* way that is not easy for those 100 eyes to understand. For example, I personally am not comfortable with understanding how the special Semaphore handling in Process>>terminate works, but I can easily follow how primitiveEnterCriticalSection just looking at the code [3].
Points 2 & 3 might possibly be addressed by having new primitiveWaitReturned *always* return true, so if the process is terminated while waiting, the assignment to signalConsumed doesn't occur...
critical: mutuallyExcludedBlock signalConsumed := false. [ signalConsumed := self primitiveWaitReturned. blockValue := mutuallyExcludedBlock value ] ensure: [ signalConsumed ifTrue: [self signal] ]. ^blockValue
where primitiveWait (https://git.io/vuDjd) is copied and (just guessing) the marked line added...
primitiveWaitReturned | sema excessSignals activeProc inInterpreter | sema := self stackTop. "rcvr" "==>>" self pop: argumentCount + 1 thenPush: objectMemory trueObject. "<<==" excessSignals := self fetchInteger: ExcessSignalsIndex ofObject: sema. excessSignals > 0 ifTrue: [self storeInteger: ExcessSignalsIndex ofObject: sema withValue: excessSignals - 1] ifFalse: inInterpreter := instructionPointer >= objectMemory startOfMemory. activeProc := self activeProcess. self addLastLink: activeProc toList: sema. self transferTo: self wakeHighestPriority from: CSWait. self forProcessPrimitiveReturnToExecutivePostContextSwitch: inInterpreter]
which I guess could be added quickly if Esteban could compile the latest pharo-spur-vm ;)
cheers -ben
Won't work, there's no guarantee thread has actually ran and signalConsumed been assigned the primitive result after Semaphore resumed the waiting thread, before a higher priority thread runs and terminates it. (which is exactly the case handled by special code in #terminate)
Cheers, Henry
2016-01-17 20:58 GMT+01:00 Eliot Miranda eliot.miranda@gmail.com:
Thanks Denis,
On Mon, Jan 11, 2016 at 6:54 AM, Denis Kudriashov dionisiydk@gmail.com wrote:
I write it such way:
primitiveEnterCriticalSectionOnBehalfOf: activeProcess [owningProcess ifNil: [owningProcess := activeProcess. ^false]. owningProcess = activeProcess ifTrue: [^true]. self addLast: Processor activeProcess. activeProcess suspend] valueUnpreemptively
primitiveTestAndSetOwnershipOfCriticalSectionOnBehalfOf: activeProcess [owningProcess ifNil: [owningProcess := activeProcess. ^false]. owningProcess = activeProcess ifTrue: [^true]. ^nil] valueUnpreemptively
This looks good. Do you have methods you've written? I'd like to use yours as they have your time stamp.
I only added it to renamed class LockOwnership (attached). So you can only copy it to original class which will reset timestamps
2016-01-11 14:58 GMT+01:00 Denis Kudriashov dionisiydk@gmail.com:
Eliot I found that there are no methods for simulation code (if it right name for it). So stepping over new primitives failed (which is infinite recursion for Pharo case)
Context>>doPrimitive: primitiveIndex method: meth receiver: aReceiver args: arguments ... "Mutex>>primitiveEnterCriticalSection Mutex>>primitiveTestAndSetOwnershipOfCriticalSection" (primitiveIndex = 186 or: [primitiveIndex = 187]) ifTrue: [| active effective | active := Processor activeProcess. effective := active effectiveProcess. "active == effective" value := primitiveIndex = 186 ifTrue: [aReceiver primitiveEnterCriticalSectionOnBehalfOf: effective] ifFalse: [aReceiver primitiveTestAndSetOwnershipOfCriticalSectionOnBehalfOf: effective]. ^(self isPrimFailToken: value) ifTrue: [value] ifFalse: [self push: value]].
How this methods should be implemented?
On Mon, Jan 11, 2016 at 10:54 PM, Denis Kudriashov dionisiydk@gmail.com wrote:
I write it such way:
primitiveEnterCriticalSectionOnBehalfOf: activeProcess [owningProcess ifNil: [owningProcess := activeProcess. ^false]. owningProcess = activeProcess ifTrue: [^true]. self addLast: Processor activeProcess. activeProcess suspend] valueUnpreemptively
primitiveTestAndSetOwnershipOfCriticalSectionOnBehalfOf: activeProcess [owningProcess ifNil: [owningProcess := activeProcess. ^false]. owningProcess = activeProcess ifTrue: [^true]. ^nil] valueUnpreemptively
@Dennis, Why not... primitiveEnterCriticalSectionOnBehalfOf: activeProcess <primitive: 186> self primitiveFailed
I had been meaning to ask what this code at the top of the primitive was for... argumentCount > 0 ifTrue: [criticalSection := self stackValue: 1. "rcvr" activeProc := self stackTop] ifFalse: [criticalSection := self stackTop. "rcvr" activeProc := self activeProcess].
but took a wild guess its to support onBehalfOf type calls. So I tried...
| cs p1 p2 p3 | cs := CriticalSection new. p1 := [ cs critical: [ Transcript crShow: 1 ] ] newProcess priority: 41. p2 := [ cs critical: [ Transcript crShow: 2 ] ] newProcess priority: 41. p3 := [ cs critical: [ Transcript crShow: 3 ] ] newProcess priority: 41. cs primitiveEnterCriticalSectionOnBehalfOf: p3. p1 resume. p2 resume. p3 resume. cs primitiveExitCriticalSection.
and indeed the result is... 3 1 2
cheers -ben
2016-02-06 8:16 GMT+01:00 Ben Coman btc@openinworld.com:
@Dennis, Why not... primitiveEnterCriticalSectionOnBehalfOf: activeProcess <primitive: 186> self primitiveFailed
I had been meaning to ask what this code at the top of the primitive was for... argumentCount > 0 ifTrue: [criticalSection := self stackValue: 1. "rcvr" activeProc := self stackTop] ifFalse: [criticalSection := self stackTop. "rcvr" activeProc := self activeProcess].
but took a wild guess its to support onBehalfOf type calls.
Nice. What about tryAcquire (187)? Does it implemented same way?
On Mon, Jan 11, 2016 at 9:58 PM, Denis Kudriashov dionisiydk@gmail.com wrote:
Eliot I found that there are no methods for simulation code (if it right name for it). So stepping over new primitives failed (which is infinite recursion for Pharo case)
Context>>doPrimitive: primitiveIndex method: meth receiver: aReceiver args: arguments ... "Mutex>>primitiveEnterCriticalSection Mutex>>primitiveTestAndSetOwnershipOfCriticalSection" (primitiveIndex = 186 or: [primitiveIndex = 187]) ifTrue: [| active effective | active := Processor activeProcess. effective := active effectiveProcess. "active == effective" value := primitiveIndex = 186 ifTrue: [aReceiver primitiveEnterCriticalSectionOnBehalfOf: effective] ifFalse: [aReceiver primitiveTestAndSetOwnershipOfCriticalSectionOnBehalfOf: effective]. ^(self isPrimFailToken: value) ifTrue: [value] ifFalse: [self push: value]].
How this methods should be implemented?
2016-01-11 13:59 GMT+01:00 Denis Kudriashov dionisiydk@gmail.com:
Hello. I publish slice 17373.
I introduce LockOwnership class which implements VM primitives as:
- acquire
- tryAcquire
- release
(It comment saves copyright from CriticalSection)
I don't think that is proper. The code is the same, just some names have changed. I'll bet the primitives were funded by 3DICC too, so credit where credit is due. cheers -ben
2016-01-11 17:47 GMT+01:00 Ben Coman btc@openinworld.com:
(It comment saves copyright from CriticalSection)
I don't think that is proper. The code is the same, just some names have changed. I'll bet the primitives were funded by 3DICC too, so credit where credit is due.
But copyright is mentioned in class comment
Hi Denis,
On Mon, Jan 11, 2016 at 5:58 AM, Denis Kudriashov dionisiydk@gmail.com wrote:
Eliot I found that there are no methods for simulation code (if it right name for it). So stepping over new primitives failed (which is infinite recursion for Pharo case)
Find the Squeak code attached. Merge it with the Pharo code (renaming ContextPart to COntext of course; something we'll do in Squeak eventually :-) )..
Context>>doPrimitive: primitiveIndex method: meth receiver: aReceiver args: arguments ... "Mutex>>primitiveEnterCriticalSection Mutex>>primitiveTestAndSetOwnershipOfCriticalSection" (primitiveIndex = 186 or: [primitiveIndex = 187]) ifTrue: [| active effective | active := Processor activeProcess. effective := active effectiveProcess. "active == effective" value := primitiveIndex = 186 ifTrue: [aReceiver *primitiveEnterCriticalSectionOnBehalfOf*: effective] ifFalse: [aReceiver *primitiveTestAndSetOwnershipOfCriticalSectionOnBehalfOf*: effective]. ^(self isPrimFailToken: value) ifTrue: [value] ifFalse: [self push: value]].
How this methods should be implemented?
2016-01-11 13:59 GMT+01:00 Denis Kudriashov dionisiydk@gmail.com:
Hello. I publish slice 17373 https://pharo.fogbugz.com/f/cases/17373/Mutex-should-be-based-on-VM-primitives-and-implement-critical-methods-from-Semaphore .
I introduce LockOwnership class which implements VM primitives as:
- acquire
- tryAcquire
- release
(It comment saves copyright from CriticalSection)
Mutext uses it as ownership instance variable to implement critical methods correctly:
- critical:
- critical:ifLocked:
- critical:ifError:
- critical:ifCurtailed:
For the integration process old Mutex instance variables are not removed. So loading this code should not broke current Mutex instances. But when we integrate it Mutex will have only variable ownership.
If you look critical implementation you will understand why I not like semantic of primitives. it should be inverted in future. #acquire should return true when ownership is acquired right now.
Process>>terminate now detects waiting on LockOwnership and ask it to handle wait termination.Then LockOwnership inject right value into lock state variable. Such variables should be marked with special pragma <lockAt: #ownershipVariableName tracksStateAt: 1> "index of local variable" Method can contain mulpible pragmas to reference all ownerships in method. ReadWriteLock for example needs this.
2016-01-11 13:30 GMT+01:00 Henrik Johansen henrik.s.johansen@veloxit.no :
On 08 Jan 2016, at 4:25 , Ben Coman <btc@openInWorld.com btc@openinworld.com> wrote:
On Fri, Jan 8, 2016 at 9:39 PM, Ben Coman btc@openinworld.com wrote:
On Fri, Jan 8, 2016 at 5:42 PM, stephane ducasse stephane.ducasse@gmail.com wrote:
I have a (stupid) question. Is the code running without the primitives? Are the code below the primitives correct? I asked that because we can have 100 eyes and brains on the smalltalk level and far less on the VM primitive level.
Because:
- Concurrency bugs can be subtle and the *exact* conditions can be
hard to reproduce for debugging. For example, the solution to a couple of problems with Delay [1] [2] were solved by moving away from Semaphore>>critical: to use signalling.
- The in-image atomicity of determining whether a signal was actually
consumed or not during process suspension/termination is awkward. Its seems hard to *really* know for sure it right (but I haven't looked in depth into Denis' latest proposals.)
- The existing in-image implementation of Semaphore>>critical messes
around in Process>>terminate in a *special* way that is not easy for those 100 eyes to understand. For example, I personally am not comfortable with understanding how the special Semaphore handling in Process>>terminate works, but I can easily follow how primitiveEnterCriticalSection just looking at the code [3].
Points 2 & 3 might possibly be addressed by having new primitiveWaitReturned *always* return true, so if the process is terminated while waiting, the assignment to signalConsumed doesn't occur...
critical: mutuallyExcludedBlock signalConsumed := false. [ signalConsumed := self primitiveWaitReturned. blockValue := mutuallyExcludedBlock value ] ensure: [ signalConsumed ifTrue: [self signal] ]. ^blockValue
where primitiveWait (https://git.io/vuDjd) is copied and (just guessing) the marked line added...
primitiveWaitReturned | sema excessSignals activeProc inInterpreter | sema := self stackTop. "rcvr" "==>>" self pop: argumentCount + 1 thenPush: objectMemory trueObject. "<<==" excessSignals := self fetchInteger: ExcessSignalsIndex ofObject: sema. excessSignals > 0 ifTrue: [self storeInteger: ExcessSignalsIndex ofObject: sema withValue: excessSignals - 1] ifFalse: inInterpreter := instructionPointer >= objectMemory startOfMemory. activeProc := self activeProcess. self addLastLink: activeProc toList: sema. self transferTo: self wakeHighestPriority from: CSWait. self forProcessPrimitiveReturnToExecutivePostContextSwitch: inInterpreter]
which I guess could be added quickly if Esteban could compile the latest pharo-spur-vm ;)
cheers -ben
Won't work, there's no guarantee thread has actually ran and signalConsumed been assigned the primitive result after Semaphore resumed the waiting thread, before a higher priority thread runs and terminates it. (which is exactly the case handled by special code in #terminate)
Cheers, Henry
_,,,^..^,,,_ best, Eliot
Hi Denis,
ignore this: I'm confused. I will catch up soon :-/
On Sun, Jan 17, 2016 at 11:52 AM, Eliot Miranda eliot.miranda@gmail.com wrote:
Hi Denis,
On Mon, Jan 11, 2016 at 5:58 AM, Denis Kudriashov dionisiydk@gmail.com wrote:
Eliot I found that there are no methods for simulation code (if it right name for it). So stepping over new primitives failed (which is infinite recursion for Pharo case)
Find the Squeak code attached. Merge it with the Pharo code (renaming ContextPart to COntext of course; something we'll do in Squeak eventually :-) )..
Context>>doPrimitive: primitiveIndex method: meth receiver: aReceiver args: arguments ... "Mutex>>primitiveEnterCriticalSection Mutex>>primitiveTestAndSetOwnershipOfCriticalSection" (primitiveIndex = 186 or: [primitiveIndex = 187]) ifTrue: [| active effective | active := Processor activeProcess. effective := active effectiveProcess. "active == effective" value := primitiveIndex = 186 ifTrue: [aReceiver *primitiveEnterCriticalSectionOnBehalfOf*: effective] ifFalse: [aReceiver *primitiveTestAndSetOwnershipOfCriticalSectionOnBehalfOf*: effective]. ^(self isPrimFailToken: value) ifTrue: [value] ifFalse: [self push: value]].
How this methods should be implemented?
2016-01-11 13:59 GMT+01:00 Denis Kudriashov dionisiydk@gmail.com:
Hello. I publish slice 17373 https://pharo.fogbugz.com/f/cases/17373/Mutex-should-be-based-on-VM-primitives-and-implement-critical-methods-from-Semaphore .
I introduce LockOwnership class which implements VM primitives as:
- acquire
- tryAcquire
- release
(It comment saves copyright from CriticalSection)
Mutext uses it as ownership instance variable to implement critical methods correctly:
- critical:
- critical:ifLocked:
- critical:ifError:
- critical:ifCurtailed:
For the integration process old Mutex instance variables are not removed. So loading this code should not broke current Mutex instances. But when we integrate it Mutex will have only variable ownership.
If you look critical implementation you will understand why I not like semantic of primitives. it should be inverted in future. #acquire should return true when ownership is acquired right now.
Process>>terminate now detects waiting on LockOwnership and ask it to handle wait termination.Then LockOwnership inject right value into lock state variable. Such variables should be marked with special pragma <lockAt: #ownershipVariableName tracksStateAt: 1> "index of local variable" Method can contain mulpible pragmas to reference all ownerships in method. ReadWriteLock for example needs this.
2016-01-11 13:30 GMT+01:00 Henrik Johansen <henrik.s.johansen@veloxit.no
:
On 08 Jan 2016, at 4:25 , Ben Coman <btc@openInWorld.com btc@openinworld.com> wrote:
On Fri, Jan 8, 2016 at 9:39 PM, Ben Coman btc@openinworld.com wrote:
On Fri, Jan 8, 2016 at 5:42 PM, stephane ducasse stephane.ducasse@gmail.com wrote:
I have a (stupid) question. Is the code running without the primitives? Are the code below the primitives correct? I asked that because we can have 100 eyes and brains on the smalltalk level and far less on the VM primitive level.
Because:
- Concurrency bugs can be subtle and the *exact* conditions can be
hard to reproduce for debugging. For example, the solution to a couple of problems with Delay [1] [2] were solved by moving away from Semaphore>>critical: to use signalling.
- The in-image atomicity of determining whether a signal was actually
consumed or not during process suspension/termination is awkward. Its seems hard to *really* know for sure it right (but I haven't looked in depth into Denis' latest proposals.)
- The existing in-image implementation of Semaphore>>critical messes
around in Process>>terminate in a *special* way that is not easy for those 100 eyes to understand. For example, I personally am not comfortable with understanding how the special Semaphore handling in Process>>terminate works, but I can easily follow how primitiveEnterCriticalSection just looking at the code [3].
Points 2 & 3 might possibly be addressed by having new primitiveWaitReturned *always* return true, so if the process is terminated while waiting, the assignment to signalConsumed doesn't occur...
critical: mutuallyExcludedBlock signalConsumed := false. [ signalConsumed := self primitiveWaitReturned. blockValue := mutuallyExcludedBlock value ] ensure: [ signalConsumed ifTrue: [self signal] ]. ^blockValue
where primitiveWait (https://git.io/vuDjd) is copied and (just guessing) the marked line added...
primitiveWaitReturned | sema excessSignals activeProc inInterpreter | sema := self stackTop. "rcvr" "==>>" self pop: argumentCount + 1 thenPush: objectMemory trueObject. "<<==" excessSignals := self fetchInteger: ExcessSignalsIndex ofObject: sema. excessSignals > 0 ifTrue: [self storeInteger: ExcessSignalsIndex ofObject: sema withValue: excessSignals - 1] ifFalse: inInterpreter := instructionPointer >= objectMemory startOfMemory. activeProc := self activeProcess. self addLastLink: activeProc toList: sema. self transferTo: self wakeHighestPriority from: CSWait. self forProcessPrimitiveReturnToExecutivePostContextSwitch: inInterpreter]
which I guess could be added quickly if Esteban could compile the latest pharo-spur-vm ;)
cheers -ben
Won't work, there's no guarantee thread has actually ran and signalConsumed been assigned the primitive result after Semaphore resumed the waiting thread, before a higher priority thread runs and terminates it. (which is exactly the case handled by special code in #terminate)
Cheers, Henry
_,,,^..^,,,_ best, Eliot
Hi ben
thanks for your comments. I feel better.
Stef
On 08 Jan 2016, at 14:39, Ben Coman btc@openinworld.com wrote:
On Fri, Jan 8, 2016 at 5:42 PM, stephane ducasse stephane.ducasse@gmail.com wrote:
I have a (stupid) question. Is the code running without the primitives? Are the code below the primitives correct? I asked that because we can have 100 eyes and brains on the smalltalk level and far less on the VM primitive level.
Because:
- Concurrency bugs can be subtle and the *exact* conditions can be
hard to reproduce for debugging. For example, the solution to a couple of problems with Delay [1] [2] were solved by moving away from Semaphore>>critical: to use signalling.
- The in-image atomicity of determining whether a signal was actually
consumed or not during process suspension/termination is awkward. Its seems hard to *really* know for sure it right (but I haven't looked in depth into Denis' latest proposals.)
- The existing in-image implementation of Semaphore>>critical messes
around in Process>>terminate in a *special* way that is not easy for those 100 eyes to understand. For example, I personally am not comfortable with understanding how the special Semaphore handling in Process>>terminate works, but I can easily follow how primitiveEnterCriticalSection just looking at the code [3].
- Its faster (for me, the least argument given that.
btw, Looking at the following... CriticalSection>>critical: aBlock ^self primitiveEnterCriticalSection ifTrue: [aBlock value] ifFalse: [aBlock ensure: [self primitiveExitCriticalSection]]
without intimate VM knowledge but seeing Eliot recently say "invocations of methods with primitives [...] are not suspension points, unless their primitives fail" -- I'm paranoid that since #ensure: primitive 198 always fail, there might be some some small window for a race where a process might be terminated before #primitiveExitCriticalSection can be executed. I'll take a refutation of this to improve the method comment.
The following instils more confidence...
CriticalSection2>>critical: aBlock | reEntered | reEntered := false. [ reEntered := self primitiveEnterCriticalSection. aBlock value ] ensure: [ reEntered ifFalse: [ self primitiveExitCriticalSection] ]
but performance is reduced... "#('Mutex -> 282,511 per second' 'Monitor -> 479,217 per second' 'CriticalSection -> 729,049 per second' 'CriticalSection2 -> 571,631 per second')"
Then again, any termination from within a critical section is anyhow problematic since potentially data structures are left in a indeterminate state.
[1] http://forum.world.st/Super-fast-delay-td4787257.html [2] https://pharo.fogbugz.com/default.asp?13755 [3] https://git.io/vuDnA
cheers -ben
Stef
Hi Ben,
On Thu, Jan 7, 2016 at 10:39 AM, Ben Coman btc@openinworld.comwrote:
On Fri, Jan 8, 2016 at 1:20 AM, Eliot Miranda eliot.miranda@gmail.com wrote:
and here's a version with a better class comment
On Thu, Jan 7, 2016 at 9:12 AM, Eliot Miranda eliot.miranda@gmail.com wrote:
Hi Denis, Hi Clément, Hi Frank,
On Thu, Jan 7, 2016 at 5:34 AM, Clément Bera bera.clement@gmail.com wrote:
Hello,
Eliot, please, you told me you had the code and Denis is interested.
It uses 3 primitives for performance.
Forgive the delay. I thought it proper to ask permission since the code was written while I was at Qwaq. I'm attaching the code in a fairly raw state, see the attached. The code is MIT, but copyright 3DICC.
It is a plugin replacement for Squeak's Mutex, and with a little ingenuity could be a replacement for Squeak's Monitor. It is quicker because it uses three new primitives to manage entering a critical section and setting the owner, exiting the critical section and releasing the owner, and testing if a critical section, entering if the section is unowned. The use of the primitives means fewer block activations and ensure: blocks in entering and exiting the critical section, and that's the actual cause of the speed-up.
You can benchmark the code as is. Here are some results on 32-bit Spur, on my 2.2GHz Core i7
{Mutex new. Monitor new. CriticalSection new} collect: [:cs| | n | n := 0. [cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. n ] bench]
{Mutex new. Monitor new. CriticalSection new} collect: [:cs| | n | n := 0. cs class name, ' -> ', [cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. n ] bench]
#( 'Mutex -> 440,000 per second. 2.27 microseconds per run.' 'Monitor -> 688,000 per second. 1.45 microseconds per run.' 'CriticalSection -> 1,110,000 per second. 900 nanoseconds per run.')
This is great Eliot. Thank you and 3DICC. After loading the changeset into Pharo-50515 (32 bit Spur) I get the following results on my laptop i5-2520M @ 2.50GHz
#('Mutex -> 254,047 per second' 'Monitor -> 450,442 per second' 'CriticalSection -> 683,393 per second')
In a fresh Image "Mutex allInstances basicInspect" lists just two mutexes...
- NetNameResolver-->ResolverMutex
- ThreadSafeTranscript-->accessSemaphore
I hate myself for getting distracted but I'm finding this is un. One can migrate to the new representation using normal Monticello loads by
In the first version redefine Mutex and Monitor to subclass LinkedList and have their owner/ownerProcess inst var first (actually third after firstLink & lastLink), and add the primitives.
In the next version check that all Mutex and Monitor instanes are unowned and then redefine to discard excess inst vars
Let me test this before committing, and see that all tests are ok.
cheers -ben
Replacement is probably trivial; rename Mutex to OldMutex, rename CriticalSection to Mutex, recompile. But there are lots of mutexes in the system and these are potentially owned. Transforming unowned ones is trivial, but transforming owned ones is, I think, impossible. But at least in my system there are no owned mutexes or monitors.
Frank (or anyone else), would you be interested in creating a replacement for Squeak's Monitor based on CriticalSection?
Here are the two business methods: CriticalSection methods for mutual exclusion critical: aBlock "Evaluate aBlock protected by the receiver." ^self primitiveEnterCriticalSection ifTrue: [aBlock value] ifFalse: [aBlock ensure: [self primitiveExitCriticalSection]]
critical: aBlock ifLocked: lockedBlock "Answer the evaluation of aBlock protected by the receiver. If it is already in a critical section on behalf of some other process answer the evaluation of lockedBlock." ^self primitiveTestAndSetOwnershipOfCriticalSection ifNil: [lockedBlock value] ifNotNil:[:alreadyOwner| alreadyOwner ifTrue: [aBlock value] ifFalse: [aBlock ensure: [self primitiveExitCriticalSection]]]
and the primitives: CriticalSection methods for private-primitives primitiveEnterCriticalSection "Primitive. The receiver must be unowned or owned by the current process to proceed. Answer if the process is owned by the current process." <primitive: 186> self primitiveFailed "In the spirit of the following" "[owner ifNil: [owner := Processor activeProcess. ^false]. owner = Processor activeProcess ifTrue: [^true]. self addLast: Processor activeProcess. Processor activeProcess suspend] valueUnpreemptively"
primitiveExitCriticalSection "Primitive. Set te receiver to unowned and if any processes are waiting on the receiver then proceed the first one, indicating that the receiver is unowned." <primitive: 185> self primitiveFailed "In the spirit of the following" "[owner := nil. self isEmpty ifFalse: [process := self removeFirst. process resume]] valueUnpreemptively"
primitiveTestAndSetOwnershipOfCriticalSection "Primitive. Attempt to set the ownership of the receiver. If the receiver is unowned set its owningProcess to the activeProcess and answer false. If the receiver is owned by the activeProcess answer true. If the receiver is owned by some other process answer nil." <primitive: 187> self primitiveFail "In the spirit of the following" "[owner ifNil: [owningProcess := Processor activeProcess. ^false]. owner = Processor activeProcess ifTrue: [^true]. ^nil] valueUnpreemptively"
2016-01-07 13:24 GMT+01:00 Denis Kudriashov dionisiydk@gmail.com: > > > Hello. > > I hear about new Monitor implementation based on new primitives. > Where to get it?
On Fri, Jan 8, 2016 at 9:39 PM, Ben Coman btc@openinworld.com wrote:
btw, Looking at the following... CriticalSection>>critical: aBlock ^self primitiveEnterCriticalSection ifTrue: [aBlock value] ifFalse: [aBlock ensure: [self primitiveExitCriticalSection]]
without intimate VM knowledge but seeing Eliot recently say "invocations of methods with primitives [...] are not suspension points, unless their primitives fail" -- I'm paranoid that since #ensure: primitive 198 always fail, there might be some some small window for a race where a process might be terminated before #primitiveExitCriticalSection can be executed. I'll take a refutation of this to improve the method comment.
The following instils more confidence...
CriticalSection2>>critical: aBlock | reEntered | reEntered := false. [ reEntered := self primitiveEnterCriticalSection. aBlock value ] ensure: [ reEntered ifFalse: [ self primitiveExitCriticalSection] ]
Having spent more time considering this, I found I was off track here. If a process waiting at #primitiveEnterCriticalSection is terminated, then #primitiveExitCriticalSection is erroneously executed.
And it will be the same problem with the proposal for Case17373 based on OwnedLock>>acquire
Mutex>>critical: aBlock | lockAcquiredNotHere | <lockAt: #lock tracksStateAt: 1> lockAcquiredNotHere := true. ^[ lockAcquiredNotHere := false. lockAcquiredNotHere := lock acquire. aBlock value ] ensure: [lockAcquiredNotHere ifFalse: [lock release]].
So now I believe the original #critical: proposed by Eliot is optimal. * If #primitiveEnterCriticalSection is terminated while waiting, then the #ifFalse: is never executed. * When #primitiveEnterCriticalSection returns, the inlined #ifTrue:ifFalse cant be interrupted. * #ensure is a primitive which can't be interrupted. By the time it has done its usual "fail", which I raised concern over, actually it has already done its job to make sure #primitiveExitCriticalSection is executed.
On Fri, Jan 8, 2016 at 1:20 AM, Eliot Miranda eliot.miranda@gmail.com wrote:
On Thu, Jan 7, 2016 at 5:34 AM, Clément Bera bera.clement@gmail.com wrote:
Eliot, please, you told me you had the code and Denis is interested. It uses 3 primitives for performance.
Here are the two business methods: CriticalSection methods for mutual exclusion critical: aBlock "Evaluate aBlock protected by the receiver." ^self primitiveEnterCriticalSection ifTrue: [aBlock value] ifFalse: [aBlock ensure: [self primitiveExitCriticalSection]]
critical: aBlock ifLocked: lockedBlock "Answer the evaluation of aBlock protected by the receiver. If it is already in a critical section on behalf of some other process answer the evaluation of lockedBlock." ^self primitiveTestAndSetOwnershipOfCriticalSection ifNil: [lockedBlock value] ifNotNil:[:alreadyOwner| alreadyOwner ifTrue: [aBlock value] ifFalse: [aBlock ensure: [self primitiveExitCriticalSection]]]
and the primitives: primitiveEnterCriticalSection primitiveExitCriticalSection primitiveTestAndSetOwnershipOfCriticalSection
Reading this vm code...
primitiveExitCriticalSection | criticalSection owningProcessIndex owningProcess | criticalSection := self stackTop. "rcvr" owningProcessIndex := ExcessSignalsIndex. (self isEmptyList: criticalSection) ifTrue: [objectMemory storePointerUnchecked: owningProcessIndex ofObject: criticalSection withValue: objectMemory nilObject] ifFalse: [owningProcess := self removeFirstLinkOfList: criticalSection. objectMemory storePointerUnchecked: owningProcessIndex ofObject: criticalSection withValue: owningProcess. self resume: owningProcess preemptedYieldingIf: preemptionYields]
I suspect a problem since it does not check it is the owningProcess before setting owningProcess to nil. AFAIK, only the owning process should be able to release the mutex. To demonstrate, evaluating the following...
Transcript clear. cs := CriticalSection new. sync := Semaphore new. b1 := [ Transcript crShow: '1A'. cs primitiveEnterCriticalSection. sync wait. Transcript crShow: '1B'. cs primitiveExitCriticalSection. ]. b2 := [ Transcript crShow: '2A'. cs primitiveEnterCriticalSection. Transcript crShow: '2B'. cs primitiveExitCriticalSection. ]. b3 := [ Transcript crShow: '3A'. cs primitiveExitCriticalSection. cs primitiveEnterCriticalSection. Transcript crShow: '3B'. ]. [ b1 newProcess resume. b2 newProcess resume. 2 second wait. sync signal. ] fork
correctly produces... 1A 2A "pauses here" 1B 2B
but if the last forked block replaces b2 with b3... [ b1 newProcess resume. b3 newProcess resume. 2 second wait. sync signal. ] fork
it produces incorrectly result...
1A 3A 3B "pauses here" 1B
I believe the "cs primitiveExitCriticalSection" in b3 should raise an error. Thus primitiveExitCriticalSection might be...
primitiveExitCriticalSection argumentCount > 0 ifTrue: [ criticalSection := self stackValue: 1. "rcvr" activeProc := self stackTop ] ifFalse: [ criticalSection := self stackTop. "rcvr" activeProc := self activeProcess ]. owningProcessIndex := ExcessSignalsIndex. owningProcess := objectMemory fetchPointer: owningProcessIndex ofObject: criticalSection. owningProcess = activeProc ifFalse: self primitiveFail. "<==!!!" (self isEmptyList: criticalSection) ifTrue: [ objectMemory storePointerUnchecked: owningProcessIndex ofObject: criticalSection withValue: objectMemory nilObject] ifFalse: [ owningProcess := self removeFirstLinkOfList: criticalSection. objectMemory storePointerUnchecked: owningProcessIndex ofObject: criticalSection withValue: owningProcess. self resume: owningProcess preemptedYieldingIf: preemptionYields ]
Now I've inserted the additional check without completely understanding the code around it. I've been contemplating (self isEmptyList: criticalSection) off and on for a couple of days and I'm stumped. I only guess it has something to do with a process being suspended while inside the critical section.
Also I guess the ( argumentCount > 0) is so it can be used from the debugger ??
cheers -ben
Hi.
2016-01-17 17:47 GMT+01:00 Ben Coman btc@openinworld.com:
I suspect a problem since it does not check it is the owningProcess before setting owningProcess to nil. AFAIK, only the owning process should be able to release the mutex.
Yes. But it is not mutex. It is more abstract thing which allow implement mutex and other concurrency stuff (like ReadWriteLock). That's why I rename CriticalSection to LockOwnership in my proposals. It only contains primitives and no high level methods like #critical:.
Hi Denis,
On Jan 17, 2016, at 9:01 AM, Denis Kudriashov dionisiydk@gmail.com wrote:
Hi.
2016-01-17 17:47 GMT+01:00 Ben Coman btc@openinworld.com:
I suspect a problem since it does not check it is the owningProcess before setting owningProcess to nil. AFAIK, only the owning process should be able to release the mutex.
Yes. But it is not mutex. It is more abstract thing which allow implement mutex and other concurrency stuff (like ReadWriteLock). That's why I rename CriticalSection to LockOwnership in my proposals. It only contains primitives and no high level methods like #critical:.
Sounds good. Now would be a good time to submit your preferred versions of the primitives which answer the values that make for more natural variable names. I'm about to release new VMs, including a 64-bit Spur JIT. So I could include the new primitives.
2016-01-17 18:09 GMT+01:00 Eliot Miranda eliot.miranda@gmail.com:
Sounds good. Now would be a good time to submit your preferred versions of the primitives which answer the values that make for more natural variable names. I'm about to release new VMs, including a 64-bit Spur JIT. So I could include the new primitives.
This is excellent news Eliot! And new primitives will be nice addition.
On Mon, Jan 18, 2016 at 1:09 AM, Eliot Miranda eliot.miranda@gmail.com wrote:
Hi Denis,
On Jan 17, 2016, at 9:01 AM, Denis Kudriashov dionisiydk@gmail.com wrote:
Hi.
2016-01-17 17:47 GMT+01:00 Ben Coman btc@openinworld.com:
I suspect a problem since it does not check it is the owningProcess before setting owningProcess to nil. AFAIK, only the owning process should be able to release the mutex.
Yes. But it is not mutex. It is more abstract thing which allow implement mutex and other concurrency stuff (like ReadWriteLock). That's why I rename CriticalSection to LockOwnership in my proposals.
I find LockOwnership a bit cumbersome. I guess you don't want just Lock but to include the sense of ownership. Would you consider one of these... * Locker "A person or thing that does an action indicated by the root verb" [1] * Lockee "Less commonly added to verbs to form words meaning a person or thing that is the subject of that verb (ie, who or that does an action), especially where a passive sense of the verb is implied." [2] * OwnLock * OwnedLock
[1] https://en.wiktionary.org/wiki/-er#Suffix [2] https://en.wiktionary.org/wiki/-ee#Suffix
cheers -ben
It only contains primitives and no high level methods like #critical:.
Sounds good. Now would be a good time to submit your preferred versions of the primitives which answer the values that make for more natural variable names. I'm about to release new VMs, including a 64-bit Spur JIT. So I could include the new primitives.
2016-01-17 19:10 GMT+01:00 Ben Coman btc@openinworld.com:
I find LockOwnership a bit cumbersome. I guess you don't want just Lock but to include the sense of ownership. Would you consider one of these...
- Locker "A person or thing that does an action indicated by the
root verb" [1]
- Lockee "Less commonly added to verbs to form words meaning a
person or thing that is the subject of that verb (ie, who or that does an action), especially where a passive sense of the verb is implied." [2]
- OwnLock
- OwnedLock
To choose better class name we should take into account how we will use it. Could you suggest names for #acquire and #release in context of your class name? ("locker acquire" is not good)
If people think that simple Lock is good for this purpose we can use it.
On Mon, 18 Jan 2016 11:22:15 +0100 Denis Kudriashov dionisiydk@gmail.com wrote:
To choose better class name we should take into account how we will use it. Could you suggest names for #acquire and #release in context of your class name? ("locker acquire" is not good)
If people think that simple Lock is good for this purpose we can use it.
Perhaps better to think of an "access permission" than a "lock"
uniqueAccess acquire
accessPermission acquire
...? -KenD
2016-01-18 17:06 GMT+01:00 KenD Ken.Dickey@whidbey.com:
Perhaps better to think of an "access permission" than a "lock"
uniqueAccess acquire
accessPermission acquire
...?
It is interesting. But I don't know how to make conclusion. Maybe on Sprint I can discuss it with more people
On Tue, Jan 19, 2016 at 12:06 AM, KenD Ken.Dickey@whidbey.com wrote:
On Mon, 18 Jan 2016 11:22:15 +0100 Denis Kudriashov dionisiydk@gmail.com wrote:
To choose better class name we should take into account how we will use it. Could you suggest names for #acquire and #release in context of your class name? ("locker acquire" is not good)
If people think that simple Lock is good for this purpose we can use it.
Perhaps better to think of an "access permission" than a "lock"
uniqueAccess acquire
Maybe. Except when I think of uniqueXXX what comes to mind is the Singleton pattern.
accessPermission acquire
I think this is a stretch, and invokes thoughts of security and file access.
cheers -ben
On Wed, 20 Jan 2016 22:37:07 +0800 Ben Coman btc@openinworld.com wrote:
On Tue, Jan 19, 2016 at 12:06 AM, KenD Ken.Dickey@whidbey.com wrote:
..
Perhaps better to think of an "access permission" than a "lock"
uniqueAccess acquire
Maybe. Except when I think of uniqueXXX what comes to mind is the Singleton pattern.
Well, a semaphore is not a "lock". One does not have to use it.
Given that the intended usage is a unique semaphore per resource, one wishes to acquire a unique access permission to the resource.
resourceGuardAccessPermission seems a bit long to me.
Perhaps a unique spelling: uniqAccess acquire ?
For clarity, what are your suggestions?
On Mon, Jan 18, 2016 at 6:22 PM, Denis Kudriashov dionisiydk@gmail.com wrote:
2016-01-17 19:10 GMT+01:00 Ben Coman btc@openinworld.com:
I find LockOwnership a bit cumbersome. I guess you don't want just Lock but to include the sense of ownership. Would you consider one of these...
- Locker "A person or thing that does an action indicated by the
root verb" [1]
- Lockee "Less commonly added to verbs to form words meaning a
person or thing that is the subject of that verb (ie, who or that does an action), especially where a passive sense of the verb is implied." [2]
- OwnLock
- OwnedLock
To choose better class name we should take into account how we will use it. Could you suggest names for #acquire and #release in context of your class name? ("locker acquire" is not good)
You're right. I didn't think it through. I don't have any good alternative message names.
If people think that simple Lock is good for this purpose we can use it.
I think Lock is probably okay but survey a few more opinions. However a generic lock as introduced in most CS lessons that turned up in searches don't have an owner, so maybe my last suggestion OwnedLock is also. For example... lock := OwnedLock new. lock aquire
cheers -ben
I have a (stupid) question. Is the code running without the primitives? Are the code below the primitives correct? I asked that because we can have 100 eyes and brains on the smalltalk level and far less on the VM primitive level.
Stef
2016-01-08 13:06 GMT+01:00 stepharo stepharo@free.fr:
I have a (stupid) question. Is the code running without the primitives? Are the code below the primitives correct? I asked that because we can have 100 eyes and brains on the smalltalk level and far less on the VM primitive level.
Eliot code is used VM primitives for critical sections.
2016-01-08 13:36 GMT+01:00 Denis Kudriashov dionisiydk@gmail.com:
2016-01-08 13:06 GMT+01:00 stepharo stepharo@free.fr:
I have a (stupid) question. Is the code running without the primitives? Are the code below the primitives correct? I asked that because we can have 100 eyes and brains on the smalltalk level and far less on the VM primitive level.
Eliot code is used VM primitives for critical sections.
And it seams that without primitive it is impossible to implement #critical:ifLocked:. Current implementation is not guard against caller blocking
On 7 January 2016 at 17:12, Eliot Miranda eliot.miranda@gmail.com wrote:
Hi Denis, Hi Clément, Hi Frank,
On Thu, Jan 7, 2016 at 5:34 AM, Clément Bera bera.clement@gmail.com wrote:
Hello,
Eliot, please, you told me you had the code and Denis is interested.
It uses 3 primitives for performance.
Forgive the delay. I thought it proper to ask permission since the code was written while I was at Qwaq. I'm attaching the code in a fairly raw state, see the attached. The code is MIT, but copyright 3DICC.
It is a plugin replacement for Squeak's Mutex, and with a little ingenuity could be a replacement for Squeak's Monitor. It is quicker because it uses three new primitives to manage entering a critical section and setting the owner, exiting the critical section and releasing the owner, and testing if a critical section, entering if the section is unowned. The use of the primitives means fewer block activations and ensure: blocks in entering and exiting the critical section, and that's the actual cause of the speed-up.
You can benchmark the code as is. Here are some results on 32-bit Spur, on my 2.2GHz Core i7
{Mutex new. Monitor new. CriticalSection new} collect: [:cs| | n | n := 0. [cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n
- 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1].
cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. n ] bench]
{Mutex new. Monitor new. CriticalSection new} collect: [:cs| | n | n := 0. cs class name, ' -> ', [cs critical: [n := n + 1]. cs critical: [n := n + 1]. cs critical: [n := n
- 1]. cs critical: [n := n + 1]. cs critical: [n := n + 1].
cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. cs critical: [n := n - 1]. n ] bench]
#( 'Mutex -> 440,000 per second. 2.27 microseconds per run.' 'Monitor -> 688,000 per second. 1.45 microseconds per run.' 'CriticalSection -> 1,110,000 per second. 900 nanoseconds per run.')
Replacement is probably trivial; rename Mutex to OldMutex, rename CriticalSection to Mutex, recompile. But there are lots of mutexes in the system and these are potentially owned. Transforming unowned ones is trivial, but transforming owned ones is, I think, impossible. But at least in my system there are no owned mutexes or monitors.
Frank (or anyone else), would you be interested in creating a replacement for Squeak's Monitor based on CriticalSection?
It sounds like an interesting problem. I watched the thread unfold and I'm still not massively clear on what is required though? (Apologies for the delay in responding: my cup overfloweth at the moment, but should resume normal levels in a month or so.)
frank
On Thu, Jan 7, 2016 at 8:24 PM, Denis Kudriashov dionisiydk@gmail.com wrote:
Hello.
I hear about new Monitor implementation based on new primitives. Where to get it?
A while ago Eliot mentioned some mutex primitives. I didn't follow up then, but maybe now is the right time to try these. I can't find the post, but looking just now I found some likely candidates at [1]: - 185 primitiveExitCriticalSection - 186 primitiveEnterCriticalSection - 187 primitiveTestAndSetOwnershipOfCriticalSection
However I can't find any image side code in Squeak or Newspeak.
[1] https://github.com/pharo-project/pharo-vm/search?utf8=%E2%9C%93&q=critic...
cheers -ben
vm-dev@lists.squeakfoundation.org