<div class="gmail_quote">Hi all,<br>
this is my first post here, just joined this group so let me do a<br>
quick introduction.<br>
I have 10+ years of experience doing full-time development with<br>
VisualWorks (creating trading platforms for the European energy<br>
exchanges).<br>
Also I have near-zero experience with using Squeak or how its<br>
development process works. I'll slowly cut my teeth, starting by<br>
interacting on this mailing list.<br>
<br>
<br>
Multi threading is something I have spent a lot of quality time with,<br>
so I want to share some thoughts on the following:<br>
<div class="im"><br>
|semaphores tr|<br>
semaphores := Array new: 10.<br>
tr := ThreadSafeTranscript new.<br>
tr open.<br>
</div>1 to: 10 do: [ :index | semaphores at: index put: Semaphore new ].<br>
<div class="im"><br>
1 to: 10 do: [:i |<br>
[<br>
</div> tr nextPutAll: i printString, ' fork '; cr.<br>
<div class="im"> (semaphores at: i) signal.<br>
] fork<br>
].<br>
<br>
semaphores do: [:each | each wait ].<br>
tr show: 'all forks proccesed'; cr.<br>
<br>
<br>
</div>I have seen this pattern often (allocating a semaphore for every<br>
forked process), I usually interpret this as a signal that such code<br>
is still in its first 'make it work/make it right' stages.<br>
What a lot of people don't realize is that at its heart a semapore is<br>
a thread-safe counter/register (and if you look at the hierarchy it is<br>
implemented on you wouldn't guess that either, since the hierarchy<br>
stresses the implementation part that manages waiting processes rather<br>
than the counter aspect).<br>
<br>
So trying to take the code snippet toward 'make it abstract' territory<br>
this could be refactored to lean more on the counter aspect of<br>
semaphores and use only a single semaphore:<br>
<br>
<br>
|count sem tr|<br>
<div class="im">tr := ThreadSafeTranscript new.<br>
tr open.<br>
</div>count := 10.<br>
sem := Semaphore new.<br>
<br>
1 to: count do: [:i |<br>
[ tr nextPutAll: (i printString, ' fork\') withCRs.<br>
sem signal.<br>
] fork].<br>
<br>
count timesRepeat: [sem wait].<br>
<div class="im">tr show: 'all forks proccesed'; cr.<br>
<br>
<br>
<br>
</div>Now the above is about as far as you can go with the current Squeak<br>
and VisualWorks implementations so you can take it as a simple<br>
refactoring advise.<br>
<br>
<br>
<br>
<br>
However I want to press on a bit more (and go a bit off-topic for this<br>
list ;-) because I feel it still has a big problem: we need to<br>
maintain a 'count' and pass that between the two loops in the above<br>
example.<br>
In the current example this is not much of a problem but in more<br>
complex applications where the forking is done by yet other forked<br>
processes we will need to make 'count' thread-safe as well -- I find<br>
this very ugly, because you will need an extra semaphore just to make<br>
the original semaphore work as required.<br>
Furthermore you cannot add new forked processes once the second loop<br>
has started running.<br>
<br>
So here is an experiment I did a couple of years ago with VisualWorks:<br>
I altered the VM (just one line of its source ;-) so it would react<br>
properly to semaphores that have negative values in the<br>
'excessSignals' instance variable, and I added a method #unsignal to<br>
Semaphore that would decrease the value of that ivar.<br>
<br>
In my experiments that yielded many opportunities to simplify<br>
multiprocessing code (not only for thread synchronization but also for<br>
passing around counts in a thread-safe register!).<br>
<br>
In the above code that would allow us to 'pre-load' the semaphore at<br>
the place where the threads are created with as result that the<br>
'count' variable can be removed and the bottom loop can be removed too:<br>
<br>
<br>
|count sem tr|<br>
<div class="im">tr := ThreadSafeTranscript new.<br>
tr open.<br>
</div>sem := Semaphore forMutualExclusion. "We need one excessSignal to<br>
balance the #wait below"<br>
<div class="im"><br>
1 to: 10 do: [:i |<br>
</div> sem unsignal. "outside the forked code"<br>
[ tr nextPutAll: (i printString, ' fork\') withCRs.<br>
sem signal. "balance the unsignal"<br>
] fork].<br>
<br>
sem wait. "no loop, no need to know the count!"<br>
<div class="im">tr show: 'all forks proccesed'; cr.<br>
<br>
<br>
<br>
<br>
<br>
</div>Above was just a simple refactoring, but look at how I needed it:<br>
<br>
|sem tr|<br>
<div class="im">tr := ThreadSafeTranscript new.<br>
tr open.<br>
</div>sem := Semaphore new. "no excessSignal this time"<br>
<br>
"set up a monitoring system first(!)"<br>
[ sem wait.<br>
<div class="im"> tr show: 'all forks proccesed'; cr<br>
</div>] fork.<br>
<br>
"then create jobs (in my case I had only a single first job that would<br>
recursively create more jobs, not shown here)"<br>
<div class="im"><br>
1 to: 10 do: [:i |<br>
</div> sem unsignal.<br>
[ tr nextPutAll: (i printString, ' fork\') withCRs.<br>
sem signal.<br>
] fork].<br>
"Now that we are sure at least one job is entered balance the #wait we<br>
started out with"<br>
sem signal.<br>
<br>
<br>
Since we elided 'count' I can move the code that relied on it up in<br>
front of the thread creation code, I very much like this flavor of<br>
decoupling.<br>
<br>
<br>
I guess this illustrates that Semaphore is stuck in the 'make it work/<br>
make it right' phase for thirty years now, and that moving it into<br>
'make it abstract' territory will make lots of hairy multi-threading<br>
code much simpler to express...<br>
<br>
(And for those thinking this through: yes I did implement a thread-<br>
safe #add: and #valueWithReset on semaphore too ;-)<br>
<br>
<br>
I hope I didn't bore y'all and stray to far off-topic, but I did want<br>
to share this bit of insight I gained by tinkering with the semaphore<br>
implementation: semaphores are thread-safe counters at their heart.<br>
<br>
<br>
<br>
Cheers,<br>
<br>
Reinout<br>
-------<br>
<br>
PS: big congrats with the license cleaning milestone, this is what<br>
finally pulled me into this project :-)<br>
<div><div></div><div class="h5"><br>
<br>
_______________________________________________<br>
Pharo-project mailing list<br>
<a href="mailto:Pharo-project@lists.gforge.inria.fr">Pharo-project@lists.gforge.inria.fr</a><br>
<a href="http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project" target="_blank">http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project</a><br>
</div></div></div><br>