...instead of "Process allSubInstances". The caveat is this: process creation gets a little bit slower while accessing the list of runnable *and* waiting processes gets about 40x faster:
[ [] newProcess terminate ] bench. AFTER: '140,000 per second. 7.13 microseconds per run. 35.02 % GC time.' BEFORE: '224,000 per second. 4.47 microseconds per run. 4.38 % GC time.'
[ aProcessBrowser updateProcessList ] bench. -- allow quick updates AFTER: '4,410 per second. 227 microseconds per run. 77.34453 % GC time.' BEFORE: '118 per second. 8.47 milliseconds per run. 2.59792 % GC time.'
Why do we need this? If you have an auto-updating ProcessBrowser running in the background while doing time-sensitive benchmarks, you want a fast ProcessBrowser :-)
Please see the attached change set. Thanks to Leon and Eliot for the hint!
Best, Marcel
Hi all --
So, the overall discussion is about how to access the list of all processes to be used in tools such as the ProcessExplorer. We currently do this via "Process allSubInstances" to then filter the terminated ones. Then we put this again into a WeakArray and extract labels (for non-nil elements) for the tool.
While working on the multi-threaded OSVM, Leon noticed a slowdown during benchmarks whenever an existing ProcessBrowser was self-updating during the benchmark. While this depends on the current heap size, we were wondering whether there should be a faster way to get the list of processes like getting the list of classes for code-browsing tools.
The thing is that maintaining, for example, a WeakSet during process creation puts pressure on the GC, which might be annoying for process-intensive applications such as Tweak. Having 64-bit images already and once multi-threading is up and running for everybody, we are likely to observe a larger footprint for "Process allSubInstances" ...
... why #allSubInstances anyway? Who dares to subclass Process? :-)
Best, Marcel
Am 28.03.2024 14:39:48 schrieb Marcel marcel.taeumel@hpi.uni-potsdam.de:
...instead of "Process allSubInstances". The caveat is this: process creation gets a little bit slower while accessing the list of runnable *and* waiting processes gets about 40x faster:
[ [] newProcess terminate ] bench. AFTER: '140,000 per second. 7.13 microseconds per run. 35.02 % GC time.' BEFORE: '224,000 per second. 4.47 microseconds per run. 4.38 % GC time.'
[ aProcessBrowser updateProcessList ] bench. -- allow quick updates AFTER: '4,410 per second. 227 microseconds per run. 77.34453 % GC time.' BEFORE: '118 per second. 8.47 milliseconds per run. 2.59792 % GC time.'
Why do we need this? If you have an auto-updating ProcessBrowser running in the background while doing time-sensitive benchmarks, you want a fast ProcessBrowser :-)
Please see the attached change set. Thanks to Leon and Eliot for the hint!
Best, Marcel
From what I can tell, +1 for the idea. :-) Just two times two cents from me:
1. #all(Sub)Instances reads like a lot of global state. Given that we already have Processor, could we maybe store the list of processes there? (In a future with a truely distributed VM, this could also allow to scope a process browser to a single processor ... maybe ...) Or are those two different semantics?
(On a side note, when playing with speculative execution in SimulationStudio/Sandbox, e.g., by exploring the protocol of BlockClosure in a SimulationMethodFinder or SimulationProtocolExplorer, I often see a vast amount of irrelevant Process instances in the process browser that were created temporarily during speculative execution of a selector, will never be scheduled, and not yet been gc'ed. This is a good example of how global state access in tools becomes impractical. But on the other hand, listing all including unscheduled processes seems to be an explicit requirement to the process browser ...)
2.
... why #allSubInstances anyway? Who dares to subclass Process? :-)
Please don't get started to seal classes in Squeak. :o Openness by default! As I have proposed in another debate (we were talking about subclasses of workspaces), IMHO #allInstances(Do:) should be used with extreme caution but the default should be #allSubInstances(Do:). Similarly to how we almost always use #isKindOf: rather than #isMemberOf:. Surely there are subinstances of Process out there that customize or restrict suspending, debugging, or environment handling ... or maybe just override #printOn:. Or at least, they could be. Let's not kill those dreams unless unnecessary. :D Luckily, your implementation still supports subinstances with no extra costs.
Best, Christoph
--- Sent from Squeak Inbox Talk
On 2024-04-03T12:40:16+00:00, marcel.taeumel@hpi.de wrote:
Hi all --
So, the overall discussion is about how to access the list of all processes to be used in tools such as the ProcessExplorer. We currently do this via "Process allSubInstances" to then filter the terminated ones. Then we put this again into a WeakArray and extract labels (for non-nil elements) for the tool.
While working on the multi-threaded OSVM, Leon noticed a slowdown during benchmarks whenever an existing ProcessBrowser was self-updating during the benchmark. While this depends on the current heap size, we were wondering whether there should be a faster way to get the list of processes like getting the list of classes for code-browsing tools.
The thing is that maintaining, for example, a WeakSet during process creation puts pressure on the GC, which might be annoying for process-intensive applications such as Tweak. Having 64-bit images already and once multi-threading is up and running for everybody, we are likely to observe a larger footprint for "Process allSubInstances" ...
... why #allSubInstances anyway? Who dares to subclass Process? :-)
Best, Marcel
Am 28.03.2024 14:39:48 schrieb Marcel <marcel.taeumel(a)hpi.uni-potsdam.de>:
...instead of "Process allSubInstances". The caveat is this: process creation gets a little bit slower while accessing the list of runnable *and* waiting processes gets about 40x faster:
[ [] newProcess terminate ] bench. AFTER: '140,000 per second. 7.13 microseconds per run. 35.02 % GC time.' BEFORE: '224,000 per second. 4.47 microseconds per run. 4.38 % GC time.'
[ aProcessBrowser updateProcessList ] bench. -- allow quick updates AFTER: '4,410 per second. 227 microseconds per run. 77.34453 % GC time.' BEFORE: '118 per second. 8.47 milliseconds per run. 2.59792 % GC time.'
Why do we need this? If you have an auto-updating ProcessBrowser running in the background while doing time-sensitive benchmarks, you want a fast ProcessBrowser :-)
Please see the attached change set. Thanks to Leon and Eliot for the hint!
Best, Marcel
Hi Marcel,
On Apr 3, 2024, at 5:40 AM, Taeumel, Marcel via Squeak-dev squeak-dev@lists.squeakfoundation.org wrote:
Hi all --
So, the overall discussion is about how to access the list of all processes to be used in tools such as the ProcessExplorer. We currently do this via "Process allSubInstances" to then filter the terminated ones. Then we put this again into a WeakArray and extract labels (for non-nil elements) for the tool.
While working on the multi-threaded OSVM, Leon noticed a slowdown during benchmarks whenever an existing ProcessBrowser was self-updating during the benchmark. While this depends on the current heap size, we were wondering whether there should be a faster way to get the list of processes like getting the list of classes for code-browsing tools.
The thing is that maintaining, for example, a WeakSet during process creation puts pressure on the GC, which might be annoying for process-intensive applications such as Tweak. Having 64-bit images already and once multi-threading is up and running for everybody, we are likely to observe a larger footprint for "Process allSubInstances" ...
The GC is designed to absorb pressure ;-) Have you measured the cost of maintaining a weak set of processes? I think you’ll find the overhead is very low. The management of weak arrays is essentially to defer scanning them until all other objects have been scanned. This is cheap to do. Measurement should be preferred to speculation.
... why #allSubInstances anyway? Who dares to subclass Process? :-)
Best, Marcel
Am 28.03.2024 14:39:48 schrieb Marcel marcel.taeumel@hpi.uni-potsdam.de:
...instead of "Process allSubInstances". The caveat is this: process creation gets a little bit slower while accessing the list of runnable *and* waiting processes gets about 40x faster:
[ [] newProcess terminate ] bench. AFTER: '140,000 per second. 7.13 microseconds per run. 35.02 % GC time.' BEFORE: '224,000 per second. 4.47 microseconds per run. 4.38 % GC time.'
[ aProcessBrowser updateProcessList ] bench. -- allow quick updates AFTER: '4,410 per second. 227 microseconds per run. 77.34453 % GC time.' BEFORE: '118 per second. 8.47 milliseconds per run. 2.59792 % GC time.'
Why do we need this? If you have an auto-updating ProcessBrowser running in the background while doing time-sensitive benchmarks, you want a fast ProcessBrowser :-)
Please see the attached change set. Thanks to Leon and Eliot for the hint!
Best, Marcel
Hi Christoph --
IMHO #allInstances(Do:) should be used with extreme caution but the
default should be #allSubInstances(Do:).
Well, given that there are many cases where inheritance is used but composition would be better (see uses of #shouldNotImplement), I think that #allInstances might be the wiser choice, unless one specifically designs a hierarchy for extension-by-subclass.
Seriously: What's a valid case for subclassing Process? Somebody remembers something from the past? Process is a concern quite close to the VM.
Best, Marcel
Am 04.05.2024 um 18:26 schrieb christoph.thiede@student.hpi.uni-potsdam.de:
IMHO #allInstances(Do:) should be used with extreme caution but the default should be #allSubInstances(Do:).
On Fri, May 17, 2024 at 02:33 Marcel Taeumel via Squeak-dev < squeak-dev@lists.squeakfoundation.org> wrote:
Seriously: What's a valid case for subclassing Process? Somebody remembers something from the past? Process is a concern quite close to the VM.
Take a look at Tweak. Its whole execution mechanism is based on “script processes” which are a subclass of Process.
Vanessa
One of the important things to get right is making sure the VM code uses the properly effective approach to testing for 'processness'; in some places a very direct "class == ClassProcessOOP" is needed, in others "class isKindOf: Process" is more what you need (and is much more expensive, so don't do it in a loop!)
tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim You can swear at the keyboard and it won't be offended. It was going to treat you badly anyway
Good news, class Process still appears in the special objects array but it has not been used either in the interpreter VM or in opensmalltalk-vm for the last twenty years or so. I'm pretty sure we can nil it out in the special objects array without anyone noticing, and that would be a good thing to do in order to document the fact that it should not be used any more.
For reference see Interpreter>>primitiveResume in VMMaker-tpr.2, which (I think) shows the last reference to ClassProcess having been commented out some time before 2004:
primitiveResume
| proc | proc _ self stackTop. "rcvr" "self success: ((self fetchClassOf: proc) = (self splObj: ClassProcess))." successFlag ifTrue: [ self resume: proc ].
Overall, I would expect that it is (and should be) safe to subclass Process just as long as the order of existing instance variables does not get changed.
Dave
--- Sent from Squeak Inbox Talk
On 2024-05-17T10:00:06-07:00, tim@rowledge.org wrote:
One of the important things to get right is making sure the VM code uses the properly effective approach to testing for 'processness'; in some places a very direct "class == ClassProcessOOP" is needed, in others "class isKindOf: Process" is more what you need (and is much more expensive, so don't do it in a loop!)
tim
tim Rowledge; tim(a)rowledge.org; http://www.rowledge.org/tim You can swear at the keyboard and it won't be offended. It was going to treat you badly anyway
On 2024-05-17, at 1:32 PM, lewis@mail.msen.com wrote:
Good news, class Process still appears in the special objects array but it has not been used either in the interpreter VM or in opensmalltalk-vm for the last twenty years or so. I'm pretty sure we can nil it out in the special objects array without anyone noticing, and that would be a good thing to do in order to document the fact that it should not be used any more.
Well that certainly simplifies.
For reference see Interpreter>>primitiveResume in VMMaker-tpr.2, which (I think) shows the last reference to ClassProcess having been commented out some time before 2004:
Eeek. So very long ago...
tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Useful Latin Phrases:- Nullo metro compositum est = It doesn't rhyme.
I put System-dtl.1455 in the inbox to remove Process from the special objects array. I'll leave in in inbox for a couple of days in case there is something I am overlooking (very possible with something like this) but my hope is that cleaning this up will make class Process seem less special.
Dave
--- Sent from Squeak Inbox Talk
On 2024-05-17T14:36:37-07:00, tim@rowledge.org wrote:
On 2024-05-17, at 1:32 PM, lewis(a)mail.msen.com wrote:
Good news, class Process still appears in the special objects array but it has not been used either in the interpreter VM or in opensmalltalk-vm for the last twenty years or so. I'm pretty sure we can nil it out in the special objects array without anyone noticing, and that would be a good thing to do in order to document the fact that it should not be used any more.
Well that certainly simplifies.
For reference see Interpreter>>primitiveResume in VMMaker-tpr.2, which (I think) shows the last reference to ClassProcess having been commented out some time before 2004:
Eeek. So very long ago...
tim
tim Rowledge; tim(a)rowledge.org; http://www.rowledge.org/tim Useful Latin Phrases:- Nullo metro compositum est = It doesn't rhyme.
squeak-dev@lists.squeakfoundation.org
-
christoph.thiede＠student.hpi.uni-potsdam.de -
Eliot Miranda -
lewis＠mail.msen.com -
Marcel Taeumel -
Taeumel, Marcel
-
Tim Rowledge -
Vanessa Freudenberg