Is "params size >= 54" supposed to be a check whether its running Spur or Cog? Note that we have Smalltalk isRunningSpur for that check. Besides that, however, is the question whether this shouldn't just be on the 4.6 branch?
Dave, I've noticed this recurring theme that you don't want to move to Spur. Is it true? Why?
On Thu, Jan 28, 2016 at 10:37 PM, commits@source.squeak.org wrote:
David T. Lewis uploaded a new version of System to project The Trunk: http://source.squeak.org/trunk/System-dtl.790.mcz
==================== Summary ====================
Name: System-dtl.790 Author: dtl Time: 29 January 2016, 11:37:48.065507 pm UUID: 1ccde720-42a2-41fe-bcc4-cb9694d5af07 Ancestors: System-eem.789
Update vmStatisticsReportString to restore memory stats when running on pre-spur VM.
=============== Diff against System-eem.789 ===============
Item was changed: ----- Method: SmalltalkImage>>vmStatisticsReportString (in category 'vm statistics') ----- vmStatisticsReportString "(Workspace new contents: Smalltalk vmStatisticsReportString) openLabel: 'VM Statistics'" "StringHolderView open: (StringHolder new contents: Smalltalk vmStatisticsReportString) label: 'VM Statistics'"
| params oldSpaceEnd youngSpaceEnd memorySize fullGCs fullGCTime incrGCs incrGCTime tenureCount upTime upTime2 fullGCs2 fullGCTime2 incrGCs2 incrGCTime2 tenureCount2 str freeSize youngSize used | params := self getVMParameters. oldSpaceEnd := params at: 1. youngSpaceEnd := params at: 2. memorySize := params at: 3. fullGCs := params at: 7. fullGCTime := params at: 8. incrGCs := params at: 9. incrGCTime := params at: 10. tenureCount := params at: 11. upTime := (params at: 20) ~= 0 "utcMicrosecondClock at startupp in later Spur VMs" ifTrue: [Time utcMicrosecondClock - (params at: 20) + 500 // 1000] ifFalse: [Time eventMillisecondClock]. "Fall back on old microsecond clock; Good for 47.5 days" str := WriteStream on: (String new: 700). str nextPutAll: 'uptime '; print: (upTime / 1000 / 60 // 60); nextPut: $h; print: (upTime / 1000 / 60 \\ 60) asInteger; nextPut: $m; print: (upTime / 1000 \\ 60) asInteger; nextPut: $s; cr. str nextPutAll: 'memory '; nextPutAll: memorySize asStringWithCommas; nextPutAll: ' bytes'; cr. str nextPutAll: ' old '; nextPutAll: oldSpaceEnd asStringWithCommas; nextPutAll: ' bytes ('; print: (oldSpaceEnd / memorySize * 100) maxDecimalPlaces: 1; nextPutAll: '%)'; cr.
youngSize := params at: 44.str nextPutAll: ' young ';nextPutAll: youngSize asStringWithCommas; nextPutAll: ' bytes (';print: youngSize / memorySize * 100 maxDecimalPlaces: 1; nextPutAll: '%)'; cr.youngSize := youngSpaceEnd. "used eden"freeSize := (params at: 54) + (params at: 44) - youngSize.used := youngSize + oldSpaceEnd - freeSize.str nextPutAll: ' used ';nextPutAll: used asStringWithCommas; nextPutAll: ' bytes (';print: used / memorySize * 100 maxDecimalPlaces: 1; nextPutAll: '%)'; cr.str nextPutAll: ' free ';nextPutAll: freeSize asStringWithCommas; nextPutAll: ' bytes (';print: freeSize / memorySize * 100 maxDecimalPlaces: 1; nextPutAll: '%)'; cr.
params size >= 54 ifTrue: [ "Spur object memory"youngSize := params at: 44.str nextPutAll: ' young ';nextPutAll: youngSize asStringWithCommas; nextPutAll: ' bytes (';print: youngSize / memorySize * 100 maxDecimalPlaces: 1; nextPutAll: '%)'; cr.youngSize := youngSpaceEnd. "used eden"freeSize := (params at: 54) + (params at: 44) - youngSize.used := youngSize + oldSpaceEnd - freeSize.str nextPutAll: ' used ';nextPutAll: used asStringWithCommas; nextPutAll: ' bytes (';print: used / memorySize * 100 maxDecimalPlaces: 1; nextPutAll: '%)'; cr.str nextPutAll: ' free ';nextPutAll: freeSize asStringWithCommas; nextPutAll: ' bytes (';print: freeSize / memorySize * 100 maxDecimalPlaces: 1; nextPutAll: '%)'; cr] ifFalse: [ "Earlier VM and V3 object memory"str nextPutAll: ' young ';nextPutAll: (youngSpaceEnd - oldSpaceEnd) asStringWithCommas; nextPutAll: ' bytes (';print: (youngSpaceEnd - oldSpaceEnd / memorySize * 100) maxDecimalPlaces: 1; nextPutAll: '%)'; cr.str nextPutAll: ' used ';nextPutAll: youngSpaceEnd asStringWithCommas; nextPutAll: ' bytes (';print: (youngSpaceEnd / memoryEnd * 100) maxDecimalPlaces: 1; nextPutAll: '%)'; cr.str nextPutAll: ' free ';nextPutAll: (memoryEnd - youngSpaceEnd) asStringWithCommas; nextPutAll: ' bytes (';print: (memoryEnd - youngSpaceEnd / memoryEnd * 100) maxDecimalPlaces: 1; nextPutAll: '%)'; cr.].str nextPutAll: 'GCs '; nextPutAll: (fullGCs + incrGCs) asStringWithCommas. fullGCs + incrGCs > 0 ifTrue: [ str nextPutAll: ' ('; print: (upTime / (fullGCs + incrGCs)) maxDecimalPlaces: 1; nextPutAll: ' ms between GCs)' ]. str cr. str nextPutAll: ' full '; nextPutAll: fullGCs asStringWithCommas; nextPutAll: ' totalling '; nextPutAll: fullGCTime asStringWithCommas; nextPutAll: ' ms ('; print: (fullGCTime / upTime * 100) maxDecimalPlaces: 1; nextPutAll: '% uptime)'. fullGCs = 0 ifFalse: [str nextPutAll: ', avg '; print: (fullGCTime / fullGCs) maxDecimalPlaces: 1; nextPutAll: ' ms']. str cr. str nextPutAll: ' incr '; nextPutAll: incrGCs asStringWithCommas; nextPutAll: ' totalling '; nextPutAll: incrGCTime asStringWithCommas; nextPutAll: ' ms ('; print: (incrGCTime / upTime * 100) maxDecimalPlaces: 1; nextPutAll: '% uptime), avg '; print: (incrGCTime / incrGCs) maxDecimalPlaces: 1; nextPutAll: ' ms'; cr. str nextPutAll: ' tenures '; nextPutAll: tenureCount asStringWithCommas. tenureCount = 0 ifFalse: [str nextPutAll: ' (avg '; print: incrGCs // tenureCount; nextPutAll: ' GCs/tenure)']. str cr.LastStats ifNil: [LastStats := Array new: 6] ifNotNil: [ upTime2 := upTime - (LastStats at: 1). fullGCs2 := fullGCs - (LastStats at: 2). fullGCTime2 := fullGCTime - (LastStats at: 3). incrGCs2 := incrGCs - (LastStats at: 4). incrGCTime2 := incrGCTime - (LastStats at: 5). tenureCount2 := tenureCount - (LastStats at: 6).
str nextPutAll: self textMarkerForShortReport ; nextPutAll: (fullGCs2 + incrGCs2) asStringWithCommas. fullGCs2 + incrGCs2 > 0 ifTrue: [ str nextPutAll: ' ('; print: upTime2 // (fullGCs2 + incrGCs2); nextPutAll: ' ms between GCs)'. ]. str cr. str nextPutAll: ' uptime '; print: (upTime2 / 1000.0) maxDecimalPlaces: 1; nextPutAll: ' s'; cr. str nextPutAll: ' full '; nextPutAll: fullGCs2 asStringWithCommas; nextPutAll: ' totalling '; nextPutAll: fullGCTime2 asStringWithCommas; nextPutAll: ' ms ('; print: (fullGCTime2 / upTime2 * 100) maxDecimalPlaces: 1; nextPutAll: '% uptime)'. fullGCs2 = 0 ifFalse: [str nextPutAll: ', avg '; print: (fullGCTime2 / fullGCs2) maxDecimalPlaces: 1; nextPutAll: ' ms']. str cr. str nextPutAll: ' incr '; nextPutAll: incrGCs2 asStringWithCommas; nextPutAll: ' totalling '; nextPutAll: incrGCTime2 asStringWithCommas; nextPutAll: ' ms ('; print: (incrGCTime2 / upTime2 * 100) maxDecimalPlaces: 1; nextPutAll: '% uptime), avg '. incrGCs2 > 0 ifTrue: [ str print: (incrGCTime2 / incrGCs2) maxDecimalPlaces: 1; nextPutAll: ' ms' ]. str cr. str nextPutAll: ' tenures '; nextPutAll: tenureCount2 asStringWithCommas. tenureCount2 = 0 ifFalse: [str nextPutAll: ' (avg '; print: incrGCs2 // tenureCount2; nextPutAll: ' GCs/tenure)']. str cr.]. LastStats at: 1 put: upTime. LastStats at: 2 put: fullGCs. LastStats at: 3 put: fullGCTime. LastStats at: 4 put: incrGCs. LastStats at: 5 put: incrGCTime. LastStats at: 6 put: tenureCount.
^ str contents!
Is "params size >= 54" supposed to be a check whether its running Spur or Cog? Note that we have Smalltalk isRunningSpur for that check. Besides that, however, is the question whether this shouldn't just be on the 4.6 branch?
The size of the parameter array varies in different VM code bases.
Dave, I've noticed this recurring theme that you don't want to move to Spur. Is it true? Why?
No, that is not true.
I like to maintain backward compatibility where possible, and I find that a little bit of effort on that front can prevent a great deal of pain and confusion down the road. Many years of maintaining OSProcess over a wide range image versions says I am not wrong. There is no need any ConfigurationOf or SqueakMap updates, you can pretty much load it into any image and it just works.
Dave
On Thu, Jan 28, 2016 at 10:37 PM, commits@source.squeak.org wrote:
David T. Lewis uploaded a new version of System to project The Trunk: http://source.squeak.org/trunk/System-dtl.790.mcz
==================== Summary ====================
Name: System-dtl.790 Author: dtl Time: 29 January 2016, 11:37:48.065507 pm UUID: 1ccde720-42a2-41fe-bcc4-cb9694d5af07 Ancestors: System-eem.789
Update vmStatisticsReportString to restore memory stats when running on pre-spur VM.
=============== Diff against System-eem.789 ===============
Hi David,
On Jan 29, 2016, at 10:25 AM, David T. Lewis lewis@mail.msen.com wrote:
Is "params size >= 54" supposed to be a check whether its running Spur or Cog? Note that we have Smalltalk isRunningSpur for that check. Besides that, however, is the question whether this shouldn't just be on the 4.6 branch?
The size of the parameter array varies in different VM code bases.
Dave, I've noticed this recurring theme that you don't want to move to Spur. Is it true? Why?
No, that is not true.
I like to maintain backward compatibility where possible, and I find that a little bit of effort on that front can prevent a great deal of pain and confusion down the road. Many years of maintaining OSProcess over a wide range image versions says I am not wrong. There is no need any ConfigurationOf or SqueakMap updates, you can pretty much load it into any image and it just works.
I get that and I don't want to debilitate developing packages that run I both variants. But in the case of memory management the two systems are very different. Spur can't support object age. Spur provides pinning. The two have very different garbage collectors and compactors. In developing policy code to control the gcs it will inevitably require a different approach to effectively control either. So in the realm of GC (including this, which is reporting GC stats) I think it makes sense to let them diverge. AFAIA OSProcess doesn't depend on GC control facilities so it shouldn't affect you.
Where it may effect you is that pinning could conceivably allow you to lift up parameter processing to create the pointer arrays from that rather complex primitive into the image, given an addressOf: prim. Just as I hope the threaded FFI will make it possible to interact with output pipes much more reliably and efficiently.
Dave
_,,,^..^,,,_ (phone) best, Eliot
On Thu, Jan 28, 2016 at 10:37 PM, commits@source.squeak.org wrote: David T. Lewis uploaded a new version of System to project The Trunk: http://source.squeak.org/trunk/System-dtl.790.mcz
==================== Summary ====================
Name: System-dtl.790 Author: dtl Time: 29 January 2016, 11:37:48.065507 pm UUID: 1ccde720-42a2-41fe-bcc4-cb9694d5af07 Ancestors: System-eem.789
Update vmStatisticsReportString to restore memory stats when running on pre-spur VM.
=============== Diff against System-eem.789 ===============
Hi David,
<snip>
I get that and I don't want to debilitate developing packages that run I both variants. But in the case of memory management the two systems are very different. Spur can't support object age. Spur provides pinning. The two have very different garbage collectors and compactors. In developing policy code to control the gcs it will inevitably require a different approach to effectively control either. So in the realm of GC (including this, which is reporting GC stats) I think it makes sense to let them diverge. AFAIA OSProcess doesn't depend on GC control facilities so it shouldn't affect you.
Way off topic, but speaking of OSProcess and Spur, I'm very much looking forward to trying RemoteTask on 64 bit Spur. I am expecting it to be very memory efficient even for extremely large object memories, due to the much improved memory layout that should localize changes to the object memory and reduce GC impact.
Dave
Where it may effect you is that pinning could conceivably allow you to lift up parameter processing to create the pointer arrays from that rather complex primitive into the image, given an addressOf: prim. Just as I hope the threaded FFI will make it possible to interact with output pipes much more reliably and efficiently.
Dave
_,,,^..^,,,_ (phone) best, Eliot
On Jan 29, 2016, at 12:39 PM, David T. Lewis lewis@mail.msen.com wrote:
Hi David,
<snip>
I get that and I don't want to debilitate developing packages that run I both variants. But in the case of memory management the two systems are very different. Spur can't support object age. Spur provides pinning. The two have very different garbage collectors and compactors. In developing policy code to control the gcs it will inevitably require a different approach to effectively control either. So in the realm of GC (including this, which is reporting GC stats) I think it makes sense to let them diverge. AFAIA OSProcess doesn't depend on GC control facilities so it shouldn't affect you.
Way off topic, but speaking of OSProcess and Spur, I'm very much looking forward to trying RemoteTask on 64 bit Spur. I am expecting it to be very memory efficient even for extremely large object memories, due to the much improved memory layout that should localize changes to the object memory and reduce GC impact.
Dave
Where it may effect you is that pinning could conceivably allow you to lift up parameter processing to create the pointer arrays from that rather complex primitive into the image, given an addressOf: prim. Just as I hope the threaded FFI will make it possible to interact with output pipes much more reliably and efficiently.
Then I need to write Mac make files, which is the only thing standing between me and a new release that includes the 63-bit sour Cogit.
Dave
_,,,^..^,,,_ (phone) best, Eliot
On Fri, Jan 29, 2016 at 11:43 AM, Eliot Miranda eliot.miranda@gmail.com wrote: <snip>
But in the case of memory management the two systems are very different. Spur can't support object age. Spur provides pinning. The two have very different garbage collectors and compactors. In developing policy code to control the gcs it will inevitably require a different approach to effectively control either. So in the realm of GC (including this, which is reporting GC stats) I think it makes sense to let them diverge.
So, would creating separate classes/pluggable instances for controlling garbageCollection be worthwhile? One is used for cog/nonSpur, one for cog/Spur, one for (maybe) interpreter/Spur?
Wouldn't that let it be very focused, while still giving full relevance for the other forks as needed? Or am I just missing the point here?
-cbc
squeak-dev@lists.squeakfoundation.org
-
Chris Cunningham -
Chris Muller -
David T. Lewis -
Eliot Miranda