... at http://www.mirandabanda.org/files/Cog/VM/VM.r3018/.
The main difference here is fixes to memory segment allocation on Spur. Linux and Mac OS can now grow the heap to 2.9Gb or there abouts, and Win XP can grow to 1.8Gb. I'm interested in reports on other Windows versions that don't have XP's 2Gb user process address space limit.
Here's my test script: | them | them := OrderedCollection new. [[them addLast: (ByteArray new: 16000000). Transcript cr; print: (Smalltalk vmParameterAt: 3) / (1024*1024.0) maxDecimalPlaces: 1; flush] repeat] on: OutOfMemory do: [:ex| 2 to: them size by: 2 do: [:i| them at: i put: nil. Smalltalk garbageCollect]]. Transcript cr; print: (Smalltalk vmParameterAt: 3) / (1024*1024.0) maxDecimalPlaces: 1; flush. them := nil. Smalltalk garbageCollect. Transcript cr; print: (Smalltalk vmParameterAt: 3) / (1024*1024.0) maxDecimalPlaces: 1; flush
And the README is: CogVM binaries as per VMMaker.oscog-eem.779/r3018.
Rewrite memory allocation on linux for Spur. Arrange that the heap can grow above 2Gb without any large initial alloc.
Rewrite platforms/unix/vm/sqUnixSpurMemory.c to stand alone. Use it in place of sqMacMemory.c with Spur on Mac OS. Now Spur can grow the heap to 2.9Gb on both linux (CentOS 5.3) and Mac OS X (10.6.8).
Rewrite Spur memory allocation on win32 similarly to unix. Can now allocate up to 1.8Gb on Windows XP (which has a 2Gb address space limit). Add a flag to indicate if the win32 exe is running as a console app and don't write to the in-window console if so.
Fix some sign issues with free space tallying to allow Spur to shrink memory and answer via primitiveVMParameter heap sizes above 2Gb.
Add longPrintInstancesOf:/longPrintInstancesWithClassIndex: for debugging.
Suggest reading a suitable README when the linux VM fails to spawn the heartbeat thread.
Add parameter 54 on Spur to answer totalFreeOldSpace.
Hi Eliot
On Windows 7 Ultimate 32bit, on a laptop with 3GB RAM, tested in normal & Safe Mode your test maxed out around 1870MB using the Cog Spur v3018.
Hope this helps
Paul
here is the data from one of the runs: 62.2 78.2 94.2 110.2 126.2 142.2 158.2 174.2 190.2 206.2 222.2 238.2 254.2 270.2 286.2 302.2 318.2 334.2 350.2 366.2 382.2 398.2 414.2 430.2 446.2 462.2 478.2 494.2 510.2 526.2 542.2 558.2 574.2 590.2 606.2 622.2 638.2 654.2 670.2 686.2 702.2 718.2 734.2 750.2 766.2 782.2 798.2 814.2 830.2 846.2 862.2 878.2 894.2 910.2 926.2 942.2 958.2 974.2 990.2 1006.2 1022.2 1038.2 1054.2 1070.2 1086.2 1102.2 1118.2 1134.2 1150.2 1166.2 1182.2 1198.2 1214.2 1230.2 1246.2 1262.2 1278.2 1294.2 1310.2 1326.2 1342.2 1358.2 1374.2 1390.2 1406.2 1422.2 1438.2 1454.2 1470.2 1486.2 1502.2 1518.2 1534.2 1550.2 1566.2 1582.2 1598.2 1614.2 1630.2 1646.2 1662.2 1678.2 1694.2 1710.2 1726.2 1742.2 1758.2 1774.2 1790.2 1806.2 1822.2 1838.2 1854.2 1870.2 1886.2 974.2 62.2
Eliot Miranda-2 wrote
... at http://www.mirandabanda.org/files/Cog/VM/VM.r3018/.
The main difference here is fixes to memory segment allocation on Spur. Linux and Mac OS can now grow the heap to 2.9Gb or there abouts, and Win XP can grow to 1.8Gb. I'm interested in reports on other Windows versions that don't have XP's 2Gb user process address space limit.
Here's my test script: | them | them := OrderedCollection new. [[them addLast: (ByteArray new: 16000000). Transcript cr; print: (Smalltalk vmParameterAt: 3) / (1024*1024.0) maxDecimalPlaces: 1; flush] repeat] on: OutOfMemory do: [:ex| 2 to: them size by: 2 do: [:i| them at: i put: nil. Smalltalk garbageCollect]]. Transcript cr; print: (Smalltalk vmParameterAt: 3) / (1024*1024.0) maxDecimalPlaces: 1; flush. them := nil. Smalltalk garbageCollect. Transcript cr; print: (Smalltalk vmParameterAt: 3) / (1024*1024.0) maxDecimalPlaces: 1; flush
And the README is: CogVM binaries as per VMMaker.oscog-eem.779/r3018.
Rewrite memory allocation on linux for Spur. Arrange that the heap can grow above 2Gb without any large initial alloc.
Rewrite platforms/unix/vm/sqUnixSpurMemory.c to stand alone. Use it in place of sqMacMemory.c with Spur on Mac OS. Now Spur can grow the heap to 2.9Gb on both linux (CentOS 5.3) and Mac OS X (10.6.8).
Rewrite Spur memory allocation on win32 similarly to unix. Can now allocate up to 1.8Gb on Windows XP (which has a 2Gb address space limit). Add a flag to indicate if the win32 exe is running as a console app and don't write to the in-window console if so.
Fix some sign issues with free space tallying to allow Spur to shrink memory and answer via primitiveVMParameter heap sizes above 2Gb.
Add longPrintInstancesOf:/longPrintInstancesWithClassIndex: for debugging.
Suggest reading a suitable README when the linux VM fails to spawn the heartbeat thread.
Add parameter 54 on Spur to answer totalFreeOldSpace.
-- best, Eliot
-- View this message in context: http://forum.world.st/New-Cog-VMs-available-tp4764823p4764840.html Sent from the Squeak VM mailing list archive at Nabble.com.
With appropriate settings [1], one can get 3GB on Windows. (I haven't tested this on Cog but once upon time used it in another project.) I don't know if this extra GB is worth it to anyone; given the other platforms provide above 2GB hopefully the bugs related to signed versus unsigned comparisons are already flushed out.
[1] http://msdn.microsoft.com/en-us/library/bb613473(v=VS.85).aspx
On Wed, Jun 25, 2014 at 4:13 PM, Eliot Miranda eliot.miranda@gmail.com wrote:
... at http://www.mirandabanda.org/files/Cog/VM/VM.r3018/.
The main difference here is fixes to memory segment allocation on Spur. Linux and Mac OS can now grow the heap to 2.9Gb or there abouts, and Win XP can grow to 1.8Gb. I'm interested in reports on other Windows versions that don't have XP's 2Gb user process address space limit.
Here's my test script: | them | them := OrderedCollection new. [[them addLast: (ByteArray new: 16000000). Transcript cr; print: (Smalltalk vmParameterAt: 3) / (1024*1024.0) maxDecimalPlaces: 1; flush] repeat] on: OutOfMemory do: [:ex| 2 to: them size by: 2 do: [:i| them at: i put: nil. Smalltalk garbageCollect]]. Transcript cr; print: (Smalltalk vmParameterAt: 3) / (1024*1024.0) maxDecimalPlaces: 1; flush. them := nil. Smalltalk garbageCollect. Transcript cr; print: (Smalltalk vmParameterAt: 3) / (1024*1024.0) maxDecimalPlaces: 1; flush
And the README is: CogVM binaries as per VMMaker.oscog-eem.779/r3018.
Rewrite memory allocation on linux for Spur. Arrange that the heap can grow above 2Gb without any large initial alloc.
Rewrite platforms/unix/vm/sqUnixSpurMemory.c to stand alone. Use it in place of sqMacMemory.c with Spur on Mac OS. Now Spur can grow the heap to 2.9Gb on both linux (CentOS 5.3) and Mac OS X (10.6.8).
Rewrite Spur memory allocation on win32 similarly to unix. Can now allocate up to 1.8Gb on Windows XP (which has a 2Gb address space limit). Add a flag to indicate if the win32 exe is running as a console app and don't write to the in-window console if so.
Fix some sign issues with free space tallying to allow Spur to shrink memory and answer via primitiveVMParameter heap sizes above 2Gb.
Add longPrintInstancesOf:/longPrintInstancesWithClassIndex: for debugging.
Suggest reading a suitable README when the linux VM fails to spawn the heartbeat thread.
Add parameter 54 on Spur to answer totalFreeOldSpace.
-- best, Eliot
On 26 Jun 2014, at 8:05 , Ryan Macnak rmacnak@gmail.com wrote:
With appropriate settings [1], one can get 3GB on Windows. (I haven't tested this on Cog but once upon time used it in another project.) I don't know if this extra GB is worth it to anyone; given the other platforms provide above 2GB hopefully the bugs related to signed versus unsigned comparisons are already flushed out.
[1] http://msdn.microsoft.com/en-us/library/bb613473(v=VS.85).aspx
For that, you’d also need a binary with the Large Address Aware bit set in the COFF-header, which the released binary does not have. It’s basically just a checkbox that yes, I assure you my application only treats addresses as unsigned, so please hand me more than 2GB of memory. Which is also the way to get >2GB with 32bit executables on 64-bit Windows as well, so it might be worthwhile finding the proper compiler flag.
Cheers, Henry
PS. Here’s the transcript from Windows 7 x64 on a modified version of released binary with the only change being LAA bit set to on manually:
60.7 60.7 76.7 92.7 108.7 124.7 140.7 156.7 172.7 188.7 204.7 220.7 236.7 252.7 268.7 284.7 300.7 316.7 332.7 348.7 364.7 380.7 396.7 412.7 428.7 444.7 460.7 476.7 492.7 508.7 524.7 540.7 556.7 572.7 588.7 604.7 620.7 636.7 652.7 668.7 684.7 700.7 716.7 732.7 748.7 764.7 780.7 796.7 812.7 828.7 844.7 860.7 876.7 892.7 908.7 924.7 940.7 956.7 972.7 988.7 1004.7 1020.7 1036.7 1052.7 1068.7 1084.7 1100.7 1116.7 1132.7 1148.7 1164.7 1180.7 1196.7 1212.7 1228.7 1244.7 1260.7 1276.7 1292.7 1308.7 1324.7 1340.7 1356.7 1372.7 1388.7 1404.7 1420.7 1436.7 1452.7 1468.7 1484.7 1500.7 1516.7 1532.7 1548.7 1564.7 1580.7 1596.7 1612.7 1628.7 1644.7 1660.7 1676.7 1692.7 1708.7 1724.7 1740.7 1756.7 1772.7 1788.7 1804.7 1820.7 1836.7 1852.7 1868.7 1884.7 1900.7 1916.7 1932.7 1948.7 1964.7 1980.7 1996.7 2012.7 2028.7 2044.7 2060.7 2076.7 2092.7 2108.7 2124.7 2140.7 2156.7 2172.7 2188.7 2204.7 2220.7 2236.7 2252.7 2268.7 2284.7 2300.7 2316.7 2332.7 2348.7 2364.7 2380.7 2396.7 2412.7 2428.7 2444.7 2460.7 2476.7 2492.7 2508.7 2524.7 2540.7 2556.7 2572.7 2588.7 2604.7 2620.7 2636.7 2652.7 2668.7 2684.7 2700.7 2716.7 2732.7 2748.7 2764.7 2780.7 2796.7 2812.7 2828.7 2844.7 2860.7 2876.7 2892.7 2908.7 2924.7 2940.7 2956.7 2972.7 2988.7 3004.7 3020.7 3036.7 3052.7 3068.7 3084.7 3100.7 3116.7 3132.7 3148.7 3164.7 3180.7 3196.7 3212.7 3228.7 3244.7 3260.7 3276.7 3292.7 3308.7 3324.7 3340.7 3356.7 3372.7 3388.7 3404.7 3420.7 3436.7 3452.7 3468.7 3484.7 3500.7 3516.7 3532.7 3548.7 3564.7 3580.7 3596.7 3612.7 3628.7 3644.7 3660.7 3676.7 3692.7 3708.7 3724.7 3740.7 3756.7 3772.7 3788.7 3804.7 3820.7 3836.7 3852.7 1948.7 44.7
Thanks Ryan, Thanks Henrik,
On Thu, Jun 26, 2014 at 1:17 AM, Henrik Johansen < henrik.s.johansen@veloxit.no> wrote:
On 26 Jun 2014, at 8:05 , Ryan Macnak rmacnak@gmail.com wrote:
With appropriate settings [1], one can get 3GB on Windows. (I haven't tested this on Cog but once upon time used it in another project.) I don't know if this extra GB is worth it to anyone; given the other platforms provide above 2GB hopefully the bugs related to signed versus unsigned comparisons are already flushed out.
[1] http://msdn.microsoft.com/en-us/library/bb613473(v=VS.85).aspx
For that, you’d also need a binary with the Large Address Aware bit set in the COFF-header, which the released binary does not have. It’s basically just a checkbox that yes, I assure you my application only treats addresses as unsigned, so please hand me more than 2GB of memory. Which is also the way to get >2GB with 32bit executables on 64-bit Windows as well, so it might be worthwhile finding the proper compiler flag.
I found this:
"It depends on your toolchain.
If you are using cygwin, i don't know. But for Mingw, a quick google for "LARGEADDRESSAWARE gnu ld" turns up "--large-address-aware". You have to pass it down to ld through gcc by '-Wl,--large-address-aware'. That is if you are using mingw with gnu ld. But beware, this is still 32-Bit, so somewhere below 4GB (3GB?) is you max.. Win64 support is still unfinished AFAIK (or in the latest and greatest CVS snapshots + still itchy).
Thank you in advance.
Greetings Jan"
Cheers, Henry
PS. Here’s the transcript from Windows 7 x64 on a modified version of released binary with the only change being LAA bit set to on manually:
60.7 60.7 76.7 92.7 108.7 124.7 140.7 156.7 172.7 188.7 204.7 220.7 236.7 252.7 268.7 284.7 300.7 316.7 332.7 348.7 364.7 380.7 396.7 412.7 428.7 444.7 460.7 476.7 492.7 508.7 524.7 540.7 556.7 572.7 588.7 604.7 620.7 636.7 652.7 668.7 684.7 700.7 716.7 732.7 748.7 764.7 780.7 796.7 812.7 828.7 844.7 860.7 876.7 892.7 908.7 924.7 940.7 956.7 972.7 988.7 1004.7 1020.7 1036.7 1052.7 1068.7 1084.7 1100.7 1116.7 1132.7 1148.7 1164.7 1180.7 1196.7 1212.7 1228.7 1244.7 1260.7 1276.7 1292.7 1308.7 1324.7 1340.7 1356.7 1372.7 1388.7 1404.7 1420.7 1436.7 1452.7 1468.7 1484.7 1500.7 1516.7 1532.7 1548.7 1564.7 1580.7 1596.7 1612.7 1628.7 1644.7 1660.7 1676.7 1692.7 1708.7 1724.7 1740.7 1756.7 1772.7 1788.7 1804.7 1820.7 1836.7 1852.7 1868.7 1884.7 1900.7 1916.7 1932.7 1948.7 1964.7 1980.7 1996.7 2012.7 2028.7 2044.7 2060.7 2076.7 2092.7 2108.7 2124.7 2140.7 2156.7 2172.7 2188.7 2204.7 2220.7 2236.7 2252.7 2268.7 2284.7 2300.7 2316.7 2332.7 2348.7 2364.7 2380.7 2396.7 2412.7 2428.7 2444.7 2460.7 2476.7 2492.7 2508.7 2524.7 2540.7 2556.7 2572.7 2588.7 2604.7 2620.7 2636.7 2652.7 2668.7 2684.7 2700.7 2716.7 2732.7 2748.7 2764.7 2780.7 2796.7 2812.7 2828.7 2844.7 2860.7 2876.7 2892.7 2908.7 2924.7 2940.7 2956.7 2972.7 2988.7 3004.7 3020.7 3036.7 3052.7 3068.7 3084.7 3100.7 3116.7 3132.7 3148.7 3164.7 3180.7 3196.7 3212.7 3228.7 3244.7 3260.7 3276.7 3292.7 3308.7 3324.7 3340.7 3356.7 3372.7 3388.7 3404.7 3420.7 3436.7 3452.7 3468.7 3484.7 3500.7 3516.7 3532.7 3548.7 3564.7 3580.7 3596.7 3612.7 3628.7 3644.7 3660.7 3676.7 3692.7 3708.7 3724.7 3740.7 3756.7 3772.7 3788.7 3804.7 3820.7 3836.7 3852.7 1948.7 44.7
On 26 June 2014 00:13, Eliot Miranda eliot.miranda@gmail.com wrote:
... at http://www.mirandabanda.org/files/Cog/VM/VM.r3018/.
The main difference here is fixes to memory segment allocation on Spur. Linux and Mac OS can now grow the heap to 2.9Gb or there abouts, and Win XP can grow to 1.8Gb. I'm interested in reports on other Windows versions that don't have XP's 2Gb user process address space limit.
Cool! CI's duly updated, and the latest build (http://build.squeak.org/job/SqueakTrunkOnSpur/16/) shows great improvement. It completes the entire test suite without crashing. I do see this:
sqAllocateMemorySegmentOfSizeAboveAllocatedSizeInto mmap: Cannot allocate memory sqAllocateMemorySegmentOfSizeAboveAllocatedSizeInto mmap: Cannot allocate memory sqAllocateMemorySegmentOfSizeAboveAllocatedSizeInto mmap: Cannot allocate memory sqAllocateMemorySegmentOfSizeAboveAllocatedSizeInto mmap: Cannot allocate memory sqAllocateMemorySegmentOfSizeAboveAllocatedSizeInto mmap: Cannot allocate memory sqAllocateMemorySegmentOfSizeAboveAllocatedSizeInto mmap: Cannot allocate memory
and there are 45 failing tests, but that's significant improvement!
frank
On 26 June 2014 00:13, Eliot Miranda eliot.miranda@gmail.com wrote:
... at http://www.mirandabanda.org/files/Cog/VM/VM.r3018/.
The main difference here is fixes to memory segment allocation on Spur. Linux and Mac OS can now grow the heap to 2.9Gb or there abouts, and Win XP can grow to 1.8Gb. I'm interested in reports on other Windows versions that don't have XP's 2Gb user process address space limit.
Windows 8.1:
44.7 60.7 76.7 92.7 108.7 124.7 140.7 156.7 172.7 188.7 204.7 220.7 236.7 252.7 268.7 284.7 300.7 316.7 332.7 348.7 364.7 380.7 396.7 412.7 428.7 444.7 460.7 476.7 492.7 508.7 524.7 540.7 556.7 572.7 588.7 604.7 620.7 636.7 652.7 668.7 684.7 700.7 716.7 732.7 748.7 764.7 780.7 796.7 812.7 828.7 844.7 860.7 876.7 892.7 908.7 924.7 940.7 956.7 972.7 988.7 1004.7 1020.7 1036.7 1052.7 1068.7 1084.7 1100.7 1116.7 1132.7 1148.7 1164.7 1180.7 1196.7 1212.7 1228.7 1244.7 1260.7 1276.7 1292.7 1308.7 1324.7 1340.7 1356.7 1372.7 1388.7 1404.7 1420.7 1436.7 1452.7 1468.7 1484.7 1500.7 1516.7 1532.7 1548.7 1564.7 1580.7 1596.7 1612.7 1628.7 1644.7 1660.7 1676.7 1692.7 1708.7 1724.7 1740.7 1756.7 1772.7 1788.7 908.7 44.7
frank
vm-dev@lists.squeakfoundation.org