[Vm-dev] Re: [Pharo-project] out of memory - cog on windows

Andreas Raab andreas.raab at gmx.de
Fri Apr 22 13:01:16 UTC 2011


FWIW, the reason for the 512MB limit has to do with Windows memory 
layout. When you're running applications that load dynamic libraries 
(directly or indirectly such as when using ODBC which loads further 
DLLs) Windows needs (sometimes a lot of) address space to map these DLLs 
in order to load them[*]. When the VM starts it grabs MAX_VIRTUAL_MEMORY 
from the OS which will consequently not be available to map DLLs into. 
We have found that depending on the libraries in use and depending on 
the overall system utilization, loading DLLs would fail seemingly "at 
random" which, after further investigation, we were able to track to 
reserving too much address space upfront. We were able to show 
experimentally, that changing the limit from 1GB to 512MB would on some 
machines make all the difference.

[*] This is true in particular for libraries that create more threads as 
the default Windows policy is to create threads with the stack size of 
the application executable. Thus a 1MB default stack in the application 
will by default create all further threads to be allocated with a 1MB 
stack size. Of course all of this is subject to various other conditions.

In the end we concluded that 512MB is a reasonable size for most apps 
and with 512MB we've never seen these random failures. You can increase 
the limit by recompiling the VM, but if you ship your app in diverse 
environments you should be aware of the potential issues.

Cheers,
   - Andreas

On 4/21/2011 22:44, Alain_Rastoul wrote:
>   
>
>
> Apparently, the vm allocates MAX_VIRTUAL_MEMORY and reduces by steps 
> of 128M until it
> succeeds.
> so I suppose it could be possible to allocate 2Gb and see how it runs 
> on a 1Gb system and if this is not too much stress for
> the system (thinking of the pagefile) ?
> Tudor is your vm a cog or non cog vm ?
>
>     "Mariano Martinez Peck" <marianopeck at gmail.com
>     <mailto:marianopeck at gmail.com>> a écrit dans le message de news:
>     BANLkTi=Tved13_KL7quOzemOXPnCcfWw+g at mail.gmail.com
>     <mailto:BANLkTi=Tved13_KL7quOzemOXPnCcfWw+g at mail.gmail.com>...
>
>     ------------------------------------------------------------------------
>
>
>     On Thu, Apr 21, 2011 at 10:21 PM, Tudor Girba
>     <tudor.girba at gmail.com <mailto:tudor.girba at gmail.com>> wrote:
>
>         Hi,
>
>         Should I to understand that the only way to enable more memory
>         is to recompile the VM? Does that mean that there is no way to
>         pass this information as a parameter like we can on Mac?
>
>
>     As far as I know you can pass a parameter, but even so, it won't
>     be able to allocate more than 512MB.
>     I can compile the VM for you with this change in 5 minutes. But I
>     am not sure that such simple code would make it work. I think such
>     limit is there because of something. Otherwise, it sounds stupid
>     imposing a limit just because.
>
>         The problem is that I cannot recompile the VM because I have
>         no access to a Windows machine. Is there one available that
>         provides more memory?
>
>
>     I don't think so, but start cc'ing the VM mailing list. You'd
>     probably receive more help.
>
>     Cheers
>
>     Mariano
>
>         Cheers,
>         Doru
>
>
>         On 21 Apr 2011, at 22:09, Alain_Rastoul wrote:
>
>         > Hi Tudor,
>         >
>         > There is a constant in sqWin32Alloc.h (platforms\win32\vm) :
>         > #define MAX_VIRTUAL_MEMORY 512*1024*1024
>         > you can change it to whatever you want and rebuild the vm,
>         > for exzmple give all the available memory less 256 M .
>         >
>         > HTH
>         >
>         > Regards
>         > Alain
>         >
>         > "Tudor Girba" <tudor.girba at gmail.com
>         <mailto:tudor.girba at gmail.com>> a écrit
>         > dans le message de news:
>         03B9389F-C719-44D0-B106-2AC78B120F56 at gmail.com...
>         > Hi,
>         >
>         > We have no specific startUp: methods in Moose. In any case,
>         the issue with
>         > opening the image does not seem to be related to startUp:.
>         >
>         > Is it really true that the Cog VM is limited to 512MB of memory?
>         >
>         > Cheers,
>         > Doru
>         >
>         >
>         > On 21 Apr 2011, at 14:27, Luc Fabresse wrote:
>         >
>         >> Hi Doru,
>         >>
>         >> 2011/4/21 Tudor Girba
>         >> <tudor.girba at gmail.com <mailto:tudor.girba at gmail.com>>
>         >> Hi,
>         >>
>         >>
>         >>
>         >> On Apr 21, 2011, at 14:06, Mariano Martinez Peck
>         >> <marianopeck at gmail.com <mailto:marianopeck at gmail.com>> wrote:
>         >>
>         >>>
>         >>>
>         >>> On Thu, Apr 21, 2011 at 1:58 PM, Tudor Girba
>         >>> <tudor.girba at gmail.com <mailto:tudor.girba at gmail.com>> wrote:
>         >>>> Hi again,
>         >>>>
>         >>>> I did not say what the problem was :). The problem was
>         that when
>         >>>> opening the image on Windows, he got a Space is low
>         message and the
>         >>>> image was not usable (see attachment).
>         >>>
>         >>> That's weird. Does moose have something on the startup
>         list?   something
>         >>> that can be bothering there?
>         >>
>         >> Not that I know of. Is there a way to check this?
>         >>
>         >> Classes should be registered using Smalltlak
>         addToStartUpList: aClass
>         >> Then aClass class>>#startUp: is executed at startup.
>         >> So implementors of #startUp: on Moose classes should give
>         you the answer.
>         >>
>         >> #Luc
>         >>
>         >>
>         >> Actually, using exactly the same process some days ago he
>         produced an
>         >> image of 190MB. This one works just fine on Windows. The
>         only difference
>         >> between the two is the size of the loaded data.
>         >>
>         >> It would be really bad news if the Windows vm would be so
>         severely limited
>         >> in terms of memory.
>         >>
>         >> Cheers,
>         >> Doru
>         >>
>         >>
>         >>>
>         >>> On Mac it worked just fine.
>         >>>>
>         >>>> Cheers,
>         >>>> Doru
>         >>>>
>         >>>>
>         >>>
>         >>>>
>         >>>>
>         >>>>
>         >>>> On 21 Apr 2011, at 12:52, Tudor Girba wrote:
>         >>>>
>         >>>>> Hi,
>         >>>>>
>         >>>>> I received a question from someone running a 200MB image
>         on Windows
>         >>>>> using Cog 2361.
>         >>>>>
>         >>>>> If I open the image on Mac, it works just fine.
>         Unfortunately, I do
>         >>>>> not have a Windows machine around, and I cannot test but
>         I believe it
>         >>>>> should be solvable by increasing the allocated memory.
>         >>>>>
>         >>>>> On Mac, I would run it with: ./Croquet -memory 1500m
>         >>>>>
>         >>>>> Can anyone help me with the right incantation for Windows?
>         >>>>>
>         >>>>> Cheers,
>         >>>>> Doru
>         >>>>>
>         >>>>>
>         >>>>> --
>         >>>>> www.tudorgirba.com <http://www.tudorgirba.com>
>         >>>>>
>         >>>>> "What we can governs what we wish."
>         >>>>>
>         >>>>>
>         >>>>>
>         >>>>> <Space is low.png>
>         >>>>
>         >>>> --
>         >>>> www.tudorgirba.com <http://www.tudorgirba.com>
>         >>>>
>         >>>> "Yesterday is a fact.
>         >>>> Tomorrow is a possibility.
>         >>>> Today is a challenge."
>         >>>>
>         >>>>
>         >>>>
>         >>>
>         >>>
>         >>>
>         >>>
>         >>> --
>         >>> Mariano
>         >>> http://marianopeck.wordpress.com
>         >>>
>         >>
>         >
>         > --
>         > www.tudorgirba.com <http://www.tudorgirba.com>
>         >
>         > "Beauty is where we see it."
>         >
>         >
>         >
>         >
>         >
>         >
>         >
>         >
>
>         --
>         www.tudorgirba.com <http://www.tudorgirba.com>
>
>         "If you interrupt the barber while he is cutting your hair,
>         you will end up with a messy haircut."
>
>
>
>
>
>     -- 
>     Mariano
>     http://marianopeck.wordpress.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20110422/baeba860/attachment-0001.htm


More information about the Vm-dev mailing list